Prospectus Summarization Using BertSum

This code is for NLP final report Prospectus Summarization Using BertSum

Goal

We aim to use a deep-learning model called BertSum to summarize IPO prospectus. IPO prospectus is an important source of information. It provides information on business model, competition, risks and opportunities, and financial situations. However, it is usually a very long legal document. It could be very time-consuming to go through the whole legal document. Therefore, in order to help generate a big picture of the company, we tried to use BertSum to further condense summary part. This would be helpful especially when you are not familiar with the company and the industry that it operates in.

Web Scraping and Data Cleaning

See .\getprospectus\getprospectus.py. You need the Company Name as well as CIK to get the 424B prospectus from U.S. SECURITIES AND EXCHANGE COMMISSION website. Commonly, it will create 6 text files:

• raw prospectus
• raw summary
• cleaned prospectus
• cleaned summary
• cleaned prospectus with [CLS][SEP] tokens
• cleaned summary with [CLS][SEP] tokens

TextRank & TextTeaser

We compare the performance of BertSum with TextRank & TextTeaser. In ./textrank and ./textteaser folder, you will see code, raw text and summarized text. Notice that this two models are just for reference. We would like to show how powerful BertSum is when comparing with obsolete algos.

|models|TextTeaser & TextRank|BertSum| |---|---|---| |Categories|Extractive Only|Extractive & Abstractive| |Unit of Summarization|Sentences|Flexible| |Dealing with Polysemy Problem|No|Yes| |Training series for different cases|No|Yes|

Preparation

We borrowed the idea from Fine-tune BERT for Extractive Summarization and Text Summarization with Pretrained Encoders. The author has given us well-trained models and preprocessed data already. We can directly download from her Google Drive.

Download the processed data

Pre-processed data, unzip the zipfile and put all .pt files into ./bertdata

Trained models

CNN/DM Extractive, unzip the zipfile and put .pt file into ./models/ext

CNN/DM Abstractive, unzip the zipfile and put .pt file into ./models/abs

Package Requirements

Python 3.6

torch==1.1.0 pytorch_transformers tensorboardX multiprocess pyrouge

Usage

Extractive

python train.py -task ext -mode testtext -testfrom MODELPATH -textsrc RAWTEXTPATH -resultpath OUTPUTPATH -logfile LOGPATH -visible_gpus -1

Absractive

python train.py -task abs -mode testtext -testfrom MODELPATH -textsrc RAWTEXTPATH -resultpath OUTPUTPATH -logfile LOGPATH -visible_gpus -1