Python3 implementation of the Schwartz-Hearst algorithm for extracting abbreviation-definition pairs
Extraction of abbreviation-definition pairs
Version: 0.2.5
This is a Python3 implementation of the Schwartz-Hearst algorithm for identifying abbreviations and their corresponding definitions in free text[1].
The original implementation is in Java, and Vincent Van Asch created a Python2 implementation at
http://www.cnts.ua.ac.be/~vincent/scripts/abbreviations.py
- NB: As of March 2019 this link appears to be dead.
This version outputs a Python dictionary of abbreviation:definition pairs.
Installation for command-line use
pip install -r requirements.txtUsage
From the command line
python abbreviations/schwartz_hearst.py
Installation as a module
python3 setup.py install or
pip install abbreviations
Usage
from abbreviations import schwartz_hearst # By default, the most recently encountered definition for each term is returned pairs = schwartzhearst.extractabbreviationdefinitionpairs(doc_text='The emergency room (ER) was busy') pairs = schwartzhearst.extractabbreviationdefinitionpairs(filepath='
# If multiple definitions are encountered for each term, you might want to return the most common for each pairs = schwartzhearst.extractabbreviationdefinitionpairs(doctext='...', mostcommon_definition=True) # ... or you might want to return the first encountered definition for each pairs = schwartzhearst.extractabbreviationdefinitionpairs(doctext='...', firstdefinition=True) # when using a longer text, the format is line-separated sentences: import nltk sentences = nltk.senttokenize(longertext) pairs = schwartzhearst.extractabbreviationdefinitionpairs(doc_text='\n'.join(sentences))
[1] A. Schwartz and M. Hearst (2003) A Simple Algorithm for Identifying Abbreviations Definitions in Biomedical Text. Biocomputing, 451-462.
