Text Classification Baseline

Pipeline for fast building text classification baselines with TF-IDF + LogReg.

Usage

Instead of writing custom code for specific text classification task, you just need:

• install pipeline:

script
pip install text-classification-baseline

• run pipeline:
• either in terminal:

script
text-clf-train --pathtoconfig config.yaml

• or in python:

import text_clf
model, targetnamesmapping = textclf.train(pathto_c)

NOTE: more about config file here.

No data preparation is needed, only a csv file with two raw columns (with arbitrary names):

• text
• target

The target can be presented in any format, including text - not necessarily integers from 0 to n_classes-1.

Config

The user interface consists of two files:

• config.yaml - general configuration with sklearn TF-IDF and LogReg parameters
• hyperparams.py - sklearn GridSearchCV parameters

Change config.yaml and hyperparams.py to create the desired configuration and train text classification model with the following command:

• terminal:

script
text-clf-train --pathtoconfig config.yaml

• python:

import text_clf
model, targetnamesmapping = textclf.train(pathto_c)

Default config.yaml:

seed: 42 pathtosave_folder: models experiment_name: model

data
data:
  traindatapath: data/train.csv
  testdatapath: data/test.csv
  sep: ','
  text_column: text
  targetcolumn: targetname_short
preprocessing
(included in resulting model pipeline, so preserved for inference)
preprocessing:
  lemmatization: null  # pymorphy2
tf-idf
tf-idf:
  lowercase: true
  ngram_range: (1, 1)
  max_df: 1.0
  min_df: 1
logreg
logreg:
  penalty: l2
  C: 1.0
  class_weight: balanced
  solver: saga
  n_jobs: -1
grid-search
grid-search:
  dogridsearch: false
  gridsearchparams_path: hyperparams.py

NOTE: grid search is disabled by default, to use it set dogridsearch: true.

NOTE: tf-idf and logreg are sklearn TfidfVectorizer and LogisticRegression parameters correspondingly, so you can parameterize instances of these classes however you want. The same logic applies to grid-search which is sklearn GridSearchCV parametrized with hyperparams.py.

Output

After training the model, the pipeline will return the following files:

• model.joblib - sklearn pipeline with TF-IDF and LogReg steps
• targetnames.json - mapping from encoded target labels from 0 to nclasses-1 to it names
• config.yaml - config that was used to train the model
• hyperparams.py - grid-search parameters (if grid-search was used)
• logging.txt - logging file

Additional functions

• textclf.tokenfrequency.gettokenfrequency(pathtoconfig) -
  get token frequency of train dataset according to the config file parameters

Only for binary classifiers:

• textclf.prroccurve.getprecisionrecallcurve(pathtomodel_folder) -
  get precision and recall metrics for precision-recall curve
• textclf.prroccurve.getroccurve(pathtomodelfolder) -
  get false positive rate (fpr) and true positive rate (tpr) metrics for roc curve
• textclf.prroccurve.plotprecisionrecallcurve(precision, recall) -
  plot precision-recall curve
• textclf.prroccurve.plotroc_curve(fpr, tpr) -
  plot roc curve
• textclf.prroccurve.plotprecisionrecallf1curvesfor_thresholds(precision, recall, thresholds) -
  plot precision, recall, f1-score curves for probability thresholds

Requirements

Python >= 3.6

Citation

If you use text-classification-baseline in a scientific publication, we would appreciate references to the following BibTex entry:

@misc{dayyass2021textclf,
    author       = {El-Ayyass, Dani},
    title        = {Pipeline for training text classification baselines},
    howpublished = {\url{https://github.com/dayyass/text-classification-baseline}},
    year         = {2021}
}