RuleOpt

Optimization-Based Rule Learning for Classification

RuleOpt is an optimization-based rule learning algorithm designed for classification problems. Focusing on scalability and interpretability, RuleOpt utilizes linear programming for rule generation and extraction. An earlier version of this work is available in our journal paper.

The Python library ruleopt is capable of extracting rules from ensemble models, and it also implements a novel rule generation scheme. The library ensures compatibility with existing machine learning pipelines, and it is especially efficient for tackling large-scale problems.

Here are a few highlights of ruleopt:

Efficient Rule Generation and Extraction: Leverages linear programming for scalable rule generation (stand-alone machine learning method) and rule extraction from trained random forest and boosting models.
Interpretability: Prioritizes model transparency by assigning costs to rules in order to achieve a desirable balance with accuracy.
Integration with Machine Learning Libraries: Facilitates smooth integration with well-known Python libraries scikit-learn, LightGBM, and XGBoost, and existing machine learning pipelines.
Extensive Solver Support: Supports a wide array of solvers, including HiGHS, Gurobi and CPLEX.

Installation

To install ruleopt, use the following pip command:

pip install ruleopt

Usage

To use ruleopt, you need to initialize the ruleopt class with your specific parameters and fit it to your data. Here's a basic example:

from sklearn.modelselection import traintest_split
from sklearn.datasets import load_iris
from ruleopt import RUGClassifier from ruleopt.rule_cost import Gini from ruleopt.solver import HiGHSSolver
Set a random state for reproducibility
random_state = 42
Load the Iris dataset
X, y = loadiris(returnX_y=True)
Split the dataset into training and testing sets
Xtrain, Xtest, ytrain, ytest = traintestsplit(
    X, y, testsize=0.2, randomstate=random_state
)
Define tree parameters
treeparameters = {"maxdepth": 3, "class_weight": "balanced"}
solver = HiGHSSolver() rule_cost = Gini()
Initialize the RUGClassifier with specific parameters
rug = RUGClassifier(
    solver=solver,
    randomstate=randomstate,
    maxrmpcalls=20,
    rulecost=rulecost,
    **tree_parameters,
)
Fit the RUGClassifier to the training data
rug.fit(Xtrain, ytrain)
Predict the labels of the testing set
ypred = rug.predict(Xtest)