CARTE:
Pretraining and Transfer for Tabular Learning

This repository contains the implementation of the paper CARTE: Pretraining and Transfer for Tabular Learning.

CARTE is a pretrained model for tabular data by treating each table row as a star graph and training a graph transformer on top of this representation.

Colab Examples (Give it a test):

• CARTERegressor on Wine Poland dataset
• CARTEClassifier on Spotify dataset

Other datasets are available for testing: datasets

[!WARNING]

This library is currently in a phase of active development. All features are subject to change without prior notice. If you are interested in collaborating, please feel free to reach out by opening an issue or starting a discussion.

01 Install 🚀

The library has been tested on Linux, MacOSX and Windows.

CARTE-AI can be installed from PyPI:

pip install carte-ai
pip install huggingface_hub

Post installation check

After a correct installation, you should be able to import the module without errors:

import carte_ai

02 CARTE-AI example on sampled data step by step ➡️

1️⃣ Load the Data 💽

import pandas as pd
from carteai.data.loaddata import *
num_train = 128  # Example: set the number of training groups/entities random_state = 1  # Set a random seed for reproducibility Xtrain, Xtest, ytrain, ytest = winapl(numtrain, random_state) print("Wina Poland dataset:", Xtrain.shape, Xtest.shape)

2️⃣ Convert Table 2 Graph 🪵

The basic preparations are:

• preprocess raw data
• load the prepared data and configs; set train/test split
• generate graphs for each table entries (rows) using the Table2GraphTransformer
• create an estimator and make inference

import fasttext from huggingfacehub import hfhub_download from carte_ai import Table2GraphTransformer

modelpath = hfhubdownload(repoid="hi-paris/fastText", filename="cc.en.300.bin")
preprocessor = Table2GraphTransformer(fasttextmodelpath=model_path)
Fit and transform the training data
Xtrain = preprocessor.fittransform(Xtrain, y=ytrain)
Transform the test data
Xtest = preprocessor.transform(Xtest)

3️⃣ Make Predictions🔮

For learning, CARTE currently runs with the sklearn interface (fit/predict) and the process is:

• Define parameters
• Set the estimator
• Run 'fit' to train the model and 'predict' to make predictions

from carte_ai import CARTERegressor, CARTEClassifier
Define some parameters
fixed_params = dict()
fixedparams["nummodel"] = 10 # 10 models for the bagging strategy
fixedparams["disablepbar"] = False # True if you want cleanness
fixedparams["randomstate"] = 0
fixed_params["device"] = "cpu"
fixedparams["njobs"] = 10
fixedparams["pretrainedmodelpath"] = configdirectory["pretrained_model"]
Define the estimator and run fit/predict
estimator = CARTERegressor(**fixed_params) # CARTERegressor for Regression estimator.fit(X=Xtrain, y=ytrain) ypred = estimator.predict(Xtest)
Obtain the r2 score on predictions
score = r2score(ytest, y_pred) print(f"\nThe R2 score for CARTE:", "{:.4f}".format(score))

03 Reproducing paper results ⚙️

➡️ installation instructions setup paper

04 Contribute to the package 🚀

➡️ read the contributions guidelines

05 Star History ⭐️

06 CARTE-AI references 📚

@article{kim2024carte,
  title={CARTE: pretraining and transfer for tabular learning},
  author={Kim, Myung Jun and Grinsztajn, L{\'e}o and Varoquaux, Ga{\"e}l},
  journal={arXiv preprint arXiv:2402.16785},
  year={2024}
}