🌟 SplitLight: Explore Your RecSys Dataset and Split

SplitLight is a lightweight framework for auditing recommender-system datasets and evaluating splitting results. Its main goal is to help you produce trustworthy data preprocessing and splits and justify split choices via transparent, data-driven diagnostics. SplitLight can be used in Jupyter/Python scripts for comprehensive analysis and offers an easy-to-use Streamlit UI for interactive exploration, health checks, and side-by-side comparisons.

Why SplitLight?

• Trustworthy evaluation — Poor or inconsistent train/validation/test splits lead to overoptimistic metrics and non-reproducible research. SplitLight helps you detect leakage, cold-start issues, and distribution shifts before training.
• Transparent diagnostics — Instead of treating the split as a black box, you get concrete stats: shared interactions, temporal overlap, leaked targets, cold user/item shares, and temporal deltas between input and target.
• Flexible workflow — Use the Streamlit app for ad-hoc audits, or call src/stats and src/splits from your own pipelines and notebooks (see the demo notebook).

SplitLight in a data-preparation pipeline. From the raw dataset to split subsets, SplitLight audits data, flags problems, and enables side-by-side comparison of alternative splits to justify the selected evaluation protocol.

[!NOTE]

See short video walkthrough of SplitLight motivation and usage.

Quick Start

pip install -r requirements.txt
export PYTH
export SEQSPLITSDATA_PATH=$(pwd)/data

• Requirements file: requirements.txt
• Your datasets live under data/ (see layout below).

Data Layout

SplitLight expects each dataset under data/<DatasetName>/ with either a raw.csv (original schema) or preprocessed.csv (standard schema).

• raw.csv (optional): original column names are defined in runs/configs/dataset/<DatasetName>.yaml
• preprocessed.csv: standardized columns: userid, itemid, timestamp (seconds)
• After splitting, a per-split subfolder contains: train.csv, validationinput.csv, validationtarget.csv, testinput.csv, testtarget.csv

data/
├── Beauty/
│   ├── raw.csv                          # optional
│   ├── preprocessed.csv
│   └── leave-one-out/                   # example split folder
│       ├── train.csv
│       ├── validation_input.csv
│       ├── validation_target.csv
│       ├── test_input.csv
│       └── test_target.csv
└── Diginetica/
    ├── preprocessed.csv
    └── GTS-q09-valbytime-target_last/
        ├── train.csv
        ├── validation_input.csv
        ├── validation_target.csv
        ├── test_input.csv
        └── test_target.csv

Streamlit UI

Launch the app for interactive dataset and split audits.

export PYTH
export SEQSPLITSDATA_PATH=$(pwd)/data
streamlit run SplitLight.py

For better experience, zoom out the page to adjust to your screen size.

What you can explore:

• Core and temporal statistics per subset and vs. reference
• Interactions distribution over time
• Repeated consumption patterns (non-unique and consecutive repeats)
• Temporal leakage: shared interactions, overlap, and “leakage from future”
• Cold-start exposure of users and items
• Compare splits side-by-side and analyze time-gap deltas between input and target

What SplitLight Checks

| Category | Description | | ----------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Dataset and Subsets | Analyze raw and preprocessed data in terms of core and temporal statistics and compare. Identify repeated consumption patterns. Visualize interactions distribution over time. | | Subsets and Splits | Analyze split data in terms of core and temporal statistics and compare subsets with full data. Identify and visualize presence of data leakage. Quantify and visualize user and item cold start. | | Compare splits | Compare different splits in terms of core and temporal statistics. Identify distribution shifts for target subset. |

You can also run these checks manually using functions from the src/stats module for custom analyses or integration into your own pipelines (see demo notebook).

Streamlit Summary Page

What It Provides

• Instant snapshot of dataset quality and split integrity
• Compact visualization of core, temporal, and leakage statistics
• Color-coded signals to highlight potential issues at a glance

• 🟢 OK — within expected bounds
• 🟡 Need Attention — mild irregularity detected
• 🔴 Warning — potential data issue or leakage risk

Configuration

Project Structure (Key Parts)

• src/stats/ — Core diagnostics: base (core/temporal stats), leaks, cold, duplicates, temporal, plots. Use these in scripts or notebooks for custom analyses.
• streamlit_ui/pages/ — Streamlit pages for load, Summary, core/temporal stats, repeated consumption, leakage, cold start, and split comparison.
• runs/ — CLI entrypoints and Hydra configs: preprocess.py, split.py, trainrs.py; configs under runs/configs/ (dataset, split, preprocess, trainrs, model).

FAQ

• Q: Can I use Parquet files?

• Q: Do I need raw.csv?

• Q: What time unit is timestamp?

• Q: I only have raw interaction logs. How do I start?

• Q: How do I use SplitLight in my own Python code?

See runs/README.md for more detailed explanation on CLI tools and experimental setup for splitting results in /data dir.

• Config: runs/configs/preprocess.yaml
• Dataset column mapping: runs/configs/dataset/.yaml
• Output: data//preprocessed.csv

Split your dataset using Leave-One-Out (LOO) or Global Time Split (GTS) strategies. See src/splits.py for implementation details.

@misc{splitlight2026,
      title={SplitLight: An Exploratory Toolkit for Recommender Systems Datasets and Splits}, 
      author={Anna Volodkevich and Dmitry Anikin and Danil Gusak and Anton Klenitskiy and Evgeny Frolov and Alexey Vasilev},
      year={2026},
      eprint={2602.19339},
      archivePrefix={arXiv},
      primaryClass={cs.IR}
}