QuantPits

An advanced, production-ready quantitative trading system built on top of Microsoft Qlib. This system provides a complete end-to-end pipeline for weekly and daily frequency trading, featuring modular architecture, multi-instance isolation (Workspaces), ensemble modeling, execution analytics, and interactive dashboards.

📄 Read our paper: arXiv:2604.11477

🌐 中文版本 (README_zh.md)

Note: We welcome contributions! If you find a bug or have a feature suggestion, please feel free to open an Issue or submit a Pull Request.

🚀 Key Features

• Multi-Workspace Isolation: Spin up independent "Pits" for different markets (e.g., CSI300, CSI500) or configurations without duplicating code.
• Component-Based Pipeline:

- Train & Predict: Support for both full and incremental training on multiple models (LSTM, GRU, Transformers, LightGBM, GATs). - Brute Force & Ensemble: High-performance (CuPy accelerated) brute force combination finding, multidimensional Out-Of-Sample (OOS) pool filtering, and intelligent signal fusion. - Orders & Execution: Generate actionable buy/sell signals with TopK/DropN logic and analyze micro-friction (slippage, delay costs). - Extensible Broker Adapters: Decoupled settlement parser supporting arbitrary broker terminal formats (e.g., Guotai Junan).

• Rich Observability: Two interactive streamlit dashboards for macro portfolio performance and micro rolling health monitoring.
• Resilient Infrastructure: Automatic checkpoints, JSON tracking for model registries, and daily/weekly logs.
• OOM-RL Intelligent Feedback Loop: Multi-agent deep analysis + LLM Critic decision-making + sandboxed Playground execution, closing the loop from analysis to automated optimization.

📂 Architecture Overview

The system strictly decouples the Engine (Code) from the Workspace (Config & Data):

QuantPits/
├── docs/                   # Detailed system manuals (00-08, 30+, 70)
├── ui/                     # Streamlit interactive dashboards
│   ├── dashboard.py        # Macro performance app
│   └── rolling_dashboard.py# Temporal strategy health app
├── quantpits/              # Core logic engine and components
│   └── scripts/            # Pipeline execution scripts
│
└── workspaces/             # Isolated trading instances
    └── Demo_Workspace/     # Example configured instance
        ├── config/         # Trading bounds, model registry, cashflow
        ├── data/           # Order logs, holding logs, daily amount
        ├── output/         # Model predictions, fusion results, reports
        ├── mlruns/         # MLflow tracking logs
        └── run_env.sh      # Environment activation script

🛠️ Quick Start

1. Requirements

Ensure you have a working installation of Qlib. The engine officially supports Python 3.8 to 3.12. Then install the extra dependencies:

pip install -r requirements.txt
(Optional) Install the engine as a package for global access:
pip install -e .

(Note: For GPU-accelerated brute force combinatorial backtesting, install cupy-cuda11x or cupy-cuda12x depending on your local CUDA version)

2. Prepare Market Data

Before running the engine, ensure you have the required Qlib dataset downloaded to your local machine (e.g., ~/.qlib/qlibdata/cndata):

# Example: Download 1D data for the Chinese market
python -m qlib.run.getdata qlibdata --targetdir ~/.qlib/qlibdata/cn_data --region cn --version v2

Note: This dataset contains massive historical market data. The initial download may require tens of GBs of disk space and a considerable amount of time. Please be patient. For daily frequency data, you can also reference this repo: https://github.com/chenditc/investment_data

By default, all scripts read the data from ~/.qlib/qlibdata/cndata. To override this on a per-workspace basis, uncomment and modify the relevant lines in run_env.sh:

export QLIBDATADIR="/path/to/your/qlib_data"
export QLIB_REGION="cn"   # or "us"

3. Activate a Workspace

Every action must be performed within the context of an active workspace. We provide a Demo_Workspace to get you started:

cd QuantPits/
source workspaces/DemoWorkspace/runenv.sh

Optionally validate the workspace configs before running a pipeline:

python -m quantpits.tools.validateworkspace --workspace workspaces/DemoWorkspace --read-only

4. Run the Pipeline

Once activated, you can execute the minimal routine loop using the quantpits scripts (or you can simply run make run-daily-pipeline from the repository root):

# 0. Update Daily Market Data
Note: This engine assumes underlying Qlib data has been updated (e.g., via external Cron). 
If not, update it first.
1. Train models (Required for first-time setup or retraining)
python -m quantpits.scripts.static_train --full
2. Generate predictions from existing models
python -m quantpits.scripts.static_train --predict-only --all-enabled
3. Fuse predictions using your combo configs
python -m quantpits.scripts.ensemble_fusion --from-config-all
4. Process previous week's live trades (Post-Trade)
python -m quantpits.scripts.prodposttrade
5. Generate new Buy/Sell orders based on current holdings
python -m quantpits.scripts.order_gen
6. Deep Analysis (recommended weekly)
python -m quantpits.scripts.rundeepanalysis

5. Launch Dashboards

To view the interactive analytics of your active workspace:

# Portfolio Execution and Holding Dashboard
streamlit run ui/dashboard.py
Rolling Strategy Health & Factor Drift Dashboard
streamlit run ui/rolling_dashboard.py

🏗️ Creating a New Workspace

To spin up a new strategy for a different index (e.g., CSI 500), use the scaffolding utility:

python -m quantpits.scripts.init_workspace \
  --source workspaces/Demo_Workspace \
  --target workspaces/CSI500_Base

This will cleanly clone the configuration files and generate empty data/, output/, and mlruns/ directories, completely isolated from your other trading environments. You then simply source workspaces/CSI500Base/runenv.sh to start operating in it.

📖 Documentation

For a deep dive into each module, refer to the documentation in docs/:

• 70_WALKTHROUGH.md (End-to-End Walkthrough — Start Here!)
• 00SYSTEMOVERVIEW.md (System Architecture & Workflows)
• 01TRAININGGUIDE.md
• 02BRUTEFORCE_GUIDE.md
• 03ENSEMBLEFUSION_GUIDE.md
• 30ROLLINGTRAINING_GUIDE.md (Rolling Training: Sliding Window Training)
• 50-54 OOM-RL series: Multi-Agent Deep Analysis, LLM Critic, Playground execution & promotion
• ...and more.

All documentation is available in both Chinese and English (docs/en/).

📜 License

• Codebase: MIT License.
• Research Paper (Publication/): Creative Commons Attribution-ShareAlike 4.0 International (CC-BY-SA 4.0) License.