Applied Machine Learning for Trading: EURUSD Forecasting with Time Series Transformer

⚠️ Important Disclaimer

This project is for educational and research purposes only. It is not financial advice.

The software, models, and data provided in this repository are intended to demonstrate concepts in algorithmic trading and machine learning. They are not designed or tested for use in live trading environments.

High Risk: Trading in financial markets is inherently risky. Any use of this code, or concepts derived from it, for live trading is done at your own risk. You are solely responsible for any financial losses you may incur.
No Warranty: This software is provided "as is" without warranty of any kind, express or implied. The author makes no guarantees about its performance, accuracy, or profitability.
Regulatory Status (EU AI Act): The AI models within this repository could be subject to regulations such as the EU AI Act if used in a commercial or live trading context. By downloading or using this code, you acknowledge that you are responsible for ensuring your own compliance with all applicable laws and regulations.
Not for Professional Use: This is not a commercial-grade trading tool. Do not use this software with real money.

Short Description

This project develops and fine-tunes a TimeSeriesTransformer model to forecast EURUSD 5-minute closing prices. It serves as a modern, attention-based counterpart to a baseline LSTM model developed in a separate repository. The primary goal is to create a robust predictive "tool" that can generate signals for an algorithmic trading strategy and to compare its performance against a traditional recurrent architecture.

Project Objective

The objective is to build, evaluate, and systematically optimize a TimeSeriesTransformer model for short-term currency exchange rate prediction. This project documents the experimentation process required to fine-tune the model on a large financial dataset and provides a clear performance baseline that can be directly compared to other architectures like LSTMs.

Dataset

Asset: EURUSD (Euro / US Dollar)
Frequency: 5-minute intervals
Optimal Period: 10 years. An experiment was conducted using 20 years of historical data, but performance degraded. This suggests that the market regimes from 15-20 years ago are less relevant for predicting current market behavior, and a more focused 10-year dataset yields a more accurate model.
Source: EURUSD5m10Yea.csv (Included in the repository)
Columns used: Timestamp (derived), Close

Features

Systematic hyperparameter tuning documented in EXPERIMENTS.md.
A definitive training script (TrainTransformerEURUSD_Model.ipynb) that uses the optimal hyperparameters.
A robust PyTorch training loop with validation-based early stopping to prevent overfitting.
Export of final, synchronized model artifacts (.pth, .pkl, .json) ready for deployment.

Technologies Used

Python 3.9+
PyTorch
Hugging Face Transformers & Accelerate
Pandas & NumPy
Scikit-learn
Matplotlib
TQDM
Joblib

Setup and Installation

Clone the repository:

git clone https://github.com/ilahuerta-IA/applied-ml-trading-transformer-eurusd.git
    cd applied-ml-trading-transformer-eurusd

(Recommended) Create a virtual environment:

python -m venv venv
    source venv/bin/activate  # On Windows: venv\Scripts\activate

Install dependencies:

pip install -r requirements.txt

Usage and Model Training

The final, optimized training process is contained within the Jupyter Notebook: TrainTransformerEURUSD_Model.ipynb.

This notebook is pre-configured with the best hyperparameters found during the research phase (documented in EXPERIMENTS.md). To generate the final, deployable set of model artifacts, simply run all cells in the notebook. Upon completion, a new, perfectly synchronized set of deployment artifacts will be saved into the Models/ directory.

Performance Comparison vs. LSTM

This project's primary value is in its direct comparison to a well-optimized LSTM model from a parallel research effort. After comprehensive tuning and a definitive retraining run with a valid configuration, the final TimeSeriesTransformer model demonstrates superior predictive accuracy and robustness.

| Model | Test Set MAE (EURUSD) | Test Set MAE (Pips) | Test Set RMSE (EURUSD) | | :---------------------------------- | :-------------------- | :------------------ | :--------------------- | | Optimized LSTM (V1.0) | 0.000237 | 2.37 | 0.000417 | | TimeSeriesTransformer (Final) | 0.000203 | 2.03 | 0.000318 |

(Note: The LSTM result is sourced from the reference repository.)

Final Conclusion

The final optimized TimeSeriesTransformer model achieved a Test Set Mean Absolute Error of 0.000203 (~2.03 pips).

This result is superior to the LSTM baseline in two key ways:

Higher Accuracy: The MAE is 14% lower than the LSTM's, indicating a more precise average prediction.
Greater Robustness: The RMSE is 24% lower, indicating the Transformer makes significantly fewer large, erroneous predictions, which is critical for risk management in a live trading environment.

This project successfully demonstrates that for this large, high-frequency financial dataset, the modern attention-based Transformer architecture provides a measurable performance edge over a traditional, highly-optimized recurrent neural network.

Exported Model Artifacts & Deployment Notes

After running the final training notebook, a complete and synchronized package for deployment is saved in the Models/ directory.

besttransformermodel.pth: The trained model's state dictionary (the weights).
target_scaler.pkl: The fitted StandardScaler object for data normalization.
model_config.json: A JSON file containing the model's architectural hyperparameters.

Critical Note on `CONTEXTLENGTH` and `lags``sequence`

A key architectural requirement of the Hugging Face TimeSeriesTransformer is that the CONTEXTLENGTH (the main lookback window) must be greater than the maximum value in the lagssequence.

During initial experiments, a ValueError was encountered because a configuration with CONTEXTLENGTH=30 was incompatible with a lagssequence containing a lag of 21. The final, optimal model was retrained with a valid configuration (CONTEXTLENGTH=30 and lagssequence=[1, 2, 3, 4, 5, 6, 7]), which resolved this issue. This is a critical check for anyone adapting this model.

A Note on Versioning and Reproducibility

To avoid potential library versioning conflicts (e.g., scikit-learn InconsistentVersionWarning) or model configuration mismatches, the recommended approach for deployment is to re-run the TrainTransformerEURUSD_Model.ipynb notebook in your target environment. This generates a fresh set of artifacts (.pth, .pkl, and .json) that are guaranteed to be compatible.