TradeMaster: An RL Platform for Trading

</div>

TradeMaster is a first-of-its kind, best-in-class open-source platform for quantitative trading (QT) empowered by reinforcement learning (RL), which covers the full pipeline for the design, implementation, evaluation and deployment of RL-based algorithms.

:octocat: Hiring: multiple fully-funded PhD and RA positions are available under the supervision of Dr. Sun Shuo (first author of TradeMaster paper) at HKUST(GZ). Please send him an email if you are interested!

:star: What's NEW! :alarm_clock:

Update	Status
Add EarnHFT, Market-GAN and MacroHFT	:hammer: Updated on 25 Feb 2025
Add FinAgent and EarnMore	:hammer: Updated on 29 Oct 2024
Update TradeMaster Website (Market Simualtor)	:hammer: Updated on 21 Sep 2023
Update TradeMaster Website (Market Dynamics Modeling Tool)	:wrench: Updated on 7 July 2023
Support automatic feature generation and selection	:hammer: Update tutorial on 11 May 2023
Release TradeMaster Python package	:whale: Updated on 11 May 2023
Build TradeMaster website	:whale: Available at here on 23 April
Write TradeMaster software documentation	:speech_balloon: Updated on 11 April 2023
Release Colab version	:speech_balloon: Updated on 29 March 2023
Incldue HK Stock and Future datasets	:compass: Updated #131 #132 on 27 March 2023
Support Alpha158	:hammer: Updated #123 #124 on 20 March 2023
Release TradeMaster 1.0.0	:octocat: Released v1.0.0 on 5 March 2023

Outline

TradeMaster: An RL Platform for Trading

Overview

TradeMaster is composed of 6 key modules: 1) multi-modality market data of different financial assets at multiple granularity; 2) whole data preprocessing pipeline; 3) a series of high-fidelity data-driven market simulators for mainstream QT tasks; 4) efficient implementations of over 13 novel RL-based trading algorithms; 5) systematic evaluation toolkits with 6 axes and 17 measures; 6) different interfaces for interdisciplinary users.

Installation

Here are the installation tutorials for different operating systems and docker:

Tutorial

We provide tutorials covering core features of TradeMaster for users to get start with.

Algorithm	Dataset	Market	Task	Code Link
EIIE	DJ 30	US Stock	Portfolio Management	tutorial
DeepScalper	BTC	Crypto	Intraday Trading	tutorial
SARL	DJ 30	US Stock	Portfolio Management	tutorial
PPO	SSE 50	China Stock	Portfolio Management	tutorial
ETEO	Bitcoin	Crypto	Order Execution	tutorial
Double DQN	Bitcoin	Crypto	High Frequency Trading	tutorial

We also provide a colab version of these tutorials that can be run directly. (colab tutorial)

Useful Script

TradeMaster Sandbox

White paper
Market dynamics modeling
Website API
1. Agent training Demo
2. Market dynamics modeling tool
3. Market simulator web service

Dataset

Dataset	Data Source	Type	Range and Frequency	Raw Data	Datasheet
S&P500	Yahoo	US Stock	2000/01/01-2022/01/01, 1day	OHLCV	SP500
DJ30	Yahoo	US Stock	2012/01/01-2021/12/31, 1day	OHLCV	DJ30
BTC	Kaggle	Foreign Exchange	2000/01/01-2019/12/31, 1day	OHLCV	FX
Crypto	Kaggle	Crypto	2013/04/29-2021/07/06, 1day	OHLCV	Crypto
SSE50	Yahoo	China Stock	2009/01/02-2021/01/01, 1day	OHLCV	SSE50
Bitcoin	Binance	Crypto	2021/04/07-2021/04/19, 1min	LOB	Binance
Future	AKshare	Future	2023/03/07-2023/03/28, 5min	OHLCV	Future
HS30	AKShare	HK Stock	1988/12/30-2023/03/27, 1day	OHLCV	HS30

Dates are in YY/MM/DD format.

OHLCV: open, high, low, and close prices; volume: corresponding trading volume; LOB: Limit order book.

Users can download data of the above datasets from Google Drive or Baidu Cloud (extraction code:x24b)

Model Zoo

TradeMaster provides efficient implementations of the following algorithms:

DeepScalper based on Pytorch (Shuo Sun et al, CIKM 22)

OPD based on Pytorch (Fang et al, AAAI 21)

DeepTrader based on Pytorch (Wang et al, AAAI 21)

SARL based on Pytorch (Yunan Ye et al, AAAI 20)

ETEO based on Pytorch (Lin et al, 20)

Investor-Imitator based on Pytorch (Yi Ding et al, KDD 18)

EIIE based on Pytorch (Jiang et al, 17)

Classic RL based on Pytorch and Ray:
PPO A2C Rainbow SAC DDPG DQN PG TD3

Visualization Toolkit

TradeMaster provides many visualization toolkits for a systematic evaluation of RL-based quantitative trading methods. Please check this paper and repository for details. Some examples are as follows:

PRIDE-Star is a star plot containing normalized score of 8 key financial measures such total return (TR) and Sharpe ratio (SR) to evaluate profitability,risk-control and diversity:

File Structure

| TradeMaster
| ├── configs
| ├── data
| │   ├── algorithmic_trading 
| │   ├── high_frequency_trading  
| │   ├── order_excution          
| │   └── porfolio_management
| ├── deploy
| │   ├── backend_client.py
| │   ├── backend_client_test.py 
| │   └── backend_service.py        
| │   ├── backend_service_test.py  
| ├── docs
| ├── figure
| ├── installation
| │   ├── docker.md
| │   ├── requirements.md
| ├── tools
| │   ├── algorithmic_trading          
| │   ├── data_preprocessor
| │   ├── high_frequency_trading
| │   ├── market_dynamics_labeling
| │   ├── missing_value_imputation  
| │   ├── order_excution  
| │   ├── porfolio_management  
| │   ├── __init__.py      
| ├── tradmaster       
| │   ├── agents   
| │   ├── datasets 
| │   ├── enviornments 
| │   ├── evaluation 
| │   ├── imputation 
| │   ├── losses
| │   ├── nets
| │   ├── preprocessor
| │   ├── optimizers
| │   ├── pretrained
| │   ├── trainers
| │   ├── transition
| │   ├── utils
| │   └── __init__.py     
| ├── unit_testing
| ├── Dockerfile
| ├── LICENSE
| ├── README.md
| ├── pyproject.toml
| └── requirements.txt

Publications

A multimodal foundation agent for financial trading: Tool-augmented, diversified, and generalist (KDD 2024)

MacroHFT: Memory augmented context-aware reinforcement learning on high frequency trading (KDD 2024)

Reinforcement learning with maskable stock representation for portfolio management in customizable stock pools (WWW 2024)

EarnHFT: Efficient hierarchical reinforcement learning for high frequency trading (AAAI 2024)

Market-GAN: Adding control to financial market data generation with semantic context (AAAI 2024)

TradeMaster: A holistic quantitative trading platform empowered by reinforcement learning (NeurIPS 2023)

Mastering stock markets with efficient mixture of diversified trading experts (KDD 2023)

PRUDEX-Compass: Towards Systematic Evaluation of Reinforcement Learning in Financial Markets (Transactions on Machine learning Research 2023)

Reinforcement Learning for Quantitative Trading (Survey) (ACM Transactions on Intelligent Systems and Technology 2023)

Deep Reinforcement Learning for Quantitative Trading: Challenges and Opportunities (IEEE Intelligent Systems 2022)

DeepScalper: A Risk-Aware Reinforcement Learning Framework to Capture Fleeting Intraday Trading Opportunities (CIKM 2022)

Commission Fee is not Enough: A Hierarchical Reinforced Framework for Portfolio Management (AAAI 21)

News

[AI科技评论]TradeMaster最新发布沙盒工具箱，探索量化交易的全新维度
[AAAI Summer Syposium]Call for participation on RL4Fintech syposium in Singapore
[知乎]用强化学习在金融市场上赚钱？南洋理工发布全新基于强化学习的量化交易平台TradeMaster
[机器之心]南洋理工发布量化交易大师TradeMaster，涵盖15种强化学习算法
[运筹OR帷幄]量化交易大师TradeMaster
[Medium]Introduction on TradeMaster
[AAAI-23 Bridge Program]Host a tutorial on reinforcement learning for quantitative trading with TradeMaster

Team

This repository is developed and maintained by AMI group at Nanyang Technological University.
We have positions for software engineer, research associate and postdoc. If you are interested in working at the intersection of RL and quantitative trading, feel free to send us an email with your CV.