Project case study
ML - Finance · 2025
S&P 500 Index Replication with Neural Networks
Neural network index-tracking pipeline that reconstructs S&P 500 returns using a small subset of constituents, optimizing both tracking error and portfolio efficiency under a reproducible, time-aware evaluation protocol.
Highlights
- Converted 360 constituent price series into log returns, standardized inputs, and ranked stocks by Pearson correlation with the index to form a candidate universe.
- Trained a compact Keras MLP on top-correlated subsets and tuned subset size and hyperparameters via randomized search with a time-based validation split and early stopping.
- Replicated index behavior using ~30 stocks with RMSE ≈ 0.00799 on test returns, while explicitly trading off accuracy vs. portfolio efficiency (1 - n/360).
Gallery
What
Built a neural network system to replicate S&P 500 index dynamics using only a small subset of its 360 constituents.
Optimized for real-world index tracking by combining normalized RMSE (tracking error proxy) with an explicit efficiency term that rewards smaller portfolios.
How
Transformed daily prices into log returns, aligned time indices, and standardized features with StandardScaler to stabilize training across diverse stocks.
Ranked constituents by absolute Pearson correlation with the index to create an ordered universe and trained models on the top-k subsets.
Implemented a time-aware validation split using the most recent portion of training data as validation, then tuned subset size and MLP hyperparameters with randomized sampling over learning rate, activation, batch size, and hidden units.
Applied early stopping and fixed random seeds via tf.keras.utils.set_random_seed to ensure reproducible training and evaluation.
Results
Achieved strong out-of-sample tracking with RMSE ≈ 0.00799 on test log returns while using ~30 stocks.
Produced a compact replication approach that explicitly balances tracking accuracy with portfolio simplicity via an efficiency-aware objective.
Validated end-to-end reproducibility with fixed seeds and consistent grading on held-out test data, and visualized cumulative reconstructed index paths against true S&P 500.