Energy Price Prediction Using a Foundation Model
Google recently announced a new version of TimesFM, their foundation model for multivariate forecasting.
It made me curious how well a foundation model could perform compared to a model I created previously to predict energy prices using multivariate data. This model was designed and trained specifically for this task, whereas Google’s model is obviously designed to generalise across many tasks. Google’s model is also zero-shot, so no training required.
I ran the same evaluation data through TimesFM, and padded with historical context for larger window requirements where necessary.
I ran several variants of evaluation, mainly playing with different context sizes and univariate vs covariate. The results are very interesting.
| Model | Context | MAE | RMSE | MAPE |
|---|---|---|---|---|
| TimesFM-3 + covariates | 720h | 3.4817 | 5.1719 | 6.92% |
| TimesFM-3 | 720h | 3.5706 | 5.2848 | 7.06% |
| TimesFM-3 | 336h | 3.6960 | 5.4096 | 7.20% |
| TimesFM-3 | 168h | 3.8897 | 5.6891 | 7.53% |
| Original model | 18h | 3.9795 | 5.4034 | 7.31% |
| TimesFM-3 | 48h | 4.7620 | 6.7300 | 8.90% |
| TimesFM-3 | 24h | 5.8517 | 8.4167 | 10.59% |
| TimesFM-3 | 18h | 6.6226 | 9.1475 | 12.10% |
My first run was a like-for-like comparison using 18 hours of historical price context. TimesFM performed quite poorly, not even beating the yesterday persistence baseline from my original work.
When you extend the historical context to multiple days, however, TimesFM starts performing really well. With enough context, historical price alone is enough for TimesFM to beat my custom model. Adding covariates improves things a little further, reaching an MAE of 3.4817 compared with 3.9795 for my original model.
I was quite surprised at how little additional information TimesFM needs. My original model was specifically trained for this problem and used historical prices alongside generation, demand and weather data. TimesFM can beat it without being trained on this dataset at all, using only a sufficiently long history of the target variable.
I’m considering fine-tuning next to see if the results can be improved further, but this took almost no time at all to set up. Rather helpfully, TimesFM also has an MLX backend, which means it runs very nicely on my MacBook Pro.