A. K. Mackay
Contents

1.4

§1 Selected work

EU energy consumption forecasting

An end-to-end machine learning system, from Spark data processing to a public forecasting app on Google Cloud.

2025

Databricks, PySpark, PyTorch, XGBoost, MLflow, FastAPI, Docker, Cloud Run, Streamlit, Tableau

Goal

To show the full lifecycle of a machine learning system rather than to build the best possible forecaster: data processing, training, experiment tracking, packaging, deployment and a public interface.

Pipeline

  • Data. Eurostat energy statistics loaded into a Databricks SQL warehouse and preprocessed with PySpark and Pandas.
  • Models. PyTorch neural networks and XGBoost regressors, evaluated on temporal splits so the test years always lie in the future, tuned with cross-validation and tracked in MLflow.
  • Serving. The selected model is wrapped in a FastAPI service, containerised with Docker and deployed to Google Cloud Run, with Artifact Registry, IAM and Secret Manager configured for it.
  • Interfaces. A Streamlit app queries the API live; a Tableau story presents the exploratory analysis.
Streamlit dashboard with a map of Europe coloured by predicted energy consumption per capita in 2030, and a line chart of Spain's forecast for the next ten years.Streamlit dashboard with a map of Europe coloured by predicted energy consumption per capita in 2030, and a line chart of Spain's forecast for the next ten years.
The public Streamlit app. Forecasts come from a FastAPI service on Cloud Run; the first request can be slow while the container cold-starts.