A. K. Mackay
Contents

1.2

§1 Selected work

Micro-LLM

A Llama-style language model written from scratch in PyTorch, small enough to pre-train on a laptop, then fine-tuned with reinforcement learning.

2026 – present

PyTorch, tiktoken, PPO, RLHF

Why

To understand how large language models work by building every part of one, instead of calling an API.

What is in it

  • Model. A Llama-style decoder-only Transformer written in PyTorch, with RMSNorm and SwiGLU feed-forward layers.
  • Data. The TinyStories dataset, tokenised with tiktoken’s byte-pair encoding.
  • Pre-training. A training loop with checkpointing and evaluation, sized so the test configuration pre-trains in about 30 minutes on a laptop GTX 1050.
  • Alignment. A PPO stage that fine-tunes the pre-trained model towards a target sentiment: the RLHF recipe used for large models, at small scale. It builds directly on the reinforcement learning work from my MSc.
Line chart of training and validation cross-entropy loss over 2,000 steps, falling from 6.0 to about 3.3.Line chart of training and validation cross-entropy loss over 2,000 steps, falling from 6.0 to about 3.3.
Pre-training on TinyStories. Cross-entropy falls from 6.0 to about 3.3 over 2,000 steps, with validation loss tracking training loss closely.
Grid of bar charts comparing a laptop-sized and a Colab-sized model on parameters, size, training time, accuracy, loss, perplexity, prefill latency and decoding throughput.Grid of bar charts comparing a laptop-sized and a Colab-sized model on parameters, size, training time, accuracy, loss, perplexity, prefill latency and decoding throughput.
Laptop and Colab configurations. The 17M-parameter model trains in about 30 minutes; the 51M-parameter model trains for 3.6 hours and cuts perplexity from roughly 390 to 25.