SouL Codesmith · Jaipur, India · Python & PyTorch

BUILDING ML from first principles

Self-teaching machine learning the slow, deep way — rebuilding the machinery instead of taking it on faith, and shipping every step of it.

Currently — extending Karpathy's micrograd into Micrograd++

Selected work ↓ GitHub ↗

gradient descent — live click to drop · β = 0.86 · lr = 0.012

01 — Selected work

What I've made

2026 — actively building Python · scalar autograd · from scratch

Micrograd++

A scalar automatic-differentiation engine built from scratch — extending Andrej Karpathy's micrograd while working through his Zero to Hero series. Not a wrapper around PyTorch: computation-graph tracking, arithmetic ops, tanh, and reverse-mode backpropagation through a topologically sorted graph — hand-built, unit-tested, explained in the README.

d = a·b + a → ∂d/∂a = b + 1 = 4 the chain rule, running on a graph I built myself.

View on GitHub ↗

What is Micrograd++, really?

A neural network is a pile of simple math wired together — millions of small numbers (“knobs”) that the computer keeps adjusting until its answers stop being wrong. Micrograd++ is my hand-built version of the machinery that decides which way to turn each knob. The easiest way to explain it is cooking.

01

Cook the dish — the forward pass

Ingredients go in, steps happen, a dish comes out. In a network, numbers flow through the wiring and an answer comes out.
02

Taste it — the loss

One single number that says how bad the dish is. Networks call it the loss. Lower means better. This is the number all of learning cares about.
03

Ask every ingredient — the gradient

For every single knob, one question: “if you had been slightly different, would the dish have scored better?” The answer — which way, and how much — is called a gradient.
04

Adjust everything a little — learning

Turn every knob a tiny step in its helpful direction, cook again, taste again. Repeat thousands of times. That process is gradient descent — it's what the live demo at the top of this page is doing.

Here's the trick that makes it all possible: answering question 03 by re-cooking the whole dish once per ingredient would be impossibly slow. Backpropagation answers it for every knob at once — by walking backwards from the taste using one old piece of math, the chain rule. That backwards walk is exactly what Micrograd++ builds.

The real example from the repo: d = a·b + a. White arrows are step 01 — values flowing forward. Gold arrows are step 03 — gradients flowing back. Watch a: it ends up with gradient 4 because it reaches d by two paths, through b (×3) and straight in (+1). Adding up every path like that is backpropagation.

So where does Micrograd++ fit?

PyTorch — the library professionals use — does steps 03 and 04 in a single line: loss.backward(). Micrograd++ is me building that one line myself: a tiny automatic-differentiation engine that records every operation into a graph, then runs the chain rule backwards through it. Same math as PyTorch, roughly 100× slower — but every line of it is mine, and I can explain all of it.

Understand, don't just use

Once you've built the graph walker yourself, backward() is code you could have written — not a spell you recite.

Small enough to fully hold

The whole engine is a few hundred lines. Every operation built and tested by hand — next milestone: verifying every gradient against PyTorch itself.

The honest record

The repo grows in public, commit by commit — including the broken days. That's the whole point of learning in public.

02 — Knowledge

What I know

Deep over broad — every item below is backed by notes, code, or a repo.

Python The daily driver — everything I build, I build in it.
Backpropagation Derivatives, the chain rule, and manual gradients over computational graphs.
Autograd internals How loss.backward() actually works — because I've built one.
PyTorch Tensors, shapes and devices; training loops; nn.Module, parameters, optimizers.
scikit-learn Classical ML fundamentals — the baseline before the neural nets.

03 — Languages & tools

What I build with

Small, understandable tools over big magic ones.

Machine learning

Python PyTorch scikit-learn NumPy

Workflow

Git GitHub Jupyter Graphviz Obsidian

System

macOS Homebrew

04 — Learning

What I'm learning

Research papers I like research — reading papers closely and rebuilding what they describe. Attention and interpretability are first in the queue.
Neural Networks: Zero to Hero Karpathy's lecture series — deep in the micrograd lecture right now.
Mathematics for ML Linear algebra, calculus, probability — the parallel rigor track.
Extending Micrograd++ PyTorch-verified gradient tests, more activations, Neuron→Layer→MLP, training XOR.

All of it documented as it happens — notes cross-linked in an Obsidian vault, code shipped to GitHub. The record is the point.

05 — Journal

What I learned today.

Short, honest notes from the daily build — what clicked, what broke, what fixed it. Written my way, updated as I go.

Miss a day? The raw record — every commit, every fix — lives on GitHub.

BUILDING ML from first principles

Micrograd++

Cook the dish — the forward pass

Taste it — the loss

Ask every ingredient — the gradient

Adjust everything a little — learning

So where does Micrograd++ fit?

Understand, don't just use

Small enough to fully hold

The honest record