I built these two explainers for my kids and anyone new to neural networks, to show how machines learn and make decisions. Here, you can teach a small network to drive and see how your examples and feedback shape its behaviour.
Take the tour, or pick a method under Train a model. Each method starts from an untrained network and ends with the same test: 500 m of road it has never seen.
An untrained network has random connections, so its moves are random too: in the tour, it drives straight into a parked car. Learning means adjusting those connections, called weights, until its moves avoid collisions. The three methods differ in where the feedback comes from: your answers, your driving, or points and penalties.
You see a road scene. The network receives two numbers from a radar:
Each is 1 for a car 10 m ahead, down to 0.13 at 80 m, and 0 when the lane is clear. It receives these numbers directly. It does not recognise cars from the road image.
The two inputs feed three neurons in the hidden layer, and those feed the output. Each connection has a weight, and each neuron a bias: a base value, added to what comes in.
The network has two views. Weights shows the weights themselves: green positive, magenta negative, thicker means stronger. Impact shows what each connection contributes right now: its weight times the value it carries. Green pushes the next neuron up, magenta pushes it down.
With Flashcards and Copycat, the output is one number between 0 and 1. Above 0.5, the network presses Move! and changes lane.
Trial & Error has two outputs, one per move: Stay and Move!. Each estimates the points that move will earn from here on, so they are scores, not probabilities. The car takes the higher one.
You label 8 road situations with the right move. The network studies 4 of them, the training set. The other 4, the validation set, are only checked. They help us see whether it can apply what it learned to examples it did not study.
Training runs in cycles called epochs. Each cycle, the network works out the output error and adjusts its weights and biases backwards through the layers. This is called backpropagation. The other four cards check whether it can apply what it learned to other examples. Training stops once those answers are right and further improvement is small, or after 10 cycles.
This is supervised learning: learning from examples with answers attached, as in the digit demo. The network has no idea what is right: it learns your answers. An incorrect label on a training card can teach it the wrong move. Incorrect labels on validation cards can disrupt the learning process.
You drive 200 m, changing lane with Move!. Every 10 m, what the radar sees becomes a question and your move its answer, and the network learns from it straight away. A collision is learned as the opposite move.
This is imitation learning, or behavioural cloning. Your driving examples matter: if you only change lanes at the last moment, it may learn that habit too. It has only practised the situations you showed it.
Nobody shows it the right move. It practises alone, and every move earns points or penalties:
Collisions cost by far the most. Unnecessary lane changes cost points. Together, these rules encourage the car to move only when needed, and early enough.
At first, half its moves are random, to explore its options; the share falls to 2% over its first 400 decisions, on a fixed schedule. It remembers what it saw, what it did, and the points it got, and keeps replaying samples of those memories, adjusting its weights so each score better predicts the points that move really leads to.
This is reinforcement learning: learning from the consequences of actions. The method used here is called Deep Q-learning. The network discovers how to earn points. We decide what earns them.
Deep Q-learning made its name on old Atari video games. In 2015, DeepMind showed that the same network architecture, trained separately for each game, could learn to play 49 Atari games from the screen and the score alone, playing many as well as a professional games tester. DeepMind went on to combine deep networks with reinforcement learning in AlphaGo, which in 2016 beat Lee Sedol, one of the world's best Go players. Its successor, AlphaGo Zero, learned Go by playing against itself alone. The story is told in the free documentary AlphaGo – The Movie.
Whatever the method, the result is the same kind of thing: fewer than 20 numbers, its weights and biases. They hold a strategy for this task only: two lanes, parked cars, and a radar that sees 80 m ahead.
It cannot anticipate obstacles beyond its view. Passing one test road also does not guarantee success on every road. Once trained, its weights stay fixed: when it drives on, it uses what it learned, and learns nothing new.
In the digit demo, the training examples shape what the network recognises. Here, your cards, your driving or the points shape how it behaves. In both, learning changes the weights, and those weights change the answer.