Sigmoid neurons and gradient descent.
Learn through illustrated explanations, worked examples, practice questions and quick revision.
Before you start
Review perceptrons from Week 1, derivatives, the chain rule and dot products.
By the end: Calculate a sigmoid output, a loss, a gradient and one update.
Check the notation
y: target; ŷ or f(x): prediction; b or w₀: bias; η: learning rate. Here θ can mean the vector of all parameters rather than the threshold used in Week 1.
Study sections
Follow the order or choose a sectionThe Sigmoid Neuron — a Smooth Perceptron
A perceptron says a hard “yes” or “no”. A movie rated 0.51 → like, 0.49 → dislike. That is too harsh. The sigmoid neuron replaces the s…
02The Supervised Learning Setup, Loss and Guess Work
Before learning any weights we need to say clearly what we are learning and how we measure mistakes . Every machine learning problem in…
03Gradient Descent — Learning by Walking Downhill
Imagine you are on a foggy hill and want to reach the valley. You can’t see far, but you can feel which way the ground slopes under you…
04Representation Power of a Network of Sigmoid Neurons
In Week 1, a network of perceptrons could represent any Boolean function exactly. Now the big claim: a network of sigmoid neurons with…