REVISION AND PRACTICE

How to Solve Week 3 Questions

On this page 9 headings

Swipe long formulas, tables or code sideways to see the rest.

Practise past-paper questions

Try each question, then reveal the answer and worked solution.

18 of 18 questions

Start here Week 3 gives the biggest numerical questions in Quiz 1: a full forward pass, the softmax, the cross-entropy loss, and one or two backprop gradients. This page has every Week 3 question from five past papers (2025 T1, T2, T3 and 2026 T1, T2). Keep the five formulas below in front of you. All answers were checked with code.

The five backprop formulas (softmax + cross-entropy):

$\nabla a_L = \hat{y} - y$ (y = one-hot true label)
$\nabla W_k = \nabla a_k \cdot h_{k-1}^{T} \to$ one entry: $\partial L/\partial W_k,ij = (\nabla a_k)_{i} \times h_{k-1,j}$
$\nabla b_k = \nabla a_k$
$\nabla h_{k-1} = W_k^{T} \nabla a_k$
$\nabla a_{k-1} = \nabla h_{k-1} \odot g'(a_{k-1})$ ; for $\operatorname{sigmoid} g' = h(1 - h)$

Each layer: weights $=$ (neurons in this layer) × (neurons in the layer before), biases $=$ neurons in this layer. Add over all layers. If the question says “ignore biases” or “number of weights”, leave biases out.

Past-paper question

2026 T1 Quiz 1Q52 marks

Question: A regression network has input $x \in \mathbb{R}^{10}, 2$ hidden layers of 5 sigmoid neurons each, and 1 output neuron (sigmoid). Enter the number of parameters.

Reveal answer and solution

Answer: 91

How to solve:

Layer $1: 10 \times 5 = 50$ weights + 5 biases $= 55$
Layer $2: 5 \times 5 = 25$ weights + 5 biases $= 30$
Output: $5 \times 1 = 5$ weights + 1 bias $= 6$
Total $= 55 + 30 + 6 =$ 91

Past-paper question

2025 T2 Quiz 1Q72–734 + 4 marks

Question: A 5-class classifier has input $\in \mathbb{R}^{20}, 2$ hidden layers of 10 sigmoid neurons each, and 5 softmax outputs; cross-entropy loss (use ln); ignore biases. Q72: Find the number of weights. Q73: If all weights have the same value, find the cross-entropy loss for any data point (2 decimals; −1 if not enough information).

Reveal answer and solution

Answer: Q72: 350 · Q73: 1.61 (key range 1.59–1.63)

How to solve:

$Q72: 20\times 10 + 10\times 10 + 10\times 5 = 200 + 100 + 50 =$ 350.
Q73: if all weights are equal (and no biases), every neuron in a layer gets exactly the same inputs with the same weights → all 5 output scores are equal.
Softmax of 5 equal scores $= 1/5$ each, whatever the true class.
Loss $= -\ln (1/5) = \ln 5 =$ 1.609. (So the answer is not −1 — it can be computed.)

Type 2 — Forward pass, softmax and cross-entropy

Learn the method

Layer by layer: $a = W\cdot h (+ b)$ , then $h = \sigma (a)$ . Keep 4 decimals.
Softmax: exponentiate each score, divide by the sum.
Cross-entropy $= -\ln$ (probability of the true class).
Softmax trick: $\hat{y}_{1} = e^a_{1} / (e^a_{1} + e^a_{2} + \ldots )$ . Setting this to a target lets you solve for $a_{1}$ .

Past-paper question

2025 T1 Quiz 1Q75–781 + 1 + 1 + 1 marks

Question: A 3-class network: input $x \in \mathbb{R}^{3}, 3 \operatorname{sigmoid}$ neurons in hidden layer 1, 2 in hidden layer 2, 3 softmax outputs, no biases. $W_{1} = [[0,1,1],[1,0,1],[1,1,0]], W_{2} = [[0,0,1],[1,1,0]], W_{3} = [[0,1],[1,1],[1,0]], x = (0, 1, 2), y = (1, 0, 0)$ . (2 decimals; −1 if not enough information.) $Q75: h_{11}$ ? $Q76: h_{21}$ ? $Q77: h_{31}$ (first softmax output)? Q78: cross-entropy loss (ln)?

Question figure (redrawn exactly from the paper) — Question figure — open to inspect the labels.

Reveal answer and solution

Answer: Q75: 0.95 · Q76: 0.68 · Q77: 0.26 · Q78: 1.33 (key ranges 0.91–0.99, 0.63–0.71, 0.22–0.30, 1.29–1.37)

How to solve:

$a_{1} = W_{1}x: row 1 = 0\cdot 0 + 1\cdot 1 + 1\cdot 2 = 3; row 2 = 1\cdot 0 + 0\cdot 1 + 1\cdot 2 = 2; row 3 = 1\cdot 0 + 1\cdot 1 + 0\cdot 2 = 1 \to a_{1} = (3, 2, 1)$ .
$h_{1} = \sigma (a_{1}) = (0.9526, 0.8808, 0.7311) \to$ $h_{11} = 0.95$ .
$a_{2} = W_{2}h_{1}: row 1 = h_{13} = 0.7311; row 2 = h_{11} + h_{12} = 1.8334$ .
$h_{2} = \sigma (a_{2}) = (0.6750, 0.8622) \to$ $h_{21} = 0.68$ .
$a_{3} = W_{3}h_{2}: row 1 = h_{22} = 0.8622; row 2 = h_{21} + h_{22} = 1.5372; row 3 = h_{21} = 0.6750$ .
$e^a_{3} = (2.368, 4.651, 1.964)$ , sum $= 8.984 \to \hat{y} = (0.2636, 0.5178, 0.2186) \to$ $h_{31} = 0.26$ .
True class is 1 → loss $= -\ln 0.2636 =$ 1.33.

Past-paper question

2025 T3 Quiz 1Q53–541 + 1 marks

Question: Input $x = 2 \to$ one sigmoid hidden neuron $(W_{1} = 1) \to 3 \operatorname{softmax}$ outputs with $W_{2} = (w_{211}, w_{221}, w_{231}) = (-1, 1, 2)$ . No biases; $y = (0, 1, 0)$ . Q53: output of the hidden layer? Q54: cross-entropy loss (ln)? (2 decimals)

Reveal answer and solution

Answer: Q53: 0.88 · Q54: 1.28 (key ranges 0.84–0.92, 1.22–1.32)

How to solve:

$a_{1} = 1 \times 2 = 2 \to h = \sigma (2) = \frac{1}{1 + 0.1353} =$ 0.8808.
$a_{2} = (-1, 1, 2) \times 0.8808 = (-0.8808, 0.8808, 1.7616).$
$e^a_{2} = (0.4145, 2.4128, 5.8217)$ , sum $= 8.6490 \to \hat{y} = (0.0479, 0.2790, 0.6731)$ .
True class 2 → loss $= -\ln 0.2790 =$ 1.277.

Past-paper question

2026 T1 Quiz 1Q123 marks

Question: Binary classifier: x → two sigmoid hidden neurons $h_{1} (w_{1} = 1, b_{11} = 0)$ and $h_{2} (w_{2} = -1, b_{12} = 0) \to \operatorname{sigmoid}$ output with $v_{1} = 1, v_{2} = 1, b_{21} = 1$ . Cross-entropy loss, $x = 1, y = 1$ . Compute $\hat{y}$ (2 decimals).

Reveal answer and solution

Answer: 0.88 (key range 0.86–0.90)

How to solve:

$h_{1} = \sigma (1 \times 1 + 0) = \sigma (1) = 0.7311.$
$h_{2} = \sigma (-1 \times 1 + 0) = \sigma (-1) = 0.2689.$
Output pre-activation $= 1 \times 0.7311 + 1 \times 0.2689 + 1 =$ 2.0 (note $0.7311 + 0.2689 = 1$ , since $\sigma (1) + \sigma (-1) = 1$ ).
$\hat{y} = \sigma (2) =$ 0.881.

Past-paper question

2026 T2 Quiz 1Q143 marks

Question: A 3-class softmax output has pre-activations $a_L = (a_{1}, 0, 0)$ . Find $a_{1}$ so that $\hat{y}_{1} = 0.5$ exactly (2 decimals).

Reveal answer and solution

Answer: 0.69 (key range 0.67–0.71)

How to solve:

$\hat{y}_{1} = e^a_{1} / (e^a_{1} + e^{0} + e^{0}) = e^a_{1} / (e^a_{1} + 2).$
$Set = 0.5: e^a_{1} = 0.5 e^a_{1} + 1 \to 0.5 e^a_{1} = 1 \to e^a_{1} = 2$ .
$a_{1} = \ln 2 =$ 0.693.

Past-paper question

2026 T2 Quiz 1Q133 marks

Question: House-price regression with inputs $x_{1}$ (overall quality) and $x_{2}$ (living area). Hidden layer: 3 sigmoid neurons ( $W_{1} \in \mathbb{R}^{3}$ ˣ $^{2}, b_{1} \in \mathbb{R}^{3}$ ). Output: one linear neuron ( $W_{2} \in \mathbb{R}^{1}$ ˣ $^{3}, b_{2}$ ). Gradient descent starts with all $W_{1} = 0$ , all $W_{2} = 100$ and all biases 0. What price does the network give for quality 8 and area 3000?

Reveal answer and solution

Answer: 150

How to solve:

$W_{1} = 0$ and $b_{1} = 0 \to$ every hidden pre-activation $= 0$ (the inputs 8 and 3000 don’t matter).
Each hidden output $= \sigma (0) = 0.5$ .
$\hat{y} = 100 \times 0.5 + 100 \times 0.5 + 100 \times 0.5 + 0 =$ 150.

Type 3 — Backprop gradients

Learn the method

Do the forward pass first and keep every value.
Output gradient: $\operatorname{softmax} + CE \to \hat{y} - y$ . Sigmoid output + CE (binary) → also $\hat{y} - y$ . Linear output $+ \frac{1}{2}(\hat{y} - y)^{2} \to \hat{y} - y$ .
Weight into the output: (output gradient) × (input to that weight).
Weight further down: multiply by the weight on the path, then by h(1 − h), then by the input. If one hidden unit feeds several outputs, add all paths.

Past-paper question

2025 T1 Quiz 1Q79–802 + 2 marks

Question: Same network as Q75–78 above. Using cross-entropy loss, compute $\nabla w_{311}$ (the gradient of $w_{11}$ of $W_{3}$ ) and $\nabla w_{211}$ (w_{11} of $W_{2}$ ). (Use 2 decimals.)

Reveal answer and solution

Answer: $\nabla w_{311} = -0.50 \cdot \nabla w_{211} = 0.15$ (key ranges −0.54 to −0.46, 0.11–0.19)

How to solve:

$\nabla a_{3} = \hat{y} - y = (0.2636 - 1, 0.5178, 0.2186) = (-0.7364, 0.5178, 0.2186).$
$\nabla w_{311} = (\nabla a_{3})_{1} \times h_{21} = -0.7364 \times 0.6750 =$ −0.497.
$\nabla h_{2} = W_{3}^{T} \nabla a_{3}. W_{3}^{T} = [[0,1,1],[1,1,0]] \to (\nabla h_{2})_{1} = 0.5178 + 0.2186 = 0.7364.$
$(\nabla a_{2})_{1} = 0.7364 \times h_{21}(1 - h_{21}) = 0.7364 \times 0.6750 \times 0.3250 = 0.1616.$
$\nabla w_{211} = (\nabla a_{2})_{1} \times h_{11} = 0.1616 \times 0.9526 =$ 0.154.

Past-paper question

2025 T3 Quiz 1Q55–562 + 2 marks

Question: Same network as Q53–54 above $(x = 2, W_{1} = 1, W_{2} = (-1, 1, 2), y = (0, 1, 0))$ . Using cross-entropy loss, compute the gradient of $w_{231}$ and of $W_{1}$ (2 decimals).

Reveal answer and solution

Answer: $\nabla w_{231} = 0.59 \cdot \nabla W_{1} = 0.12$ (key ranges 0.54–0.64, 0.06–0.18)

How to solve:

$\nabla a_{2} = \hat{y} - y = (0.0479, 0.2790 - 1, 0.6731) = (0.0479, -0.7210, 0.6731).$
$\nabla w_{231} = (\nabla a_{2})_{3} \times h = 0.6731 \times 0.8808 =$ 0.593.
$\nabla h = \sum$ (weight × output gradient) $= (-1)(0.0479) + (1)(-0.7210) + (2)(0.6731) = -0.0479 - 0.7210 + 1.3462 = 0.5773$ .
$\nabla a_{1} = 0.5773 \times h(1 - h) = 0.5773 \times 0.8808 \times 0.1192 = 0.0606.$
$\nabla W_{1} = \nabla a_{1} \times x = 0.0606 \times 2 =$ 0.121.

Past-paper question

2026 T1 Quiz 1Q134 marks

Question: Same network as Q12 above ( $x = 1, y = 1, \operatorname{sigmoid}$ output, cross-entropy). Compute $\partial L/\partial w_{1}$ (2 decimals).

Reveal answer and solution

Answer: −0.02 (key range −0.04 to 0.00)

How to solve:

For a sigmoid output with cross-entropy $L = -[y \ln \hat{y} + (1 - y) \ln (1 - \hat{y})], \partial L/\partial$ (output pre-activation) $= \hat{y} - y = 0.8808 - 1 = -0.1192$ .
Path to $w_{1}$ : output $\to v_{1} \to h_{1} \to w_{1}$ .
$\partial L/\partial w_{1} = (\hat{y} - y) \times v_{1} \times h_{1}(1 - h_{1}) \times x = -0.1192 \times 1 \times (0.7311 \times 0.2689) \times 1 = -0.1192 \times 0.1966 =$ −0.0234.

Past-paper question

2025 T2 Quiz 1Q70–714 + 4 marks

Question: Regression network: one input x → two sigmoid hidden neurons (weights $w_{1}, w_{2}$ ) → linear output (weights $w_{3}, w_{4}$ ). No biases. $L = \frac{1}{2}(\hat{y} - y)^{2}$ . Given $w_{1} = 1, w_{2} = 1, w_{3} = 8, w_{4} = -4, x = -\ln 3, y = 1 + 1/\ln 3$ . $Q70: \hat{y}$ ? $Q71: \partial L/\partial w_{1}$ ?

Reveal answer and solution

Answer: $Q70: \hat{y} = 1 \cdot Q71: 1.5$

How to solve:

Both hidden units $get w\cdot x = -\ln 3 \to \sigma (-\ln 3) = \frac{1}{1 + e^{\\ln 3}} = \frac{1}{1 + 3} =$ 0.25.
$\hat{y} = 8 \times 0.25 + (-4) \times 0.25 = 2 - 1 =$ 1.
$\hat{y} - y = 1 - (1 + 1/\ln 3) = -1/\ln 3.$
$\partial L/\partial w_{1} = (\hat{y} - y) \times w_{3} \times h_{1}(1 - h_{1}) \times x = (-1/\ln 3) \times 8 \times (0.25 \times 0.75) \times (-\ln 3).$
The ln 3 terms cancel and the two minus signs make a plus: $8 \times 0.1875 =$ 1.5.

Past-paper question

2026 T2 Quiz 1Q173 marks

Question: A 10-class softmax network with cross-entropy loss $L = -\log \hat{y}$ _ℓ. On an example with true class $\ell , \hat{y}$ _ $\ell = 0.99$ . What is $\partial L/\partial a_{L,\ell }$ ? (2 decimals)

Reveal answer and solution

Answer: −0.01 (key range −0.03 to 0.01)

How to solve:

$\partial L/\partial a_{L,i} = \hat{y}_{i} - 𝟙(i = \ell ).$
$For i = \ell : 0.99 - 1 =$ −0.01 (tiny, because the network is already almost right).

Type 4 — Concept questions on network design

Learn the method

Linear (identity) activations everywhere → the whole network is one matrix: $W_{3}W_{2}W_{1}x$ . No extra power from depth.
Identical neurons (same weights and bias) → same output and same gradient → they stay identical (symmetry).
All-zero weights in a layer → no gradient flows below that layer, but that layer’s own weights still get gradients.
Regression → 1 linear output + squared error; classification → softmax + cross-entropy.
ReLU( $\alpha z$ ) = α·ReLU(z) for $\alpha > 0$ (scaling passes through ReLU), but not for sigmoid.

Past-paper question

2025 T2 Quiz 1Q624 marks

Question: A regression network: 3 hidden layers of 10 neurons each, input $\in \mathbb{R}^{10}$ , output $\in \mathbb{R}, f: \mathbb{R}^{10} \to \mathbb{R}$ , no biases, all activations linear. True or false: “The network is just computing a linear function of the input and can be represented as $f(x) = w^{T}x$ for some $w \in \mathbb{R}^{10}$ .” (True · False · Insufficient data)

Reveal answer and solution

Answer: True

How to solve:

With linear activations, $f(x) = W_{4}W_{3}W_{2}W_{1}x$ .
$W_{4}$ is 1×10, the others 10×10 → the product is a single 1×10 matrix — call it $w^{T}$ .
$So f(x) = w^{T}x \to$ True.

Past-paper question

2026 T2 Quiz 1Q163 marks

Question: A network: input (50) → 64 sigmoid → 32 sigmoid → 10 softmax, cross-entropy. If you use the identity function $(h_{i} = a_{i})$ in both hidden layers but keep softmax at the output, which is true?

It can learn non-linear boundaries because softmax is non-linear. (b) It is equivalent to a single softmax regression $\hat{y} = O(W'x + b')$ with $W' = W_{3}W_{2}W_{1}$ . (c) It will fail to train because gradients can’t flow through linear layers. (d) Training forces $W_{1}, W_{2}, W_{3}$ to become identity matrices.

Reveal answer and solution

Answer: (b)

How to solve:

$a_{3} = W_{3}(W_{2}(W_{1}x + b_{1}) + b_{2}) + b_{3} = (W_{3}W_{2}W_{1})x +$ (some vector) $= W'x + b'$ .
So the output is $\operatorname{softmax}(W'x + b')$ — a single softmax regression → (b) ✔.
1. Softmax only turns scores into probabilities; the class boundaries stay linear ✘. (c) Gradients flow fine through linear layers ✘. (d) Nothing forces identity matrices ✘.

Past-paper question

2025 T3 Quiz 1Q503 marks

Question: A network built for 10-class image classification is adapted to predict the price of a car (regression). Which changes are essential or standard?

Make the final activation linear. (b) Use one output neuron. (c) Change the loss from categorical cross-entropy to mean squared error. (d) Use accuracy as the main metric. (e) Keep softmax to scale the price.

Reveal answer and solution

Answer: (a), (b), (c)

How to solve:

A price is one real number → one output neuron, linear output, squared error loss ✔✔✔.
Accuracy is for classes, not prices ✘. Softmax squashes outputs into probabilities that add to 1 — useless for a price ✘.

Past-paper question

2026 T2 Quiz 1Q83 marks

Question: A hidden layer has n sigmoid neurons, all with identical weight vectors w and identical bias b. Which are true?

After one gradient-descent step the neurons will have different weights. (b) For any input x, all n neurons produce the same output. (c) During backprop the gradients of all n neurons are always zero. (d) The layer is functionally equivalent to a single neuron.

Reveal answer and solution

Answer: (b) and (d)

How to solve:

Same w, same b, same input → same $\sigma (w^{T}x + b) \to (b)$ ✔.
They receive the same gradient, so they stay identical after every step → (a) ✘ (if the next layer’s weights are also equal, which the question implies).
The gradients are equal, not zero → (c) ✘.
n copies of one function $=$ one neuron → (d) ✔.

Past-paper question

2025 T3 Quiz 1Q61–622 + 2 marks

Question: One hidden layer (sigmoid) and a softmax output. $W_{1}$ is random, the output weights $W_{2}$ are all zeros, biases zero, trained with SGD. After one iteration: Q61 — is $W_{1}$ unchanged? Q62 — is $W_{2}$ unchanged?

Reveal answer and solution

Answer: $Q61: Yes \cdot Q62: No$

How to solve:

$\nabla h_{1} = W_{2}^{T} \nabla a_{2} = 0$ , because $W_{2} = 0$ . $So \nabla a_{1} = 0$ and $\nabla W_{1} = 0 \to$ $W_{1}$ stays the same.
$\nabla W_{2} = \nabla a_{2} h_{1}^{T}$ . $\nabla a_{2} = \hat{y} - y \ne 0$ (softmax of all-zero scores is uniform, not one-hot), and $h_{1} > 0 (\operatorname{sigmoid}) \to \nabla W_{2} \ne 0 \to$ $W_{2}$ changes.

Past-paper question

2026 T1 Quiz 1Q34 marks

Question: Scalar input x, hidden layer $h_{1j} = a(b_{1j} + w_{1j}x)$ for $j = 1, 2, 3$ , output $y = b_{21} + w_{21}h_{11} + w_{22}h_{12} + w_{23}h_{13}$ . Multiply $b_{11}$ and $w_{11}$ by a constant $\alpha \ne 0$ and divide $w_{21}$ by α. Which is correct?

Output unchanged with ReLU for $\alpha > 0, x \in \mathbb{R}$ . (b) Output unchanged with ReLU for $\alpha < 0$ . (c) Output unchanged with sigmoid for $\alpha > 0$ . (d) The contribution $w_{21}h_{11}$ is unchanged for all x and all $\alpha \ne 0$ .

Reveal answer and solution

Answer: (a)

How to solve:

New contribution $= (w_{21}/\alpha ) \cdot a(\alpha (b_{11} + w_{11}x))$ .
ReLU, $\alpha > 0$ : ReLU( $\alpha z$ ) = α·ReLU( $z$ ) → (w_{21}/α)·α·ReLU( $z$ ) = w_{21}ReLU(z) → unchanged ✔.
ReLU, $\alpha < 0$ : ReLU( $\alpha z$ ) = |α|·ReLU(−z), a different function ✘ (e.g. $z = 1, \alpha = -1$ : ReLU( $-1$ ) = 0 but ReLU( $1$ ) = 1).
Sigmoid: $\sigma (2z)/2 \ne \sigma (z)$ (e.g. $z = 0: \sigma (0)/2 = 0.25$ but $\sigma (0) = 0.5$ ) ✘. (d) fails for the same reasons ✘.

Practice set (new questions)

P1. Input 8, one hidden layer of 6, output 3, with biases. Parameters?

8\times 6 + 6 + 6\times 3 + 3 = 48 + 6 + 18 + 3 = 75.

P2. Softmax scores (a, 0) for 2 classes. Which a gives

\hat{y}_{1} = 0.8

eᵃ/(eᵃ + 1) = 0.8 \to eᵃ = 4 \to a = \ln 4 = 1.386.

P3. All weights equal, no biases, 4 classes. Cross-entropy?

\ln 4 = 1.386.

P4.

\nabla a_{2} = (0.3, -0.5, 0.2), h_{1} = 0.6

(one hidden unit).

\nabla

of the weight to output 2?

-0.5 \times 0.6 = -0.3.

P5. In P4, the weights from

h_{1}

to the outputs are (1, 2, −1). Find

\nabla h_{1}

and

\nabla a_{1} (\operatorname{sigmoid})

\nabla h_{1} = 0.3 - 1.0 - 0.2 = -0.9. \nabla a_{1} = -0.9 \times 0.6 \times 0.4 = -0.216.

P6.

W_{1} = 0, b_{1} = 0, 4 \operatorname{sigmoid}

hidden units, linear output with all weights 10, bias 2. Output for any input?

4 \times 10 \times 0.5 + 2 = 22.

YOUR STUDY PROGRESS

Finished this section? Save your progress.