Unit 2: Gradient descent: how networks learn
About this unit
Training a network means lowering one number, the average error over the training data, by repeatedly nudging every weight a little in the direction that reduces it fastest.
By the end, you can: Define a network's cost over its training data and explain how gradient descent lowers it by stepping against the gradient.
Modules
5 modulesLearn
- Measuring how wrongArticle · The cost: squared error on one example, averaged over the training set, as a function of the weights and biases.Not started
- Which way is downArticle · The gradient, the update θ ← θ − η∇C, the learning rate, and why the cost must be smooth.Not started
- Tune the step sizeExperiment · Run gradient descent on a bowl with small, medium and large learning rates.Not started
- Learning as going downhillReview · Cost, gradient, update rule, learning rate and local minima together.Not started