04. Introduction to Neural Networks

In Lecture 4 we progress from linear classifiers to fully-connected neural networks. We introduce the backpropagation algorithm for computing gradients and briefly discuss connections between artificial neural networks and biological neural networks.

• Keywords: Neural networks, computational graphs, backpropagation, activation functions, biological neurons

• slides

---

Backpropagation

Backprop is a recursive appllication of chain rule. Let's see an example.

---

Patterns in backward flow

---

Gradients for vectorized code

---

Vectorized operations

To be added: what is Jacobian

---

Modularized implementation

---

Summary