Yes, you should understand backprop (2016)
Yes you should understand backpropAndrej Karpathy7 min read·Dec 19, 2016–48ListenShareWhen we offered CS231n (Deep Learning class) at Stanford, we intentionally designed the programming assignments to include explicit calculations involved in backpropagation on the lowest level. The students had to implement the forward and the backward pass of each layer in raw numpy. Inevitably, some students complained on the class message boards:“Why do we have to write the backward pass when frameworks in the real world, such as TensorFlow, compute them for you automatically?”This is seemingly a perfectly sensible appeal – if you’re never going to write backward passes once the class is over, why practice writing them? Are we just torturing the students for our own amusement? Some easy…
