A slow week with a new book
Slowly moving forward, realizing I really didn't know anything about matrix derivatives
While working through the practicum for Deep Learning with Pytorch, week 2 I realized I really didn’t understand anything about taking derivatives with respect to derivatives. I found some notes from Stanford’s CS231n that were useful: http://cs231n.stanford.edu/vecDerivs.pdf.
A little bit of hunting also pointed me to a paper by Jeremy Howard and Terrence that’s supposed to introduce matrix calculus particularly well, which I’ll take alook at right after finishing this post: https://arxiv.org/pdf/1802.01528v2.pdf.
Finally, I started prototyping a symbolic python library that explicitly lays out the results of matrix multiplication as symbols, to confirm I’ve understood what I’ve been reading and being able to validate my results quickly.
My copy of Linear Algebra and Learning from Data by Prof. Gilbert Strang finally arrived, and I’ve been enjoying reading it (with somewhat slower progress than I would have liked).
They’re publishing an Indian edition of the book that can be directly ordered from the publishers at http://www.wellesleypublishers.com/; they carefully pack and mail the book directly to minimize extraneous costs.