Explaining deep learning

[Rashomon]

I'm still trying to understand how RNN's work. As it happened I've been reading Eisenbud's commutative algebra with a view to AG at the same time. He mentions ideals of 2×2 minors. I wonder if that’s a useful tool for understanding the "attention windows" in Karpathy's blogpost.



I don't really like the brain metaphor part of NN's (at least not for reasoning about what the tool will do). "Take on arbitrary coefficients" feels better to me. But the minors would then need to be organized as the RNN is, and then compute something useful about that space.