Interesting paper on a new family of deep ANNs: Neural Ordinal Differential Equations

[Maggette]

Hi,



just stumbled on this kind of research

here



Sounds interesting. Anybody an opinion?



Edit: somehow thinks that could be interesting in a time series context.

Thx

[finanzmaster]

Well, if Raj addresses this topic then it might be worthy to have a closer look.



However, the more stuff I (have to) learn, the more I recall an outstanding professor, who taught us the strategic management. In particular, he emphasized that most of successfuly companies adhere not to the strategy of the 1st step but rather to a strategy of the (fast) 2nd step.



Concretely it means that I will watch Raj's video to grasp the main idea quickly, however, I will dwell only after this new approach shows a couple of successful applications (and moreover, after they implement it in Keras )

[bullero]

So this is basically just a network with infinite number of layers?



Edit: I mean, what is the "new" idea here besides additional number of mathematical layers. Sounds like a pure mathematical trick where one changes the problem domain from optimization to finite difference world. Am I missing something?

[nikol]

deviation:



NN pruning improves things

https://www.engadget.com/2019/05/06/mit-researchers-discover-neural-subnetworks/



It is like with multi-variate fit, where parameters are added/removed after they are found to be significant or irrelevant

[finanzmaster]

>So this is basically just a network with infinite number of layers?

Well, yes, but as Raj means in his video: one may (probably) apply the whole arsenal of ODE stuff to train such networks.

[bullero]

Yeah that's basically changing the problem domain from optimisation to ODE solving