ANN for learning curve dynamics

[Strange]

Consistent with the prevailing weather, there is a paper Learning Curve Dynamics with Artificial Neural Networks by Alexei Kondratyev. I don't know much about ANN (or any ML beyond simple linear regressions), but it appears that this methodology can (a) fit historical data that does not necessarily follow normal distribution and (b) potentially add other features into the model.



Has anyone tried to reproduce the results in this paper? Is there an easy way I can play with my datasets and ANN?



PS. given the little I know of ANN, there got to be huge potentially for overfitting the data

[ronin]

On quick reading, the model is:



Given the levels of all points on the curve and the move of one point on the curve, each point on the curve either ticks up, ticks down or stays the same. Regress on historical data.



So if one point moves up, presumably neighbouring points will move up by a siilar amount, and points further away less so. All capped and floored to plus minus one tick. PCA capped and floored, in other words.



The ANN angle looks a little bit contrived to me.

[nikol]

For very quick start



https://machinelearningmastery.com/introduction-python-deep-learning-library-tensorflow/



Disclaimer: the site helps with quick start, but at some deeper points you might encounter "lose ends" or simply wrong statements there.



There are many examples in github.



At some point I switched to keras.tensorflow on CUDA. Really fast. But all-in-all it is perhaps better to rent one of those AI-clouds with pre-installation.



Today, Google's Tensorflow is most famous, however, yesterday I read some stats that 70% of all publications of 2019 use PyTorch (dev by FB).



To overcome overfitting problem you could use blend of models to MC-generate training sample and then use "transfer learning" techniques (no single recipe) to train on smaller and real sample. It is sort of "deep learning".



ANN is good to use when you want to reduce operational hurdle, not to improve your knowledge about things (there are some exceptions though).



Example: imagine, you calibrated data with model-A, but tomorrow it stops working and you substitute it with model-B. It takes time to: identify that something is wrong, search for replacement, implement/test/put into production. With ANN basically there is no transition, you just incrementally train (transition learning) your model such that it adopts new reality.



PS. ANN does not "understand" the problem. It just fits into it and you don't know exactly why... It involves a lot of intuition, but it grows with experience.

[Strange]

Just heard a podcast with the guy, not sure what to make of it. His main claim is that "it matches non-gaussian dynamics better than PCA", though I still don't understand how do you back out a fair curve give the current set of points.



@nicol, thanks - i will try playing with it for a little bit

[nikol]

for fun:

https://antoinesavine.com/2019/11/01/deep-learning-derivatives-pricing/



In principle, one don't need ANN to do this trick - apply integrated sigmoid to option prices as function of strikes. That's one function. If you apply many nodes (~ANN layer) the fit will become better. Same as increase of polynomial degree.

[tabris]

I am skeptical. Basically, at lower frequencies, the non gaussian dynamics would be too noisy to render ANN useful for 1 step ahead forecasting and at higher frequencies, the linearity should make PCA a better and simpler choice. Agree with others on this that ANN for this problem just sounds a bit gimmicky. There are possibly/arguably better techniques to achieve what the paper wanted to do out there now.

[mtsm]

Some people from a Eurozone bank mentioned to me that they could not really reproduce that stuff. Not sure how hard they tried. They are a trustworthy crowd though, in the weeds of this stuff.

[nikol]

Short recipes:



- Normalization and input/output transformations are the key. What happens inside is a bit of enigma (not completely of course).

- Start with some very simple network architecture and then make it more complex.

- Gradual increase of complexity is even possible (it is part of learning transfer techniques). This can save overall training time.

- All modern ANN libraries allow intermediate saving of the model (net topology + node weights), use it and keep track of nodes changes, not only output residuals (or whatever metrics in use).



I like this visualization demo. Simple but effective learning kit.



https://playground.tensorflow.org/

[nikol]

https://www.risk.net/awards/7216346/rising-star-in-quant-finance-blanka-horvath-aitor-muguruza-and-mehdi-tomas