Big Data and Deep Learning, a technology revolution in trading or yet another hype?

[chiral3]

@rftx713: For some time, in other writings, I've been drawing parallels between modern consumption and Marshall McLuhan's famous phrase "The medium is the message". In modern times I think of Uber. Uber isn't about the driver. In most major cities (except London...) the driver is an over-educated kid punting his many start-ups who is not very good with directions. Uber is about the app. Full stop. It cracked me up to hear Travis Kalanick talking about autonomous cars years ago. When people think of Uber they think of the app. They think of pushing that button and seeing the cars on the map. The driver is a notch up in importance from the gas cap.



Several years ago the quant angle was - as an example, for music - to find songs that maximize hook appeal and maximize listener fatigue (fits a Heavyside-looking function). This results in the highest purchase-repurchase rates. There's been a shift to how we consume, though. Like the taxi driver, the player and the artist have been replaced. Of course this creates cross sell opportunities, abstracts currencies (via the endowment effect and availability heuristic), and generates massive trainable datasets.

[katastrofa]

I don't think generating songs will ever be a major application of smart algos (dare I say AI?), because there is and will be a lot of humans who WANT to write songs and will do it for peanuts just so that they get a shot at fame and recognition. And this is before we even think about reaching for the artistic creations of other species, e.g. whales.



Accounting, however...

[Maggette]

"People who talk about "big data" in public remind me of conversations about sex I had when I was 12. Everyone talks about it; very few people are actually doing it. "



Couldn't agree more. I actually do work with the hyped bid data technology stack (Spark, HBase, Hive, ELK, and some Flink) for a big customer. The hype drives bussiness my way.



I think we do have to make a distinction here: doing trivial ETL like stuff can be done and is done with "big data" technology. There it can add value and can replace expensive solutions.



But when people talk about "big data" most of the times they think of the predictive analytics part of the business. And here I only have a experienced a couple of use cases (like one couple ..2!!!!) that really added commercial value.



All the other stuff I workded on felt and was handled like an R&D project.



IMHO big data is often a lot of noise....

[jslade]

"In ten years, it's pretty feasible that some variant of recurrent-nets will be able to generate spit out mediocre sitcom episodes or formulaic pop songs on demand. That's a *really* big deal. "



You can already spit out formulaic pop songs using something as simple as LZW compression as the core predictor algorithm. In fact, you can barf out pretty convincing classical music this way. Video games already do this. This trick has existed since the 80s at least.



You can't make dweeb learning barf out mediocre sitcom episodes, and won't be able to in 10 years. You should know better than to say something this preposterous.



I have a theory that strength of claims about DL are inversely proportional to actual knowledge of the technology. The people who actually invented it: Hinton, LeCun, Bottou: they are most emphatically not saying crazy things like this, even though there are decent motivations for them to hype it up. In fact, Yann has gone on the record that people should chill out with the crazy claims, rightly drawing the parallel with the first AI winter.



DL and "AI" is presently looking a lot like "nanotech" was in the 90s and early 00s. Where's my self replicating nanobots?

[pj]

> Where's my self replicating nanobots?

Running around the house.

Annoying daddy nanobot.

[katastrofa]

http://www.reuters.com/article/us-exchanges-surveillance-ai-idUSKCN12P0FJ

[rod]

jslade: "The people who actually invented it: Hinton, LeCun, Bottou: they are most emphatically not saying crazy things like this, even though there are decent motivations for them to hype it up. In fact, Yann has gone on the record that people should chill out with the crazy claims, rightly drawing the parallel with the first AI winter."



Long before the AI Winter, back in 1956, Shannon wrote the following 1-page paper:



[img]/User%20Files/2119/Bandwagon%201956.jpg[/img]



Still applicable.

[chiral3]

Rod, good stuff.

[finanzmaster]

Indeed, an awesome essay!



My 2cents:



0) "that the basic results of the subject are aimed in a very specific direction, a direction that is not necessarily relevant to such fields as psychology, economics"...

Interestingly, that Kelly criterion is actually Kelly-Shannon.



1) "a through understanding of the mathematical foundation is surely a prerequisite to other application".

Well, I read somewhere (cannot find a link anymore) that if one would bother with proving everything about convolutional neural networks, that there would be no such impressive practical applications.



2) "a few first rate research papers are preferable to a large number that are poorly conceived or half-finished".

For nowadays I would state it even more radical: a few first class (passionate) researchers are preferable to a grey mass of lazy mediocres.

[chiral3]

George Holz. Hello from China.





Fcuk the US



Disrupt Bitchez