Why is the quant community skeptical about sentiment models?
-
Tradenator
- Posts: 0
- Joined: Thu Jan 01, 2004 12:00 am
Why is the quant community skeptical about sentiment models?
https://bankunderground.co.uk/2016/10/07/inflation-compensation-and-risk-sentiment/
-
TonyC
- Posts: 0
- Joined: Thu Jan 01, 2004 12:00 am
Why is the quant community skeptical about sentiment models?
> wonderful discussion in the comments thread -
-> "I went to Harvard" vs "I went to Stanford"
> vs "I went to Berkeley".
Old joke ... A cashier in a Boston supermatket sees a 20 yr old with 15 items in the 10 item or less checkout line. "Hey kid. Whatsa madda? Yaz went to Harvard and yuz can't count, or yaz went to MIT and yuz can't read?"
-> "I went to Harvard" vs "I went to Stanford"
> vs "I went to Berkeley".
Old joke ... A cashier in a Boston supermatket sees a 20 yr old with 15 items in the 10 item or less checkout line. "Hey kid. Whatsa madda? Yaz went to Harvard and yuz can't count, or yaz went to MIT and yuz can't read?"
flaneur/boulevardier/remittance man/energy trader
-
Rashomon
- Posts: 0
- Joined: Thu Jan 01, 2004 12:00 am
Why is the quant community skeptical about sentiment models?
Econometricus: The primary problem for me is that "sentiment" is glossed as 1-dimensional. I don't care what you label your variable; sentences are not 1-dimensional.
In training any model you have to get the Y and the X. To get lots of X (and thus convergence / accuracy estimates) let's say you scrape Wikipedia. Then you have to have a human accurately code the Y. That means both building an ontology (happy? sad? snidely undercutting 90% of readers with oblique stylistic cues?) and getting someone(s) to label all your X accurately (try doing this yourself with unknown future requirements and then ask how well a subordinate/mechTurk will do it for you when they don't care).
You can then pump Y~X through a fancy model (say a deepnet), but what good are killer fits to which words are *TRULY* happy and sad?
There are more problems depending on what people are trying to do with it. With twitter and stocks, most tweeters don't have the money to trade, they may not be trading the S&P (which is what Derwent traded iirc), and they're not tweeting as they trade. (Some people do seem to do this---with unknown size---but is it worth fading their move minutes after the fact?)
In the case of modelling terrorism by analyzing message boards text, the Y is really small. If I were granting money I would ask Harvard instead of MIT to come up with signals of who is about to go on a sniper/bomber rage.
In training any model you have to get the Y and the X. To get lots of X (and thus convergence / accuracy estimates) let's say you scrape Wikipedia. Then you have to have a human accurately code the Y. That means both building an ontology (happy? sad? snidely undercutting 90% of readers with oblique stylistic cues?) and getting someone(s) to label all your X accurately (try doing this yourself with unknown future requirements and then ask how well a subordinate/mechTurk will do it for you when they don't care).
You can then pump Y~X through a fancy model (say a deepnet), but what good are killer fits to which words are *TRULY* happy and sad?
There are more problems depending on what people are trying to do with it. With twitter and stocks, most tweeters don't have the money to trade, they may not be trading the S&P (which is what Derwent traded iirc), and they're not tweeting as they trade. (Some people do seem to do this---with unknown size---but is it worth fading their move minutes after the fact?)
In the case of modelling terrorism by analyzing message boards text, the Y is really small. If I were granting money I would ask Harvard instead of MIT to come up with signals of who is about to go on a sniper/bomber rage.
"My hands are small, I know, but they're not yours, they are my own. And they're, not yours, they are my own." ~ Jewel