Type I vs Type II error

Jurassic · Post by **Jurassic** » Mon Jun 22, 2020 3:20 pm

Interview question: which does a quantitative researcher worry more about, type I errors or type II errors, and why?

I couldnt get a good answer out for this

nikol · Post by **nikol** » Mon Jun 22, 2020 3:31 pm

Incomplete information is the largest worry. If it is incomplete, you build bad or may be wrong model and make bad or likely wrong decisions. Decisions are based on model approvals by hypotheses test, hence you have I and II types of error.

Jurassic · Post by **Jurassic** » Mon Jun 22, 2020 3:45 pm

@nikol I dont really understand you after the first sentence.

Also Im not sure whether this question is aiming towards talking about the parameters to say a linear regression are generated by hypothesis or you could hypothesis test the sharpe ratio of a strategy for significance.

nikol · Post by **nikol** » Mon Jun 22, 2020 4:05 pm

I made my best to understand your difficult to understand question (incomplete information), therefore, I made my own interpretation (model) and, hence here is my type I error.

But still I think I was right.

PS. Ha, you edited question which made me missing the target.

gmetric_Flow · Post by **gmetric_Flow** » Mon Jun 22, 2020 4:08 pm

A type I error occurs when the null hypothesis is mistakenly rejected, which would prompt us to use a signal without predictive power. A type II error would fail to reject the null for a signal, thus we would omit the signal from our strategy. Type I would expose us to trading risk (without the perceived compensation) whereas type II would be an error of omission, thereby losing an opportunity. I'd say losing money is worse than losing an opportunity to make money. Opportunities are abound, money not so much, so I'd say type I is more worrisome.

nikol · Post by **nikol** » Mon Jun 22, 2020 5:22 pm

@geometric_Flow

Your line of proof is more correct than mine, but your conclusion must be just the reverse -

type-II is linked to money loss (model is wrong, but you make trade and lose), while

type-I is linked to the loss of opportunity (model is correct, but you miss the trade).

Jurassic · Post by **Jurassic** » Mon Jun 22, 2020 6:34 pm

@gmetric_flow I think thats a great answer

one follow up: what would the hypothesis testing be with regards?

gmetric_Flow · Post by **gmetric_Flow** » Mon Jun 22, 2020 7:55 pm

@Jurassic, Apologies I failed to specify - the null hypothesis would be that the signal is random, i.e., it has no predictive value. Thus an error of commission (type I) would be worse than an error of omission (type II).

It's a rather basic overview, but Aronson's Evidenced-Based Technical Analysis has a section on error types and the like (the book has been recommended on here before).

Of course if you flip the null hypothesis around, you also should flip the error type that is most troublesome.