Type I error
The definition of Type I statistical error from wikipedia:
Type I error, also known as an “error of the first kind”, an α error, or a “false positive”: the error of rejecting a null hypothesis when it is actually true. In other words, this is the error of accepting an alternative hypothesis (the real hypothesis of interest) when the results can be attributed to chance. Plainly speaking, it occurs when we are observing a difference when in truth there is none (or more specifically – no statistically significant difference).
A false positive normally means that a test claims something to be positive, when that is not the case. For example, a test saying a woman is pregnant when she is actually not pregnant is an example of a false positive.
In risk capital, I like to define Type I errors as those where an investor rejects what would have been a good investment because he can’t tell that it’s different than a bad one. I call it a Type I error because the prevailing mindset in investing is “this is a bad investment”. Thus, a false positive (the conclusion that my hypothesis is right when, in fact, it isn’t) occurs when I discard a plan as bad when it is good.
If you’re an investor and want to argue that the primary hypothesis in your culture is “this is a good investment”, don’t fret, just add another “I” to “Type I” errors when you read this post (and the opposite when you see Type II). That is, this article would be about your firm committing Type II errors…
This is not entirely the investors’ fault, as entrepreneurs contribute to the situation: the great ones sometimes don’t make it clear that they are, in fact, the “next Google”, while the not-so-great ones (another post) do their best to look like the great ones.
But if we set that fact aside for a moment, and concentrate on the investor, why does my great plan to conquer the world receive the tepid “let me think about it” instead of the “here’s $20mm at whatever valuation you like” that it deserves?
If we go back to the original use of the term, in the field of statistics, we’ll find that researchers look to eliminate either Type I or Type II – notice the “either / or” proposition. In applied statistics, we can eliminate Type I errors and allow Type II, or vice versa, but not both. In order to eliminate both, our information system must be at least as complex as the universe it studies and cover all the individuals (become a census) . In the real world, the expense of a census is prohibitive, thus, we need to allow for one or the other kind of errors.
Back to investing.
Because investors have finite resources (mainly time, but also money, arguably) and are looking for capital appreciation and returns, they should have a system that doesn’t allow Type II errors (the false negative or the bad investment that looked good) . The cost of this is the existence of Type I errors (the good investment that looked bad). Right?
Wrong. A search through the literature clearly demonstrates that few (if any) investors approach anywhere near a zero Type II error – failure rates for risk capital are normally greater than 50% (that is, the investors are committing Type II errors more than half the time). In fact, most will freely admit that their game is a numbers one – out of 10 investments, 1 or 2 support the entire portfolio.
So, given that we’re really bad at eliminating the Type II errors from our screening systems, shouldn’t we pay closer attention to eliminating the Type I errors? In other words, given that we invest in failures most of the time, shouldn’t we make sure that the good investments get funded?
I freely recognize the enormous simplifications in my argument. It’s similar to the simplification set forth in most of my classes in mechanics (infinitely rigid, frictionless, flat surface) and economics (ceteris paribus). At the very least, there’s room enough to invite an expert on control theory or process control into the fray.
And it sure would be fun/profitable to be the investor who figures this one out…
