From: John Conover <john@email.johncon.com>

Subject: Re: forwarded message from Kimberly Bodelson

Date: Thu, 23 Jan 1997 02:48:56 -0800

I made a typo! "sigma, we have more drops, (October of 1927 and 1989 for example,)" should obviously been 1987, (October 19'th to be exact, and the 1927 date was October 29'th.) My apologies-I was typing out of memory. So did I make any other mistakes? Well, I wondered about that, and had johncon re-run the data to make sure. Some interesting facts: Since 2 January, 1900, to date, there have been 53 declines of more than 10% in the market indices. Of those 53, 15 were more than 25%. To put that into perspective, every 2 years we would expect a decline of 10%, and every 6 years, a decline of 25%, or so[1]. And how do the declines work? That is interesting. In the biz days following the 1929 crash, the market dropped about 10%, (doesn't sound like much does it?) However, the market then went out of steep decline and into a slow grinding decline until 1932, at which time it had dropped about 90%!!!! (The DJIA was 381 the date of the 1929 crash, dropping 30, points, or so in the next few days, ending at 41 in 1932, ie., about a 10 to 1 drop.) And how long did it take to get back to 381? Not until 1954. (To put that into perspective, radio became popular, the TV was invented, we went through WWII, the baby boomer generation was conceived, the Korean War started and almost finished, and then the market regained its losses.) Is this an anomaly? No. If you invested in 1968, you would have to wait until 1982 to make any money, (not to mention how much you lost in the high inflation of the era while you were waiting those 14 years-don't forget indices are not in real dollars, whatever those are.) And then there was 73-74, which was only a 2 year bear market, and, of course, 19 October, 1987, which lasted about 2-3 years, or so[2]. Interestingly, there is more known about the American equity markets than any other actuary[3]. The only things we have more knowledge about is the speed of light, (now to 11 decimal places, and possibly 12,) and the element silicon, (for example, the band gap of silicon is known to 9, and possibly 10 decimal places.) John [1] The root mean square, ie. one sigma, of the market indices runs about 3% on a day-to-day time scale, or so. So we would expect a 9%, or 3 sigma, drop in the indices every every 746 business days, or about 2.87 years-if the distribution of the increments was represented accurately as a Gaussian distribution. (2 years, or 520 biz days is 2.9 sigma for the same numbers, so we are close.) However, we would expect a 25% drop, ie., about 8 sigma every 3 x 10^15 days, or about about a thousand trillion years!!!!! (Which was my statement earlier, that to 3 sigma we have a good model, beyond that ...) Note that these sorts of things drive the numerical people up the wall. They would attack the 3% number with a vengeance, and not the 3 sigma error of 3%, (which is considered "reasonable" accuracy in statistics.) They would point out that these things are very sensitive, and the Gaussian distribution highly non-linear at these levels, so itty bitty errors make big differences. For example, a 3% error in the 3% rms value would account for the discrepancy. And how many days of data do we have to have for our measurements to have less than a 3% error in the 3% value? 188 days worth, to a 95% confidence level, which is the confidence level the political polesters use. (And most statistics folks consider that schlocky methodology.) So, you think the market has been rising since 1982, right? Is it a short term "bubble" or is it a genuine long term phenomena. How many days of data do we have to have for a 95% confidence level that the market is really rising, and not just a bubble? The market would have to have this trend over 824862 days, or about 3 millenia!!!!!! (We aren't sure what the Babylonian markets were doing, but 3003 BC was a bumper crop for barley, of which 85% was used to make beer. They kept records in cuneiform clay tablets, which were organized into a relational database, like a general ledger, much like we use today, only on clay tablets instead of magnetic media.) Babylonian affairs aside, the point is that one must be very careful about what one says about markets, and their mastery thereof. As a general number, trends in markets assert their self over a period that is measured in decades, not years[2]. As you can see, multi-year run ups (and run downs,) in the indices are common, and statistically understandable-at least to 3 sigma limits, or so. [2] The duration of bear/bull markets is interesting. The nature of things fractal is that they have these long time intervals above, or below where they should be, (not periodic, but cyclic.) If we assume a Gaussian distribution of the increments, these "intervals" when averaged over a "sufficiently long time," will have probability of duration that decrease with the reciprocal of the square root of time, which is a very sluggish function-thats why they last so long. Further, the number of bear or bull markets in a given time interval is expected to be proportional to the square root of the time interval. Note that although we can't predict when they will happen, (there stochastic, and that means we can't,) we can tell a lot about their statistical properties, which is what programmed trading is all about. And what does this mean about the current market? The DJIA is about twice what it should be to fit into the data of the rest of the century. And are such things to be expected? Yes, they are, because we would also would expect that the range of the indices in a bull minus bear markets to increase with the square root of time. [3] Of these things, the equity markets are the most difficult to measure because they are stochastic. For example, the rms is about 3%, or so, on a day to day basis. Squaring this number is approximately the growth of the market in a day, or about 0.0009, or about a tenth of a percent, in very rough numbers. Rms is the scale with which one measures randomness. So, the problem is how do you measure a tenth of a percent in something that is rattling around 3%? The answer is it takes a lot of data. For example, suppose you wanted to be 99% confident that you measured 0.0009 to and error level that is within 1%, (not an unreasonable expectation,) it would take 82,486,132 days, (or about a third of a million years,) of data!!!! (We have several centuries of data, BTW. The Spanish were good chroniclers, as were the Dutch in their bull/bear market of tulip bulbs, which eventually went bust-which is the 3'rd most highly studied fractal, next to the 1927 and 1987 market busts.) Just to give you a feel for such things, if we drop our confidence level requirements to 50%, and and our error requirements to 0.0009, it would require only 494 days, or about 2 years of data. So, in a 2 year run up, we are only 50% confident that it is really even a run up, ie., a crap shoot, if we bet on that. BTW, the way these things are handled by the programmed traders is to calculate P for the equation f = 2P - 1. Then a statistical estimate is made on the accuracy of P, and P is reduced by this amount, which essentially takes the uncertainty of measurement into the stochastic process of the stock's day-to-day rattling around. (For example, ASND has been running very hot, with a measured P of 0.6. Suppose we did a statistical estimate, and found that over the time interval, we were 90% confident that this value was not below 0.57, we would play P as 0.57 * 0.9 = 0.513. So we skirted the problem of not having enough data with risk management. However, this will only work in bubbles, and if the long term market P is above about 0.52, which it isn't, so there is still some risk involved, but it is minimal, and deceasing as the PT's gain more data with time. As you can see, they don't gamble-they reduced their long term uncertainty risk to over 3 sigma.) -- John Conover, john@email.johncon.com, http://www.johncon.com/

Copyright © 1997 John Conover, john@email.johncon.com. All Rights Reserved. Last modified: Fri Mar 26 18:55:20 PST 1999 $Id: 970123024859.2183.html,v 1.0 2001/11/17 23:05:50 conover Exp $