From: John Conover <john@email.johncon.com>
Subject: Re: rms of indices
Date: 3 Jan 2001 09:58:09 -0000
Note that this does not validate the analytical method, (tsinvest.)
What it does validate is being able to synthesize the stocks in a
market, (tsinvestsim,) at least such that the analytical method can
not determine whether the data is empirical or fabricated.
So, the synthesizer, (tsinvestsim,) is used to produce very precise
characteristics of stocks in a market, (which can be verified
theoretically,) and the analytical method, (tsinvest,) tested to see
if it produces the expected theoretical values-to gain confidence in
the analytical method.
Its a long drawn out process, (struggling through the numbers, the
simulation has to run for a minimum of 4 centuries of daily data for a
minimum of 300 stocks-all as dictated by the tsshannoneffective
program,) and involves running corner solutions in a three sided
envelope, (non-volatile, optimal, and volatile, for stocks that are
the same, and different, in all combinations thereof; and then the
same scenario for stocks with persistent and anti-persistent
characteristics.)
The results of the testing process take about 2 weeks of a dedicated
Pentium class machine to produce, and are tabulated on the tests page,
http://www.johncon.com/ntropix/tests.html, where they are compared
against the theoretical numbers, (and referenced in the first
paragraph of the quality assurance page,
http://www.johncon.com/ntropix/QA.html.)
Note that the entire QA and testing process dependeds on the utilities
listed on http://www.johncon.com/ntropix/utilities.html.
John
BTW, the validation process used is by no means fool proof, (its
circular logic if you look at it.) But what it does do is lend
confidence to the analytical methods used prior to empirical
verification.
If you are a glutton for punishment, try something like this with
MACD, etc.
John Conover writes:
> This is why the second paragraph of the quality assurance page,
> http://www.johncon.com/ntropix/QA.html, works.
>
> Note that all that was done was to extract the analytical data from a
> set of stocks, (three numbers for each stock,) and use those numbers
> in an inverse function, (simulation,) to reconstruct the original
> data-and then analyze the reconstructed data, and compare the original
> analytical data with the reconstructed analytical data.
>
> John
>
> John Conover writes:
> > And, then if you are a real glutton for punishment, you can write a
> > program that does a point-by-point reconstruction of the indice using
> > the formula:
> >
> > G = (1 + rms)^P * (1 - rms)^(1 - P)
> >
> > for each P and rms in both files.
> >
> > You will find that the indice can be reconstructed, (differing only by
> > a scaling constant,) from its statistics-a validation of the model
> > used.
> >
> > John
> >
> > BTW, there are a lot of tricks that can be used from the
> > http://www.johncon.com/ntropix/utilities.html page. The tsgainwindow
> > does about the same thing, but a geometic progression has to be done
> > on the output.
> >
> > John Conover writes:
> > > BTW, another interesting thing to do is:
> > >
> > > tsshannonwindow -w 100 -a -b -c -d -e -f -g -h data_file
> > >
> > > which calculates the Shannon entropy, using different methods. Vary
> > > the -w argument.
> > >
> > > The first thing one notices is that the Shannon entropy is fractal,
> > > (unless the -w argument is set to many thousands-consistent with the
> > > tsshannoneffective program,) and all the methods give values that
> > > are very close to the theoretical value:
> > >
> > > P = ((avg / rms) + 1) / 2
> > >
> > > even though some of the methods don't measure the avg or rms at all,
> > > (for example, some just count the number of ups, and downs, and
> > > compute the entropy from ups / (downs + ups) in the time interval
> > > defined by the -w argument.) Others assume that the absolute value of
> > > the movements are the same as the rms.
> > >
> > > Its kind of an interesting exercise.
> > >
> > > John
> > >
> > > BTW, the manual page for the tsshannonwindow program is at:
> > > http://www.johncon.com/ndustrix/archive/utilities/tsshannonwindow.txt, and the
> > > output is a standard Unix tab delimited file. So, use "cut -f1,2" and
> > > "cut -f1,3" and so on to get different files of the different methods
> > > of calculating the Shannon probability. Its kind of interesting to
> > > make a plot of them overlayed. The gist of it is that stocks increase
> > > in value, (or decrease,) more based on the ratio of the number of up
> > > movements to down movements in an interval than by the magnitude of
> > > the movements in an interval. Kind of counter intuitive, (and a good
> > > empirical statement of the fractal nature of such things.)
> > >
> > > John Conover writes:
> > > > Just as kind of an FYI, Blake LeBaron,
> > > > (http://www.ssc.wisc.edu/~blebaron/,) one of the NLDS, (chaos,)
> > > > theorist pointed out in 1991, that for some reason, market crashes are
> > > > always preceded by an increase in the root mean square of the daily
> > > > marginal returns of the indices. (Which is vary characteristic of
> > > > bifurcations in NLDSs.)
> > > >
> > > > If you use the tsrmswindow program, (from
> > > > http://www.johncon.com/ntropix/utilities.html,) on the historical
> > > > database of the DJIA, S&P500, and NASDAQ, it seems to be true. For
> > > > example:
> > > >
> > > > tsfraction data_file | tsrmswindow -w 100
> > > >
> > > > where data_file is the time series for the NASDAQ, will make a plot
> > > > file that shows that for several years, the rms values have been
> > > > running about 3X their average value-averaged since 1971. A like
> > > > scenario happened in the late 20's to the DJIA and S&P. Likewise for
> > > > the other crashes and crash'ets of the 20'th century.
> > > >
> > > > John
> > > >
> > > > BTW, as nearly as I can tell, the US equity market has degenerated
> > > > enough such that 5-10% of the US's net wealth has went up in smoke.
> > > > About 3-5 trillion bucks have been lost, (depending on who is doing
> > > > the counting,) and the US net wealth is estimated at about 50 trillion
> > > > bucks-about a forth to half of the world's net wealth, (Re: the US
> > > > FED-I have no idea how they measure that; what is the value of the
> > > > nuclear weapons arsenal? How is it depreciated?) Its a fairly sizable
> > > > chunk of the world's net wealth.
> > > >
--
John Conover, john@email.johncon.com, http://www.johncon.com/