From: John Conover <john@email.johncon.com>

Subject: Quantitative Analysis of Non-Linear High Entropy Economic Systems IV

Date: 14 Feb 2002 07:38:46 -0000

As mentioned in Section I, Section II and Section III, much of applied economics has to address non-linear high entropy systems-those systems characterized by random fluctuations over time-such as net wealth, equity prices, gross domestic product, industrial markets, etc.

The dynamics of non-linear high entropy systems are probabilistic in nature, and require a suitable methodology for analysis.

Note: the C source code to all programs used is available from the NtropiX Utilities page, or, the NdustriX Utilities page, and is distributed under License.

The file

contains the time series of GE's daily close equity price history,
from January 2, 1962, through, March 2, 2002, (10,113 days,)
inclusive. The data is from Yahoo!'s Historical Prices database, and
is adjusted for splits.*ge.jan.02.1962-mar.1.2002*

The file was converted to a Unix flat file database with the
program:

**
***csv2tsinvest* GE *ge.csv* > *ge.jan.02.1962-mar.1.2002*

Analysis:

Analyzing the time series for the GE equity price time series:

**
***tsfraction* *ge.jan.02.1962-mar.1.2002* | *tsavg* -p
0.000507
*tsfraction* *ge.jan.02.1962-mar.1.2002* | *tsrms* -p
0.015451
*tsgain* -p *ge.jan.02.1962-mar.1.2002*
1.00038773455

where the average of the marginal incrments,

, the root-mean-square of the marginal
increments, **avg =
0.000507**

, and the gain
of the marginal increments, **rms = 0.015451**

, and:**g**

**
avg 0.000507
--- + 1 -------- + 1
rms 0.015451
P = ------- = ------------ = 0.51640670507 .........(1.24)
2 2
**

are used in the file,

,
which contains the single record:*sim*

**
GE, p = 0.51640670507, f = 0.015451
**

and making the simulation file for GE equity price:

**
***tsinvestsim* *sim* 10113 > *ge.jan.02.1962-mar.1.2002.tsinvestsim*

where the file,

is the simulated GE equity price time series with identical
statistics.*ge.jan.02.1962-mar.1.2002.tsinvestsim*

The value of the variables will be used in analysis of the dynamics
of GE's equity price, a non-linear high entropy economic system, by
comparing the dynamics of the empirical data with the dynamics of both
the simulated data and its theoretical abstraction,

.**constant * erf (1 / sqrt (t))**

The file

contains the time series of GE's daily close equity price history,
from January 2, 1962, through, March 2, 2002, (10,113 days,)
inclusive. The data is from Yahoo!'s Historical Prices database, and
is adjusted for splits.*ge.jan.02.1962-mar.1.2002*

The file was converted to a Unix flat file database with the
program:

**
***csv2tsinvest* GE ge.csv > *ge.jan.02.1962-mar.1.2002*

The cumulative probability of the duration of the increases and
decreases in the average of the marginal increments,

, of GE's equity price:**avg**

**
#!/bin/sh
interval=1
while [ $interval -le 4048 ]
do
***tsmath* -l *ge.jan.02.1962-mar.1.2002* | *tslsq* -o | \
*tsderivative* | *tsavgwindow* -w $interval | *tsrms* -p >> \
*ge.jan.02.1962-mar.1.2002.avgwindow*
*tsmath* -l *ge.jan.02.1962-mar.1.2002.tsinvestsim* | *tslsq* -o | \
*tsderivative* | *tsavgwindow* -w $interval | *tsrms* -p >> \
*ge.jan.02.1962-mar.1.2002.tsinvestsim.avgwindow*
interval=`expr $interval + 1`
done

which uses the

program to calculate the average of the marginal increments over the
time interval, *tsavgwindow*

, for all
intervals from **interval**

to
**1**

, and the **4048**

program to calculate the deviation of the average, for GE's equity
price.*tsrms*

Figure I is a plot of the cumulative probability of the duration of
the increases and decreases in the average of the marginal increments,

, of GE's equity price, and its
simulation, with the theoretical value, **avg**

. The frequency of the durations of the
increases and decreases of up to 4,048 trading days, (about 16
calendar years,) are shown.**0.015451 * erf
(1 / sqrt (t))**

Figure II is a plot of the cumulative probability of the duration
of the increases and decreases in the average of the marginal
increments,

, of GE's equity price,
and its simulation, with the theoretical value, **avg**

. The frequency of the durations
of the increases and decreases of up to 253 trading days, (about a
calendar year,) are shown. Figure II the same as Figure I, but shows
only the frequency of short durations of increases and decreases for
clarity.**0.015451
* erf (1 / sqrt (t))**

Figure III is a plot of the cumulative probability of the duration
of the increases and decreases in the average of the marginal
increments,

, of GE's equity price,
and its simulation, with the theoretical value, **avg**

. The frequency of the durations
of the increases and decreases of 253 trading days, (about a calendar
year,) to 4,048 trading days, (about 16 calendar years,) are shown.
Figure III is the same as Figure I, but shows only the frequency of
long durations of increases and decreases for clarity. Since the data
set size consisted of only 10,113 trading days, the empirical
cumulative probability is smaller than the theoretical value by
**0.015451
* erf (1 / sqrt (t))**

at 4,048 trading days.**0.015451 * erf (1 / sqrt (10113)) = 0.015451 * 0.011968
= 0.000184917568**

Figures I, II, and III, are the cumulative probability of the
duration of the increases and decreases in the average of the marginal
increments,

, of GE's equity
price. For example, at the end of any 500 trading day interval, (about
2 calendar years,) the average of the marginal increments would be
within about, plus, or minus, **avg**

of
its mean value of **0.0008**

for one
standard deviation, about 68%, of the time.**0.000507**

The cumulative probability of the duration of the increases and
decreases in the deviation of the marginal increments,

, of GE's equity price:**rms**

**
#!/bin/sh
interval=1
while [ $interval -le 4048 ]
do
***tsmath* -l *ge.jan.02.1962-mar.1.2002* | *tslsq* -o | \
*tsderivative* | *tsrmswindow* -w $interval | *tsmath* -s 0.015451 | *tsrms* -p >> \
*ge.jan.02.1962-mar.1.2002.rmswindow*
*tsmath* -l *ge.jan.02.1962-mar.1.2002.tsinvestsim* | *tslsq* -o | \
*tsderivative* | *tsrmswindow* -w $interval | *tsmath* -s 0.015451 | *tsrms* -p >> \
*ge.jan.02.1962-mar.1.2002.tsinvestsim.rmswindow*
interval=`expr $interval + 1`
done

which uses the

program to calculate the deviation of the marginal increments over the
time interval, *tsrmswindow*

, for all
intervals from **interval**

to
**1**

, the **4048**

program to subtract the mean value of the deviation of the marginal
increments, *tsmath*

, and the **rms**

program to calculate the deviation of the deviation for GE's equity
price.*tsrms*

Figure IV is a plot of the cumulative probability of the duration
of the increases and decreases in the deviation of the marginal
increments,

, of GE's equity price,
and its simulation, with the theoretical value, **rms**

. The frequency of the durations
of the increases and decreases of up to 4,048 trading days, (about 16
calendar years,) are shown.**0.015451
* erf (1 / sqrt (t))**

The apparent discrepancy between the empirical and theoretical values of the cumulative probability of the duration of the increases and decreases in the deviation of the marginal increments is an artifact of the interaction between the windowing construct used for measuring the deviation over time intervals and a 13 year cyclic clustering phenomena in the time series. See Appendix I for particulars.

Figure V is a plot of the cumulative probability of the duration of
the increases and decreases in the deviation of the marginal
increments,

, of GE's equity price,
and its simulation, with the theoretical value, **rms**

. The frequency of the durations
of the increases and decreases of up to 253 trading days, (about a
calendar year,) are shown. Figure V the same as Figure IV, but shows
only the frequency of short durations of increases and decreases for
clarity.**0.015451
* erf (1 / sqrt (t))**

Figure VI is a plot of the cumulative probability of the duration
of the increases and decreases in the deviation of the marginal
increments,

, of GE's equity price,
and its simulation, with the theoretical value, **rms**

. The frequency of the durations
of the increases and decreases of 253 trading days, (about a calendar
year,) to 4,048 trading days, (about 16 calendar years,) are shown.
Figure VI is the same as Figure IV, but shows only the frequency of
long durations of increases and decreases for clarity. Since the data
set size consisted of only 10,113 trading days, the empirical
cumulative probability is smaller than the theoretical value by
**0.015451
* erf (1 / sqrt (t))**

at 4,048 trading days.**0.015451 * erf (1 / sqrt (10113)) = 0.015451 * 0.011968
= 0.000184917568**

Figures IV, V, and VI, are the cumulative probability of the
duration of the increases and decreases in the deviation of the
marginal increments,

, of GE's equity
price. For example, at the end of any 500 trading day interval, (about
2 calendar years,) the deviation of the marginal increments would be
within about, plus, or minus, **rms**

of
its mean value of **0.00075**

for one
standard deviation, about 68%, of the time.**0.015451**

The cumulative probability of the duration of the increases and
decreases in the likelihood of an up movement in the marginal
increments,

, of GE's equity price:**P**

**
#!/bin/sh
interval=1
while [ $interval -le 4048 ]
do
***tsshannonwindow* -a -w $interval *ge.jan.02.1962-mar.1.2002* | \
*tsmath* -s 0.51640670507 | *tsrms* -p >> *ge.jan.02.1962-mar.1.2002.Pwindow*
*tsshannonwindow* -a -w $interval *ge.jan.02.1962-mar.1.2002.tsinvestsim* | \
*tsmath* -s 0.51640670507 | *tsrms* -p >> *ge.jan.02.1962-mar.1.2002.tsinvestsim.Pwindow*
interval=`expr $interval + 1`
done

which uses the

program to calculate the likelihood of an up movement,
*tsshannonwindow*

, of the marginal increments over the
time interval, **P**

, for all
intervals from **interval**

to
**1**

, the **4048**

program to subtract the mean value of the likelihood of an up movement
of the marginal increments, *tsmath*

, and the
**P**

program to calculate the deviation of the likelihood of an up movement
of the marginal increments.*tsrms*

Figure VII is a plot of the cumulative probability of the duration
of the increases and decreases in the likelihood of an up movement in
the marginal increments,

, of GE's
equity price, and its simulation, with the theoretical value,
**P**

. The
frequency of the durations of the increases and decreases of up to
4,048 trading days, (about 16 calendar years,) are shown.**0.51640670507 * erf (1 / sqrt (t))**

Figure VIII is a plot of the cumulative probability of the duration
of the increases and decreases in the likelihood of an up movement in
the marginal increments,

, of GE's
equity price, and its simulation, with the theoretical value,
**P**

. The
frequency of the durations of the increases and decreases of up to 253
trading days, (about a calendar year,) are shown. Figure VIII the same
as Figure VII, but shows only the frequency of short durations of
increases and decreases for clarity.**0.51640670507 * erf (1 / sqrt (t))**

Figure IX is a plot of the cumulative probability of the duration
of the increases and decreases in the likelihood of an up movement in
the marginal increments,

, of GE's
equity price, and its simulation, with the theoretical value,
**P**

. The
frequency of the durations of the increases and decreases of 253
trading days, (about a calendar year,) to 4,048 trading days, (about
16 calendar years,) are shown. Figure IX is the same as Figure VII,
but shows only the frequency of long durations of increases and
decreases for clarity. Since the data set size consisted of only
10,113 trading days, the empirical cumulative probability is smaller
than the theoretical value by **0.51640670507 * erf (1 / sqrt (t))**

at 4,048 trading days.**0.51640670507 * erf (1 /
sqrt (10113)) = 0.51640670507 * 0.011968 =
0.00618035545**

Figures VII, VIII, and IX, are the cumulative probability of the
duration of the increases and decreases in the likelihood of an up
movement in the marginal increments,

,
of GE's equity price. For example, at the end of any 500 trading day
interval, (about 2 calendar years,) the likelihood of an up movement
in the marginal increments would be within about, plus, or minus,
**P**

of its mean value of
**0.025**

for one standard
deviation, about 68%, of the time.**0.51640670507**

The cumulative probability of the duration of the increases and
decreases in the gain of the marginal increments,

, of GE's equity price:**g**

**
#!/bin/sh
interval=1
while [ $interval -le 4048 ]
do
***tsgainwindow* -w $interval *ge.jan.02.1962-mar.1.2002* | \
*tsmath* -s 1.00038773455 | *tsrms* -p >> *ge.jan.02.1962-mar.1.2002.gwindow*
*tsgainwindow* -w $interval *ge.jan.02.1962-mar.1.2002.tsinvestsim* | \
*tsmath* -s 1.00038773455 | *tsrms* -p >> *ge.jan.02.1962-mar.1.2002.tsinvestsim.gwindow*
interval=`expr $interval + 1`
done

which uses the

program to calculate the gain, *tsgainwindow*

, of the
marginal increments over the time interval,
**g**

, for all intervals from
**interval**

to **1**

,
the **4048**

program to subtract the mean value of the gain of the marginal
increments, *tsmath*

, and the **g**

program to calculate the deviation of the gain of the marginal
increments.*tsrms*

Figure X is a plot of the cumulative probability of the duration of
the increases and decreases in the gain of the marginal increments,

, of GE's equity price, and its
simulation, with the theoretical value, **g**

. The frequency of the durations of
the increases and decreases of up to 4,048 trading days, (about 16
calendar years,) are shown.**0.015451 *
erf (1 / sqrt (t))**

Figure XI is a plot of the cumulative probability of the duration
of the increases and decreases in the gain of the marginal increments,

, of GE's equity price, and its
simulation, with the theoretical value, **g**

. The frequency of the durations of the
increases and decreases of up to 253 trading days, (about a calendar
year,) are shown. Figure XI the same as Figure X, but shows only the
frequency of short durations of increases and decreases for
clarity.**0.015451 * erf
(1 / sqrt (t))**

Figure XII is a plot of the cumulative probability of the duration
of the increases and decreases in the gain of the marginal increments,

, of GE's equity price, and its
simulation, with the theoretical value, **g**

. The frequency of the durations of the
increases and decreases of 253 trading days, (about a calendar year,)
to 4,048 trading days, (about 16 calendar years,) are shown. Figure
XII is the same as Figure X, but shows only the frequency of long
durations of increases and decreases for clarity. Since the data set
size consisted of only 10,113 trading days, the empirical cumulative
probability is smaller than the theoretical value by
**0.015451 * erf
(1 / sqrt (t))**

at 4,048 trading days.**0.015451 * erf (1 / sqrt (10113)) = 0.015451 * 0.011968
= 0.000184917568**

Figures X, XI, and XII, are the cumulative probability of the
duration of the increases and decreases in the gain of the marginal
increments,

, of GE's equity price. For
example, at the end of any 500 trading day interval, (about 2 calendar
years,) the gain of the marginal increments would be within about,
plus, or minus, **g**

of its mean
value of **0.0011**

for one standard
deviation, about 68%, of the time.**1.00038773455**

As a side bar, none other than Sir Isacc Newton, referenced
in Harvard
Magazine's The
Damn'd South Sea, was foiled by the dynamics of speculative
markets-as were many more recently in the An equity increasing in price, generously, for two years is no indication that the good fortune will continue. About 16%, one deviation, of the time, it would be expected
that GE's equity gain, That's the good news. The bad news is that at 1,012 trading
days, (or about 2 years later,) the chances that GE's equity
price would still be maintaining the factor of 2 growth in value
would be about |

How much would a typical equity's price on the US exchanges be expected to fluctuate in a year?

From Section III, Equation (3.1):**
rms * sqrt (t) .....................................(3.1)
**

which is a formula for the deviation of the magnitude of
*bubbles* for the Brownian motion/random walk equivalent of
equity prices. It is, also, the *range*,

, often used in **R***Range/Scale*,
(Hurst,) analysis.

Converting to log-normal characteristics, the maximum, and minimum, respectively would be:

**
+rms * sqrt (t)
V0 * e ..............................(4.1)
-rms * sqrt (t)
V0 * e ..............................(4.2)
**

where

is the deviation of the
marginal increments in an equity's price,
**rms**

is the starting value of an equity
price, and **v0**

, is some time in the
future.**t**

The median value for the daily marginal increments in an equity
price on the US exchanges is about

, so for **rms =
0.02**

trading days in a calendar year, a **t = 253***typical* equity's minimum
and maximum price would be:

**
+0.02 * sqrt (253)
V0 * e = 1.37454047383 ...........(4.3a)
-0.02 * sqrt (253)
V0 * e = 0.727515864 .............(4.4a)
**

A *typical* equity's price maximum, divided by its minimum,
in a year, will be

, or
about a factor of **1.8893615142**

, where
**2***typical* means a standard deviation, (i.e., about 68% will be
less, 32%, more.)

Its interesting, because from "*Reuter*", April 27, 1997,
Peter Lynch was quoted as saying:

... Stocks are volatile ... the average stock listed on the New York Stock Exchange fluctuates 50 percent between its annual high and low ...

Although non-linear high entropy economic systems typically exhibit
long term exponential growth, that does not mean they can not go
*bust*. An interesting question is how long a public company,
on average, is listed on an exchange.

It is a difficult analytical problem because of the discontinuities; companies have different initial public offering equity prices, there are splits which effectively reduce the IPO price, and when a company's equity price falls below one dollar, it is de-listed. Not to mention that companies get de-listed when they are acquired, too.

However, some work on http://www.google.com/ indicated
that after adjusting for splits, a *reasonable* estimate for
the median, (half more, half less,) value of the IPO price for US
equities in the Twentieth Century was about two dollars.

For the Twentieth Century, the median value, (half more, half
less,) of the average and deviation of the daily marginal increments
in equity prices for those listed on the US exchanges was about

, and, **avg = 0.0004**

, respectively.**rms =
0.02**

As an interesting side bar, from Section I, Equation (1.24), and, Equation (1.18):
meaning that most, (actually, the median,) of the equities of US public companies in the Twentieth Century grew maximally optimal. |

The following

program file, *tsinvestsim*

, was
made for 500 companies:*exchange*

**
0, p = 0.51, i = 2
1, p = 0.51, i = 2
.
.
.
498, p = 0.51, i = 2
499, p = 0.51, i = 2
**

where the company names are

and all the company's stocks are statistically
identical, i.e., they all start at an IPO price of two dollars, and
are maximally optimal with **0, 1, ... 498,
499**

,
**avg = 0.0004**

, and **rms = 0.02**

.**P =
0.51**

The C source code to the

program had to be modified for the simulation so that whenever a
company's equity value dropped below a dollar, it had to be
de-listed. The following construct was added to the *tsinvestsim*

C source code, version 1.3, dated 2001/12/07 10:05:09, at line
588:*tsinvestsim*

**
if (sum < (double) 1.0) /* stock's value less than a buck? */
{
sum = (double) 0.0; /* yes, de-list the stock */
stock->sum = (double) 0.0;
stock->f = (double) 0.0;
}
**

which, if an equity's value is ever less than one dollar, set's it
to zero, forever. A simulation of

trading days, or 22 calendar years of 253 trading
days in a calendar year:**5,566 = 22 *
253**

**
./***tsinvestsim*

exchange 5566 > *result*

was run, and the last 500 records of the output file,

, searched for equities with zero
value. There were **result**

out of the
**206**

, or
**500**

of the companies had been
de-listed in the 22 calendar year simulation.**41.2%**

The simulation duration of 22 calendar years was chosen because
during the Twentieth Century, the median time a company was listed on
the US exchanges was 22 years. Half were listed for less, half
more. The

simulation number is
within about 20% of the empirical number. (Although
**41.2%**

of the companies in the simulation
had failed by 22 years, one had an equity price of
**41%**

, and the median value of
the 500 equities was **$1333.721702**

, compared
with a calculated value of
**$3.207836**

from Section
I, Equation
(1.20). Almost a perfect log-normal distribution.)**$3.04430985883**

As a side bar, this is an intuitive result. Using round numbers, during the Twentieth Century a company had a 50% chance of being de-listed every 20 years, or so. So, at the end of 20 years, 50% of the companies would remain. At the end of 40 years, 25% would remain, and at the end of 60, 12.5%, and 6.25% at 80 years, and after a century, 3.125%. There are 30 companies in the DJIA. Of the original that
were in the DJIA in 1900, it would be expected that that
The exponential characteristics of Equation
(1.20) would tend to imply that a company's equity value
would increase forever. That would be true if |

The

C source code is available in the tsinvest
distribution available from the NtropiX site.*tsinvestsim*

As a footnote addendum to this section, (October 19, 2010,) the same is true for the hedge fund industry, too. According to The
Quants: How a New Breed of Math Whizzes Conquered Wall Street
and Nearly Destroyed It, Scott Patterson, Crown Business, a
division of Random House, Inc., New York, New York, 2010, ISBN
978-0-307-45337-2, (which is highly recommended reading,) hedge
funds return, on average, about 20% a year in good times, (to a
one-time maximum as high as 80%.) This means the average and
deviation of the daily marginal increments of a typical hedge
fund would be about
where the hedge fund names are A simulation of
yields 47.29%, ( The hedge fund industry will have a major " And, that is true for any non-linear high entropy economic
system, including recessions in the GDP, (see: List of
recessions in the United States, where the recessions
lasting more than one year since the Great Depression are, 1929,
1937, 1973, 1980, and, 2009, which happened, on average, every
20 years,) currency exchange rates, etc. The "mechanism" of the
panics is a characteristic of the random process itself-an
economic system will experience exponential growth, and sooner
or later, the random risks will But, doesn't this contradict the paradigm of John Kelly's Kelly criterion? The answer is no. Note that all of the simulations in this
section required a " |

As a *qualitative* approximation model, (i.e., simplified
conceptual interpretation for Brownian motion fractals, using the stochastic
calculus, which is not adequate for *quantitative*
analysis,) the economic system, (equity value, GDP, currency exchange
rate, etc.) is always increasing or decreasing, randomly, (i.e.,
"*bubbles*," always increasing or decreasing, randomly.) The
probability of a bubble lasting at least

many years is, approximately,
**t**

. For example, the
probability of a positive, (or negative,) parts of the, (Juglar,) business cycle
lasting at least 4 years is **1 / sqrt (t)**

, (half of the positive or negative parts of the
business cycles would last less, half more); the chances of a business
cycle "bubble" lasting at least twenty five years would be
**1 / sqrt (4) =
50%**

, and so on. How big
the "bubble" will be, in one year, (if it lasts that long,) is
probabilistic, too, with a standard
deviation of approximately **1 / sqrt (25) = 20%**

for many/most economic systems. For example, 68%
of the time, the economic system movement will be less than +/- 30% in
one year; in four years, less than +/- 60%.**0.3 * sqrt
(t)**

Or, modeling the five positive or negative parts of the business cycles, (each lasting a median of four years, half lasting more, half lasting less,) over an typical 20 year period, one of the business cycles, on average, would have a negative economic system movement of at least a standard deviation, (assuming one out of five is 20% of the time, and is about equal to 16%, which is the the value of the standard deviation-not being precise, this is a simple "thought" model, technically, with a 50% chance,) or a movement of about 60%, which is very close to the 50% loss simulated, above.

Note the simplicity of the economic model-which only assumes that unpredictable things happen to an an economic system, (economy, equity value, etc.,) in a random fashion, which moves the economic system from its current value or state. Once that is assumed, the durability of the economic system is defined by the stochastic calculus.

An analysis of the deviation of the marginal increments of GE's equity price indicates that the deviation is not stable:

**
***tsmath* -l *ge.jan.02.1962-mar.1.2002* | *tslsq* -o | \
*tsderivative* | *tsrmswindow* -w 50 > *ge.jan.02.1962-mar.1.2002.rmswindow-50*

where the

program is used to calculate the deviation of the marginal increments
over a moving window of *tsrmswindow*

trading
days for GE's equity price.**50**

Figure XIII is a plot of the deviation, (root-mean-square,) of the
marginal increments,

, of GE's equity
price, using a **rms**

trading day moving
window for the calculation of the deviation.**50**

Apparently, there is cyclic phenomena occurring about every

trading days, (about 13 calendar
years,) causing the calculation of the deviation of the deviation of
the marginal increments of GE's equity price to be too large
everywhere except the **3,300***dips* at

days in Figure
IV and Figure
VI. The phonomena is quite complex:**3,300**

Marginal Increment | Date |
---|---|

0.066667 |
1962-07-02 |

-0.089744 |
1974-08-23 |

0.070423 |
1974-08-26 |

-0.065789 |
1974-09-04 |

-0.066667 |
1974-09-09 |

0.065574 |
1974-09-16 |

0.088235 |
1974-10-09 |

-0.174757 |
1987-10-19 |

0.082353 |
1987-10-20 |

-0.068293 |
1987-10-22 |

-0.084656 |
1987-10-26 |

0.064426 |
1987-11-05 |

-0.083333 |
1988-01-08 |

-0.106904 |
2001-09-17 |

-0.065502 |
2001-09-20 |

0.124352 |
2001-09-24 |

Note that:

The values are quite large-for example, on 1987-10-19, GE's equity price suffered a

drop in value; a**17.4757%**

deviation incident, which has a probability of happening that is indistinguishable from zero-yet it did happen.**0.174757 / 0.015451 = 11.3104006213**The large excursions in high volatility tend to cluster, usually occuring for a few weeks to a few months.

They happen infrequently, and possibly by serendipity, are cyclic in nature.

The mechanisms creating periods of high volatility are not clearly understood, and create substantial leptokurtosis in the frequency distribution of the marginal increments as shown in Section I, Figure II. The prevailing explanations are:

They are created by structural phenomena-they usually occur in the fourth calendar quarter when taxation requirements are addressed by investors.

Or, the probability distribution is more complex than is generally thought-tending to be produced by a non-linear mechanism, having Cauchy distribution like characteristics.

Or, possibly, they are the result of complex non-linear dynamical system characteristics, which can

*phase lock*to structural phenomena.

The high volatility days that are inconsistent with the statistical model used occurred in 16 out of the 10,113 days between January 2, 1962, through, March 2, 2002. There are several options available to address the discrepancies created:

Remove the 16 offenders from the data set.

Reduce their values to be consistent with the model.

Shuffle the 16 offenders around in the data such that their effects do not cluster, and are not synchronous with any window size used in the analysis.

The latter is preferred since much of the analytical data will
remain the same, (i.e., the average, deviation, and gain, of the
marginal increments,

,
**avg**

, and,
**rms**

, respectively, will not change.)**g**

A software strategy to shuffle the marginal increments in a time series would be to generate a new first column in the data set's tabular time series consisting of random numbers, sort on the random numbers, and then remove the column, leaving the original data set in a shuffled random order:

**
***tsgaussian* 10113 > *random*
*tsfraction* *ge.jan.02.1962-mar.1.2002* > *fraction*
paste *random* *fraction* | sort -n | \
cut -f2 > *ge.jan.02.1962-mar.1.2002.shuffled*
while [ $interval -le 6325 ]
do
*tsmath* -s 0.000507 *ge.jan.02.1962-mar.1.2002.shuffled* | \
*tsrmswindow* -w $interval | *tsmath* -s 0.015451 | \
*tsrms* -p >> *ge.jan.02.1962-mar.1.2002.rmswindow.shuffled*
done

where the

program is used to generate a file,
*tsgaussian*

, consisting of 10113
random numbers. The *random*

program is used to make a file,
*tsfraction*

, of the marginal
increments of GE's equity price. The two files are tabularized into a
single file with two columns using the Unix *fraction**paste* command,
numerically sorted on the first column of random numbers using the
Unix *sort* command, and then the first column removed, using
the Unix *cut* command, leaving only the marginal increments of
GE's equity price, in random order. The remainder of the proceedure is
identical to the process that created the graphs in Figure
IV and Figure
VI.

Figure XIV is a plot of the cumulative probability of the duration
of the increases and decreases in the deviation of the shuffled
marginal increments,

, of GE's equity
price with the theoretical value, **rms**

. The frequency of the durations of the
increases and decreases of up to 506 trading days, (about two calendar
years,) are shown.**0.015451 * erf (1 /
sqrt (t))**

As a side bar, many quantitative analyst-sometimes called
Peter Lynch is accredited with the proverb "making money on Wall Street is easy-keeping it is the hard part." A very small chance of something happening, given enough time, will happen, and that's why its hard. Mitigating risk is how one keeps the money. But we have just seen how difficult it is to get accurate
metrics on risk. Most market technicians use a set of
cross-checks on the metrics of risk. For example, there were
10,113 trading days represented in the time series for GE's
equity price, between January 2, 1962, and March 1, 2002. That
means that the largest daily movement would be a
Its an important concept since Continuing on, there should be 27 out of 10,000
For 2 sigma events, there should be 460 out of 10,000-and the
460'th largest movement was
Our conclusion is that the model seems to hold fairly well for slightly past 2 sigma events-possibly even to 3 sigma-but beyond 3 sigma, and into 4 sigma, we have to increase our risk assessment metric for an investment in GE's equity by at least a factor of 3. In |

Why the discrepancy?

**
***tsmath* -l *ge.jan.02.1962-mar.1.2002* | *tslsq* -o | *tsroot* -l | \
*tsscalederivative* > *ge.jan.02.1962-mar.1.2002*
*tsinvestsim* -n 1000 *sim* 10113 | *tsmath* -l | tslsq -o | *tsroot* -l | \
*tsscalederivative* > *tsinvestsim.jan.02.1962-mar.1.2002*

Figure XIV is a plot of the *persistence* in the daily
returns of GE's equity price. Note that there is significant short
term persistence, (about 0.54,) meaning that there is a 54% chance of
GE's equity price doing tomorrow, what it did today. This is probably
a short term inefficiency, (in the sense of the EMH, the Efficient
Market Hypothesis.) However, there is also an indication of a
period of *anti-persistence* of about

days, too-which is probably related to structural
issues, (for example, a high probability of a cyclic downward
**e^2.7 =
15***spike* every year would do this; probably near late October,
when mutual funds account for capital and income gains for tax
reasons, which can be seen in TABLE I.)

This type of persistence, (actually a cyclic issue,) would increase the likelihood of large swings, and they would tend to cluster. In addition, the leptokurtosis shown in the analysis of GE's equity price in Section I, would tend to increase the relative number of large movements in price.

It would be advantageous to develop a methodology to
*quantify* the persistence/leptokurosis/root inefficiencies of
a time series.

In the following analysis, the historical time series of the DJIA,
NASDAQ, and S&P500 indices through December 5, 2002 was obtained
from Yahoo!'s database of equity
Historical Prices, (ticker
symbols *^DJI*, *^IXIC*, and, *^SPC*,
respectively,) in *csv* format-which was converted to a Unix
database format using the

program. The converted filename were
*csv2tsinvest*

,
*djia1900-2002*

, and
*nasdaq1984-2002*

, respectively.*sp1928-2002*

**
***tsmath* -l djia1900-2002 | *tsroot* -l > djia.jan.02.1900-dec.05.2002
*tsmath* -l nasdaq1984-2002 | *tsroot* -l > nasdaq.oct.11.1984-dec.05.2002
*tsmath* -l sp1928-2002 | *tsroot* -l > sp.jan.03.1928-dec.05.2002
*tsinvestsim* -n 1000 sim 100000 | tsmath -l | *tsroot* -l > simulation.100000

where the file

contains:**sim**

**
dummy, p = 0.51
**

and is a simulation, for comparison, where the average of the
marginal increments,

, and
the root-mean-square of the marginal increments, **avg = 0.0004**

, giving a likelihood of an up movement,
**rms =
0.02**

, of
**P**

-the dynamic characteristics of a
**0.51***typical* equity's value on the US exchanges.

Using the

to calculate the linear least squares fit approximation of the first
*tslsq*

trading days,
(approximately one calendar month):**e^3 = 20.0855369232**

**
egrep '^[0-2]\.' djia.jan.02.1900-dec.05.2002 | ***tslsq* -p
-4.915572 + 0.539041t
egrep '^[0-2]\.' nasdaq.oct.11.1984-dec.05.2002 | *tslsq* -p
-4.665253 + 0.558219t
egrep '^[0-2]\.' sp.jan.03.1928-dec.05.2002 | *tslsq* -p
-4.884043 + 0.534678t
egrep '^[0-2]\.' simulation.100000 | *tslsq* -p
-4.138124 + 0.498988t

and plotting for

trading days, (about a calendar quarter):**e^4.14709512761 =
63.25**

Figure XV is a Log-Log plot of the Deviation from the mean, and
least squares fit, of the Brownian motion/random walk equivalent of
the DJIA, (from January 2, 1900, through, December 5, 2002,) the
NASDAQ, (from October 11, 1984, through, December 5, 2002,) the
S&P500, (from January 3, 1928, through, December 5, 2002,) and a
100,000 trading day simulation of a typical equity's value from the US
exchanges. The slope of the lines in the graph is the *root* of
the deviation-i.e., for the simulation, it would be 0.5, (meaning
square root,) or root mean square mathematics should be used,
(implying infinite entropy, uncertainty, or unpredictability.)

Of interest is the linearity of the empirical metrics-at least for a calendar quarter of daily closes.

Considering only the dynamics, the deviation of the indices' Brownian motion/random walk equivalent for the DJIA, NASDAQ, and S&P500 from their mean values would be:

**
(e^-4.915572) * (t^0.539041) = 0.00733152306 * t^0.539041
(e^-4.665253) * (t^0.558219) = 0.00941686546 * t^0.558219
(e^-4.884043) * (t^0.534678) = 0.00756636131 * t^0.534678
**

respectively, compared with the values derived using the

and *tsfraction*

programs:*tsrms*

**
0.011028 * t^0.5
0.014448 * t^0.5
0.011367 * t^0.5
**

respectively. (Note that the exponents represent a system with a Hausdorff
fractal dimension following a power law relationship-for a
Gaussian/normal distribution of the marginal increments, the
relationship is

; larger than
**1 / 0.5 = 2**

represents a time series with
marginal increments that have a deviation that is non-stable, or
diverging-a Pareto-Levy
distribution-which does have implications; see: Lessons
of the Fall: A Stunning Collapse and Weighty Morals to the
Story.)**0.5**

And, how much difference is there between the actual, empirical theoretic, and standard deviation approximation for the DJIA, NASDAQ, and S&P500? As an informal, (the deviation is not stable, and diverges,) pictographic interpretation:

**
***tsmath* -l djia1900-2002 | *tsroot* > djia.jan.02.1900-dec.05.2002.a
*tsmath* -l nasdaq1984-2002 | *tsroot* > nasdaq.oct.11.1984-dec.05.2002.a
*tsmath* -l sp1928-2002 | *tsroot* > sp.jan.03.1928-dec.05.2002.a

and plotting:

Figure XVI is a plot of the "*deviation*" from the mean,
least squares fit of variables, and standard deviation approximation
of the Brownian motion/random walk equivalent of the DJIA, (from
January 2, 1900, through, December 5, 2002,) the NASDAQ, (from October
11, 1984, through, December 5, 2002,) the S&P500, (from January 3,
1928, through, December 5, 2002,) for 512 trading days, (about two
calendar years.) The top three graphs are the NASDAQ, and the DJIA and
S&P500 are the bottom six, or approximately a +/- 10% error at two
calendar years.

Remember, as a mathematical expediency, the analysis uses the Brownian motion/random walk equivalent of the time series for the indices-to convert back to a log-normal distribution, find the median values, and multiply the median values by the exponentiated values in Figure XVI.

-- John Conover, john@email.johncon.com, http://www.johncon.com/

Copyright © 2002-2004 John Conover, john@email.johncon.com. All Rights Reserved. Last modified: Mon Jan 8 18:17:29 PST 2001 $Id: 020303194059.19689.html,v 1.0 2010/10/20 10:23:59 conover Exp $