From: John Conover <john@email.johncon.com>
Subject: Re: Poll on market efficiency
Date: 5 Aug 2000 03:41:26 -0000
Hi Jim. Regarding my reply of yesterday regarding your question on
how/why entropy and market efficiency are related, I got some free
time today, and did a simulation that might lend some insight.
The simulation and analytical tools are Open Source, (from
http://www.johncon.com/ntropix/ and http://www.johncon.com/ndustrix/,)
so you can compile them, and verify the results. For the sake of
simplicity, only the EMH model was used in the simulation.
Here is what I did:
1) Using the tsinvestsim[1] program, with no persistence, and a
51% chance of moving up on any given day, (and a 49% of moving
down,) with a daily volatility of 2%, (I choose these values
because they are "typical" of stocks on the US exchanges,) for a
million days, yielded a stock price time series, that measured an
average gain per day of 0.04%, a volatility of 2%, which
translates to a daily gain of 0.02% per day. The stock has maximal
gain, and lowering OR raising the risk, (risk = volatility,)
results in less growth in price. This is a baseline stock for
comparison[2]. There was no persistence, as measured by the
tsrootmeanscale program[3].
2) By using the tsinvest[4] program, it can be verified that it
can do no better than the original time series in the growth of a
stock's value over time.
3) By iterating the tsinvestsim program parameters, but with
persistence[5], to find new values that give identical statistics
to 1), (above, and verifying as such, as in [1], [2], and [3],)
the values are verified as being the same, for all practical
purposes, (and the persistence, as measured by the tsrootmeanscale
program,) is, as specified.
4) But by using the tsinvest program, as before, it can be seen
that it can do substantially better than the original time series
in the growth of a stock's value over time. Using the same
proceedure as in 1) and 2), the output time series from the
tsinvest program has an average gain per day of 0.24%, and a
volatility of 2%, which translates to a daily gain of 0.2% per
day[6]. Additionally, there was NO persistence in the output time
series as measured by the tsrootmeanscale program, (i.e., the
entropy has been raised to infinity, meaning it is 100%
efficient, at least in the sense of the EMH.)
If we define efficiency as the ratio of annual gains, then, (with 253
trading days per year,) the inefficiency of the original time series
would be 1.0002^253 / 1.002^253 = 1.052 / 1.66 = 63%.
The 5% persistence, (e.g., lowering the time series entropy,) also
lowered the stock price efficiency by about a third, (in this very
simple case, assuming the paradigm of the EMH, which may or may not be
representative of real market conditions.)
John
[1] The tsinvestsim program is part of the tsinvest program suite from
http://www.johncon.com/ntropix/. It uses a random number generator,
(the one supplied in the Linux 2.2.12 kernel, which is not the best,
but adequate for the needs here.) All the program does is produce a
time series of a stock's price by implementing the EMH, which, in a
nutshell, is a compound interest algorithm-with variable interest
rates, (which can be negative, or positive, depending on price
movements,) but must be a random process that has a Gaussian/Normal
distribution, (for consistency with the paradigm of the EMH.) The mode
or mean of the distribution can be offset such that there is a long
term exponential gain in a stock's price, which seems consistent with
the empiricals of stocks on the US exchanges over the last
century. Persistence can be included in the simulation by adding the
last movement in the time series, multiplied by an appropriate value,
such that when added to the current movement, there is a better than
50/50 chance that the resulting current movement will be the same as
the last. Bottom line, the program is a stock price simulator, with
EMH characteristics, and allows the variables, (volatility,
exponential growth, persistence, etc.,) to be altered.
[2] The record used for tsinvestsim was "my_name, p = 0.51", and the
characteristics were measured by using the tsfraction program, (which
just subtracts the current value of the stock, from the last value of
the stock, and divides this by the last value, ie., it is the marginal
revenue per day-which is the exact opposite of the way tsinvestsim
constructed the times series,) and the output of the tsfraction
program piped to both tsavg, and tsrms, which computes the average,
and root mean square of the marginal returns.
[3] The tsrootmeanscale program is not sophisticated-it just catalogs
like sequential movements in a time series, (many use an FFT to do the
same thing; the program's only redeeming value is that it is
conceptually simple.) All it does is, when it gets two like movements,
it increments the value in the first element of an array-three, it
increments the second, and so on. It then divides the values in the
array by the total increments. If the time series under consideration
is a Brownian motion fractal, then the sequence in the array would be
0.5, 0.5, 0.5 ..., but if there is a persistence, of say a 55% chance
of what happened the previous day happening again today, then the
series would be 0.55, 0.55, 0.55 ...; and if there is short term
persistence, (i.e., Markovian,) then it would look something like
0.55, 0.53, 0.51, 0.50. (Again, 0.55 was chosen because it is
"typical" of stocks in the US exchanges.)
[4] The tsinvest program is just a monolithic program that compiles
the statistics as in [1], [2], [3], and [5], on a stock price time
series, attempts to time the market, (there are other options, but for
the sake here, only the -d5 option is used-which instructs the program
to attempt to time the market based on persistence measurements, using
the EMH paradigm.)
[5] The record used for tsinvestsim was "my_name, , p = 0.5088, f =
0.02, h = 0.55", which gives a 55% chance of what happened today
happening again tomorrow, the same volatility, and the same gain, as
in 1). The persistence is short term Markovian, which will be shown to
be inefficient, even though the measured characteristics, (other than
the persistence,) are are not statistically different than in 1),
above, (I chose them to be exact, but there are round off, and data
set size considerations-even at a million records, that get in the
way.)
[6] The program was only invested for about half the time, and the
coprocessor overflowed at record 812,983-it was invested in 507,488 of
those days. What I needed was a very large data set size, (see the
tsshannoneffective program as to how large these data sets have to
be,) and so what I did was pick a value-a million-records that I knew
would overflow the coprocessor late in the simulation, and then stop
the simulation the record before. The time series from the tsinvest
program has "holes" in it when it was not invested, so I removed those
records; otherwise, I would have about a third of the records with
zero gain, and zero risk, distorting the statistics. Removing the
records has about the same effect as giving the program many more
stocks to chose from, that are all identical.
jim blair writes:
>
> <conover@rahul.net> wrote in message news:so6fdp78o0b94@corp.supernews.com...
> > FWIW, efficiency in the sense of the EMH, (depending on who is telling
> > the story, of course,) gets larger as the system's entropy gets
> > larger, (the EMH assumes infinite entropy, i.e., 100% efficiency, as a
> > first order approximation; .....etc.
>
> I have thought over your clam of a relationship between "efficiency" and
> entropy (=disorder), but fail to see any connection. If anything, I would
> expect a reverse relationship (if any). Greater order connected to greater
> efficiency?
>
> At any rate, one problem with entropy is the need to define a "zero level".
> That is, only changes in entropy can be measured.
>
> In chemistry, the 3rd Law establishes (defines) the zero level of entropy.
> But what is it in economics/market/or whatever it is we are trying to relate
> to?
>
--
John Conover, john@email.johncon.com, http://www.johncon.com/