From: John Conover <john@email.johncon.com>

Subject: Re: Poll on market efficiency

Date: 5 Aug 2000 03:41:26 -0000

Hi Jim. Regarding my reply of yesterday regarding your question on how/why entropy and market efficiency are related, I got some free time today, and did a simulation that might lend some insight. The simulation and analytical tools are Open Source, (from http://www.johncon.com/ntropix/ and http://www.johncon.com/ndustrix/,) so you can compile them, and verify the results. For the sake of simplicity, only the EMH model was used in the simulation. Here is what I did: 1) Using the tsinvestsim[1] program, with no persistence, and a 51% chance of moving up on any given day, (and a 49% of moving down,) with a daily volatility of 2%, (I choose these values because they are "typical" of stocks on the US exchanges,) for a million days, yielded a stock price time series, that measured an average gain per day of 0.04%, a volatility of 2%, which translates to a daily gain of 0.02% per day. The stock has maximal gain, and lowering OR raising the risk, (risk = volatility,) results in less growth in price. This is a baseline stock for comparison[2]. There was no persistence, as measured by the tsrootmeanscale program[3]. 2) By using the tsinvest[4] program, it can be verified that it can do no better than the original time series in the growth of a stock's value over time. 3) By iterating the tsinvestsim program parameters, but with persistence[5], to find new values that give identical statistics to 1), (above, and verifying as such, as in [1], [2], and [3],) the values are verified as being the same, for all practical purposes, (and the persistence, as measured by the tsrootmeanscale program,) is, as specified. 4) But by using the tsinvest program, as before, it can be seen that it can do substantially better than the original time series in the growth of a stock's value over time. Using the same proceedure as in 1) and 2), the output time series from the tsinvest program has an average gain per day of 0.24%, and a volatility of 2%, which translates to a daily gain of 0.2% per day[6]. Additionally, there was NO persistence in the output time series as measured by the tsrootmeanscale program, (i.e., the entropy has been raised to infinity, meaning it is 100% efficient, at least in the sense of the EMH.) If we define efficiency as the ratio of annual gains, then, (with 253 trading days per year,) the inefficiency of the original time series would be 1.0002^253 / 1.002^253 = 1.052 / 1.66 = 63%. The 5% persistence, (e.g., lowering the time series entropy,) also lowered the stock price efficiency by about a third, (in this very simple case, assuming the paradigm of the EMH, which may or may not be representative of real market conditions.) John [1] The tsinvestsim program is part of the tsinvest program suite from http://www.johncon.com/ntropix/. It uses a random number generator, (the one supplied in the Linux 2.2.12 kernel, which is not the best, but adequate for the needs here.) All the program does is produce a time series of a stock's price by implementing the EMH, which, in a nutshell, is a compound interest algorithm-with variable interest rates, (which can be negative, or positive, depending on price movements,) but must be a random process that has a Gaussian/Normal distribution, (for consistency with the paradigm of the EMH.) The mode or mean of the distribution can be offset such that there is a long term exponential gain in a stock's price, which seems consistent with the empiricals of stocks on the US exchanges over the last century. Persistence can be included in the simulation by adding the last movement in the time series, multiplied by an appropriate value, such that when added to the current movement, there is a better than 50/50 chance that the resulting current movement will be the same as the last. Bottom line, the program is a stock price simulator, with EMH characteristics, and allows the variables, (volatility, exponential growth, persistence, etc.,) to be altered. [2] The record used for tsinvestsim was "my_name, p = 0.51", and the characteristics were measured by using the tsfraction program, (which just subtracts the current value of the stock, from the last value of the stock, and divides this by the last value, ie., it is the marginal revenue per day-which is the exact opposite of the way tsinvestsim constructed the times series,) and the output of the tsfraction program piped to both tsavg, and tsrms, which computes the average, and root mean square of the marginal returns. [3] The tsrootmeanscale program is not sophisticated-it just catalogs like sequential movements in a time series, (many use an FFT to do the same thing; the program's only redeeming value is that it is conceptually simple.) All it does is, when it gets two like movements, it increments the value in the first element of an array-three, it increments the second, and so on. It then divides the values in the array by the total increments. If the time series under consideration is a Brownian motion fractal, then the sequence in the array would be 0.5, 0.5, 0.5 ..., but if there is a persistence, of say a 55% chance of what happened the previous day happening again today, then the series would be 0.55, 0.55, 0.55 ...; and if there is short term persistence, (i.e., Markovian,) then it would look something like 0.55, 0.53, 0.51, 0.50. (Again, 0.55 was chosen because it is "typical" of stocks in the US exchanges.) [4] The tsinvest program is just a monolithic program that compiles the statistics as in [1], [2], [3], and [5], on a stock price time series, attempts to time the market, (there are other options, but for the sake here, only the -d5 option is used-which instructs the program to attempt to time the market based on persistence measurements, using the EMH paradigm.) [5] The record used for tsinvestsim was "my_name, , p = 0.5088, f = 0.02, h = 0.55", which gives a 55% chance of what happened today happening again tomorrow, the same volatility, and the same gain, as in 1). The persistence is short term Markovian, which will be shown to be inefficient, even though the measured characteristics, (other than the persistence,) are are not statistically different than in 1), above, (I chose them to be exact, but there are round off, and data set size considerations-even at a million records, that get in the way.) [6] The program was only invested for about half the time, and the coprocessor overflowed at record 812,983-it was invested in 507,488 of those days. What I needed was a very large data set size, (see the tsshannoneffective program as to how large these data sets have to be,) and so what I did was pick a value-a million-records that I knew would overflow the coprocessor late in the simulation, and then stop the simulation the record before. The time series from the tsinvest program has "holes" in it when it was not invested, so I removed those records; otherwise, I would have about a third of the records with zero gain, and zero risk, distorting the statistics. Removing the records has about the same effect as giving the program many more stocks to chose from, that are all identical. jim blair writes: > > <conover@rahul.net> wrote in message news:so6fdp78o0b94@corp.supernews.com... > > FWIW, efficiency in the sense of the EMH, (depending on who is telling > > the story, of course,) gets larger as the system's entropy gets > > larger, (the EMH assumes infinite entropy, i.e., 100% efficiency, as a > > first order approximation; .....etc. > > I have thought over your clam of a relationship between "efficiency" and > entropy (=disorder), but fail to see any connection. If anything, I would > expect a reverse relationship (if any). Greater order connected to greater > efficiency? > > At any rate, one problem with entropy is the need to define a "zero level". > That is, only changes in entropy can be measured. > > In chemistry, the 3rd Law establishes (defines) the zero level of entropy. > But what is it in economics/market/or whatever it is we are trying to relate > to? > -- John Conover, john@email.johncon.com, http://www.johncon.com/

Copyright © 2000 John Conover, john@email.johncon.com. All Rights Reserved. Last modified: Fri Aug 4 21:42:38 PDT 2000 $Id: 000804213922.4718.html,v 1.0 2001/11/17 23:05:50 conover Exp $