Forecast: The Markets Are Still Predictable

For ages people have been trying to forecast the stock market. What started a hundred years ago with paper charts and pencil lines has developed into ever more sophisticated computerized market forecasting tools. Many indicators are available today. The thing is, the more popular they are, the less useful they become. Thus the ever bigger and bigger guns are needed to exploit the market inefficiencies.

As the markets become more computerized, with the response time now measured in milliseconds, the question arises, is it still possible to forecast the stock market for days or weeks in advance, or have they become totally efficient and thus impossible to forecast?

Markets' Predictability

Over the past year we have recorded the predictability of over a hundred equities by the machine learning system, which forecasts the future movement curve of the market based on past history. Its algorithms constantly look for patterns in the markets, make and test conjectures, and provide an objective measure of the strength of such conjectures and create a daily stock forecast for six different time horizons. This bootstrapping, self-learning system is constantly evolving, as new data is added daily and a better machine-derived model is found.

Our quality control indicator, the measure of predictability (P) accompanies each market forecast. P, a correlation coefficient between the predicted move and the actual one, ranges between minus 1 (the actual move was opposite to the forecast) through zero (no relation between the forecast and the actual), to plus 1 (the actual move was exactly as forecast).

Results

Then how predictable are the markets? Here are some observations from the recent records:

Over the last year the average predictability of the top 100 most predictable markets in our system (note: markets = stocks, indexes, commodities and currencies) was 0.53.

Some markets were on average more predictable than the others. Fig. 1 shows daily predictability of Disney (DIS) stock forecast and Fig. 2 of DAX index (^GDAXI). One can see that DAX index was more predictable than Disney stock.

Each market had its own predictability curve, which was not necessarily synchronized with other markets.

Over the last year there were long periods of predictability interspersed with a few short unpredictability spikes.

Among other observations: The long term market forecast was better, i.e., more reliable than the shorter one.

Just as there were waves in the market prices, there were also waves in predictability.

Click to enlarge.

Discussion and Conclusions

Some markets were on the average more predictable than the others, meaning that the latter markets were more "efficient." (Some stocks are practically unpredictable for other reasons. They were screened out of the system at an earlier stage).

Sometimes a shocking news can drastically affect all markets, regardless of the news relevance to the specific market, as what for instance happened between last May 11 and May 24 when the news were dominated by European sovereign debt fears and disappointing U.S. data. The spike of unpredictability understandably affects the German DAX index and even the seemingly unrelated stock as Disney, see Figs. 1 and 2.

The longer term forecast was more predictable than the shorter one. The reason is, the short term forecast is more affected by the market noise created by the daily news stream. The longer term forecast reflects the deeper fundamental trends.

We conclude that in spite of the proliferation of computerization and the algo-trading, the markets still exhibit classic chaotic behavior, and are largely predictable.

The reasons are, in our opinion:

• Differences in valuating the equity between different models and different market players, be it humans or the machines. What seems undervalued according to one model can appear overvalued to another.
• Time horizon factor: What seems overvalued in the short time horizon can appear undervalued in the longer view.
• The ever-present element of uncertainty in every news event, and the difficulty to quantify its effect on the market forecast.
• Human factor: Even when given equal access to the news, different people react differently. Human mind apparently can't process objectively the constant information stream, and tends to react to just the latest headline in the news, until the next item catches the attention. The psychological dynamics of fear and greed can lead to irrational decisions.

All these factors result in waves in prices, which can be detected and exploited by the more objective computer models. By monitoring predictability one can get advance warning that the market paradigm change is in progress.

Disclosure: I have no positions in any stocks mentioned, and no plans to initiate any positions within the next 72 hours.