Prediction: The future of the USA stock market

Based on a theory of cooperative herding and imitation working both in bullish as well as in bearish regimes, we have detected the existence of a clear signature of herding in the decay of the US S&P500 index since August 2000 with high statistical significance, in the form of strong log-periodic components.

Please refer to the following paper for a detailed description: D. Sornette and W.-X. Zhou, The US 2000-2002 Market Descent: How Much Longer and Deeper? Quantitative Finance 2 (6), 468-481 (2002) (e-print at http://arXiv.org/abs/cond-mat/0209065).

For a general presentation of the underlying concepts, theory, empirical tests and concrete applications, with a discussion of previous predictions, see Why Stock Market Crash?.

NEW: (Evidence of Fueling of the 2000 New Economy Bubble by Foreign Capital Inflow: Implications for the Future of the US Economy and its Stock Market ), this new paper attempts to construct a coherent analysis of the US stock market linking technical analysis of the type presented below to macroeconomic thinking. We combine a macroeconomic analysis of feedback processes occurring between the economy and the stock market with a technical analysis of more than two hundred years of the DJIA to investigate possible scenarios for the future, three years after the end of the bubble and deep into a bearish regime. We also detect a log-periodic power law (LPPL) accelerating bubble on the EURO against the US dollar and the Japanese Yen. In sum, our analyses is in line with our previous work on the LPPL "anti-bubble" representing the bearish market that started in 2000.

This figure shows 8 years of the evolution of the Japanese Nikkei index and 7 years of the USA S&P500 index, compared to each other after a translation of 11 years has been performed. The years are written on the horizontal axis (and marked by a tick on the axis) where January 1 of that year occurs. This figure illustrates an analogy noted by several observers that our work has made quantitative. The oscillations with decreasing frequency which decorate an overall decrease of the stock markets are observed only in very special stock markets regimes, that we have terms log-periodic "anti-bubbles". By analyzing the mathematical structure of these oscillations, we quantify them into one (or several) mathematical formula(s) that can then be extrapolated to provide the prediction shown in the two following figures. Note that extrapolating is often a risky endeavor and needs to be justified. In our case, the extrapolations, which give the forecasts, are based on the belief that these equations offered below embody the major forces in the market at the macroscopic scale. This leads to the possibility of describing several probable scenarios. We do not believe in the existence of deterministic trajectories but we aim at targetting the most probable future paths.

Fig. 1 shows the predictions of the future of the US S&P 500 index performed on Aug.24,2002. The continuous line is the fit and its extrapolation, using our theory capturing investor herding and crowd behavior. The theory takes into account the competition between positive feedback (self-fulfilling sentiment), negative feedbacks (contrariant behavior and fundamental/value analysis) and inertia (everything takes time to adjust). Technically, we use what we call a "super-exponential power-law log-periodic function" derived from a first order Landau expansion of the logarithm of the price. The dashed line is the fit and its extrapolation by including in the function a second log-periodic harmonic. The two fits are performed using the index data from Aug.9,2000 to Aug.24 2002 that are marked as black dots. The blue dots show the daily price evolution from Aug.25,2002 to July 17,2003. The large (respectively small) ticks in the abscissa correspond to January 1st (respectively to the first day of each quarter of each year.

Fig. 2 shows the new predictions of the future of the US S&P 500 index using all the data from Aug.9,2000 to Jul.17,2003, illustrated by (continuous and dashed) black lines. Again, the continuous line is the fit and its extrapolation using the super-exponential power-law log-periodic function derived from the first order Landau expansion of the logarithm of the price, while the dashed line is the fit and its extrapolation by including in the function a second log-periodic harmonic. We also present the two previous fits (red lines) performed on Aug.24,2002 (shown in Fig. 1) for comparison, so as to provide an estimation of the sensitivity of the prediction and of its robustness as the price evolves. The blue dots show the daily price evolution from Aug.9,2000 to Jul.17,2003. The large (respectively small) ticks in the abscissa correspond to January 1st (respectively to the first day of each quarter) of each year.

Fig. 3 shows the predictions of the future of the US S&P 500 index applying the so-called 'zero-phase' Weierstrass-type function, which is another child of our general theory of imitation and herding between investors. As for the previous figures, our theory takes into account the competition between positive feedback (self-fulfilling sentiment), negative feedbacks (contrariant behavior and fundamental/value analysis) and inertia (everything takes time to adjust). This 'zero-phase' Weierstrass-type function adds one additional ingredient: it attempts to capture the existence of 'critical' points within the anti-bubble, corresponding to accelerating waves of imitation within the large scale unraveling of the herding anti-bubble. The continuous black line is the forward prediction using all the data from Aug.9,2000 to July, 17,2003, while the dashed black line is the retroactive prediction using the data from Aug.9,2000 to Aug.24,2002. Both lines are reconstructed and extrapolated from the fits to a six-term zero-phase Weierstrass-type function. We also present the two previous fits (red lines) performed on Aug.24,2002 (shown in Fig. 1) for comparison. The blue dots show the daily price evolution from Aug.9,2000 to July, 17,2003. The large (respectively small) ticks in the abscissa correspond to January 1st (respectively to the first day of each quarter) of each year.

The striking development observed in the last update on June, 19, is confirmed. The 'zero-phase' Weierstrass-type function, which up to May, 18, 2003 included selected a series of downward critical crashes, is now selecting as the dominant critical points the bullish accelerations. The formula is thus deciphering the coexistence of two sets of critical points: (i) the crashes previously recognized which have punctuated the descent in the last three years and (ii) the bursts of upward accelerating rallies. This formula is however not rich enough in its present version to capture these two sets simultaneously and has to choose between the two, as a result of their relative strengths. This new twist does not change fundamentally our prediction of a drastic turn in the very near future towards a systematic downward trajectory till the summer of 2004. The question posed by the insight provided by this figure 3 is whether this will turn out as a result of a crash following a strong rally in the next two months or so. This crash will then be followed by a longer and continuous price depreciation.

Fig. 3bis is a modification of the 'zero-phase' Weierstrass-type function, which contains only odd-terms in the expansion (this will be elaborated upon in a future technical communication). By this trick, the odd-zero-phase Weierstrass-type function is able to describe simultaneously the two sets of critical points. The continuous black line is the forward prediction using all the data from Aug.9,2000 to July, 17,2003, while the dashed black line is the retroactive prediction using the data from Aug.9,2000 to Aug.24,2002. Both lines are reconstructed and extrapolated from the fits to a six-term odd-zero-phase Weierstrass-type function. We also present the two previous fits (red lines) performed on Aug.24,2002 (shown in Fig. 1) for comparison. The blue dots show the daily price evolution from Aug.9,2000 to July 17,2003. The large (respectively small) ticks in the abscissa correspond to January 1st (respectively to the first day of each quarter) of each year.

In conclusion, the coexistence of the strong downward crashes and upward rallies in the overall anti-bubble regime suggests to us that the market is completely dominated by sentiment, confidence and lack thereof and byherding. These mechanisms are amplifying any news, perturbation or rumor spreading in the network of investors.

Fig. 4 extends figures 1 and 2 by performing a sensitivity analysis on the simple log-periodic formula (continuous lines in figures 1 and 2), in order to assess the reliability and range of uncertainty of the prediction. Using the fit shown in black solid lines in figure 2, we have generated 10 realizations of an artificial S&P500 by adding GARCH noise to the black solid line. GARCH means "generalized auto-regressive conditional heteroskedasticity". It is a process often taken as a benchmark in the financial industry and describes the fact that volatility is persistent. The innovations of the used GARCH noise have been drawn from a Student distribution with 3 degrees of freedom with a variance equal to that of the residuals of the fit of the real data by the black continuous curve, to ensure the agreement between these synthetic time series and the known properties of the empirical distribution of returns. Using the GARCH noise improves on our previous synthetic tests of last month by using a more realistic correlated noise process. The fits are shown as the bundle of 10 curves in magenta. This bundle of predictions is coherent and suggests a good robustness of the prediction. The typical width of the blue dots give a sense of the variability that can be expected around this most probable scenario. The real S&P500 price trajectory is shown as the red wiggly line.

Fig. 5 extends figures 1 and 2 by performing a sensitivity analysis on the log-periodic formula with a second log-periodic harmonic (dashed lines in figures 1 and 2), in order to assess the reliability and range of uncertainty of the prediction. Using the fit shown in dashed solid lines in figure 2, we have generated 10 realizations of an artificial S&P500 by adding the GARCH noise (described in the previous caption of Fig. 5) to the dashed solid line. We have then fitted each of these 10 synthetic noisy clones of the S&P500 by our log-periodic formula. This yields the 10 curves shown here in magenta. This test shows that the log-periodic formula with a second log-periodic harmonic (dashed lines in figures 1 and 2) is also providing stable scenarios: the precise timing of the highs and lows remain robust with respect to the realization of the noise. The real S&P500 price trajectory is shown as the red wiggly line.

Fig. 6 extends figure 3 by performing a sensitivity analysis on the 'zero-phase' Weierstrass-type function, in order to assess the reliability and range of uncertainty of the prediction. Using the fit shown in black solid lines in figure 3, we have generated 10 realizations of an artificial S&P500 by adding A GARCH noise to the black solid line. GARCH means "generalized auto-regressive conditional heteroskedasticity". It is a process often taken as a benchmark in the financial industry and describes the fact that volatility is persistent. The innovations of the used GARCH noise have been drawn from a Student distribution with 3 degrees of freedom with a variance equal to that of the residuals of the fit of the real data by the black continuous curve, to ensure the agreement between these synthetic time series and the known properties of the empirical distribution of returns. Using the GARCH noise improves on our previous synthetic tests of last month by using a more realistic correlated noise process. We have then fitted each of these 10 synthetic noisy clones of the S&P500 (shown as the blue dots) by our 'zero-phase' Weierstrass-type function. This yields the narrow bundle of 10 curves shown here in magenta. This bundle of predictions is very coherent and suggests a good robustness of the prediction. The typical width of the blue dots give a sense of the variability that can be expected around this most probable scenario. The real S&P500 price trajectory is shown as the red wiggly line.

Fig. 6bis is the same as Fig. 6 but for the odd-zero-phase Weierstrass function shown in Fig. 3bis.

Fig. 7 analyses the VIX index by fitting it with our simple log-periodic formula. The VIX index is one of the world's most popular measures of investors' expectations about future stock market volatility (that is, risk). See http://www.cboe.com/micro/vixvxn/introduction.asp. For historical data, see http://www.cboe.com/micro/vixvxn/specifications.asp. The VIX time series is shown as the red wiggly curve. We have followed the same procedure as for figures 4-6: (i) we fit the real VIX data with our simple log-periodic formula; (ii) we then generate 10 synthetic time series by adding GARCH noise to the fit; (iii) we redo a fit of each of the 10 synthetic time series by the simple log-periodic formula and thus obtain the bundle of 10 predictions shown as the magenta lines. Strikingly, we first observe that our log-periodic formula is able to account quite well for the behavior of the VIX index, strengthening the evidence that the market is presently in a strong herding (anti-bubble) phase. Note also the rather good stability of the predictions, suggesting a reasonable reliability.

If the good professors' predictive methods are valid, they should be able to withstand testing in which all market data after some arbitrary date in the past is ignored, and "predictions" made from that point. Those predictions would then be compared with the actual data.

For example, if Jan. 1, 2001 were taken as a starting point, would these methods have been to predict the behavior of the market between then and today?

What about 9-11 you say? Proponents of these rigorous technical analysis methods would say that if 9-11 had not happened, "something else" would have. Or, nothing, for that matter. To them, the market operates as it pleases, while the fundamentalists fit real world events to the market after the fact. The tail wags the dog, in other words.

I also notice the fractal image on the book's cover. It's not clear what they mean to imply by the choice of this pretty picture, but it should be noted that "fractal behavior" (i.e. nested waves and cycles) can disappear at the most inconvenient times as the system shifts into a chaotic mode, or into a totally different wave/cycle pattern.

Actually, you can check something like you are asking for, whether his methods, given earlier data, can predict more recent results.

Go to the original for this article, Prediction: The future of the USA stock market , and you will see a pull down list "Other Predictions". It has copies of his predictions, dating back as early as October 31. 2002. They predict that the market will rise from then, to a peak in the first half of 2003, before falling all through later 2003, and early 2004.

The portion of his prediction that has already transpired was pretty good, and his predictions of what is still the future have not changed substantially.

I was about to post almost the exact same thing. All the TA stuff I've seen, is guys drawing lines on charts for support, etc.

It's like making a general statement "I get 19 miles per gallon in my car, it's got 26,000 on it, so I think it will go until it has 160,000 before I need to trade it in"

Way too general and ignoring fundamentals, like the air filter, transmission, etc. Even if all the conditions are excellen for making a product, there still needs to be buyers. The fundamentals of the market now are quite bad. New technology acquisitions are being deferred because theres still alot of the tech from the bubble that can be had for pennies on the dollar. If lending capital dries up, it's not good. And lending capital is based in large part on the overall health of the economy. The snake is eating its tail.

My, my, something touched a nerve for you.

The stock market is not just a simple reflection of the strength of the economy or business fundamentals. There are also some group behaviours going on, some herding, which is what the cycles are somtimes used to show.

That there is such group behaviour going on seems self evident to me. If you find that possibility threatening, then I'll wager a pint that this reflects more on you than it does the stock market.

However, since I am not paid nor qualified to be your analyst, either financial nor psychological, I will not be able to assist you in resolving that conundrum.

Using a validity baseline from Oct 02 thru mid-July 03 is ridiculous.

If the good professors were even interested in demonstrating the validity of their method, they would (very easily, btw) have run annual backtests of prediction vs. result for the last 40 or 50 years, or at least back to the start of the SP500 futures (1982).

Without such backtesting, no matter how spiffy the analysis, any conclusion drawn about the validity of the predictive method is at best shaky and more likely simply worthless.

Backtesting before the year 2000 is meaningless on this model. It is not a model for all seasons. It is a model for post-bubble collapse. He makes references to testing this model on the aftermath of other bubbles, though I haven't checked these references, so can't comment on their persuasiveness.

If you haven't realized by now that it's a model for a particular phase, or season, of the market, then I suspect you aren't reading closely enough. Perhaps you just skimmed it, and brought out the usual attacks on anyone who fits some curve to stock prices and then projects that curve.

Of course, so far as I see here, his model also doesn't predict when we are no longer in the post-bubble collapse phase, at which point his model will die a horrible death. That's part of the challenge imposed by the fractal nature of markets - just when you've got the pattern figured out, it up and changes on you.

And this of course limits ones ability to get filthy rich off this model, even if it were everything it was cracked up to be. If you bet the farm on it, leveraged to the hilt, then when the model breaks down, you're one of the ones jumping out of a 40-th story penthouse window.

When someone like myself reads his model as saying "It's going to be a sad sad Christmas", we're saying more than is in his model. We are also saying that it's our opinion that his model will still be worth the pixels its painted on, in another five months. We're looking at a variety of other fundamentals, tea leaves or whatever we use, and reaching the conclusion that we are still in this phase. Reasonable people can and do disagree on that matter; the ones that are right will be rewarded, with money from the losers pockets.

His model doesn't say that this is what will happen over the next year; it says that this would be what happened, if the current phase continues. Even if the authors claim that their model is stating what must be over the next year, rather than what would be if the model happened to still hold, one still can't find fault with the model on that account, just with the authors hubris.

Like any model, to extend it from "what would happen if the world would kindly consent to continue following the pattern I've noticed" to "this is it, this is your destiny; do not attempt to adjust the controls, we are in charge", would be an abuse of the model. And to claim that anyone finding this model, or any other model, interesting, is so abusing models is to create a paper tiger.

Find something more challenging to put down next time.

How curious, then, that no analysis of the other recent great post-bubble period, specifically Mar 73 through 1975 was included. Sure, no futures available, but so what -- one can easily generate an implied futures price by any of several methods.

The article, in any practical sense, is worthless: the data set is far too small, ridiculously so, nor is there any indication of what one's strategic approach to the mkt should be, in order to achieve some sort of profit. Models (sic) with exactly one data point or data period are instances of either the dishonest or incompetent use of statistical analysis, are purest BS. Sorry if you disagree (and, please note, I made no ad hominem comments about you. Do try to return the favour).

Aha - after a nights sleep, it seems obvious what this pattern is showing us: deepening involvement.

Have you ever followed behind a big truck and noticed that you could tell how heavily loaded it was, by the frequency of its vibrations when it went over bumps. A backend that wiggles quickly is a sign of a lightly loaded truck; a backend that moves ponderously is a sign of a heavily loaded truck.

What we're seeing here is a market moving with increasing ponderousness. More and more of the economic and financial markets are getting caught up in the pattern, causing the waves to run deeper, run slower, over time.

If the pattern continued unabated, then it would lead to the premature implosion of the Universe. I suspect that we have unanimous consent here that this pattern won't continue that far. So now we're just disagreeing on when the pattern will break down, cease applying to reality. Last month, next year, next decade, it never applied in the first place; pick a number, place your bets.

Some of my colleagues (I'm in Silicon Valley high tech) got caught up in the ".com" boom of the late '90s. They have already washed ashore, in the waves of the last three years. I'm in the tech boom of the last couple decades; my boat is taking on water, but still afloat, though we have had to throw alot of stuff (and jobs) overboard, to keep it from sinking already.

Many folks on FR have not yet gotten any closer to this storm than musing over the evening weather report showing rain and wind in far off lands.

But, back to this model, what it is showing that in a post bubble market, the collapse cuts increasingly deeply into various associated markets, moving bigger and slower as it gains mass.

Until one day, some other pattern begins to take hold. But this model doesn't help us identify that day, so far as I can tell.

How curious, then, that no analysis of the other recent great post-bubble period, specifically Mar 73 through 1975 was included

The authors do study other post-bubble collapses. What I posted doesn't analyze it, but the book Why Stock Markets Crash? referred to in this posting claims to:

Sornette probes major historical precedents, from the decades-long "tulip mania" in the Netherlands that wilted suddenly in 1637 to the South Sea Bubble that ended with the first huge market crash in England, in 1720, to the Great Crash of October 1929 and Black Monday in 1987, to cite just a few. He concludes that most explanations other than cooperative self-organization fail to account for the subtle bubbles by which the markets lay the groundwork for catastrophe.

I take it that you are saying that the only practical sense in which this article would be useful would be if it helped one make money in the market. If it can't do that, it is worthless BS.

I also find value in models (and essays, pictures, cartoons, FR postings, ...) that seem to capture some element of where we are now, or what has happened so far, even if I can't just plug it into my Excel spreadsheet and retire to a South Pacific island next year.

Not every expression must guarantee profit to have worth.

Sure, mathematical models and charts can be mechanically extended to appear to predict future events. But in doing so, one is just showing where the model goes. The model itself does not predict when the model will break down, cease applying to reality. Sooner or later, they all break down.

Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.