Financial markets are more efficient than some speculators may want to believe. When it comes to predicting the performance of markets, everyone wants an edge—an advantage that sets them apart from the competition. Getting such an edge is achievable, but it’s never going to be easy and it will be impossible to maintain over time, according to research from the University of California San Diego’s Rady School of Management.

“Essentially, what this work shows is that our financial markets are pretty efficient, thus it’s hard to make outsized and lasting profits from trading,” said Allan Timmermann, a distinguished professor of finance at UC San Diego’s Rady School of Management and the Dr. Harry M. Markowitz Endowed Chair in Finance and Investing. “It’s not impossible, in the sense that market participants—investors, daytraders, hedge funds—are constantly on the lookout for predictable patterns that they can exploit through their trading activities. There are some corners of the market where there is some predictability that can be exploited, but since everybody’s looking for it, the advantages of finding it are short-lived.”

The study will appear in a forthcoming issue in The Journal of Finance. It reveals that in the United States stock market, “pockets,” or short-term periods of significant predictability, tend to be interspersed with longer periods of little to no evident predictability. Predictability, when it appears, tends to be self-extinguishing because whatever edge an investor may have will be emulated by others.

The paper documents this empirically using a flexible time-varying parameter model designed to track how pockets of return predictability come and go.

“Everyone’s looking for valuable information sources and competition is fierce,” Timmerman said. “If you expect predictable results and stable performance, you’re likely to be disappointed sooner or later. Professional investors like hedge fund managers can sometimes outperform the market, but typically not for long. There’s a reason funds state that past performance is no guarantee for future performance.”

Strategies used by investment managers include using satellite pictures of Home Depot parking lots to predict the company’s quarterly earnings. Other investment companies have used helicopters to survey the level of oil in strategic reserves in hopes of estimating how much oil is going to be released and its effect on oil prices. These strategies have helped companies gain edges for short amounts of time until they inspire imitations, attracting new investment flows, which in turn, causes “the edge” to self-destruct.

“It’s like playing peek-a-boo with the markets,” said Timmermann. “Now you see it, now you don’t. All the funds, all the investors and their owners are looking for the same weak signals that will help them predict future prices. If the signal was too strong, everyone would exploit it and everyone would become wealthy very quickly. In reality, having an edge in financial markets is very hard to keep up because of the free-flowing capital and the strong competition.”

Financial markets may be highly efficient, but they are by no means infallible. Even professional investors have to address the same extremely difficult questions such as how will the economy emerge from the COVID-19 pandemic and how long will the war in Ukraine last and what will be its impact on commodity prices, inflation and economic growth.

“No one has good answers to these questions and so financial markets have to process new information just like all of us,” Timmerman said. “This learning process can give rise to predictable patterns, exactly because each episode has some uniqueness and is never a replica of the past.”

Timmermann co-authored the study, Pockets of Predictability, with Leland E. Farmer of the University of Virginia and Lawrence Schmidt from the Massachusetts Institute of Technology.