Why do professional money managers so often fail to outperform a simple index fund? Why do stock prices seem to move unpredictably, defying even the most sophisticated forecasting models? And why, despite decades of research, does no one seem to have a foolproof system for consistently beating the market?
These questions lie at the heart of one of the most influential and controversial ideas in modern finance: the Efficient Market Hypothesis (EMH). Born from the observation that stock prices appear to follow a random walk, refined through decades of empirical testing, and challenged by the rise of behavioural finance and artificial intelligence, the EMH has shaped how academics, investors, and policymakers think about financial markets.
At its core, the hypothesis makes a bold claim: in an efficient market, prices at any moment “fully reflect” all available information. If that is true, then no amount of analysis, whether technical, fundamental, or even algorithmic, can consistently generate returns above the market average, after adjusting for risk.
What Did Economists Believe?
Before the efficient market hypothesis took hold, the prevailing view among investors and many economists was that stock prices could be predicted, at least to some extent, using past price patterns or fundamental analysis. Technical analysts believed that chart patterns, head and shoulders, moving averages, support and resistance levels, could forecast future price movements. Fundamental analysts pored over company financials, industry trends, and macroeconomic data to identify undervalued stocks.
But the evidence was mixed. Early studies, such as those by the French mathematician Louis Bachelier in 1900, had already noted that commodity prices seemed to fluctuate randomly. Bachelier’s doctoral thesis, Théorie de la Spéculation, proposed that the expected profit from speculation is zero, implying that prices follow a martingale: a process where future price changes are unpredictable given past information. His work, however, remained largely unnoticed for decades.
In the 1950s and 1960s, the advent of computers allowed researchers to analyse stock market data systematically. Maurice Kendall, a British statistician, examined weekly changes in British industrial share prices and found almost no serial correlation. He famously concluded that “the series looks like a wandering one, almost as if once a week the Demon of Chance drew a random number … to determine the next week’s price.” These findings puzzled economists: if prices were random, what explained them? And if they were random, why did so many people believe they could predict them?
The stage was set for a theory that could reconcile the apparent randomness of prices with the notion of rational, competitive markets.
The Man Behind the Idea
The architect of the efficient market hypothesis as we know it today is Eugene Fama. A young PhD student at the University of Chicago in the early 1960s, Fama was fascinated by the behaviour of stock prices. In his 1965 paper “The Behavior of Stock Market Prices,” he presented evidence that daily returns on the Dow Jones Industrial Average exhibited no significant autocorrelation: the statistical property that would allow past returns to predict future returns. He also noted that large price changes tended to be followed by large changes of unpredictable sign, consistent with the arrival of new information rather than systematic patterns.
But Fama’s most enduring contribution came in 1970 with his seminal review, “Efficient Capital Markets: A Review of Theory and Empirical Work.” In that paper, he defined an efficient market as one “in which prices always ‘fully reflect’ available information.” He then laid out a taxonomy that has become standard in finance:
- Weak form efficiency: Prices reflect all historical price and volume information. Technical analysis is useless.
- Semi‑strong form efficiency: Prices reflect all publicly available information (earnings announcements, stock splits, news, etc.). Fundamental analysis cannot generate abnormal returns.
- Strong form efficiency: Prices reflect all information, public and private. Even insiders cannot earn abnormal profits.
Fama’s 1970 paper also formalised the “fair game” model, which states that expected returns conditional on available information are zero after adjusting for risk. In other words, the deviation of a security’s return from its expected return is unpredictable. This property is the essence of market efficiency: no trading strategy based on the information set can generate excess profits.
Crucially, Fama emphasised that the EMH is not a “one‑size‑fits‑all” truth but a hypothesis that must be tested jointly with a model of equilibrium expected returns. This is the joint‑hypothesis problem: any test of efficiency is also a test of the asset‑pricing model used to define “normal” returns. If a test finds abnormal returns, it could mean the market is inefficient, or it could mean the model is wrong. This ambiguity would later become a central battleground in the debate over EMH.
The Core Idea Explained: Random Walks, Martingales, and Fair Games
To understand the EMH, we need to grasp the underlying mathematics. The simplest version is the random walk model, which assumes that successive price changes are independent and identically distributed. If prices follow a random walk, then the best forecast of tomorrow’s price is today’s price (plus a constant drift). In 1965, Paul Samuelson published “Proof That Properly Anticipated Prices Fluctuate Randomly,” showing that in a market where investors rationally anticipate future prices, the current price must equal the expected future price discounted for risk. Consequently, price changes are unpredictable: they are a “fair game.”
A fair game means that the expected excess return (the return minus the risk‑free rate) is zero when conditioned on past information. This is the martingale property, which is weaker than a random walk. A martingale requires only that expected price changes are zero; it allows for dependence in higher moments, such as volatility clustering. The random walk is a special case where the entire distribution of returns is independent over time.
Why would markets be efficient? The logic is simple and powerful: if prices did not fully reflect available information, then there would be unexploited profit opportunities. Investors would rush to trade on the information, driving prices to the correct level. In an open, competitive market with low transaction costs, this arbitrage process should happen quickly, leaving no room for persistent mispricing.
Consider a hypothetical scenario: Suppose a company’s earnings announcement will be released tomorrow, and a handful of investors know that earnings will be unexpectedly high. If the market were inefficient, these insiders could buy shares today at the current price and sell after the announcement, pocketing a profit. But as they buy, they bid up the price. In a competitive market, the price will rise until the expected profit from trading on the inside information is eliminated. By the time the announcement is made, the price already reflects the good news. Thus, the announcement itself causes no further movement.
This “no‑free‑lunch” intuition is the bedrock of the EMH. It implies that stock prices are the best available estimates of fundamental value, given all information.
Key Extensions: Anomalies and Multi‑Factor Models
The EMH did not remain unchallenged. As empirical research accumulated, a series of “anomalies” emerged: patterns in stock returns that seemed inconsistent with the simple market‑beta version of the Capital Asset Pricing Model (CAPM). (For a detailed explanation of CAPM, see our previous article: The Capital Asset Pricing Model: How Finance Learned to Price Risk.) The CAPM, developed by William Sharpe, John Lintner, and Jan Mossin in the 1960s, predicts that the only determinant of expected return is a stock’s beta (its sensitivity to the overall market). But researchers found that other factors appeared to explain returns as well.
- Size effect: In 1981, Rolf Banz showed that small‑capitalisation stocks had higher average returns than large‑cap stocks, even after controlling for beta.
- Value effect: In 1977, Sanjoy Basu found that stocks with low price‑to‑earnings ratios (value stocks) outperformed high‑P/E (growth) stocks.
- Momentum effect: In 1993, Narasimhan Jegadeesh and Sheridan Titman documented that stocks that performed well over the past 3–12 months tended to continue performing well.
- Post‑earnings announcement drift: Ray Ball and Philip Brown (1968) found that stocks with positive earnings surprises continue to drift upward for months after the announcement, a phenomenon that seems to contradict semi‑strong efficiency.
These anomalies posed a dilemma: were they evidence of market inefficiency, or did they simply indicate that the CAPM was an incomplete model of risk? Eugene Fama and Kenneth French took the latter view. In a series of influential papers (1992, 1993), they proposed a three‑factor model that added size and book‑to‑market (value) factors to the market factor. The Fama‑French three‑factor model did a much better job of explaining the cross‑section of average returns, absorbing many of the anomalies.
Yet the debate continued. Behavioural economists, led by Richard Thaler, Robert Shiller, and others, argued that the anomalies were genuine evidence of irrational investor behaviour. They pointed to psychological biases such as overconfidence, loss aversion, and herd mentality that could cause prices to deviate from fundamental values. Robert Shiller’s 1981 paper, “Do Stock Prices Move Too Much to Be Justified by Subsequent Changes in Dividends?” provided a powerful critique: stock prices were far more volatile than could be explained by changes in future dividends, suggesting that investor sentiment played a large role.
The Challenge / Critique
If the EMH is true, then prices should be unpredictable. But if markets are so efficient that no one can profit from information, why would anyone spend resources to gather information? This paradox was formalised by Sanford Grossman and Joseph Stiglitz in their 1980 paper, “On the Impossibility of Informationally Efficient Markets.” They showed that if markets were perfectly efficient, meaning prices fully reflect all information, then no one would have an incentive to acquire costly information. But if no one acquires information, prices cannot reflect it. Therefore, a perfectly efficient market cannot exist in equilibrium. There must be some degree of inefficiency to compensate information gatherers.
This insight led to the concept of “noisy rational expectations” equilibrium, where prices reveal information imperfectly, and informed traders earn a return that covers their information costs. The market is efficient in the sense that no one can earn abnormal profits without incurring costs, but it is not perfectly efficient in the sense of fully reflecting all information instantaneously.
The Grossman‑Stiglitz critique is now widely accepted. It implies that the EMH should be interpreted as a benchmark, not a literal description. In practice, markets are “efficient enough” that most investors cannot consistently beat them after accounting for transaction costs, but they are not so efficient that there is no role for active management or information gathering.
Another major challenge came from the behavioural finance movement. Daniel Kahneman and Amos Tversky’s prospect theory (1979) showed that individuals systematically deviate from rationality in predictable ways: they are loss averse, they overweigh small probabilities, and they use mental shortcuts (heuristics) that lead to biases. In financial markets, these biases can lead to phenomena like excessive trading, herding, and bubbles.
Robert Shiller’s work on “irrational exuberance” (2000) documented how speculative bubbles, such as the dot‑com bubble of the late 1990s, appeared to be driven by investor psychology rather than fundamentals. For behavioural economists, these episodes are clear evidence against the EMH. For proponents, they are rare exceptions that do not undermine the general usefulness of the hypothesis.
The empirical battleground has also shifted to the predictability of returns. In the 1980s and 1990s, researchers found that variables such as dividend yields, earnings‑price ratios, and interest rate spreads could forecast stock returns over multi‑year horizons. For example, a high dividend yield today tends to predict higher future returns, and a low dividend yield predicts lower returns. To an EMH advocate, this is not a violation of efficiency but rather a reflection of time‑varying expected returns: when expected returns are high, prices are low, and vice versa. To a critic, it is evidence that prices deviate from fundamental values and slowly mean‑revert.
Modern Relevance: AI, Algorithmic Trading, and the Future of Efficiency
In the twenty‑first century, the debate over market efficiency has taken new turns. The rise of algorithmic trading and artificial intelligence has transformed financial markets, raising questions about whether machines can achieve super‑human levels of efficiency, or whether they introduce new kinds of inefficiencies.
Algorithmic Trading and High‑Frequency Markets
Algorithmic trading now accounts for a majority of trading volume in many markets. High‑frequency trading (HFT) firms use ultra‑fast algorithms to execute trades in microseconds, exploiting tiny price discrepancies. Proponents argue that HFT makes markets more efficient by improving liquidity and reducing bid‑ask spreads. Critics worry about flash crashes, such as the one on May 6, 2010, when the Dow Jones Industrial Average plunged nearly 1,000 points in minutes before recovering, partly due to algorithmic interactions.
The presence of such automated trading does not necessarily contradict the EMH. If algorithms quickly eliminate mispricings, they make markets more efficient. But if they react in herding ways, they can create temporary dislocations. The key question is whether the returns to algorithmic trading represent compensation for speed and infrastructure (costs) or genuine excess profits.
Artificial Intelligence and Machine Learning
Artificial intelligence, particularly machine learning, has also been applied to stock prediction. Studies such as Abunasser et al. (2023) have shown that sophisticated algorithms like Gaussian process regressors can achieve near‑perfect in‑sample predictions. But out‑of‑sample, the performance often degrades: a reminder that the EMH’s core insight about unpredictability remains relevant.
Machine learning models can identify complex patterns that human analysts might miss, but they can also overfit to historical noise. The real test is whether such models can deliver consistent risk‑adjusted returns after transaction costs. So far, evidence suggests that while some quantitative funds succeed, their success is not easily replicated and may stem from unique data or execution advantages rather than a fundamental violation of efficiency.
Cryptocurrencies: A New Frontier
What about cryptocurrencies? The wild volatility of Bitcoin and other digital assets has prompted many to question whether these markets are efficient. Some studies find evidence of weak‑form efficiency in crypto markets, while others document significant anomalies. The debate is ongoing, and it is not yet clear whether crypto markets will converge to the same kind of efficiency seen in mature equity markets. One factor is the immaturity of these markets: low liquidity, regulatory uncertainty, and the dominance of retail investors may create temporary inefficiencies that could be exploited.
The Future of Market Efficiency
One of the most profound modern extensions of the EMH comes from the intersection of information economics and machine learning. In an era of big data and AI, the cost of processing information has fallen dramatically. If information becomes cheap, does that mean markets become more efficient? Or does it mean that the only sustainable source of profit is the ability to process data faster than others, leading to a technological arms race? These questions are at the forefront of contemporary finance research.
The Grossman‑Stiglitz insight still applies: some level of inefficiency must persist to reward information acquisition. As technology evolves, the nature of that inefficiency changes, but it does not disappear. Markets may become more efficient in incorporating widely available data, but new forms of private information (or new ways of processing public information) may create temporary advantages for the fastest, smartest participants.
Why Does EMH Still Matter?
After more than half a century, the efficient market hypothesis remains the central organising principle of modern finance. It has profoundly shaped investment practice: the rise of index funds and passive investing is a direct outgrowth of the belief that active managers cannot consistently beat the market after costs. Today, trillions of dollars are managed in passive strategies, and the EMH has become part of the conventional wisdom taught in business schools around the world.
Yet the hypothesis is not without its critics. The anomalies literature, behavioural finance, and the Grossman‑Stiglitz impossibility theorem have all shown that markets are not perfectly efficient. Prices can deviate from fundamental values for extended periods, and some investors, such as corporate insiders and, occasionally, skilled quantitative funds, do seem to earn abnormal returns.
Perhaps the most balanced view is that markets are generally efficient, but not perfectly efficient. The EMH is best thought of as a useful benchmark, a null hypothesis that forces researchers to carefully identify genuine inefficiencies rather than simply assuming them. As the legendary economist John Maynard Keynes once said, “The market can remain irrational longer than you can remain solvent.” In other words, even if you think you have found a mispricing, it may persist long enough to bankrupt you before it corrects.
For the ordinary investor, the practical lesson of the EMH is simple: trying to beat the market is a fool’s errand for most people. The costs of research, trading, and taxes typically outweigh any potential gains. The smarter approach is to diversify, keep costs low, and stay invested for the long term. In this sense, the EMH has changed the world not by proving that markets are always right, but by demonstrating that the pursuit of market‑beating returns is largely a game of chance: a game where the house (the market) almost always wins.
So, can you beat the stock market? Perhaps a tiny handful of exceptionally skilled or lucky individuals can, at least for a while. But for the vast majority, the evidence suggests that the best strategy is to accept that the market knows what it knows, and to invest accordingly.
Did you find this article helpful? Share it with someone who loves economics. And remember, at MASEconomics, we make complex ideas simple.
