Background reading
A survey of behavioral finance (Barberis & Thaler, 2003)
1. Introduction
The document “A Survey of Behavioral Finance” by Nicholas Barberis and Richard Thaler
explores the role of behavioral finance as an alternative to the traditional finance paradigm,
particularly examining cases where not all agents in the financial markets are rational. This
approach to finance departs from traditional models that assume agents act
rationally—processing information correctly according to Bayes' law and making decisions
that maximize expected utility. Behavioral finance argues that some financial market
phenomena are better understood by relaxing these rationality assumptions, thus accounting
for common psychological biases and the structural limits of arbitrage. The survey is divided
into a discussion of two central pillars: limits to arbitrage and psychology. These pillars form
the foundation for behavioral finance and offer insight into various persistent market
anomalies, from stock prices and individual trading behaviors to corporate finance decisions.
2. Limits to Arbitrage
The first major section, Limits to Arbitrage, addresses why rational traders are often unable
to correct mispricings in the market, even when they recognize these discrepancies. In an
efficient market where agents are rational, security prices should reflect their "fundamental
value"—the discounted sum of future cash flows, priced with the correct risk-adjusted rate.
This concept underpins the Efficient Market Hypothesis (EMH), which suggests that prices
are “right” and that there is “no free lunch,” meaning no risk-free arbitrage opportunities exist
for investors. Behavioral finance challenges this, arguing that various forces can prevent
arbitrageurs from taking advantage of irrational behaviors and that, consequently, prices may
diverge from fundamental values for extended periods.
Behavioral finance counters the assumption that mispricings will always be corrected by
rational traders. It argues that even when prices diverge from fundamental values, arbitrage
strategies may not be practical or effective due to risks and costs. Key risks and costs
outlined include:
● Fundamental Risk: Arbitrageurs attempting to exploit mispricing must often hold
risky assets over time. For example, if a stock price drops below its estimated
fundamental value, buying it assumes that fundamental value will eventually drive the
price up. However, unforeseen negative news could cause further declines, leading
to losses. Arbitrageurs typically hedge by shorting substitute securities, but these
substitutes often have imperfect correlations, meaning risks cannot be fully
eliminated.
● Noise Trader Risk: A unique risk in behavioral finance, noise trader risk involves the
unpredictability of irrational traders' future actions. For instance, if noise traders drive
a stock price below its fundamental value, there is no guarantee that they will not
become even more pessimistic, pushing prices further away from the expected
equilibrium. This potential for increased mispricing means arbitrageurs risk deeper
short-term losses and might face forced liquidation, especially when managing
capital on behalf of others, due to their clients' loss aversion or loss of confidence.
● Implementation Costs: These include a variety of transactional and institutional
constraints that make arbitrage costly or impractical. Transaction costs like
commissions, bid-ask spreads, and market impact reduce potential profits, especially
in high-frequency strategies. Short-sale constraints further complicate the ability to
, exploit overpricing by limiting the availability of shares to borrow. Additionally, the
difficulty of identifying mispricings due to limited information, and high costs in
researching or structuring complex trades, add further barriers. Arbitrage positions
may also require expensive hedging instruments, further increasing implementation
costs.
The document presents evidence for these limits to arbitrage through real-world cases:
● Twin Shares: The case of Royal Dutch and Shell demonstrates mispricing when the
market value of Royal Dutch stock, which should be proportional to Shell’s value by a
fixed ratio, deviated by as much as 35%. Arbitrageurs attempting to profit by
exploiting this imbalance faced substantial noise trader risk, as pessimistic sentiment
could further lower Royal Dutch’s price relative to Shell’s, worsening short-term
losses. Despite low fundamental risk and few implementation costs, noise trader risk
made arbitrage difficult and prolonged the mispricing.
● Index Inclusions: The price of stocks often jumps permanently upon being added to
major indexes like the S&P 500, even though inclusion does not change a firm’s
fundamentals. This mispricing arises from demand by index funds. Here, arbitrage is
constrained by fundamental risk, as finding a perfect substitute security is
challenging, and by noise trader risk, where prices may continue rising due to
continued buying pressure. The case of Yahoo’s stock price doubling after S&P 500
inclusion illustrates these constraints.
● Internet Carve-Outs: The Palm-3Com carve-out case involved a situation where the
valuation of 3Com, which owned 95% of Palm, implied a negative value for 3Com’s
other assets. Arbitrageurs attempting to short Palm to capture the mispricing faced
high costs due to overwhelming demand for short positions, illustrating how
short-sale constraints and borrowing fees can impede arbitrage.
These cases underscore that, contrary to traditional finance theory, mispricings can persist
when rational traders face substantial limits to arbitrage.
3. Psychology
The second major component, Psychology, investigates the cognitive biases and deviations
from rationality that impact investor beliefs and preferences, ultimately influencing market
outcomes. Behavioral finance draws on cognitive psychology to better understand and
model these deviations. Key biases impacting belief formation and decision-making are:
● Overconfidence: Investors tend to be overconfident, often underestimating the
likelihood of negative outcomes and overestimating their predictive accuracy. This
leads them to take excessive risks or trade more frequently, despite poor calibration
in their forecasts. Overconfidence manifests as narrow confidence intervals for future
estimates and is particularly evident among professional investors, such as traders
and fund managers.
● Optimism and Wishful Thinking: Many individuals exhibit unrealistic optimism
regarding their abilities and future outcomes, leading them to take greater risks than
rational models would suggest. For example, investors may have overly positive
views on stock performance, especially during bullish markets, which can drive prices
above fundamental values.
● Representativeness Heuristic: Investors often rely on representativeness, or
judging probability based on similarity to a stereotype, leading to biases like base
, rate neglect and sample size neglect. Base rate neglect means that investors may
ignore the overall probability of an event, focusing instead on specific characteristics.
For example, they might attribute a strong quarter’s performance to skill rather than
luck, failing to account for the average performance rate.
● Anchoring and Availability: Investors may anchor their judgments to initial,
potentially arbitrary values, adjusting insufficiently when new information is
introduced. Availability bias also impacts judgments, as investors place undue weight
on recent or highly memorable information. A recent market crash, for instance,
might disproportionately influence future risk assessments.
In terms of preferences, the document examines the limitations of Expected Utility (EU)
theory in explaining investor choices under risk. Behavioral finance instead frequently
employs Prospect Theory, which accounts for two key observed behaviors: loss aversion
and probability distortion.
● Prospect Theory: In contrast to EU theory, which evaluates wealth as a whole,
Prospect Theory asserts that people evaluate gains and losses separately from a
reference point (often the current wealth level), and they are more sensitive to losses
than gains—a phenomenon known as loss aversion. Empirical studies show that
investors may reject risky gambles with equal gains and losses due to the pain of
loss being more intense than the pleasure of an equivalent gain.
● Probability Weighting: Prospect Theory also posits that people overvalue small
probabilities and undervalue large ones, explaining both a preference for lotteries
(high potential gain, low probability) and insurance (large potential loss, low
probability). This distortion helps explain why investors may engage in high-risk
investments despite low success probabilities and, conversely, avoid low-risk
investments with moderate returns.
● Ambiguity Aversion: Many individuals prefer known risks over ambiguous ones,
even when the expected outcomes are identical, a behavior that deviates from
subjective expected utility (SEU) models. This aversion becomes evident when
investors shy away from uncertain investment opportunities due to a lack of
confidence in the underlying probabilities, such as in new markets or unfamiliar
assets. In cases where investors feel competent—such as familiar sectors—they
might even prefer these "risky" bets, displaying what is known as a “preference for
the familiar.”
These psychological insights, including biases in belief formation and deviations in
preference, explain why investors might engage in seemingly irrational behavior, leading to
persistent market anomalies. By incorporating limits to arbitrage and these psychological
factors, behavioral finance provides a framework that better captures real-world financial
market behavior, offering explanations for phenomena that traditional models struggle to
account for.
4. Application: The Aggregate Stock Market
In Part 4, the document explores how behavioral finance principles can explain certain
puzzles observed in the aggregate stock market, specifically regarding the equity premium,
volatility, and predictability of returns. Each of these phenomena has historically been
challenging to explain within traditional finance frameworks, which typically rely on models
that assume rational investors.
, The Equity Premium Puzzle:
The equity premium puzzle refers to the observation that stocks have historically provided
a much higher average return compared to risk-free assets, such as government bonds.
Traditional models struggle to justify why investors demand such a large premium for holding
stocks, given the relatively low correlation between stock returns and consumption growth,
which would theoretically reduce perceived risk.
→ Behavioral finance approaches this puzzle by suggesting alternative preference models
that better align with observed behavior. One model uses prospect theory, where investors
are more sensitive to potential losses than equivalent gains. Benartzi and Thaler’s (1995)
concept of "myopic loss aversion" combines loss aversion with frequent portfolio
evaluation, suggesting that investors act as if they view stocks as riskier than they are in the
long term. This heightened sensitivity to short-term losses, combined with loss aversion,
could lead to a higher required premium for holding stocks.
→ Another approach involves ambiguity aversion, where investors prefer predictable,
familiar investments over uncertain ones. Stocks, being more ambiguous due to variable
returns, are perceived as riskier, pushing investors to demand higher returns.
The Volatility Puzzle:
Another key observation in aggregate stock markets is that stock prices exhibit high volatility,
with prices fluctuating more than can be explained by changes in dividends or earnings.
According to standard models, stock price movements should closely align with expected
dividend growth; however, in reality, the price-to-dividend ratios are highly variable.
→ Behavioral models attribute this excess volatility to investor sentiment and irrational
beliefs. Investors may react to short-term market trends, leading to excessive optimism
during market upswings and excessive pessimism during downturns, causing prices to
diverge from fundamental values. Moreover, psychological biases like overconfidence and
representativeness can amplify these swings, with investors placing too much emphasis on
recent trends rather than long-term fundamentals.
Predictability of Returns:
Traditionally, stock returns are expected to follow a random walk, meaning future returns
should be independent of past values. However, empirical evidence shows that certain
financial metrics, like the dividend-price ratio, can predict future returns. This "predictability
puzzle" challenges traditional models, as it implies that returns are not purely random but
rather somewhat forecastable.
→ Behavioral finance explains this predictability by suggesting that shifts in investor
sentiment create temporary mispricings that later revert to fundamental values. For example,
during periods of high optimism, stocks may become overvalued, only to correct in
subsequent periods as sentiment cools. Behavioral models highlight that cycles of
overvaluation and undervaluation, driven by psychological factors like overreaction and
underreaction to new information, lead to predictable patterns in stock returns.
In summary, behavioral finance provides alternative explanations for these stock market
puzzles by incorporating psychological factors into models of investor preferences and belief
formation. This approach helps to explain why the stock market exhibits such high equity
premiums, volatility, and return predictability—phenomena that have long puzzled traditional
financial theorists.
