Survivorship Bias: Testing Only on the Winners

How the bias works

Suppose you backtest a strategy on the current members of a major index over the past twenty years. Every company in that list is there because it succeeded enough to remain. The companies that failed, were acquired at a discount, or declined out of the index are absent, and so are their returns.

True universe at the start:  500 companies

Over the following 10 years:
   340 remained in the index
    70 were acquired (mixed outcomes)
    55 declined and were removed
    35 went bankrupt or were delisted worthless

Backtesting on the 340 survivors:
   You have removed 90 companies whose outcomes were
   poor or catastrophic, and kept every one that thrived.

Measured effect in published studies:
   Typically 1 to 4 percentage points of annual return
   added to equity strategies, and considerably more
   for strategies that buy distressed or cheap stocks.

Where the bias hides

The data cleaning case deserves attention because it happens innocently. Removing instruments with missing data seems like good hygiene, but instruments have missing data precisely because they stopped trading, which means the cleaning step is performing the selection.

Related selection biases

Survivorship bias

Excluding instruments that ceased to exist. The classic form described above.

Backfill bias

When a database adds a fund or instrument, it sometimes includes its prior history, which is only offered when that history is good.

Selection bias in reporting

Funds and managers report voluntarily, and those performing poorly stop reporting.

Look-ahead in universe construction

Selecting instruments by criteria that could only be evaluated later, such as current market capitalisation.

Delisting return omission

Even databases with delisted securities sometimes omit the final return, which is often a large loss.

Index reconstitution timing

Using the announcement date versus the effective date changes which instruments are in the universe on a given day.

Obtaining unbiased data

1. 1

   Use a database that retains delisted instruments

   Academic and professional datasets preserve securities after they stop trading, including the delisting return. Most free sources do not.

2. 2

   Obtain point-in-time index membership

   A historical record of which instruments were in the index on each date. Without this, any index-based universe is biased by construction.

3. 3

   Reconstruct the universe at each rebalance date

   Apply your liquidity and price filters using data available on that date, not current data.

4. 4

   Handle acquisitions explicitly

   An acquired company’s position should be closed at the acquisition price, which may be above or below the market price before the announcement.

5. 5

   Check the delisting return treatment

   Confirm whether your data includes the final value. If not, assume a substantial loss for delistings following a decline.

6. 6

   Estimate the bias if you cannot eliminate it

   Where unbiased data is unavailable, at minimum note that results are inflated and discount them accordingly rather than reporting them as if unbiased.

The crypto case

Survivorship bias is more severe in crypto than in equities because the failure rate is far higher and the history far shorter. A large majority of tokens launched in any given year are effectively worthless or untradeable within a few years.

• Backtesting a rotation strategy on today’s top 50 tokens over five years excludes everything that collapsed, which is most of what existed.
• Exchange data disappears when tokens are delisted, so historical price series for failed assets are frequently unavailable at any price.
• The surviving assets are, by construction, those that attracted sustained capital, which is exactly what any momentum or quality strategy would have selected.
• The practical response is to restrict tests to the largest and longest-lived assets, acknowledge the remaining bias, and heavily discount results from broad token universes.
• Any strategy claiming strong historical performance across a wide crypto universe should be assumed biased until the delisted assets are shown to be included.