The Pitfalls of Revised Data: Why Point-in-Time Discipline Makes or Breaks a Backtest

Georgi Kuzmanov Senior Equity Analyst & Founder Columbia MSFE · 13+ years equity research

Look at the hero image above for a moment. That ticker was photographed in 2012: Facebook at $22, Intel at $22.81 — and two names, Research In Motion and Ceradyne, that no longer exist as listed companies. A database that only knows today’s market has quietly forgotten them. A backtest built on such a database is not testing history; it is testing a cleaned-up story about history, told by the survivors.

This is the single most common way a beautiful backtest lies. It rarely lies through bad math. It lies through the data: it uses today’s value of a number that was different — or didn’t exist at all — on the day it claims to have traded. GDP prints get revised for years after release; quarterly earnings get restated; index membership lists get rewritten in place; delisted tickers evaporate. Each rewrite is individually reasonable. Collectively, they hand your simulated past a small piece of the future.

The fix is a discipline called point-in-time (PIT) data: every number enters the simulation only on the date it actually became knowable, at the value it actually had then. In the series opener we said the scoring function must use “only information that existed on the decision date” and promised to show why that clause is worth an entire article. Here it is: the formula, the exact enforcement code from our platform, and — because claims need numbers — a measurement of the bias on our own data, where a naive backtest nearly doubles the apparent strength of a real factor.

The Revised-Data Trap

Three flavors of the trap, in increasing order of subtlety:

• Revisions. Macro series are published, then re-published. First-print GDP and the figure sitting in the database three years later can differ by a full percentage point — our May 2026 Weekly Wrap covered a week in which a Q1 GDP revision itself moved the tape. A regime model trained on final values saw recessions arriving earlier and cleaner than any live trader ever did.
• Restatements and lag. A fiscal quarter ends March 31, but the 10-Q that reports it is filed weeks later. A backtest that uses the March 31 value on March 31 is trading on a document that did not exist. Restatements compound this: the “final” number in the database may be a corrected one that nobody saw for years.
• Survivorship. If your universe is “stocks that exist today,” every bankruptcy, delisting, and acquisition-at-a-loss has been silently removed from your simulated past. The disasters your strategy would have held are simply not in the data to be held.

Why is so much financial data like this? Because most consumers of a database want the best current answer, not the historically honest one. A vendor that overwrites last quarter’s figure with the restated value is serving the analyst valuing the company today — and silently sabotaging the researcher simulating a decision made two years ago. Point-in-time datasets are rarer and more expensive precisely because they must store every vintage of every number instead of just the latest, and because assembling them retroactively is somewhere between hard and impossible. The default state of financial data is revised; honesty has to be engineered in.

None of these errors announce themselves. The backtest runs, the equity curve compounds, the Sharpe ratio impresses. The lie only surfaces live, as the mysterious gap between simulated and realized returns — paid for at full size.

The Formula: Availability Date, Not Report Date

The entire discipline compresses into one inequality. A data point about entity i is usable at decision date t only if its availability date — not its reference period — has passed:

usable(i, t)  <=>  availability_date(i) <= t

  equities:  availability_date = datekey        (the SEC filing date)
             NOT calendardate                   (the fiscal period end)
  macro:     availability_date = available_from (the vintage release date)
             NOT the reference month

And the bias it prevents has a measurable size. For any signal s, compare the expected edge computed both ways:

look-ahead error  =  E[ edge | naive data ]  −  E[ edge | PIT data ]

where naive means “values keyed to the period they describe” and PIT means “values keyed to the date they became public.” If the two agree, the discipline cost you nothing. When they disagree, the difference is phantom alpha — edge your backtest claims but no trader could have captured. We compute this error on real data below.

How Our Platform Enforces Point-in-Time

The as-of merge: availability date wins

In our factor library every fundamental panel is built by one function, and its only job is to enforce the inequality above. This is the actual code (from factors.py, the factor engine behind our long/short strategies):

def _asof_panel(sf, values, month_idx):
    """month-end x ticker matrix: latest SF1 row with datekey <= month-end."""
    d = sf[['ticker', 'datekey']].copy()
    d['v'] = values.values
    d = d.dropna(subset=['v'])
    out = {}
    frame = pd.DataFrame({'datekey': month_idx})
    for tk, g in d.groupby('ticker', sort=False):
        g = g.sort_values('datekey')
        merged = pd.merge_asof(frame, g[['datekey', 'v']], on='datekey',
                               direction='backward')
        out[tk] = merged['v'].values
    return pd.DataFrame(out, index=month_idx)

The load-bearing line is the merge_asof(…, on='datekey', direction='backward'): for each month-end, take the latest filing whose SEC filing date — Sharadar’s datekey — has already passed. The fiscal period end (calendardate) is deliberately not the join key. A quarter that ended in March but was filed in May is invisible to the March and April rebalances, exactly as it was invisible to everyone trading those months.

Macro data: vintages, and the series we refuse to use

The same rule governs the macro-regime layer. Our FRED store keeps an available_from timestamp per row, and the regime function only reads rows with available_from ≤ t — the value as first published, not as later revised. More telling is what is excluded: heavily-revised series like nonfarm payrolls and industrial production are barred from the regime signal entirely, because their first prints differ too much from their final values to be trustworthy inputs. A regime detector that quietly consumed final payrolls would be a time traveler. (How the regime model works is its own article, next in this series.)

What else the contract forbids

• Non-deterministic tables. Our config maintains a hard exclusion set (FMP_UNSTABLE_TABLES) for provider tables that rewrite history in place — analyst consensus and estimate tables are the canonical offenders. You cannot ask “what was the consensus on date t?” of a table that only stores today’s consensus, so ranking signals are simply not allowed to read it.
• The 400-day staleness cap. Forward-filling the “latest filing” is correct PIT behavior — up to a point. A company that stopped filing two years ago should not still be ranked on its last 10-K, so fundamental panels forward-fill for at most 400 trading days, after which the value goes NaN and the name drops out of ranking.
• Survivorship-bias-free membership. Index membership is reconstructed month by month from a change log — today’s roster with every historical add/remove replayed backward — so the 2018 universe contains the 2018 members, including the ones that later died. Delisted tickers stay in the price panel, NaNs and all.

A Measured Bias: The Phantom Alpha in Our Own Data

How much does all this machinery actually matter? We measured it. Using our Sharadar SF1 store (228,100 filings, 2014–2026) we built two versions of the same two profitability factors over the same scoped universe — the top 600 US names by average dollar volume, 123 months of 2016–2026 — differing only in the join key: PIT keys each filing to datekey (the filing date); naive keys it to calendardate (the period end), the classic look-ahead mistake. For each we computed the monthly cross-sectional information coefficient (IC) — the rank correlation between the factor and the next month’s returns.

First, the raw material of the bias: the filing lag. Across those 228,100 filings the median gap between fiscal period end and SEC filing date is 42 days (mean 53; the 10th–90th percentile range runs 30 to 88 days). The naive backtest hands itself every filing about six weeks early — and in 52% of stock-months, the value it ranks on differs from what the market actually had.

Read the second row carefully, because it is the whole article in two numbers. Cash profitability’s true, tradeable IC is 2.88%. Give the backtest six illegal weeks of foresight and the IC jumps to 5.38% with a t-statistic of 5.24 — the signal looks nearly twice as strong, and statistically bulletproof. Almost half of the naive edge is phantom: it is the market’s reaction to the filing being counted before the filing existed. Gross profitability, a slower-moving balance-sheet ratio, inflates less (+0.82 points on 2.99 — still a 27% exaggeration). The faster and more news-like the signal, the bigger the lie: an earnings-surprise factor computed naively would be almost entirely fiction.

The Risk: PIT Is Necessary, Not Sufficient

It would be comfortable to end here: use filing dates, keep vintages, done. But point-in-time discipline only guarantees that the data you have is honestly dated. It does not guarantee the data is whole — and the holes bite too.

A survivorship-bias-free universe means delisted names remain in your panels, and their prices end in a wall of NaNs. Those NaNs have to be handled explicitly: our engines force-liquidate a holding whose price has been missing for ten consecutive trading days (at a write-off price, not a hopeful one), precisely so a dead ticker cannot zombie inside a portfolio at its last traded value. This is not hypothetical caution. A real-data defect in exactly this family — catalogued as BUG-002 in our v7 build log — slipped past the type system and was caught only by the output-sanity gate, the final check that refuses to publish dashboard files containing impossible values. The gates deserve their own article, and they get one later in this series (the series hub tracks what has shipped).

There is also an honest cost to admit: PIT discipline makes your backtests look worse. The naive row in our table is the one a marketing deck would prefer. Choosing the 2.88% over the 5.38% — structurally, in code, so no one gets to choose per-run — is the least glamorous decision in quantitative investing and the most important one. It is also why comparing backtest numbers across shops is nearly meaningless unless you know which row of that table each one is reporting.

The AlphaEdge Take

A backtest is only as honest as the calendar of its data: every number must enter on the date it became public, at the value it had then — filing dates over fiscal dates, first prints over final revisions, dead tickers included. On our own data, skipping that discipline inflates a real factor’s IC from 2.88% to 5.38% — phantom alpha you would have paid to discover live. Enforce point-in-time structurally, in the join itself, and treat any backtest that cannot name its availability dates as fiction until proven otherwise.

Georgi Kuzmanov

Senior Equity Analyst & Founder at AlphaEdge. Columbia University MSFE (2011–2013). Covering equities, macro, and geopolitics for serious investors.