Reference Class Adequacy Check

Overview

This pillar assesses whether a valid historical reference class exists for a given market. It helps determine if a prediction is based on statistical precedent or is effectively a unique, unpredictable gamble.

What It Does

The analysis systematically searches for and evaluates similar past events to form a 'reference class'. It then scores this class based on its size, the similarity of its events to the current one, and the consistency of past outcomes. A strong reference class provides a powerful statistical baseline, or 'outside view', for the prediction.

Why It Matters

It provides a crucial defense against cognitive biases like the planning fallacy, where forecasters focus too much on the unique details of a situation. By anchoring predictions to a historical base rate, it often yields more accurate probabilities than narrative-based 'inside view' analysis alone.

How It Works

First, the key attributes of the prediction market's event are defined. Then, historical data is scanned to find analogous events that share these attributes. Each potential precedent is scored for relevance, and a final 'adequacy score' is calculated based on the quantity and quality of the matches found.

Methodology

The pillar defines an event as a feature vector. It uses a weighted similarity metric (e.g., cosine similarity) to compare the current event to a historical database. A reference class is considered adequate if its size (N) is greater than 20, the average similarity score is above 0.75, and the outcome variance is low. A time-decay function is applied to down-weight older, less relevant precedents.

Edge & Advantage

This provides an edge by systematically identifying markets where emotional, narrative-driven trading has deviated far from the statistical base rate of similar historical events.

Key Indicators

• Reference Class Size

  high

  The number of comparable historical precedents found. A larger class provides a more reliable statistical sample.

• Similarity Score

  high

  A measure of how closely the past instances match the key characteristics of the current event.

• Outcome Variance

  medium

  The consistency of outcomes within the reference class. High variance suggests the outcome is inherently unpredictable, even with precedents.

Data Sources

• Academic Databases (JSTOR, etc.)

  Provides access to historical, political, and economic research papers containing structured data on past events.

• Sports Statistics Archives

  Repositories like Baseball-Reference or FiveThirtyEight offer deep historical data on sports matchups and player performance.

• Government & Economic Data

  Sources like the World Bank or FRED provide datasets for macroeconomic and geopolitical precedents.

Example Questions This Pillar Answers

• → Will a third-party candidate receive over 5% of the popular vote in the next US presidential election?
• → Will the sequel to 'Galactic Warriors' gross more than its predecessor on opening weekend?
• → Will the Federal Reserve cut interest rates in the next six months?

Tags