Historical Accuracy Weighting

Overview

This pillar analyzes your personal forecasting track record to adjust current predictions. It provides a data-driven check on your confidence levels, leading to more accurate and reliable probability estimates.

What It Does

It calculates your historical accuracy within a specific market category, typically using a Brier score. This score quantifies your past performance, revealing if you tend to be overconfident or underconfident. The pillar then generates a personal calibration factor to fine-tune your future predictions.

Why It Matters

Human intuition is often biased. This pillar grounds your predictions in personal historical data, correcting for cognitive biases and improving long-term profitability. It turns your past experience into a concrete, actionable advantage.

How It Works

First, the system gathers all your resolved predictions within a specific domain, like 'US Politics'. It then calculates your Brier score, which measures the accuracy of your probability estimates. This score is used to create a weighting factor that suggests an adjustment to your current, unresolved predictions in that same domain.

Methodology

The primary metric is the Brier Score, calculated as (1/N) * Σ(f_t - o_t)^2, where 'f' is your forecast probability and 'o' is the outcome (1 for yes, 0 for no). The analysis covers the last 50-100 resolved markets in a given category. A Calibration Factor is then derived to adjust new forecasts towards your historical mean accuracy.

Edge & Advantage

This provides a powerful defense against overconfidence by systematically adjusting your probabilities based on your demonstrated skill, not just your current feeling.

Key Indicators

• Domain-Specific Brier Score

  high

  A score from 0 to 1 measuring your historical forecast accuracy in a specific category. Lower is better.

• Historical Calibration Factor

  high

  A calculated multiplier used to adjust new predictions based on your historical accuracy.

• Confidence Delta

  medium

  The difference between your stated confidence and your historically demonstrated accuracy.

Data Sources

• User Prediction History

  Provides the user's past forecasts and their stated probabilities across various markets.

• Platform Market Resolution Data

  Provides the official outcomes (Yes/No) for past prediction markets.

Example Questions This Pillar Answers

• → Am I consistently overconfident in my tech stock predictions?
• → How should my 80% confidence in this election be adjusted based on my track record?
• → Does my historical accuracy in sports betting justify my current high-stakes predictions?

Tags