Base Rates & Bayesian Updating

A base rate is the probability of something happening before you have any specific information about the particular case. "Our new feature will succeed" is a prediction. But what is the base rate for new features succeeding? If industry data shows that 30% of new features meet their success criteria, that 30% is your starting point — not your enthusiasm, not your gut feeling, not the CEO's conviction. Bayesian updating is the process of adjusting that base rate as new evidence arrives. Good user testing moves the probability up. Poor early metrics move it down. The combination of starting from base rates and updating with evidence is how rational engineers make predictions under uncertainty.

Why Base Rates Matter

Humans are terrible at base rates. We substitute specific information for statistical information. This is called base rate neglect, and it is one of the most well-documented cognitive biases.

Scenario: An engineer says "this project will ship on time"
Question: What percentage of software projects ship on time?
Answer:   Roughly 30% (based on decades of industry data)

The engineer is not lying. They genuinely believe their project
is special. But the base rate says: without strong evidence to
the contrary, this project has a 70% chance of being late.

The engineer has inside information (their specific project plan, their team's capabilities, the scope they defined). But they are overweighting that inside information and ignoring the base rate entirely. Kahneman calls this the "inside view" versus the "outside view." The outside view (base rates) is almost always more accurate than the inside view (your feelings about your specific situation).

Bayes' Theorem in Practice

Bayes' theorem is the mathematical framework for updating probabilities:

P(H|E) = P(E|H) * P(H) / P(E)

Where:
  P(H)   = Prior probability (base rate)
  P(E|H) = Likelihood of evidence given hypothesis is true
  P(H|E) = Updated probability after seeing evidence
  P(E)   = Total probability of the evidence

You do not need to compute exact numbers. The intuition is what matters: start with the base rate, then adjust based on how surprising the evidence is.

Example: Should we build a microservices architecture?

Base rate: "What percentage of teams that adopt microservices
           regret it within 2 years?"
           Rough estimate: 40-50%

Evidence that updates upward (toward success):
  - Team has 50+ engineers (microservices make more sense at scale)
  - Strong DevOps culture already in place
  - Clear service boundaries based on domain expertise
  Updated probability of success: maybe 65%

Evidence that updates downward (toward failure):
  - Team has 5 engineers (too small for the overhead)
  - No experience with distributed systems
  - Adopting because "Google does it"
  Updated probability of success: maybe 15%

The base rate prevents you from starting at 90% because you read a blog post that made microservices sound great.

Applied to Project Estimation

Project estimation is where base rate neglect causes the most damage in engineering organizations.

Inside view (what engineers do):
  1. Break project into tasks
  2. Estimate each task
  3. Sum the estimates
  4. Add 20% buffer
  5. Present estimate with confidence

Outside view (what they should also do):
  1. Find 5 similar past projects at this company
  2. How long did they actually take vs. their estimates?
  3. Average overrun ratio: probably 1.5x to 2.5x
  4. Apply that ratio to the inside-view estimate

The inside view feels rigorous because you did detailed planning. The outside view feels lazy because you just looked at historical averages. But the outside view is consistently more accurate because it incorporates base rates from actual outcomes.

Project estimation with base rates:

  Inside view estimate: 6 weeks
  Historical overrun for similar projects: 2x
  Base-rate-adjusted estimate: 12 weeks

  "But this project is different because..."
    This is exactly what every team says.
    The base rate already accounts for projects that were
    "different." It still averages 2x overrun.

  Adjusted estimate with strong mitigating evidence:
    Team has shipped 3 similar projects recently on time -> 1.3x
    Revised estimate: 8 weeks (not 6, not 12)

Applied to Risk Assessment

Risk assessment without base rates is just storytelling. With base rates, it becomes calibrated prediction.

Risk: "Our database might fail under Black Friday load"

Without base rates:
  "We did load testing, it should be fine" (inside view)
  Assigned probability: 5%

With base rates:
  "What percentage of e-commerce sites experience database
   issues on Black Friday?"
  Base rate: roughly 20-30%
  "We did load testing" -> update downward to maybe 12%
  "We have auto-scaling configured" -> update to maybe 8%
  "We ran a full dress rehearsal last week" -> update to maybe 5%

  The base rate forced us to start from reality (20-30%)
  and justify each reduction with specific evidence.

The Pre-Mortem as Bayesian Practice

A pre-mortem asks: "Assume the project failed. Why did it fail?" This forces the team to consider evidence that updates the success probability downward — evidence they would otherwise ignore due to optimism bias.

Pre-mortem output for a platform migration:
  - "The migration took 2x longer than estimated"
    Base rate for migration overruns: very high
  - "We discovered undocumented dependencies mid-migration"
    Base rate for undocumented dependencies: near-certain
  - "The new platform had a critical bug we did not test for"
    Base rate for unexpected bugs in new platforms: high

Each of these updates the success probability downward.
A team that does a pre-mortem is implicitly doing Bayesian
updating.

Applied to Hiring

Hiring decisions are deeply susceptible to base rate neglect. Interviewers form strong impressions from a 45-minute conversation and overweight that impression relative to base rates.

Base rates for hiring:
  - Probability that a candidate who performs well in
    interviews performs well on the job: ~60-70%
  - Probability that a candidate from a top company
    will succeed at your startup: ~50% (different context)
  - Probability that a referral hire outperforms a
    non-referral hire: ~10-20% higher retention, marginal
    performance difference

Common base rate neglect in hiring:
  "They crushed the system design interview!"
  Base rate: 30% of candidates who crush system design
  interviews struggle with ambiguity in actual projects.

  "They worked at Google!"
  Base rate: Google engineers succeed at startups roughly
  50% of the time. Different environment, different skills.

Applied to Technology Decisions

Every technology adoption decision has a base rate.

"Should we adopt Kubernetes?"
  Base rate: What percentage of teams that adopt K8s
  report it was worth the investment within 1 year?
  Roughly 40-50% (many teams adopt too early or at too
  small a scale for the overhead to pay off)

  Evidence that updates upward:
    - Team runs 20+ services
    - Already using containers
    - Dedicated ops/platform team
    Updated: 70%

  Evidence that updates downward:
    - Team runs 3 services
    - No containerization experience
    - No dedicated ops team
    Updated: 20%

"Should we rewrite in a new language?"
  Base rate: What percentage of language rewrites deliver
  the expected benefits on schedule?
  Roughly 25-35%

  This base rate alone should make you pause.

Calibration: Testing Your Predictions

Calibrated prediction means that when you say something has a 70% probability, it actually happens about 70% of the time. Most people are badly calibrated — their 90% confidence predictions come true only 60-70% of the time.

Calibration exercise:

  Make 20 predictions with confidence levels:
    "CI will be green on first try" - 80%
    "This PR will get approved without changes" - 40%
    "We will ship feature X by Friday" - 60%
    "The database migration will complete in under 1 hour" - 70%

  Track outcomes. After 50+ predictions:
    If your 80% predictions come true 80% of the time,
    you are well calibrated.
    If they come true only 50% of the time,
    you are overconfident.

Most engineers are overconfident. The fix is to practice making predictions, track outcomes, and adjust. Over time, your predictions become more accurate because you internalize the base rates from your own experience.

The Update Rules

Not all evidence deserves the same weight. Strong evidence updates your probability a lot. Weak evidence updates it a little.

Strong evidence (large update):
  - Controlled experiment with clear results
  - Data from your own system under realistic conditions
  - Multiple independent sources agreeing

Moderate evidence (moderate update):
  - Case study from a similar company
  - Expert opinion from someone with direct experience
  - Single well-designed A/B test

Weak evidence (small update):
  - Blog post from someone selling the solution
  - Anecdote from a conference talk
  - Theoretical argument without empirical backing
  - "It worked at [famous company]"

Common Pitfalls

Ignoring base rates entirely: The most common error. Engineers estimate, predict, and plan as if their situation is unique. It rarely is. Always ask: "What is the base rate for this type of thing?"
Refusing to update: Starting from a base rate but then ignoring strong evidence that should change the probability. Stubbornly sticking to "projects always fail" when this specific project has strong evidence of success.
Updating on noise: Adjusting your probability after every data point, including irrelevant or unreliable ones. A single user complaint is not evidence that the feature is failing. A pattern of complaints is.
Confusing absence of evidence with evidence of absence: "We have not seen any bugs" does not mean there are no bugs. It means you have not looked hard enough, or users have not hit them yet.
Anchoring on a bad base rate: Using an irrelevant base rate is worse than using none. The base rate for "software projects" is too broad. The base rate for "database migration projects at companies our size" is much more useful.
Overweighting inside view: Detailed plans feel more reliable than statistical averages. They are not. The inside view is a story you tell yourself. The outside view is what actually happened to everyone else.

Key Takeaways

Always start with the base rate. Ask "How often does this type of thing succeed/fail?" before considering your specific case.
Update the base rate with evidence, but weight evidence by its strength. A controlled experiment moves the needle more than a blog post.
Apply base rates to project estimation, risk assessment, hiring, and technology adoption decisions.
Practice calibration by making explicit predictions with confidence levels and tracking outcomes.
The outside view (base rates from similar situations) is almost always more accurate than the inside view (your detailed plan for your specific situation).
Pre-mortems are a practical tool for Bayesian updating: they force you to consider evidence that reduces your success probability.