Back to deep dives

Noam Brown on Why AI Benchmarks Need a Compute Budget

  • Agents
  • AI For Science
  • AI Engineering, Software, And Developer Tooling
  • Policy, Governance, And Geopolitics
  • AI Infrastructure, Compute, Chips, And Energy
  • Frontier Models And Capabilities

Watch the deep dive

The problem is we're in a world now where the capability of the model is a function of how much money you put into it.

The Erdős unit distance problem is an 80-year-old math puzzle proposed by Paul Erdős in 1946. It asks how many pairs of points on a flat plane can be exactly one unit apart.

Section 1

Section 01
  • The Erdős unit distance problem is an 80-year-old math puzzle proposed by Paul Erdős in 1946. It asks how many pairs of points on a flat plane can be exactly one unit apart.
Noam Brown benchmark-budget recap thumbnail
The AGI Post recap thumbnail for Noam Brown on budget-aware AI benchmarks. Source: The AGI Post YouTube

Tags

  • Agent Runtime
  • Research Labor Productivity
  • Agent Orchestration
  • Frontier Models
  • Test-Time Compute
  • AI Policy And Governance
  • Software Reliability And Verification
  • Benchmarks And Evaluation
  • Inference Infrastructure
  • Agent Infrastructure
  • AI Safety Governance