LLMs are being proclaimed as achieving AGI because they are saturating evals that are non-agentic one-shot quizzes. However, the best definitions of AGI say it must “autonomously perform most economically valuable work better than humans”. Vanishingly little economically valuable work by humans consists of answering quizzes using a No. 2 pencil.

Below are concrete economically-valuable achievements by which AIs could be objectively evaluated. Many of these achievements will be subject to human gatekeeping and so might seem unfair. But most of these milestones should be expected on most purported pathways to ASI. If human gatekeeping can prevent many of these milestones, it probably could also prevent ASI and the doom that would purportedly accompany it.

Evals for each occupation:

• Avg number of consecutive hours an AI commercially does the job without more supervision than humans get
• Does the job commercially full-time without such supervision, at lower total amortized cost
• Displaces most humans from that job

For each category, and with no more human supervision than usually provided by a board of directors, movie studio, or literary editor:

• create one with a calendar year profit of $1M
• create one that ranks in the calendar year top 100 of that category by revenue, profit, or market value

Bonus category: media franchise with >$2B cumulative gross revenue

• fiction book
  • By 2050, will at least 25% of #1 NYT Best Selling Fiction be primarily written by AI?
• song
  • By 2050, will at least 25% of Year-End Billboard Hot 100 songs be primarily composed by AI?
• feature film
  • In early 2028, will an AI be able to generate a full high-quality movie to a prompt?

Win an award with no more human supervision than the typical human winners get.

• Academy Award: Best Original Screenplay
• Pulitzer Prize for Fiction
• Nobel Prize for Literature
  • This Metaculus prediction will turn on on human judges, not AI capabilities
• Academy Award: Best Animated Short Film
• Academy Award: Best Animated Feature Film
• Pulitzer Prize for Biography, Nonfiction, or History
• Academy Award: Best Documentary Feature Film
• Academy Award: Best Picture
• Pulitzer Prize for Investigative Journalism
• Turing Award
• Nobel Prize for Physics, Chemistry, or Physiology/Medicine
• Fields Medal
• Any Millennium Prize
• All Millennium Prizes

When will super-persuasive AI get a million people to

• join an existing religious sect or political party
• join a new religious sect or political party
• vote for it in an election

• 2024 Occupational Exposure to AI

Data are averaged estimates from multiple LLMs, and not to be considered authoritative. (It's 2024 and LLMs still can't reliably research straightforward historical data like this. Indeed, when asked to fill in this table, frontier LLMs routinely hallucinated some occupation codes.)