What is Claude Opus 4's performance on METR's task length evaluation?

Ṁ1053

Jul 31

ALL

Less than 1 hour

11%

1h to 1.5h

21%

1.5h to 2h

43%

2h to 3h

18%

At least 3h

METR's evaluation measures AI performance by the duration of tasks that models can complete with a 50% success rate. This market predicts Claude Opus 4's time horizon, as reported by METR.

If no score is provided by the end of July 2025, this market resolves as N/A. If there are multiple scores provided by METR, I'll use my best judgment. I won't trade in this market.

#AI

#Claude

Get

1,000

and

1.00

4 Comments

Sort by:

Does it count if it gets to use parallel processing?

@PhilosophyBear ~~Yes~~ See comment below.

@Loppukilpailija shouldn't you try to compare like-for-like with METR's previous results?

@JoshYou Huh. I had thought that METR's resuls already allowed for best-of-N, which would be equivalent/redudant with parallel processing, but apparently I was wrong. I redact my earlier comment and try to do an apples-to-apples comparison.

Related questions

Related questions