There's definitely some degenerate cases here which seem impossible to achieve the standard of "beat any human"

For example, type your best guess for the current location of the US President. A neutral net is almost certainly not beating the actual US President at this task.

Draft an email while offline on a working properly but 10 year old laptop without a dedicated graphics card

How about "Rate the outputs of a cutting edge LLM"? 🤭

I bet AIs won't be able to do many management tasks (give feedback, take the relationship and personality of the other person into context reasonably), or negotiate for the company buying something (for example, the company needs a CRM/payment system with many requirements and it's hard to figure out which system would fit).

These can all be somewhat done via email

I bet NO

@YonatanCale A typical worker doesn’t negotiate for the company or give feedback to direct reports. (Most people aren’t managers)

And you can’t write up an RfP for a complex product like a CRM in 10 minutes.

@JimHays Thanks. Sold my NO and I'll rethink this

@YonatanCale The “typical” part makes it really hard to know what’s allowed.

Things like “schedule a meeting between three people”, “submit an expense report”, or “make a sign for the break room that the coffee machine is broken” seem like they’d be included. But these all seem pretty easy.

Excel is mentioned in the description, but a typical worker can hardly use Excel to sum a list of numbers. Certainly not make a pivot table to analyze a dataset, or anything similar.

@JimHays "a typical worker can do" not does do

Not only do most workers not do those things, but because they don’t do them regularly, they also don’t have the necessary skills to do them well, either.

@JimHays doesn't matter, the typical worker comment only applies to the selection of the task. For any selected task, the prediction says that the neural net beats all humans.

How about, “Convince your boss that a human version of yourself showed up to your performance review over Zoom”?

Taken to its extreme (literally “any” computer task a typical white caller worker can do fast) this sounds like a Turing Test, but it’s also quite squishy what counts. I don’t think it will be interpreted this way, but hard to know where the line will be