Was GPT-4 trained in 4 months or less? | Manifold

Was GPT-4 trained in 4 months or less?

Mini

5

Ṁ52

2029

59%

chance

1D

1W

1M

ALL

If I become confident at >95%, I will evaluate this YES or NO.

This is for the base model in particular.

Get

1,000

and

1.00

Sort by:

Saltman said it cost ~100 million minimum for training gpt-4

@firstuserhere I can't tell if this is for the cluster itself or an estimate of the cost of the cluster-time

Matthew Barnett says GPT-3.5 finished training in early 2022.
https://twitter.com/MatthewJBar/status/1605509829555953667

We know that GPT-4 finished training in August 2022.
https://arxiv.org/pdf/2303.08774.pdf

My best guess is GPT-3 took 1~3 months to train: https://www.getguesstimate.com/models/22425

@MatthewBarnett

Do you have a source for this claim?

@NoaNabeshima Also I've heard it said that GPT-3.5 was a test run for the hardware used to train GPT-4, suggesting that GPT-4 came after GPT-3.5 was done.

Unclear if GPT-3.5 is literally just the largest model in the GPT-4 scaling law experiment

@NoaNabeshima Came to these estimates with @datagenproc though idk if they endorse them

@NoaNabeshima oh sorry @datageneratingprocess

Related questions

When will GPT-5 finish training?

Will there be an LLM (as good as GPT-4) that was trained with 1/100th the energy consumed to train GPT-4, by 2026?

Will GPT-4 be trained on more than 10T text tokens?

In yottaFLOPs (10^24), how much compute will GPT-4 be trained with?

Will a GPT-4 quality model be trained for under $10.000 by 2030?

What is true of the new Japanese GPT-4?

What is the main reason behind GPT-4o speed improvement relative to GPT-4 base model?

Before 2028, will anyone train a GPT-4-level model in a minute?

Will GPT-4 be trained (roughly) compute-optimally using the best-known scaling laws at the time?

Will a GPT-4 level system be trained for <$1mm by 2030?

Related questions

When will GPT-5 finish training?

What is true of the new Japanese GPT-4?

Will there be an LLM (as good as GPT-4) that was trained with 1/100th the energy consumed to train GPT-4, by 2026?

What is the main reason behind GPT-4o speed improvement relative to GPT-4 base model?

Will GPT-4 be trained on more than 10T text tokens?

Before 2028, will anyone train a GPT-4-level model in a minute?

In yottaFLOPs (10^24), how much compute will GPT-4 be trained with?

Will GPT-4 be trained (roughly) compute-optimally using the best-known scaling laws at the time?

Will a GPT-4 quality model be trained for under $10.000 by 2030?

Will a GPT-4 level system be trained for <$1mm by 2030?

Terms & Conditions•Privacy Policy•Sweepstakes Rules