ProBackend
AI Infrastructure
3 days ago5 min read

Cost Allocation for Shared GPU Clusters: OpenAI’s 2024 Compute Bill

Cost Allocation for Shared GPU Clusters: OpenAI’s 2024 Compute Bill

Epoch AI estimates OpenAI spent about $7 billion on cloud compute in 2024. The breakdown separates inference, broad R&D spending, and final training runs—and explains what those estimates leave out.

Cost Allocation for Shared GPU Clusters: What the Estimate Counts

OpenAI’s estimated 2024 compute spending was about $7 billion. That figure is useful, but it isn’t a clean ledger of every GPU-hour assigned to a named model. Epoch AI combines reported company-wide expenses with its own estimates of final training runs, and those are different kinds of evidence.

The distinction matters. The overall spending estimate includes inference, training, and research compute; the model-level estimates concern final training runs for selected releases. Experimental runs are not included in those final-run figures, even though Epoch places training and research together in a broader R&D category.

Epoch says OpenAI did not own significant amounts of AI compute in 2024. Its estimate therefore treats the spending as covering all of OpenAI’s compute resources that year. The figures are cloud compute expenses, not the upfront capital cost of building data centers. Epoch AI’s analysis explains its sources and estimation method.

How much does an LLM cost?

There isn’t one price for “an LLM.” Training a model, serving it to users, and running research experiments are separate uses of compute. Epoch’s figures make that separation visible, but they do not provide a universal model price or a complete cost for every OpenAI system.

Epoch’s 2024 company-wide breakdown starts with reports indicating $3 billion for training compute, $1.8 billion for inference, and $1 billion for research compute amortized over multiple years. Epoch assumes a two-year amortization schedule for research compute, which makes its estimated research expense incurred in 2024 $2 billion. It then groups the $3 billion training and $2 billion research amounts as $5 billion in R&D compute.

That $5 billion is not simply the cost of training released models. Epoch says the precise distinction OpenAI itself makes between training and research compute is unknown. “R&D” is Epoch’s grouping for those expenses, not a disclosed accounting category with a fully specified boundary.

How much is an LLM? The final-run estimate

The analysis separately estimates the final training runs behind models released or announced from the end of Q1 2024 through the end of Q1 2025. Epoch uses that window as a rough way to account for the lag between training and release. The models it identifies include GPT-4.5, GPT-4o, o3 (December preview), and Sora Turbo.

A final training run means the run that produced the version released to users. It excludes earlier experimental or “derisking” runs used to prepare for that final run. So a final-run estimate is narrower than the total compute invested in developing a model, and it should not be read as the all-in R&D cost.

For GPT-4.5, Epoch estimates cloud spending from cluster size, duration, and price per GPU-hour. Its low and high estimates use 40,000 and 100,000 Nvidia H100s, respectively, and a run lasting 90 to 165 days. It assumes a cloud price of $1.50 to $3 per H100-hour, noting that $2 per H100-hour matches a general industry trend. The cluster and duration estimates draw on evidence including analysis of Microsoft’s Goodyear, Arizona cluster and a third-party estimate.

For other models, Epoch estimates training compute in floating-point operations, then converts that to cloud cost using assumptions about GPU utilization and GPU-hour pricing. Rather than treating the resulting model estimates as precise point values, it uses confidence intervals. Its ranges include substantial uncertainty: for GPT-4o, for example, the low and high compute estimates are 1e25 and 5e25 FLOP. Sora Turbo’s range is wider still, from 1e24 to 1e26 FLOP.

These model-level estimates and the $5 billion R&D grouping answer different questions. One estimates final runs for selected releases; the other groups reported overall training and research spending. Adding or comparing them without regard to scope would risk counting unlike things as though they were the same accounting line.

What is cloud cost in this estimate?

Here, cloud cost means the expense of accessing computing capacity through cloud providers, rather than the purchase price of data-center infrastructure. Epoch describes OpenAI as relying on cloud companies for compute access in 2024. That’s why the estimated $7 billion is a compute-spending figure, not a valuation of GPUs or facilities owned by OpenAI.

The broad total is based on reports about OpenAI’s expenses. Epoch traces the reported breakdown to investor documents from September 2024 that were shown to journalists, and cautions that the figures should not be considered completely reliable. They also rely in part on projections for the final quarter of 2024.

Epoch notes other evidence that it considers consistent with the spending estimate: OpenAI reportedly generated around $3.7 billion in revenue in 2024 and was substantially unprofitable. That context is a plausibility check, not an independent itemized audit of compute costs.

Inference has an important boundary

The reported $1.8 billion inference figure is not necessarily all inference tied to OpenAI models. Epoch says it presumably excludes Microsoft’s spending to run licensed OpenAI models in Microsoft products, including Copilot and the Azure API. Microsoft owns the data centers used for training, research, and inference, while also licensing OpenAI models for its own products.

Epoch considers those Microsoft inference costs likely significant, but potentially smaller than OpenAI’s own inference costs, based on reports that OpenAI’s products generate more revenue than Microsoft’s equivalents. That is an estimate about a boundary in the available numbers; it is not a disclosed total for all inference served through every partner.

This is one reason “how much does an LLM cost?” has no single answer in this data. A company-wide inference line, the final training run for a named release, and a partner’s cost to serve a licensed model are not interchangeable measures.

What the figures can—and can’t—tell us

The clearest reading is modest: Epoch estimates roughly $7 billion in OpenAI cloud compute spending for 2024, with about $5 billion grouped as R&D compute under its assumptions and $1.8 billion reported for inference. The exact research amortization assumption matters, and the broad total rests partly on reports and projections rather than a public, audited GPU-by-GPU ledger.

The final training-run estimates provide another useful view, but they do not replace that company-wide accounting. They cover selected models and exclude experimental runs. Nor does this analysis estimate environmental impact, energy consumption, or a general cost per user query; those require different data and boundaries.

For anyone comparing AI compute costs, keep the accounting unit attached to the number: annual cloud spending, R&D grouping, final training run, or inference expense. Remove that label and a precise-looking dollar figure can answer the wrong question.