Compute Budgeting After the Pre-Training Plateau

Overview

The assumption that shaped AI infrastructure budgets from 2020 to 2024 — that capability comes mainly from bigger pre-training runs on more tokens — no longer describes where marginal compute goes. A growing share of frontier spend has moved to post-training (reinforcement learning, preference optimisation, verifiable-reward loops) and to inference-time reasoning, where models spend more compute per query rather than more compute per training run. For an enterprise planning capacity for 2026-28, the practical consequence is a different split between training and serving hardware, and a different risk profile.

What actually changed

Classical scaling relationships described loss as a function of parameters, data and compute, and implied a straightforward strategy: spend more, get better. Those relationships have not been repealed, but two things constrain them in practice. High-quality text data is finite, so adding tokens increasingly means adding lower-quality or synthetic tokens with diminishing returns. And the cost of each further step on the pre-training curve has grown faster than the capability gain it delivers.

What has proved more compute-efficient is teaching an existing base model to reason and to follow verifiable objectives after pre-training, and letting it spend more compute at inference time on hard problems. That is a different shape of spend: many medium-sized jobs rather than one enormous one, with a much tighter feedback loop between training and evaluation.

Why post-training absorbs budget differently

A pre-training run is a single long job that fills a cluster for weeks and is largely predictable. A post-training programme is a pipeline: generate rollouts, score them, update the policy, evaluate, repeat. It mixes inference-heavy phases (generating candidate responses) with training-heavy phases (the update), and it is driven by experiment throughput rather than by a fixed token budget.

That changes the infrastructure requirement in three ways. Utilisation is spikier, so a cluster sized for the peak sits idle between cycles unless it is shared. Inference capacity becomes part of the training system, not just the serving system. And storage and evaluation infrastructure grow in importance, because the bottleneck is often how fast you can score outputs, not how fast you can do gradient steps. The sizing detail is in sizing DPO and RLVR post-training infrastructure.

Inference-time reasoning moves the cost

Reasoning models generate substantially more tokens per query than an equivalent non-reasoning model, because the chain of intermediate work is itself generated output. A query that once produced 300 tokens may now produce several thousand. Serving cost scales roughly with generated tokens, so a reasoning deployment can cost several times a conventional one for the same user-visible answer count.

This is the single most under-modelled item in 2026 enterprise AI budgets. It also reshapes serving architecture: long generations make decode throughput and KV cache capacity the binding constraints, which is exactly what drives the prefill-decode split described in disaggregated inference. Budget in tokens generated per day, not queries per day.

How to split a fixed capital budget

The row most Indian enterprises occupy is the first, and the honest reading of it is that the training cluster many organisations budgeted for in 2024 was the wrong purchase. The defensible capital spend is inference capacity sized to real token volume, plus a modest, well-utilised adaptation cluster. The decision frame is developed further in the AI infrastructure buying guide for CIOs.

A budgeting method that survives contact

Work in workload units, not hardware units. For serving, estimate daily queries, average generated tokens per query including reasoning traces, and peak-to-average ratio; convert to required decode throughput and then to GPUs using measured tokens per second on your model and precision, never on vendor peak figures. For adaptation, estimate experiments per quarter rather than a token count, since post-training capacity is governed by iteration speed.

Then apply three stress tests. What happens if generated tokens per query triple because a reasoning mode is enabled by default? What happens if the model you standardised on is superseded in nine months — is the hardware still useful? And what happens if utilisation is half your plan, which is the common outcome for adaptation clusters? A budget that survives all three is a budget worth defending.

What this means for Indian buyers

India’s public compute programmes have concentrated capacity for exactly the workloads most organisations should not be building themselves. The IndiaAI Mission’s empanelled GPU capacity — roughly 38,000 GPUs, with plans reported toward around 100,000 by the end of 2026 and subsidised rates near a dollar per GPU-hour — is a reasonable place to run bursty adaptation and experimentation without buying a cluster that idles.

The complement to that is owned inference capacity for steady-state, latency-sensitive and regulated workloads, where data residency and predictable unit cost favour on-premises. That split — rent the spiky training, own the steady serving — is the most defensible 2026 posture for a mid-size Indian enterprise, and it is consistent with the TCO arithmetic in on-prem vs cloud GPU TCO.

Frequently asked questions