Indian-Language Adaptation: Tokenizers, Data and GPU Planning

Overview

Adapting a model for Indian languages is not the same exercise as adapting it for a domain, and the difference is mostly about tokenization. A model whose tokenizer was built for English fragments Devanagari, Tamil or Bengali text into far more tokens per word, which inflates context consumption, inference cost and training compute simultaneously. India’s sovereign model programme has addressed this at the base-model level — Sarvam, BharatGen and Gnani released models covering 22 Indian languages during 2026 — and for most enterprises the practical question is now which base to adapt and how.

Why tokenizer fertility matters so much

Fertility is the average number of tokens a tokenizer produces per word. For English on an English-trained tokenizer it is close to one. For Indic scripts on the same tokenizer it can be several times higher, because the subword vocabulary contains few Indic pieces and the text fragments into short units or even individual bytes.

Three costs follow directly. Inference cost rises roughly proportionally, since you are billed and throttled by tokens. Effective context shrinks, because the same document consumes several times more of the window. And training compute rises for the same corpus. A model that is nominally multilingual but tokenizes Hindi at four tokens per word is meaningfully more expensive to run in Hindi than in English for identical work — which is the practical case for Indic-native tokenizers rather than an ideological one.

The sovereign base-model landscape

India’s position changed materially during 2026. Under the IndiaAI Mission, with an outlay of Rs 10,372 crore, three sovereign models were launched by Sarvam AI, Gnani.ai and BharatGen, supporting 22 Indian languages and aimed at governance, voice and offline use cases. Sarvam unveiled 35-billion and 105-billion parameter models designed for cost efficiency in Indian languages, and BharatGen introduced Param2, a 17-billion parameter mixture-of-experts model across 22 languages positioned for public services.

For an enterprise, this changes the default. Rather than taking an English-centric open model and forcing Indic capability into it, the more efficient path is usually to adapt a base that already tokenizes and models Indian languages natively. That saves the vocabulary extension work entirely and typically produces better quality per unit of adaptation compute. The MoE consideration for Param2-style bases is covered in fine-tuning mixture-of-experts models.

Choosing an adaptation path

The translation-bridge row deserves a caution. It is cheap and works surprisingly well for simple tasks, but it loses code-mixing, register and culturally specific meaning, and it introduces two additional failure points. It is a reasonable pilot approach and a poor production one for anything where nuance matters.

Data strategy for low-resource languages

The instinct is to gather as much text as possible. For Indian languages this is actively harmful without filtering, because a large fraction of available Indic web text is machine-translated from English, transliterated into Latin script, or heavily code-mixed in ways that do not match the target usage. Training on it teaches the model to produce exactly those artefacts.

A workable strategy has three parts. Curate quality over quantity: a few million tokens of genuinely native, in-domain text usually beats a hundred million tokens of scraped noise. Handle code-mixing deliberately, because real Indian usage is code-mixed and a model trained only on pure-language text will handle real user input badly. And build the evaluation set first, in the actual languages and registers your users write in, since without it you cannot tell whether adaptation helped.

Sizing the work

For the common case — adapter tuning an India-trained base on enterprise data — the compute requirement is modest: a single high-VRAM node handles most jobs, and the exercise is measured in GPU-days rather than GPU-months. The memory arithmetic follows the standard LoRA pattern described in LoRA and QLoRA memory math.

Vocabulary extension with continued pre-training is a different scale entirely, needing a multi-GPU cluster for a sustained period, and it is where subsidised national capacity becomes relevant. IndiaAI has commissioned over 36,000 GPUs in Indian data centres with expansion targeting more than 100,000 by end-2026, and startups such as Sarvam have received substantial subsidised allocations — reportedly 4,096 GPUs with subsidies near Rs 100 crore. For an enterprise, that capacity is better suited to the bursty adaptation campaign than owned hardware, with owned capacity reserved for steady-state inference.

What to measure

Three metrics beyond generic benchmarks. Tokenizer fertility on your own text, per language, which directly predicts your inference cost and should inform base-model selection before any training. Task accuracy on a native-script evaluation set built from real user inputs, including code-mixed examples. And degradation in English or in other languages after adaptation, since narrow fine-tuning frequently damages capabilities outside the target.

One deployment note for regulated Indian sectors: an Indian-language model serving customers in banking, healthcare or government handles personal data in every interaction, so the residency and audit considerations apply in full. The placement logic is worked through in DPDP-ready AI infrastructure planning, and the sovereign infrastructure context in sovereign AI in India.

Frequently asked questions