Conversational Commerce in Indian Languages: GPU Sizing for Voice Retail

Overview

Conversational commerce is where India’s internet population and its language reality meet. Most of the country’s several hundred million internet users are not comfortable transacting in English, and a large share prefer speaking to typing — which has pushed retailers toward voice and vernacular chat interfaces on messaging platforms and, increasingly, on the ONDC network. The infrastructure question is specific: a voice turn runs three models in sequence within a latency budget a human will tolerate, and doing that in Hindi, Tamil or Bengali costs more than doing it in English.

What one turn actually costs

A voice commerce turn decomposes into: capture and endpointing (deciding the user stopped speaking), automatic speech recognition, intent handling by a language model with retrieval against the catalogue, response generation, and text-to-speech synthesis. Chat-only interfaces drop the first and last stages, which is why they are considerably cheaper and why many deployments start there.

Each stage consumes part of a single budget. Users tolerate roughly a second of silence before a spoken system feels broken, and endpointing alone typically consumes 200 to 500 milliseconds because the system must be confident the speaker finished. That leaves comparatively little for everything else, which is why architecture rather than raw GPU power determines whether the experience feels natural.

The latency budget

The design principle running through the table is overlap. Recognition streams while the user speaks; synthesis begins as soon as the first sentence of the response exists rather than waiting for completion. A system that serialises these stages will be two to three times slower than one that pipelines them on identical hardware, which makes engineering the larger lever over procurement.

Code-mixing is the accuracy problem

Real Indian conversational speech mixes languages within a single sentence: an English product name inside a Hindi request, a Tamil sentence with English numerals, brand names pronounced in a regional accent. Speech recognition models evaluated on clean monolingual benchmarks perform substantially worse on this, and the errors concentrate exactly where they matter — product names, quantities and numbers.

Two responses work. Bias the recogniser toward your catalogue with contextual hints, so brand and product names are far more likely to be recognised correctly. And build the evaluation set from actual recorded customer speech, stratified by language and region, rather than trusting published word error rates. The consistent finding across Indian deployments is that catalogue-specific accuracy matters more than general accuracy, and it is improvable with modest effort.

Tokenizer cost in Indic languages

The language model stage carries a hidden multiplier. Tokenizers built primarily for English fragment Devanagari, Tamil and Bengali text into several tokens per word, which raises both the compute per turn and the time to first token — directly harming the latency budget. Using a model with native Indic tokenization is therefore a latency decision as much as a quality one.

India’s sovereign model programme has made this practical: models covering 22 Indian languages with native tokenization are now available for adaptation, and the arithmetic is worked through in Indian-language adaptation. For a voice deployment, evaluating candidate models on tokens-per-word for your actual languages before benchmarking quality is a sensible first filter.

Sizing from concurrent conversations

Work from concurrency, not from daily volume. Estimate peak simultaneous conversations, which for retail follows sharp patterns — evenings, weekends, and the festive peaks that dominate Indian e-commerce. Each concurrent conversation occupies capacity across all three model tiers, and the tiers scale differently: recognition and synthesis are relatively predictable per stream, while the language model tier is governed by KV cache across concurrent sessions.

Size each tier separately and independently, since they saturate at different points, and provision the language model tier for peak concurrency with the tail in mind rather than the average. Festive planning deserves particular attention in India, where peak-to-average ratios are extreme — the pattern is covered in festive-peak AI capacity planning, and the general agent-sizing method in agentic commerce infrastructure.

Governance and the transaction boundary

Two considerations. Voice recordings and transcripts are personal data, and for a retailer they may also contain payment context; under DPDP, with penalties reaching Rs 250 crore and full enforcement expected in 2027, retention and access control for those recordings need a policy rather than a default. Keeping the speech pipeline in-country avoids a category of transfer questions entirely.

Second, the transaction boundary. A conversational system should recommend and assemble a cart, but the commitment step — payment authorisation — belongs to an explicit, unambiguous confirmation outside the conversational flow, not to a model’s interpretation of an ambiguous utterance. Misrecognition of a quantity in a noisy environment is common, and an architecture that requires confirmation before charging is both better product design and a materially smaller dispute surface. The catalogue and search infrastructure this sits on is covered in hybrid product search.

Frequently asked questions