How Many GPUs Do You Need for a 100-User Private ChatGPT (On-Prem)?

A private, on-prem ChatGPT-style assistant for 100 users typically runs on one to two datacenter GPUs. Because "100 users" rarely means 100 people typing at once — active-to-total ratios are ~1:10–1:20 — you're really sizing for ~5–15 concurrent requests. A single H200 serving a 70B model at FP8 handles that with headroom; a smaller 8B–13B model runs on a fraction of one GPU.

TL;DR — the decisions

• 100 users ≈ 5–15 concurrent (1:10–1:20 active-to-total) (Lyceum, 2026).
• 70B model, FP8 → 1× H200 (141 GB) comfortably; add a second GPU for redundancy/peak.
• 8B–13B model → a fraction of one GPU (e.g. an L40S/48 GB), the cheapest private-assistant option.
• One H100-class GPU sustains ~80–120 concurrent chat users on an FP8 mid-size model — well above 100 total.

What this sizes

The GPU/server footprint for an internal, private ChatGPT-style assistant serving ~100 employees on-prem — chosen for data control (nothing leaves your network). Fine-tuning is separate; this is serving.

The concurrency math

The trap is treating "100 users" as 100 simultaneous generations. In practice, "concurrent" means actively waiting on a response, and with a 1:10–1:20 active-to-total ratio, 100 registered users produce roughly 5–15 concurrent requests at busy times (Lyceum, 2026). Benchmarks put a single H100-class GPU at ~80–120 concurrent chat users for an FP8 mid-size model — so 100 total users is comfortably within one GPU.

Worked configurations

Table 1 — Private ChatGPT for 100 users (on-prem).

Model	Precision	GPU	Concurrency headroom	Notes
8B–13B	FP8/4-bit	1× L40S (48 GB) or slice	ample	cheapest; great for Q&A/assist
70B	FP8	1× H200 (141 GB)	~80–120 concurrent	best quality-per-GPU; recommended
70B	FP8	2× H200	redundancy + peaks	HA / growth headroom

Reading Table 1: for most 100-user deployments, one H200 running a 70B at FP8 is the sweet spot — strong quality, easily inside its concurrency budget. Add a second GPU only for high availability or planned growth.

Assumptions & scope

Interactive chat workloads; concurrency from active-to-total ratios; FP8 serving. Very long context (128k) or heavy document processing raises KV-cache memory — validate on your stack. Estimates, not a quote.

Where RDP GPU Mart fits

RDP GPU Mart builds the single- or dual-H200 servers a private on-prem assistant needs — enough VRAM for a 70B plus healthy concurrency — India-manufactured, INR-transparent, and DPDP-aware so employee and company data never leave your network. *(Configure a private-LLM server or request a quote at RDP GPU Mart.)*

FAQ

How many GPUs for a 100-person internal ChatGPT? Typically one H200 for a 70B model at FP8; a second GPU only for redundancy or growth.

Does 100 users mean 100 concurrent requests? No — active-to-total ratios of ~1:10–1:20 mean ~5–15 concurrent at peak (Lyceum, 2026).

Which model size for a private assistant? A 70B gives the best quality on one H200; an 8B–13B runs cheaper on a fraction of a GPU if lighter answers suffice.

Why on-prem for a private ChatGPT? So prompts and company data stay inside your network — the point of "private," and often a DPDP requirement.

—

Related

• How Much VRAM for a 70B / 405B LLM (Inference)?
• Right-Sizing a GPU Server for Enterprise RAG
• On-Prem vs Cloud GPU: True TCO for AI Training in India

Research log (Rule #1)

1. Lyceum (2026) — LLM tokens/sec + concurrent-user benchmarks (80–120/GPU; active-to-total). https://lyceum.technology/magazine/llm-inference-tokens-per-second-comparison-2026/ 2. DigitalOcean (2026) — vLLM GPU sizing (KV cache, concurrency). https://www.digitalocean.com/community/conceptual-articles/vllm-gpu-sizing-configuration-guide 3. apxml (2026) — model VRAM by size/precision. https://apxml.com/models/llama-3-1-405b 4. NVIDIA — H200 datasheet (141 GB). https://www.nvidia.com/en-us/data-center/h200/ 5. iternal.ai (2026) — on-prem AI hardware sizing. https://iternal.ai/hardware-sizing-guide