AI Architectures
48 articlesContext Engineering for Agentic RAG: Caching and Cost Control
An agentic RAG system retrieves repeatedly and carries results forward, so context grows across a trajectory and dominates cost. Prefix caching, compaction and retrieval budgets are what keep a deployment…
Multimodal RAG: GPU Planning for Document, Image and Video Retrieval
Most enterprise knowledge is in scanned documents, diagrams, screenshots and recordings, not clean prose. Multimodal RAG indexes those directly, and the GPU cost sits overwhelmingly in ingestion rather than in…
Long Context vs RAG in 2026: Where Retrieval Still Wins
Million-token windows did not make retrieval obsolete. Published comparisons put long-context serving at orders of magnitude more cost per query than a RAG pipeline, and multi-fact recall degrades in the…
Continual Pre-Training: When Fine-Tuning Is Not Enough
Fine-tuning teaches behaviour; continual pre-training teaches a language or a domain's underlying distribution. It costs orders of magnitude more and risks catastrophic forgetting. This sets out when it is genuinely…
Indian-Language Adaptation: Tokenizers, Data and GPU Planning
Sovereign Indian models from Sarvam, BharatGen and Gnani now cover 22 languages, and IndiaAI has commissioned over 36,000 GPUs heading toward 100,000. This covers tokenizer efficiency, data strategy and how…
Sizing the RL Fine-Tuning Loop: Rollouts, Verifiers and GPU Split
Reinforcement learning with verifiable rewards is now standard post-training, and it inverts the usual sizing assumption: most of the GPU budget goes to generating rollouts, not to gradient steps. That…
Fine-Tuning Mixture-of-Experts Models: Memory and Routing Realities
MoE models activate few parameters but must hold all experts in memory. A widely cited example needs roughly 94 GB at FP16 for inference alone, and fine-tuning adds optimiser state…
Compute Budgeting After the Pre-Training Plateau
Frontier compute is shifting from pre-training toward post-training and inference-time reasoning. That changes how an enterprise budgets GPU capacity: fewer giant runs, more RL loops and evaluation, and a different…
Training Goodput at 10,000 GPUs: Failures, MFU and Honest Throughput
Llama 3 pre-training on 16,384 GPUs saw 466 interruptions in 54 days. At that failure rate, the number that matters is goodput, not peak FLOPS. This explains MFU, failure modes,…
800 VDC Power: Preparing Training Halls for Megawatt AI Racks
Rack power went from about 40 kW in the Hopper era to roughly 120 kW with Blackwell, and 800 VDC distribution arrives with megawatt racks from 2027. This explains what…
HBM4 and the Memory Wall: What It Means for 2027 Training Clusters
HBM4 doubles the memory interface to 2048 bits and entered mass production in early 2026. Because large-model training is bandwidth-bound more often than FLOPS-bound, that step decides achieved utilisation, cluster…
Vera Rubin NVL144: What the 2026 Training Platform Changes for Cluster Design
Vera Rubin NVL144 keeps the rack as the scale-up domain but raises memory, interconnect and power together. HBM4, NVLink 6 and ConnectX-9 change how many racks a training run needs,…
GPU RDP Buyer Search Guide for Indian AI Teams
Indian AI teams evaluating GPU RDP servers should anchor every purchase decision to workload type, memory footprint, and data-residency obligations before comparing SKUs. The right GPU RDP configuration depends on…
Disaggregated Inference: Splitting Prefill and Decode in 2026
Prefill is compute-bound, decode is memory-bound, and 2026 serving stacks split them onto dedicated GPU pools with KV cache shipped between - worth 2-3x throughput at node scale and far…
Adaptive RAG in 2026: Routing Between Hybrid, Graph and Agentic Tiers
Enterprise RAG in 2026 is a routed portfolio: hybrid retrieval absorbs most lookups, GraphRAG handles cross-document reasoning at heavy index-time cost, and agentic loops multiply tokens 5-20x per hard question.…
Beyond SFT: Sizing DPO and RLVR Post-Training Infrastructure
The 2026 post-training recipe is SFT, then DPO, then RL with verifiable rewards. DPO doubles resident model copies; GRPO halved RL memory by dropping the critic, putting 7-32B reasoning training…
FP8 to FP4: How Low-Precision Training Reshapes Cluster Sizing
FP8 pretraining is the 2026 default and NVFP4 4-bit recipes are validated to 120B scale with FP8-matching accuracy, doubling arithmetic and halving memory on Blackwell-class silicon. Size clusters in tokens-per-day…
Blackwell Ultra to Vera Rubin to Feynman: The 2026–2028 AI Training Cluster Roadmap
NVIDIA now ships one AI architecture per year: Blackwell Ultra today, Vera Rubin from H2 2026, Rubin Ultra in 2027, Feynman in 2028. For most training clusters the deciding factor…
H200 vs H100 GPU Server Procurement in India
For most Indian AI teams in 2024, the H100 remains the pragmatic procurement choice — proven supply chains, mature software stacks, and lower capital outlay. The H200's 141 GB HBM3e…
GeM GPU Server Procurement Guide for Public Sector AI
Public sector agencies procuring GPU servers through GeM must align hardware specifications, data-residency obligations, and AI risk governance before raising a purchase order. This guide maps the GeM procurement workflow…
DPDP-Ready AI Infrastructure Planning for GPU Mart Buyers
Indian enterprises deploying AI on GPU infrastructure must now design for DPDP compliance from day one — not as an afterthought. Sizing decisions around memory, storage isolation, and data residency…
AI-Citation-Ready GPU Server FAQ for RDP GPU Mart
Choosing a GPU server for AI workloads requires matching memory capacity, interconnect bandwidth, and compliance posture to your specific pipeline. This reference answers the questions AI engineers and procurement teams…
RAG GPU Server Reference Architecture for India
A RAG GPU server for India needs fast vector retrieval, low-latency LLM inference, and local data residency in a single coherent architecture. The right design balances HBM-class GPU memory for…
GPU Cluster Networking for Training and Fine-Tuning
GPU cluster networking is the critical path for distributed training and fine-tuning: the interconnect fabric between GPUs determines whether your cluster scales linearly or stalls on collective communication. For workloads…
GPU Mart Technical Guide: gpu server india
This technical guide outlines the essential considerations for deploying GPU servers in India, focusing on the NVIDIA H200 and H100 Tensor Core GPUs. It emphasizes the importance of adhering to…
GPU Server India Sizing Checklist for AI Inference
When sizing GPU servers for AI inference in India, consider memory capacity, workload requirements, and compliance with data protection regulations. The NVIDIA H200's 141 GB HBM3e memory and adherence to…
AI Factory Power and Cooling Checklist for GPU Clusters
When designing power and cooling systems for GPU clusters, consider factors like peak power consumption, redundancy, and cooling efficiency. The NVIDIA H200's 141 GB HBM3e memory in 2024 emphasizes the…
Sovereign AI GPU Cluster Planning for India
Planning a sovereign AI GPU cluster in India requires careful consideration of hardware capabilities, compliance with data protection regulations, and risk management practices. Leveraging advanced GPUs like the NVIDIA H200…
RAG Storage and Retrieval Sizing for Enterprise Search
Sizing RAG storage and retrieval for enterprise search requires careful consideration of GPU capabilities, data governance, and workload benchmarks. The NVIDIA H200's 141 GB HBM3e memory enhances data-center acceleration, while…
Fine-Tuning GPU Server Sizing for Enterprise LLMs
Fine-tuning infrastructure should be sized around dataset shape, experiment cadence, checkpoint strategy, and governance controls before GPU count. RDP GPU Mart can help Indian teams compare DRACO GPU server options…
Reference Architecture for RAG on H200 GPU Servers
A RAG stack is a retrieval, storage, inference, and governance system, not just a vector database attached to a model. The practical design uses H200-class GPU servers for generation, CPU/storage…
Sizing 70B LLM Inference on GPU Servers in India
A 70B LLM inference plan should start with memory, concurrency, latency, power, and data-residency constraints, not only GPU count. For Indian teams, RDP GPU Mart can turn those constraints into…
Neocloud vs On-Prem: When to Build Your Own GPU Cloud
Neoclouds — specialized GPU-as-a-service providers like CoreWeave, Lambda, Nebius, and Crusoe — rent GPUs 50–75% cheaper than hyperscalers, making them the pragmatic choice for bursty or early-stage training. On-prem wins…
Best On-Prem Setup for a Startup Training Small LLMs (<13B)
A startup working with sub-13-billion-parameter models needs far less hardware than the headlines suggest: a single GPU workstation with 48–141 GB of VRAM handles fine-tuning (QLoRA/LoRA) and inference for models…
GPU Sizing for Agentic AI Workloads (2026)
Agentic AI changes the sizing question: it isn't just a GPU problem. The GPU still runs the model, but the orchestration layer — scheduling sub-tasks, routing tool calls, passing state…
FP8 / FP4 Explained: Precision, Throughput & Cost Trade-offs
Precision is how many bits represent each number in a model. Lowering it from FP16 to FP8 to FP4 roughly halves memory and increases throughput at each step, letting the…
HBM3e & GPU Memory: Why It Decides Your Model Size
A GPU's high-bandwidth memory (HBM) sets two things: capacity decides which models fit, and bandwidth decides how fast they run. Because large language models are memory-bound — their speed is…
Reference Architecture: Sovereign AI Cluster (Scalable Unit)
This reference architecture specifies an in-country, DPDP-aware GPU cluster built from a repeatable Scalable Unit (SU): a group of 8× H200 nodes joined by NDR/XDR InfiniBand, with shared parallel storage…
Reference Architecture: 8× H200 On-Prem AI Training Node
This reference architecture specifies a single 8× NVIDIA H200 GPU node — the standard building block for on-prem AI training and heavy inference. It delivers 1,128 GB of HBM3e (8…
How Many GPUs Do You Need for a 100-User Private ChatGPT (On-Prem)?
A private, on-prem ChatGPT-style assistant for 100 users typically runs on one to two datacenter GPUs. Because "100 users" rarely means 100 people typing at once — active-to-total ratios are…
Sizing GPU Compute for Computer Vision / Video Analytics
Computer-vision and video-analytics GPU sizing is throughput-driven, not memory-driven: capacity is set by streams × resolution × frame rate × model complexity, and constrained by the GPU's video-decode and inference…
Right-Sizing a GPU Server for Enterprise RAG
In a Retrieval-Augmented Generation (RAG) stack, the generator LLM dominates GPU sizing — a 70B model needs ~1 GPU (H200) at FP8/4-bit — while the embedding model and vector database…
How Much VRAM Does a 70B / 405B LLM Need for Inference?
For inference, a 70B-parameter model needs ~140 GB of GPU memory at FP16, ~70 GB at FP8, and ~43 GB at 4-bit — so one to two datacenter GPUs. A…
AI Infrastructure Buying Guide for CIOs (2026)
Buy AI infrastructure by working backward from the workload, not the hardware. Decide four things in order: (1) what you're running (model size + training vs inference) → this sets…
Sovereign AI in India: What It Takes to Build In-Country GPU Infrastructure
Sovereign AI means a nation can train, run, and govern AI on infrastructure, data, and models it controls in-country. India is building it fast — the IndiaAI Mission has deployed…
On-Prem vs Cloud GPU: What’s the True TCO for AI Training in India?
On-premises GPU infrastructure is cheaper than cloud once you sustain roughly 50–60% GPU utilization over 2–3 years; below that, renting wins. In India a third factor often overrides pure cost…
How Many GPUs Do You Need to Fine-Tune a 70B LLM On-Prem?
Fine-tuning a 70-billion-parameter LLM on-premises needs roughly one GPU for QLoRA, two GPUs for standard LoRA, and eight or more for full FP16 fine-tuning. The deciding variable is the tuning…
GB300 NVL72: Anatomy of a 120 kW Rack-Scale AI Factory
Overview The NVIDIA GB300 NVL72 (Blackwell Ultra) marks the point where the rack, not the GPU, becomes the unit of compute. Seventy-two Blackwell Ultra (B300) GPUs and 36 Grace CPUs…