Model and Data Governance On-Prem: Registries, Lineage and Approval Gates
Overview
An on-premises GPU cluster removes the cloud provider from your compliance story, but it also removes the governance scaffolding cloud platforms bundle in. Nothing on a bare Kubernetes-plus-Slurm stack stops an engineer training on an unapproved dataset and pushing weights straight to production. Governance on owned infrastructure therefore has to be built deliberately: a model registry as the single source of truth, dataset lineage that survives personnel changes, approval gates wired into pipelines rather than wikis, and a considered position on what the DPDP Act’s erasure right means for models already trained on personal data. This article describes a workable minimum for Indian enterprises – concrete enough to implement, honest about the hard parts.
Key takeaways
- On-prem AI platforms inherit no governance by default – registry, lineage and gates must be designed in, and they cost far less at build time than at audit time.
- The model registry should be the only path to production: every deployed model traceable to a registered version with evaluation results and a named approver.
- Dataset lineage – source, consent basis, transformations, version hashes – is the record that answers both regulator and customer questions about what trained a model.
- Approval gates work when they are pipeline steps that block promotion, not documents; each gate should emit its own evidence record.
- Erasure of personal data already trained into weights has no clean technical answer today: the defensible options are documented risk assessment, retraining cycles, output filtering, and honest retention design upstream.
Why governance is an infrastructure problem
Policy documents do not stop an ungoverned deployment; infrastructure does. If the serving layer will load any checkpoint from a shared filesystem, governance is aspirational. If it will only pull signed artefacts from a registry, governance is enforced. The same logic runs through DPDP compliance: the Act expects fiduciaries to evidence safeguards and accountability, and Significant Data Fiduciaries must verify that algorithmic systems do not endanger individuals’ rights – obligations discussed in The DPDP Act for AI Infrastructure. For banks, the RBI’s FREE-AI framework points the same direction, with model risk expectations examined in FREE-AI and Model Risk Rules.
The model registry as single source of truth
A registry entry per model version should hold: the weights artefact and its hash; the training run reference (code commit, container image digest, hyperparameters, job ID); the dataset versions consumed; evaluation results against agreed benchmarks, including safety and bias checks where relevant; the approval record with named approver and date; and current lifecycle state – registered, approved, deployed, retired. Open-source options such as MLflow run comfortably on the same cluster. The rule that gives the registry teeth is simple: serving infrastructure authenticates to the registry and refuses artefacts that are not in an approved state. One rule, enforced in the deployment path, converts the registry from documentation into control.
Dataset lineage and the consent trail
Datasets need the same discipline as code. Version every training and fine-tuning set with a content hash; record its sources and the legal basis for each – consent, legitimate use, licensed corpus, synthetic; log the transformations from raw to training-ready, especially de-identification steps; and record which model versions consumed which dataset versions. This last mapping is the pivot for erasure requests and for questions like “was our data used to train this?” from enterprise customers. Storage-level access logs complete the picture, as covered in Audit Evidence for AI Systems.
Approval gates that generate their own evidence
Gates fail as meetings and succeed as pipeline steps. A pragmatic ladder: dataset admission (legal basis and de-identification verified before data enters the training tier); training authorisation (job templates only accept admitted dataset versions); evaluation gate (promotion blocked until benchmark and safety thresholds pass); deployment approval (a human sign-off recorded in the registry for material models); and change review for prompt templates and RAG sources, which alter behaviour as surely as weights do. Right-size the ceremony to model materiality – a customer-facing credit model deserves gates a internal code-search assistant does not – but let every gate write its own record.
Right to erasure and trained models, honestly
The DPDP Act requires erasure of personal data when consent is withdrawn or purpose is served. Databases and object stores can comply cleanly. Model weights cannot: machine unlearning remains an active research field, and no production-grade technique reliably removes one individual’s influence from a large model without retraining. The defensible posture today is layered: minimise personal data entering training at all through de-identification; keep the dataset-to-model mapping so affected versions are identifiable; erase the source data and remove it from future training sets immediately; assess and document the residual risk of the trained model (memorisation likelihood, exposure surface); apply output filtering where extraction risk is real; and fold removal into the next scheduled retraining, retiring old versions on a defined cycle. State this position in your privacy documentation rather than implying erasure from weights is instant – regulators respond better to documented honesty than to quiet impossibility. This reflects general practice, not legal advice on any specific request.
Governance artefacts at a glance
| Artefact | What it records | Who consumes it |
|---|---|---|
| Model registry entry | Weights hash, run reference, evaluations, approver, lifecycle state | Auditors, serving infrastructure, model risk teams |
| Dataset version record | Content hash, sources, legal basis, transformations | DPO, auditors, data engineering |
| Dataset-to-model map | Which dataset versions trained which model versions | Erasure workflow, customer assurance |
| Gate records | Who approved what, when, against which criteria | Auditors, internal review |
| Erasure decision log | Request, affected versions, remedy chosen, residual risk | DPO, Data Protection Board if queried |
Frequently asked questions
Do we need commercial governance software?
Not to start. MLflow or an equivalent registry, versioned object storage, and CI-enforced gates cover the core. Buy tooling when scale or multi-team coordination makes the assembled version expensive to maintain.
Who should own model governance?
A joint owner model works best: platform engineering owns enforcement (registry, gates), while the DPO or risk function owns criteria. Governance owned solely by either side tends to become unenforced policy or unusable friction.
Does a fine-tuned open-weights model need registry treatment?
Yes. The moment your data shapes the weights, lineage and approval questions apply. Record the base model, its license, the fine-tuning dataset versions and evaluations like any other version.
Can we really not delete a person’s data from a model?
Selective removal from trained weights is not reliably achievable in production today. The workable remedy is erasing source data, documenting affected model versions, filtering outputs where needed, and retraining on a defined cycle.
How does this interact with RBI expectations for banks?
The FREE-AI framework and model risk guidance point to board-level accountability, inventories and lifecycle controls for AI models. A registry-and-gates platform produces exactly the records those reviews request, as published in RBI’s framework.
Ready to deploy?
Talk to an RDP architect about power, cooling and lead time.