Here’s a question that should keep every enterprise AI leader awake at night: If we’ve solved software delivery with containers and Kubernetes, why are we still manually copying gigabyte-sized model weights between storage buckets like it’s a USB stick operation from 2003?
The answer isn’t technical incompetence. It’s institutional lag. AI model artifact management has become the forgotten stepchild of cloud-native infrastructure—and the cost of that neglect compounds every quarter as models get bigger, deployments get more distributed, and the stakes climb higher.
The Gap That Breaks Silently Until It Breaks Everything
Most organizations operate AI infrastructure on Kubernetes. Yet their model delivery pipeline looks like this: Someone downloads a 140 GB quantized LLaMA-3 checkpoint from Hugging Face. They wait (usually with poor bandwidth). They manually transfer it to an S3 bucket. Then they paste a download URL into a YAML file and hope the inference pod can fetch it before a timeout.
Meanwhile, their software delivery pipeline is running laps around them. Application containers? They’re pulled from OCI registries with full versioning, cryptographic signing, security scanning, and one-click rollback. Model weights? They’re basically distributed via ad hoc scripts, copied between buckets with zero audit trails, and stored on unsecured filesystems.
“Most existing ML model storage approaches were not designed with Kubernetes-native delivery in mind, leaving a critical gap between how software artifacts are managed and how model artifacts are managed.”
This gap isn’t theoretical. It creates deployment fragility that nobody wants to talk about until 2 AM when a model rollout fails across five regions simultaneously. It introduces security risks that auditors lose sleep over. It multiplies operational overhead at scales that make sense only when you’re managing dozens of models across dozens of teams.
When Model Weights Exceed Your Entire Application
Let’s anchor this to real numbers, because vagueness is how infrastructure debt sneaks past leadership.
A single LLaMA-3 70B checkpoint: 140 GB. A multimodal frontier model from an advanced lab? Easily exceeding 1 TB. These aren’t files you version-control with standard Git. They’re not small enough to ignore network efficiency. They’re not static enough to treat as immutable artifacts without careful, reproducible tracking.
When your model weighs 1,000 times more than your inference engine, the infrastructure can’t be an afterthought. You need:
Storage at scale without breaking budgets. Distribution speed that doesn’t require waiting hours for a single model pull. Reproducibility guarantees—the ability to trace any production inference back to the exact, immutable artifact it ran against.
Three traditional approaches exist. All three fail at enterprise scale.
Git LFS (Hugging Face Hub) gives you native version control—branches, tags, commits, full lineage. But it was never designed for petabyte-scale P2P distribution or cloud-native environments. The transport layer inherits Git’s inefficiencies, and it shows.
Object storage (S3, MinIO) is the cloud provider’s native answer. It works. It’s available everywhere. Inference engines like vLLM have built-in S3 support. But object storage has a fatal weakness: it has no notion of versioning, no structured metadata, no security scanning, no supply chain provenance. You’re managing model artifacts the way we managed dependencies in 2008—by filename convention and prayer.
Distributed filesystems (NFS, CephFS) promise POSIX compatibility and low integration cost. They’re also organizational nightmares. High operational complexity. No versioning. No metadata. NFS at massive scale becomes a performance bottleneck that requires deep infrastructure expertise most teams don’t have.
None of these paths forward are sufficient because none of them treat model weights as first-class cloud-native artifacts.
The Answer Was Always There—We Just Didn’t Want to See It
What if we shipped models the same way we ship application code?
The analogy isn’t perfect. But it’s closer than anyone’s admitted.
When you deploy an application container in 2024, here’s what actually happens behind the scenes:
Developers commit code to a Git repository, managing changes through branches. At stable milestones, they tag versions. CI/CD pipelines compile, test, and validate. The output: an immutable container image. That image gets pushed to a container registry—which handles versioning, security scanning, signing, RBAC, and cryptographic provenance.
When a Kubernetes cluster needs it, the kubelet pulls the image from the registry. The image is already scanned for vulnerabilities. Its provenance is cryptographically signed. You can roll back to any previous version in seconds. You can audit exactly who deployed what, when, and why.
Model delivery should follow the same pattern:
Algorithm engineers push weights and configs to Hugging Face Hub, managing versions through commits and tags (treating it as the source-of-truth repository). CI/CD pipelines package weights, runtime configurations, and metadata into a single, immutable model artifact. That artifact gets stored in an artifact registry—the same OCI registry infrastructure already managing your application containers. Versioning, scanning, signing, all inherited from the container ecosystem.
When a Kubernetes cluster needs the model, it pulls it as an OCI artifact through a Model CSI Driver. The model mounts as a volume inside the inference container (vLLM, SGLang, whatever). The AI model is decoupled from the inference engine. You can swap engines without re-downloading the model. You can rollback, audit, trace provenance—all the practices that made container delivery reliable.
The infrastructure already exists. CNCF projects like ORAS, Harbor, and Dragonfly are actively building this. OCI already defined artifact specifications that go beyond containers. Kubernetes already supports arbitrary artifact pulling through CSI drivers.
What’s missing is adoption.
Why This Matters (And Why It Won’t Happen Fast)
This isn’t a small infrastructure optimization. This is the difference between managing AI at startup scale and managing AI at enterprise scale.
Small teams can keep shipping models through S3 scripts. The friction is annoying. The lack of versioning stings. But it doesn’t kill them. Enterprises trying to run 50 models across multiple regions? They’re hemorrhaging operational debt.
And here’s the hard truth: container orchestration took a decade to become standard, even though its benefits were obvious from 2015 onward. Model artifact management will take just as long, even though we’ve already solved the hard problems in the container ecosystem.
Why? Because it requires coordination between three separate communities: MLOps teams (who’ve been trained to think object storage is the answer), platform engineers (who own Kubernetes but don’t fully own ML infrastructure), and AI researchers (who’ve standardized on Hugging Face Hub and see it as separate from deployment).
Unifying these communities around OCI artifact registries is the right call. It’s overdue. But it requires change at every layer of the stack, and institutional change moves like continental drift.
🧬 Related Insights
- Read more: Why Open Source Contributions Aren’t Charity—They’re a $2.6 Trillion Business Move
- Read more: Running LLMs on Kubernetes? Your Infrastructure Doesn’t Protect You From Prompt Injection
Frequently Asked Questions
What does treating models as OCI artifacts actually do? It applies the same versioning, security scanning, cryptographic signing, and rollback capabilities that software containers already have to AI model weights. Instead of managing models through ad hoc scripts and S3 URLs, you manage them through the same container registry infrastructure you already own.
Will this replace my current model storage setup? Not immediately. But within 3-5 years, OCI artifact registries will become the standard for enterprise model distribution. Teams still using object storage scripts will have to migrate—the sooner you start, the less technical debt you’ll carry into the next cycle.
Why haven’t cloud providers pushed this harder? They have, but quietly. AWS, Google Cloud, and Azure all support OCI artifact registries. The real bottleneck is adoption velocity. Most teams don’t know this option exists, and education moves slowly when the status quo (object storage) isn’t causing catastrophic failures—yet.
