Guest post by Zach Deane-Mayer @ AI Insight Solutions
The frontier labs sell turnkey AI, priced by the token. The lab sets that price, picks the deprecation schedule, and can revoke your access at any time. If your business depends on AI, you want to own the stack your AI runs on. That’s sovereign AI.
Building a sovereign AI stack yourself is daunting: you need GPUs, drivers, system software, an inference engine, and a frontend. Even the strongest engineering teams can struggle to deliver a stack this complicated.
However, you don’t have to go it alone. RLC Pro AI (Rocky Linux from CIQ) now ships as a batteries-included container image: the drivers, the system dependencies, and the software for an inference backend. A neo cloud (a serverless GPU cloud) ships the GPUs. You grab the pre-baked image and run it wherever you choose. There is no stack to assemble and nothing to install.
Combine them and you have a sovereign AI stack in three steps:
- Pick a model.
- Bake the weights into an image built from the RLC Pro AI base image.
- Deploy the image to a neo cloud of your choice: this is your inference backend.
For a model, I typically start with the Gemma family: it’s small enough to run on a single GPU and smart enough to provide real value in production. Ollama serves the weights from inside the image, so inference is fully offline, and you’re off to the races with an OpenAI-compatible endpoint (e.g., you could connect it directly to your coding agents).
The whole backend for this setup is about 40 lines of code, and the GPU scales to zero when idle. If you want a chat interface, deploy Open WebUI as a second container and point it at your backend container:
This stack runs on rented hardware. The neo cloud supplies compute; everything else is yours:
- You own the model: the weights live in an image you built.
- You own the runtime: no vendor other than the neo cloud sits in the request path.
- You own the data: chat history lives in a data store you run.
- You are not locked in: move your containers to a different cloud whenever you want.
If you want to try this yourself, you can follow the step-by-step tutorial to deploy the backend or read the detailed design document. Sovereign inference used to be a whole build process. Now it's a simple pull and deploy.