Private LLM (air-gapped)
On-network large language model with zero external egress for PHI-adjacent work.
Problem
Staff wanted LLM help with documents and code, but hospital policy forbids sending anything sensitive to public AI services.
Approach
Stood up a Linux server inside the network, containerised Ollama and Open-WebUI, and served open-weight models with no internet route. Documented the PHI boundary and who may use it for what.
Architecture
Docker Compose stack; model weights pulled once then locked; per-user auth in Open-WebUI; no outbound connectivity.
By the numbers
Outcome
Secure, self-hosted AI available to approved staff; foundation for later RAG work over internal documents.
What I would tell your team
Ask me how this maps onto your EHR, your interface engine and your security review. I can walk through the data flow, the failure modes we hit at go-live, and what I would do differently the second time.
Talk about a similar system