Abstract
As large language models become more integrated into research, teaching, and administration, universities must decide which AI workloads belong in the cloud and which require stronger institutional control. This is not an either-or choice: an effective AI strategy combines cloud and on-premises systems to preserve AI sovereignty where it matters most. In this context, AI sovereignty means retaining control over infrastructure, data, models, access policies, guardrails, monitoring, and governance so that sensitive or strategic workloads remain aligned with institutional requirements while enabling transparent energy accounting and lower costs at scale.
This paper presents MindRouter, an open-source LLM inference gateway designed to help universities operate self-hosted AI services on institutional GPU infrastructure as part of a balanced AI strategy. MindRouter is designed to support sensitive workloads involving FERPA-regulated student data, controlled unclassified information (CUI), export-controlled data, and institutionally managed intellectual property (IP). The system provides a unified API layer that translates among OpenAI, Ollama, and Anthropic client protocols, routes requests across heterogeneous GPU backends using a Weighted Deficit Round Robin (WDRR) fair-share scheduler, and incorporates role-based access control, audit logging, configurable guardrails, and operational observability through real-time GPU telemetry and system-level monitoring. Deployed in production at the University of Idaho on a cluster of 40 GPUs serving more than 64 open-weight models, MindRouter shows that mid-sized institutions can operate institutionally managed AI services while retaining direct control over data, infrastructure, and operational policy. MindRouter is released under the Apache 2.0 license and is available at MindRouter.ai