The ALCF is deploying a scalable, on-premises inference framework to provide DOE researchers with direct access to foundation models for high-performance computing.
The Argonne Leadership Computing Facility has officially inaugurated a dedicated AI inference service designed to integrate large foundation models directly into high-performance computing research environments. This infrastructure allows scientists across the Department of Energy National Labs and the Genesis Mission to execute batch and interactive queries against a diverse library of open-weights models.
The service currently supports approximately 35 distinct models, including the Google Gemma series, Meta LLaMA architectures, and the OpenAI GPT-OSS family. Researchers can also deploy specialized domain-specific models, computer vision frameworks, and internal developments such as the AuroraGPT model. The system architecture leverages two primary supercomputing clusters, Sophia and Metis, to handle the computational load of these varied inference tasks.
Sophia operates on an Nvidia DGX-based system equipped with eight A100 GPUs, delivering 3.9 petaflops of FP64 performance. Metis functions as a SambaNova SN40L cluster, utilizing over 1,000 AI accelerators across 16 nodes to provide 637.5 teraFLOPS of BF16 performance. This heterogeneous hardware approach allows the facility to balance different precision requirements for scientific simulation and data analysis.
Michael Papka, director of the Argonne Leadership Computing Facility, noted that the service operates on a first-come, first-served basis for authenticated users. The facility has already processed approximately 26 billion tokens for 450 active users during the initial testing phase. The current implementation avoids aggressive throttling, as the team prioritizes identifying system bottlenecks under real-world scientific workloads.
The underlying framework is based on the Federated Inference Resource Scheduling Toolkit, or FIRST, which was detailed in a 2025 technical paper by researchers from Argonne and the University of Chicago. This toolkit integrates an Inference Gateway API, Globus Compute for task execution, and specialized model serving tools. The architecture is designed to maintain data sovereignty by keeping inference tasks on-premises rather than relying on commercial cloud providers.
Managing concurrent access remains a significant engineering challenge, particularly as researchers push for larger context windows that strain available memory. The facility handles traffic by dynamically shifting workloads between Nvidia GPUs and SambaNova XPUs, though this can lead to temporary unavailability of specific models. Future upgrades, including the integration of the upcoming Minerva and Tara supercomputers, are expected to incorporate advanced Nvidia software infrastructure to improve performance without altering the user experience.
The integration of AI inference into traditional scientific simulations creates an iterative feedback loop that may significantly accelerate discovery. By allowing researchers to query LLMs during active simulations, the facility enables real-time data analysis and model adjustment. This service-oriented approach represents a shift toward treating AI inference as a standard utility, similar to traditional HPC resource allocation.
The broader impact of this initiative lies in its ability to provide a secure, private environment for sensitive scientific data. By leveraging Globus authentication, the system allows researchers from institutions like Brookhaven National Laboratory and Pacific Northwest National Laboratory to access resources without requiring local ALCF credentials. This federated access model is critical for the Genesis Mission and the Transformational AI Models Consortium.
The facility is currently refining its scheduling logic to accommodate the distinct needs of batch versus interactive workloads. As the load increases, the team intends to use performance data to lobby for additional resources, viewing system degradation as a necessary indicator of scientific demand. The long-term goal is to establish a reliable, dial-tone-style inference service that supports the complex, long-running simulations characteristic of modern computational science.
Future development will focus on optimizing the interaction between traditional HPC simulations and AI-driven analysis. The facility plans to monitor how researchers adapt their workflows to these new capabilities, particularly as they integrate larger context windows into their research. These efforts will determine how effectively the laboratory can scale its AI infrastructure to meet the evolving requirements of the national scientific community.
