Prefill on GPUs, decode on cards
MiTAC Computing Technology Corporation, a subsidiary of MiTAC Holdings, is collaborating with Skymizer on an inference architecture that assigns compute-intensive prefill to GPUs and memory-intensive decode to Skymizer's HTX301 accelerator cards. The company said the arrangement is intended to let enterprises provision each stage for its workload while keeping model inference on premises. Rick Hwang, president of MiTAC Computing Technology, said scaling agentic AI needs cohesive infrastructure rather than fragmented systems.
An 8U prefill box, a 4U decode box
The proposed prefill stage is an 8U MiTAC G8825Z5 with AMD EPYC 9575F or 9755 processors and AMD Instinct MI350X GPUs, which transfers the Key-Value cache to the decode server over high-speed networking. The decode stage is an air-cooled 4U MiTAC G4520G6 with Intel Xeon 6 processors and eight Skymizer HTX301-4S cards, each with 256 GB of memory, for 2 TB of accelerator memory across eight PCIe slots. MiTAC said the decode server is designed to run large models such as the 2.8-trillion-parameter Kimi K3 on premises.
Racks, cooling and software
MiTAC said it is also working with AMD on the Helion Agentic AI Rack, an ORv3 44OU system intended to scale to 32,768 AMD Zen 6 CPU cores. At SC26 it is showing a 52U liquid-cooled rack of 12 G4826Z5 servers with 96 AMD Instinct MI355X GPUs, which it said achieves 50% higher GPU density than conventional deployments, and a ZutaCore proof of concept using waterless two-phase direct liquid cooling that supports thermal design power above 4,000W and a heat flux over 250W/cm², at an annualized pPUE of 1.03. Its software layer includes Canonical's Ubuntu operating system, DDN's Infinia platform, Rafay's GPU-as-a-Service orchestration and MiTAC's MiCoreView POD management software, the company said.
Booth 4615, Nov. 17-19
MiTAC is exhibiting at Booth 4615 at McCormick Place in Chicago from Nov. 17 to 19.