Microsoft will begin deploying its Maia 300 AI accelerator in 2027 and plans to scale the Maia family across multiple gigawatts of data-center capacity. The company expects Maia to become the primary inference system behind its own products and services, moving a larger share of Copilot, Teams, GitHub and Microsoft Foundry workloads onto hardware designed in-house.
The announcement is more consequential than a routine chip refresh. Microsoft is building Maia 300 around the part of generative AI that happens after a model is trained: serving answers, processing long prompts and sustaining multi-step agent jobs. At the planned scale, even modest improvements in tokens delivered per watt can change how much AI capacity Microsoft can install without adding equivalent power and cooling.
Microsoft claims Maia 300 will deliver 40% more tokens per megawatt than “leading GPU-based systems” and up to seven times the rack-level token throughput of Maia 200. Those figures are projections rather than independent benchmark results. The company did not identify the GPU systems, models, precision settings, latency targets or utilization levels behind the comparison, so buyers cannot yet reproduce it.
Maia 300 specifications
Each Maia 300 accelerator pairs a chiplet-based processor with eight HBM4 memory stacks. Microsoft lists 288 GB of memory, 18 TB/s of peak memory bandwidth and a 950-watt thermal design power. Compared with Maia 200, that represents 33% more memory capacity and 2.5 times the bandwidth, while the claimed HBM bandwidth per watt roughly doubles.
The compute chiplets use TSMC’s N2P process, while other chiplets handle memory control and I/O. Reuters reported that those supporting components use a less expensive 3-nanometer process. Splitting the design lets Microsoft reserve the newest manufacturing node for the logic most likely to benefit from it, instead of fabricating every function on the costliest process.
Maia 300 also supports enhanced MXFP4, a low-precision number format intended to reduce the memory and bandwidth required for model weights and key-value cache. Its on-die SRAM can act as a software-managed scratchpad, keeping frequently used data closer to the compute units.
That emphasis reflects a basic constraint in large-model inference: arithmetic units are useful only when weights and cached context reach them fast enough. Long-context prompts and the token-by-token decode phase are often limited by memory movement rather than raw math throughput. Microsoft’s software-defined dataflow architecture schedules compute, memory and network operations as one data path, with AI-assisted kernel optimization intended to reduce unnecessary transfers.
A 96-accelerator rack inside a 167 kW envelope
A full Maia 300 rack contains 96 accelerators across 24 four-accelerator trays. Together they provide 27.6 TB of HBM capacity and 1.7 petabytes per second of aggregate peak memory bandwidth inside a 167-kilowatt rack power envelope. Maia 200 racks carry 48 accelerators, so Microsoft is doubling chip density while integrating host processors, memory and storage into each tray.
For scale-up networking, six high-radix switch ASICs connect the accelerators inside the rack. Microsoft’s Ethernet-based transport can also extend the same programming model across racks, with as much as 750 GB/s of cross-rack bandwidth per accelerator when only part of a system is communicating outside its rack.
The topology supports several practical serving patterns. Tensor-parallel jobs can stay within one 96-accelerator rack, while expert or pipeline parallelism can span racks. Prefill and decode workers can also be separated: after a prompt is processed, its key-value state moves directly to selected decode workers without first taking a separate host-mediated path. Spare accelerator trays in another rack can cover failed hardware while the job remains on the scale-up fabric.
Cooling is split between liquid and air. Direct liquid cooling removes more than 80% of the rack’s thermal load from accelerators, voltage regulators and switch silicon. Air handles memory, storage and supporting components. Microsoft says a rack can connect to facility water or use a heat exchanger in an air-cooled data center, which gives the company more options for retrofitting existing sites.
What Microsoft’s efficiency claims do and do not show
The comparison that will attract the most attention is Microsoft’s claim of 40% more tokens per megawatt than leading GPU systems. It is a system-level measure, not a claim that one Maia 300 chip is 40% faster than one Nvidia or AMD accelerator. Model choice, quantization, batch size, response latency, network overhead and utilization can all change the result.
Reuters noted that a 96-chip Maia 300 rack uses about 167 kW, while Nvidia’s 72-GPU GB300 rack uses roughly 142 kW. Those numbers should not be read as a direct efficiency ranking. The racks contain different accelerator counts and architectures, and Microsoft has not published matched workload results. The relevant question is how many useful tokens each complete system can deliver at an acceptable latency and accuracy for a given amount of electricity.
Microsoft has disclosed no Maia 300 price and has not announced general-purpose Azure instances that customers can rent. The initial strategy is internal: Maia 200 is already serving Microsoft Copilot and the company’s MAI model work, with GitHub Copilot and Microsoft Foundry due to follow by the end of 2026. Maia 300 production deployments start next year.
Why the 2027 rollout matters
Microsoft told Reuters it could deploy more than one million Maia 300 chips in 2027. The company’s public wording is more cautious, promising multi-gigawatt deployments without committing to a unit count. Either version points to infrastructure measured at a scale where power delivery, HBM supply, advanced packaging and TSMC capacity become as important as chip design.
The plan does not mean Microsoft is abandoning Nvidia. Azure still needs a heterogeneous fleet to support customer-chosen models and software stacks, and Nvidia’s CUDA ecosystem remains a major advantage for general-purpose workloads. Maia gives Microsoft a second path for high-volume services it controls, where it can tune models, kernels, networking and rack operations together.
For Azure customers, the immediate watchpoints are concrete: whether Microsoft exposes Maia 300 through Foundry, which models and precision formats it supports, how it prices tokens or instances, and whether production latency and availability match the efficiency claims. Until those details arrive, Maia 300 is best understood as a large internal infrastructure commitment with unusually detailed hardware specifications, not yet as a GPU alternative that outside developers can buy.
