MTIA 300 Cuts Recommendation Model Comms Time 3.9x vs GPU

Meta has detailed MTIA 300, its first in-house accelerator built specifically for training ranking and recommendation models rather than large-language-model workloads. The distinction matters architecturally: recommendation models store more than 99% of their parameters in embedding tables, and distributing those tables across hundreds of accelerators generates dense AllReduce, AllToAll, and AllGather traffic that can make the interconnect the true system bottleneck.

Network integration as a first-class constraint

Rather than connecting NICs through a PCIe bus, Meta moved networking directly into the MTIA 300 package via two network chiplets. Each chiplet carries six custom 800 Gbps RDMA NICs, giving the chip twelve NICs in total and 1.2 TB/s of aggregate I/O bandwidth without a PCIe hop. The same twelve NICs handle both scale-up traffic within a rack and scale-out traffic between racks, and the bandwidth split between those two domains can be reconfigured without a hardware revision.

On a conventional GPU cluster, collective communication and matrix computation compete for the same processing resources. Meta measured more than 20% throughput degradation on GPU hardware when those two workloads overlapped. MTIA 300 eliminates that coupling with 16 dedicated message engines that handle collective operations independently of the main compute grid, including near-memory hardware for reduction operations. Meta reports that large matrix operations and collective communication can run concurrently with less than 0.5% degradation in compute throughput — a roughly 40× improvement over the GPU baseline.

HCCL: communication compiled into the chip

The hardware was co-designed with HCCL, Meta's collective-communication library. Rather than requiring the host CPU to orchestrate each collective operation at runtime, HCCL compiles collective operations into subgraphs that the chip's message engines execute autonomously. Once those instructions are loaded onto the accelerator, the host is no longer in the critical path during a training step. In production, Meta reports HCCL achieves up to 940 GB/s of intra-rack communication bandwidth on MTIA 300.

Measured results

On a 150-billion-parameter recommendation model distributed across 40 accelerators, MTIA 300 reduced total communication time by 3.9× compared with an equivalent GPU cluster — the combined effect of on-package NICs eliminating PCIe latency, dedicated message engines preventing compute interference, and HCCL's autonomous collective execution reducing host-side overhead.

Fleet context and roadmap

Meta already operates hundreds of thousands of MTIA accelerators for inference workloads; MTIA 300 extends that programme into training. The company has announced four additional generations planned over the next two years, covering ranking, recommendation, and generative-AI workloads, and has expanded a co-development partnership with Broadcom for future MTIA generations while continuing to source accelerators from AMD and NVIDIA as part of an explicit portfolio strategy.

That strategy reflects a wider shift among hyperscalers toward workload-specific AI silicon. Google continues expanding TPU generations, Amazon has grown its Trainium programme, and Microsoft is iterating on Maia — all driven by the same pressure to reduce dependence on general-purpose GPU procurement. The competitive dynamics around custom silicon and third-party supply chains make that trajectory durable.

What distinguishes MTIA 300 is the depth of co-design across compute silicon, on-package networking, and the collective-communication stack. Treating the NIC as a chiplet rather than a peripheral, and compiling communication patterns into autonomous accelerator subgraphs, reflects a systems-level architectural philosophy where the network is not infrastructure wrapped around the chip — it is part of the chip.