d-Matrix Raptor Architecture Reaches 100 TB/s Bandwidth via 3D DRAM Integration

The Santa Clara-based startup d-Matrix introduced its Raptor accelerator architecture at Hot Chips 2026, demonstrating a novel approach to the memory wall that has long constrained large language model inference. The system achieves a memory bandwidth exceeding 100 terabytes per second by bonding a logic die directly to a custom DRAM die using a 36-micron pitch vertical microbump array.

This integration strategy eliminates the need for a traditional silicon interposer, which typically serves as the primary source of power consumption in high-bandwidth memory configurations. According to company data, the Raptor architecture operates at 0.37 picojoules per bit, representing a six-fold reduction in energy expenditure compared to standard HBM3 implementations.

The design utilizes TSMC’s 4-nanometer N4 process for the logic layer, which is positioned atop the DRAM to facilitate direct liquid cooling. By maintaining a power density below 0.5 watts per square millimeter, the configuration keeps the DRAM junction temperature within operational limits despite the high-performance throughput.

To address the inherent inefficiencies of data movement, d-Matrix implemented a stream-blocking mechanism that aligns DRAM access patterns with tensor engine requirements. This process aggregates four 96-byte DRAM accesses to form three 128-byte flits, effectively eliminating the bandwidth waste associated with overfetching.

The architecture also incorporates a stream-flipping technique to reduce bit transitions on the microbump interface. By comparing consecutive flits and inverting bits when necessary, the system achieves a 20% reduction in I/O power consumption without requiring additional burst window hardware.

Reliability concerns stemming from elevated junction temperatures are mitigated through a specialized microbank design and multi-layered error correction. Each bank is limited to 5.33 megabytes, which restricts the bandwidth penalty of frequent charge refreshes to 1.37%, while a Reed-Solomon T=2 code on the logic die addresses soft-error rates.

Manufacturing yields are protected by an integrated redundancy scheme featuring 72 spare DRAM banks per chiplet. A two-level multiplexer chain allows the system to bypass defective banks while maintaining symmetric channel width and bandwidth across the entire array.

The shift from interposer-based designs to direct vertical bonding represents a fundamental change in how memory-intensive compute tasks are handled at the silicon level. By reducing the physical distance between logic and memory, d-Matrix addresses the latency and energy penalties that have historically hindered the deployment of trillion-parameter models in inference environments.

The ability to sustain 100 terabytes per second within a 32-gigabyte capacity envelope suggests a potential shift in rack-scale infrastructure for frontier models. This density allows for the processing of massive key-value caches required by long-context windows, which currently force GPU clusters into inefficient configurations.

Industry observers note that the Raptor architecture directly challenges the scaling limits of current HBM4 standards. While HBM4 remains the industry benchmark for capacity, the power-per-bit requirements of interposer-based PHYs create a physical ceiling that may necessitate alternative integration paths for future AI hardware.

The architecture delivers 13.5 times better power efficiency when measured in milliwatts per gigabyte per second compared to NVIDIA’s Rubin platform. Comparative performance metrics provided by d-Matrix indicate that Raptor achieves 20 times higher bandwidth density, suggesting that direct-bonded DRAM could redefine the power envelope for future inference clusters.

Thermal management remains a critical constraint, as the logic-on-top orientation requires precise liquid-cooling contact to prevent heat soak into the DRAM. By isolating the logic die, d-Matrix ensures that the DRAM junction temperature remains stable at 105 degrees Celsius, preventing the rapid charge loss that would otherwise necessitate more frequent and bandwidth-draining refresh cycles.

Future performance benchmarks will determine if the Raptor architecture maintains its efficiency advantages outside of controlled laboratory environments. The company intends to scale these chiplets into full-rack configurations, aiming to support three-trillion-parameter models like GLM 5.2 and Kimi K3 with improved power-to-performance ratios.