Reports say the AI semiconductor field is signaling an architectural shift. Groq, known for high-speed SRAM designs and recently effectively acquired by Nvidia, is evaluating 3D-stacked DRAM packaging for its next-gen chips to break past traditional SRAM capacity limits.
Most AI chips use 2D packaging; Groq plans 3D packaging, vertically stacking multiple dies under a scheme called 3DS. The core uses TSV through-silicon via technology to drill thousands of micro-holes in DRAM chips for vertical interconnection, delivering ultra-high memory density per unit area. Unlike HBM used in Nvidia's high-end GPUs—both use TSV stacking, but HBM pursues extreme data exchange rates, while 3DS keeps existing DRAM standard interface protocols and focuses on capacity expansion. For AI inference tasks that must load massive parameters, this is a more practical cost-efficiency approach.
Groq's LPU architecture has long drawn attention for extremely high inference speed, but its SRAM capacity is tiny, struggling with hundred-billion-parameter models on a single chip. DRAM native read/write speed is slightly below SRAM, but its large capacity is more advantageous for complex model deployment. Through 3D stacking, Groq aims to keep its speed edge while solving the capacity pain point, paving the way for ultra-large AI model inference.
ICgoodFind Summary: Groq's shift from SRAM to 3D-stacked DRAM is essentially rebalancing inference speed and model capacity. 3DS prioritizes capacity over bandwidth, creating a differentiated path from HBM—showing AI chip memory solutions are moving from single-minded speed pursuit to scenario-based tiering.
