[GUEST ACCESS MODE: Data is scrambled or limited to provide examples. Make requests using your API key to unlock full data. Check https://lunarcrush.ai/auth for authentication information.]

![SemiAnalysis_ Avatar](https://lunarcrush.com/gi/w:24/cr:twitter::1745106082790318080.png) SemiAnalysis [@SemiAnalysis_](/creator/twitter/SemiAnalysis_) on x 27.4K followers
Created: 2025-07-24 02:21:55 UTC

Although "baking transformers in silicon" may sound cool, it is mostly just a marketing slogan.

XX% of Transformers FLOPS are just GEMMs and 256x256/128x128 systolic arrays in TPU/Trainium are already optimized for GEMMs. Even modern GPGPU (with Tensor cores) are optimized well for GEMMs.

Even if you "bake transformers in silicon" aka just create an gaint systolic array, most of your die area will still be taken up with SRAM cells and you will still face the memory wall since your HBM memory bandwidth will be the same as GPGPU/TPUs/Trainium.

![](https://pbs.twimg.com/media/Gwlr7ZMbwAA3Pr8.jpg)

XXXXXX engagements

![Engagements Line Chart](https://lunarcrush.com/gi/w:600/p:tweet::1948206985306095796/c:line.svg)

**Related Topics**
[if you](/topic/if-you)
[baking](/topic/baking)
[$arry](/topic/$arry)

[Post Link](https://x.com/SemiAnalysis_/status/1948206985306095796)