[GUEST ACCESS MODE: Data is scrambled or limited to provide examples. Make requests using your API key to unlock full data. Check https://lunarcrush.ai/auth for authentication information.]  Rihard Jarc [@RihardJarc](/creator/twitter/RihardJarc) on x 50.9K followers Created: 2025-07-16 10:59:23 UTC A MUST READ INTERVIEW with a current Director at $MU on HBM & GPU/ASIC market ( $NVDA, $MU, Samsung struggles ): X. The hyperscalers $AMZN, $MSFT, $GOOGL are achieving significantly higher GPU utilization rates than they had in the past years. In 2022 and 2023, the GPU utilization rates were less than XX% The ROI on that in terms of hyperscalers getting their investment back used to be 12-18months. Now, the utilization rates are 70%-80 %, and the ROI has accelerated to only 6-9 months. X. When $NVDA's Rubin comes out, he expects even higher GPU utilization rates - 90%. With Blackwell next year, he expects the cost of training on hyperscalers to be in the $60-$70 per hour range and for inference, smaller cases to be in the $5-$10 per hour range. X. The supply of $NVDA's H100 has dramatically increased. The shortage, in his view, has been reduced mostly because of $TSM, which has improved their 4nm yield a lot as the technology has matured and as HBM availability has been good. In his words, when it comes to HBM, we are now moving from »call scarcity« to »high-volume pricing«. X. HBM has been almost 2X-2.5X year on year. Both SK Hynix as well as $MU first had yield challenges as the biggest problem is TSV technology (through-silicon via), because you have to put eight of these chips on top of each other. If you have to put XX of these on top of each other the connection between them is the biggest challenge. Both $MU and SK Hynix have become very proficient in that area. He continues to see the pricing going up slightly on the ASP for HBM. X. He gives the example of HBM use in B100. If you say eight-high stack, that's around $400-$500; if you're using five or six of those, that's around $2500-$ 3600. It is XX% of the GPU cost in most cases. X. The reason why HBM capacity has caught up in his view is not because of Samsung, but because the TSV technology has matured, and both SK Hynix and $MU have converted most of their CapEx into expanding HBM. He thinks this year might be the last year where we have a scarcity. He thinks 2026 will be even better with availability. X. The reason why Samsung has struggled with $NVDA's certification process is because of thermal issues. $NVDA B100 and H100 have very tight thermal margins because they're using more than 700W of power. Each signal requires eight Gbps per pin. X. Samsung has been struggling with the DPPM rate (defect parts per million rate), as it is significantly higher on Samsung parts compared to their competitors. The second thing they struggle with is the eye diagram. The eye diagram is the margin, and according to him, that margin has always been poor compared to SK Hynix and $MU. The reason is that $NVDA uses NVLink. The poor margin also affects the data transfer. X. According to him, it makes sense that $AVGO approved Samsung as they use less than 300W of power and have more margin, $AVGO uses four Gbps-six Gbps per pin, $NVDA uses twice that, both in power as well as data transfer rate. XX. He thinks that by 2028, $NVDA's market share will be reduced to 50-60% with most of it going to hyperscalers doing their own ASICs. He mentions $GOOGL, noting that their TPUs are at the forefront of this, as they have highly customized FP4 optimization for inferencing and have already deployed TPUs for Gemini, as well as for internal AI training. Second is $AMZN's with Trainium, while both $META and $MSFT are still a bit behind and need to catch up. If he had to make an educated guess, he thinks that $GOOGL and AWS are already serving XX% of their own internal inferencing with their custom ASICs.  XXXXXX engagements  **Related Topics** [googl](/topic/googl) [msft](/topic/msft) [amzn](/topic/amzn) [nvda](/topic/nvda) [investment](/topic/investment) [gpu](/topic/gpu) [$msft](/topic/$msft) [samsung](/topic/samsung) [Post Link](https://x.com/RihardJarc/status/1945438104745324835)
[GUEST ACCESS MODE: Data is scrambled or limited to provide examples. Make requests using your API key to unlock full data. Check https://lunarcrush.ai/auth for authentication information.]
Rihard Jarc @RihardJarc on x 50.9K followers
Created: 2025-07-16 10:59:23 UTC
A MUST READ INTERVIEW with a current Director at $MU on HBM & GPU/ASIC market ( $NVDA, $MU, Samsung struggles ):
X. The hyperscalers $AMZN, $MSFT, $GOOGL are achieving significantly higher GPU utilization rates than they had in the past years. In 2022 and 2023, the GPU utilization rates were less than XX% The ROI on that in terms of hyperscalers getting their investment back used to be 12-18months. Now, the utilization rates are 70%-80 %, and the ROI has accelerated to only 6-9 months.
X. When $NVDA's Rubin comes out, he expects even higher GPU utilization rates - 90%. With Blackwell next year, he expects the cost of training on hyperscalers to be in the $60-$70 per hour range and for inference, smaller cases to be in the $5-$10 per hour range.
X. The supply of $NVDA's H100 has dramatically increased. The shortage, in his view, has been reduced mostly because of $TSM, which has improved their 4nm yield a lot as the technology has matured and as HBM availability has been good. In his words, when it comes to HBM, we are now moving from »call scarcity« to »high-volume pricing«.
X. HBM has been almost 2X-2.5X year on year. Both SK Hynix as well as $MU first had yield challenges as the biggest problem is TSV technology (through-silicon via), because you have to put eight of these chips on top of each other. If you have to put XX of these on top of each other the connection between them is the biggest challenge. Both $MU and SK Hynix have become very proficient in that area. He continues to see the pricing going up slightly on the ASP for HBM.
X. He gives the example of HBM use in B100. If you say eight-high stack, that's around $400-$500; if you're using five or six of those, that's around $2500-$ 3600. It is XX% of the GPU cost in most cases.
X. The reason why HBM capacity has caught up in his view is not because of Samsung, but because the TSV technology has matured, and both SK Hynix and $MU have converted most of their CapEx into expanding HBM. He thinks this year might be the last year where we have a scarcity. He thinks 2026 will be even better with availability.
X. The reason why Samsung has struggled with $NVDA's certification process is because of thermal issues. $NVDA B100 and H100 have very tight thermal margins because they're using more than 700W of power. Each signal requires eight Gbps per pin.
X. Samsung has been struggling with the DPPM rate (defect parts per million rate), as it is significantly higher on Samsung parts compared to their competitors. The second thing they struggle with is the eye diagram. The eye diagram is the margin, and according to him, that margin has always been poor compared to SK Hynix and $MU. The reason is that $NVDA uses NVLink. The poor margin also affects the data transfer.
X. According to him, it makes sense that $AVGO approved Samsung as they use less than 300W of power and have more margin, $AVGO uses four Gbps-six Gbps per pin, $NVDA uses twice that, both in power as well as data transfer rate.
XX. He thinks that by 2028, $NVDA's market share will be reduced to 50-60% with most of it going to hyperscalers doing their own ASICs. He mentions $GOOGL, noting that their TPUs are at the forefront of this, as they have highly customized FP4 optimization for inferencing and have already deployed TPUs for Gemini, as well as for internal AI training. Second is $AMZN's with Trainium, while both $META and $MSFT are still a bit behind and need to catch up. If he had to make an educated guess, he thinks that $GOOGL and AWS are already serving XX% of their own internal inferencing with their custom ASICs.
XXXXXX engagements
Related Topics googl msft amzn nvda investment gpu $msft samsung