900 tons of Nvidia H100 GPUs shipped in hyperscale AI craze • The Register
This headline-grabbing determine comes from the corporate’s newest cloud and information middle market replace, which alerts a shift within the information middle funding priorities it highlighted beforehand on an ongoing foundation. The report says demand for servers geared up with eight GPUs for AI processing work has had the impact of pushing up common costs, whereas concurrently squeezing funding in different areas.
In consequence, Omdia as soon as once more lowered its estimate for 2023 annual server shipments by one other million items to 11.6 million, a 17 % decline from final yr’s determine. Nevertheless, on the similar time, the common server value has risen greater than 30 % each quarter-on-quarter and year-on-year as ultra-scalpers pour funding into these high-spec bins.
The report states that simply over 300,000 Nvidia H100 GPUs have discovered their manner onto server makers’ meeting strains, with the 900-ton determine primarily based on every GPU with a heatsink weighing round 3kg.
Since every H100 carries a hefty price ticket of about $21,000 every, this paradoxically signifies that Omdia now expects whole server market income for 2023 to achieve $114 billion, up 8 % year-over-year, regardless of unit shipments falling considerably. marked. .
This huge demand for GPU-packed AI servers is basically pushed by large-scale enterprises and cloud service suppliers, in keeping with Omdia, with the beneficiaries being authentic design producer (ODM) server makers — so-called white field distributors.
It claims that a good portion of these eight servers shipped within the second quarter had been constructed by ZT Methods, a US-based cloud server maker described as a fast-growing, privately owned firm. Proof for that is primarily based on the declare that as a lot as 17 % of Nvidia’s $13.5 billion in income within the second quarter was pushed by a single provider, and that Nvidia CEO Jensen Huang demonstrated a DGX server manufactured by ZT Methods throughout Computex Taipei.
Omdia additionally believes that a lot of the expansion in demand for eight-GPU servers in the course of the second quarter is probably going on account of Meta, primarily based on Nvidia beforehand saying that 22 % of income for the quarter was pushed by a single cloud supplier.
There’s extra of this to return. The report expects the speedy adoption of servers geared up for synthetic intelligence processing to proceed within the second half of this yr and the primary half of 2024.
Nevertheless, Omdia expects a better-than-seasonal restoration in demand for general-purpose servers in the course of the present third quarter and the subsequent as enterprise demand picks up. That is anticipated to extend income in 3Q23 by 13% yr over yr, accelerating to 29% in 4Q23.
Specifically, Omdia expects the continued deployment of eight-GPU servers to extend server market revenues by 51 % yr over yr by means of the primary half of 2024, with 1 million H100 GPUs anticipated to seek out their manner into techniques.
Nevertheless, the analyst cautions that the speedy funding in AI coaching capabilities he’s seeing amongst hyperscalers and cloud suppliers shouldn’t be confused with speedy adoption. AI use stays low total, she mentioned, citing a survey that discovered solely 18% of US adults have used ChatGPT. Generative AI at present makes up solely a small portion of enterprise cloud computing prices as nicely File I discussed lately.
There are additionally downsides to this big funding in power-hungry GPU techniques for coaching AI. One media trade govt lately warned that synthetic intelligence will “burn the world” until radical motion is taken to implement extra sustainable practices.
Talking on the Worldwide Broadcasting Convention (IBC) in Amsterdam this month, Tom Dunning, CEO and founding father of Advert Sign, mentioned that whereas we can not ask organizations to surrender the enterprise advantages of AI, it’s clear that “there may be an pressing have to To scale back its carbon affect.
Knowledge facilities are actually accountable for 2.5 to three.7 % of all carbon dioxide emissions, Dunning claimed, and AI adoption is anticipated to develop at a compound annual development fee of 37 % between now and 2030.
