I am not often one for extravagant tech and {hardware} predictions—issues can change a lot and so unpredictably that it is tough to place an excessive amount of weight behind such premonitions. However with regards to AI tech, issues transfer so shortly that what appears distant won’t truly be too far-off. Mix this with Nvidia being the one outlining the futuristic imaginative and prescient, and I take it a bit of extra severely.
Nvidia’s newest prediction, as outlined on the IEDM 2024 convention based on Dr. Ian Cutress (through TechPowerUp), is AI accelerators which can be 3D-stacked and that use—at the least partly—silicon photonics for knowledge transmission. That is, as Cutress places it, Nvidia’s imaginative and prescient of “the way forward for AI compute”.
The picture offered within the submit exhibits an AI accelerator (ie, a datacentre GPU) that is cut up vertically right into a substrate, built-in silicon photonics, GPU tiers, 3D stacked DRAM, and chilly plate.
The 2 huge improvements on this image, insofar as they might be utilized to AI accelerators, are silicon photonics and the vertical stacking for logic. The previous makes use of photons (gentle) to transmit knowledge to and from optical elements, which is quicker and makes use of much less energy for extra bandwidth than conventional electrical knowledge transmission.
Judging by the diagram, it seems like this light-based transmission expertise could be used horizontally to hook up with different accelerators.
This is @NVIDIA’s imaginative and prescient of the way forward for AI compute.Silicon photonics interposerSiPh intrachip and interchip12 SiPh connects, 3 per GPU tile4 GPU tiles per tierGPU ‘tiers’ (GPU on GPU?!?)3D Stacked DRAM, 6 per tile, fine-grainedFrom #iedm24. My guess, 2028/2029/2030… pic.twitter.com/5IsDkYSWT2December 8, 2024
Nevertheless, TechPowerUp says these accelerators function “12 SiPh [silicon photonics] connections for intrachip and interchip connections, with three connections per GPU tile throughout 4 GPU tiles per tier”. And “intra-chip connection” would appear to indicate connection between every of those tiles inside every tier, too.
The diagram says there’s an electrical (not optical) interconnect from die-to-die and tier-to-tier, which might recommend it is utilizing extra conventional By-Silicon Through (TSV) tech to get the vertical stacking executed.
Silicon photonics continues to be solely in its infancy. It’d make extra sense for Nvidia to make use of By-Silicon-Through (TSV) expertise for the vertical dimension, which basically includes creating tiny tunnels as pathways between the stacked chips. That is the expertise that enables the AMD Ryzen 7 9800X3D, for instance, to have its processor sitting on high of its cache.
Although often we see 3D stacked chips restricted to cache on logic, i.e. L3 cache on cores as per AMD’s chip, reasonably than logic on logic, which is usually recommended right here.
It seems like 4 GPU tiles will exist per GPU “tier”, and these tile tiers will probably be stacked vertically, too. Then on high of all of that, stacked DRAM. That every one sounds prefer it’d get extraordinarily toasty, and it isn’t one thing I would count on to be achieveable within the rapid future.
Regardless of the case, it is actually an fascinating image of what may be to come back, and if anybody’s capable of do it it’s going to be Nvidia. And whereas we should not infer an excessive amount of about these applied sciences making their method throughout to gaming GPUs any time quickly, it would not be unreasonable to imagine some of it’d, at some level.
If the expertise’s there and will get carried out in AI accelerators, the cheaper facets (comparable to TSV stacking) may be price including to the patron GPU combine. Gaming graphics does not require the form of bandwidth that AI processing does, although, so I feel we are able to take away photonics from the gaming equation for the foreseeable future.
And these mixed applied sciences will not even be doable for AI accelerators within the close to future, both. I feel Cutress is true: “My guess, 2028/2029/2030 minimal.”
