The race to make AI inference cheaper is pushing chip design beyond general-purpose accelerators.
We’re now moving toward silicon that is tailored to individual models.
And Google’s reported “Frozen v2” project suggests the Gemini AI model is part of the movement.
First reported by The Information, the unannounced chip would reportedly hardwire parts of Gemini’s architecture while leaving its weights updatable.
That compromise could give Google much of the efficiency of model-specific silicon without making the hardware obsolete every time Gemini changes.
A spokesperson for Google tells The New Stack, “Our teams are constantly researching and experimenting with new innovations to deliver maximum performance and efficiency for our users and customers.” A spokesperson for Google tells The New Stack, “Our teams are constantly researching and experimenting with new innovations to deliver maximum performance and efficiency for our users and customers.
While not every project moves into production, this rigorous exploration is central to our full stack approach.
By co-designing our hardware and software from the ground up, we ensure our systems are integrated and highly optimized for real-world workloads.” According to the reporting, Google hopes the chip will help relieve the AI compute crunch that’s made it harder for cloud providers to keep up with demand, while also making Gemini much cheaper and more efficient to serve.
Internal projections reportedly estimate the design could deliver six to ten times more tokens per watt than Google’s current generation of AI chips.
If the project moves forward, it would represent a different approach to AI infrastructure because Google would be building hardware specifically for Gemini.
For developers, that’s an early indication that future AI systems may be designed with much tighter integration between the model and the hardware beneath it.
Specialized silicon replaces flexibility Currently, the vast majority of AI inferences run on Nvidia GPUs or Google’s own Tensor Processing Units (TPUs).
Because these chips are intended to accommodate a wide variety of AI models, they carry a high degree of processing load.
There’s precedent for this kind of shift.
Bitcoin mining followed a similar path, starting with CPUs, then GPUs, before ultimately settling on ASICs.
AI inference may be headed in the same direction.
Training still benefits from the flexibility of GPUs, but once a model reaches production, the priority shifts to serving as many requests as possible while using less power.
Training still benefits from the flexibility of GPUs, but once a model reaches production, the priority shifts to....
