Google DeepMind has unveiled a groundbreaking new AI model named DiffusionGemma, designed to revolutionize the way local AI processing is carried out. This innovative model is part of the Gemma 4 open model family and boasts a significant speed boost over its predecessors.
A New Approach to Text Generation
Unlike traditional autoregressive models that generate text one token at a time, DiffusionGemma produces an entire block of text in parallel. This approach is more akin to image generation models, which start with static and then denoise it to create the desired content. In the case of DiffusionGemma, the model takes a field of placeholder tokens running over the canvas multiple times to generate likely tokens and use those to improve estimation of others.
A Shift in Bottleneck from Memory Bandwidth to Compute
DiffusionGemma is a Mixture of Experts (MoE) model with 26 billion parameters, but only 3.8 billion are activated during inference. This means it should fit within the 18GB RAM allotment of a high-end GPU. In testing with an RTX 5090, DiffusionGemma spits out around 700 tokens per second. With a single Nvidia H100 AI accelerator, DiffusionGemma can produce 1,000+ tokens per second.
Applications and Use Cases
Google says this approach offers a measurable boost in non-linear tasks like in-line editing, molecular sequencing, and mathematical graphing. The animation above shows how DiffusionGemma was tuned to solve Sudoku puzzles, which is a notoriously challenging task for standard autoregressive AI models because each token depends on future tokens.
Limitations and Future Directions
If diffusion is so much faster, why isn’t Google using it in big cloud-based Gemini models? Google has experimented with this, but there are a few drawbacks to text diffusion, including a higher error rate. In image diffusion models, a single badly predicted pixel doesn’t make the image useless, but language is discrete. An equivalent error in text can make a block of tokens meaningless and force you to start over to get a better output.
Availability and Optimization
DiffusionGemma is available under the same Apache 2.0 license as all the other fourth-generation Gemma models. You can download the model weights today from Hugging Face. Google says it worked with Nvidia to ensure DiffusionGemma was optimized for a variety of setups, including high-end RTX GPUs (quantized) and enterprise systems like the H100 or DGX Spark platform.
Conclusion
Google DeepMind’s release of DiffusionGemma marks an exciting new development in local AI processing. With its 4x speed boost over traditional autoregressive models, it has the potential to revolutionize the way we approach non-linear tasks and applications. While there are still limitations to be addressed, this innovative model is certainly worth exploring further.
Source: Original article