Google's Gemma 4 multi-token drafters speed up inference by up to 3×
Google released Apache 2.0 multi-token prediction drafters for Gemma 4 on May 6, 2026. A lightweight drafter shares the target model's KV cache in speculative decoding, speeding up inference by up to 3× without reducing output quality. Weights are on Hugging Face and Kaggle, with support for vLLM, SGLang, Ollama, and the AI Edge Gallery on iOS and Android.
Visit original source