A $175 single-board computer running a full multimodal AI stack, completely offline, with no cloud calls, no API fees, and no latency penalty from a network hop. That is what Google just demonstrated with LiteRT and Gemma 4 E2B on the Raspberry Pi 5, using the Reachy Mini robot as a live showcase. The combination handles voice input, object detection via YOLO, language reasoning, and text-to-speech output, all running in parallel on a single board.
The hardware-software equation that makes this work
The key enabler on the software side is LiteRT, Google AI Edge's on-device inference runtime. Through LiteRT-LM, a specialized orchestration layer on top of LiteRT, developers can deploy Gemma right out of the box, with CPU acceleration via XNNPACK ensuring the model family is highly optimized for resource efficiency and low-latency execution directly on the Raspberry Pi.
On the model side, the star is Gemma 4 E2B. The "E" stands for "effective" parameters, and the name hides a clever trick. The E2B and E4B models use per-layer embeddings (PLE) inherited from Gemma 3n. Their total parameter counts are substantially larger than the numbers in their names: E2B has roughly five billion stored parameters but only 2.3 billion effective parameters. The large embedding tables live in flash storage, not RAM, so they barely touch your working memory budget.
Here is what PLE actually does: standard transformers give each token a single embedding vector at input. PLE adds a parallel lower-dimensional conditioning pathway, producing a small dedicated vector per layer, letting each decoder layer receive token-specific information only when relevant rather than requiring everything to be frontloaded into a single embedding. The practical result is that a 2.3B-active model carries the representational depth of the full 5.1B parameter count while fitting in under 1.5 GB of memory with 2-bit quantization.
What 9 tokens/sec actually means in practice
On a Raspberry Pi 5, LiteRT-LM achieves 99 tokens/sec for prefill and 9 tokens/sec for decode, all while maintaining a remarkably low peak memory footprint of just 1432 MB. Nine decode tokens per second sounds modest, but context matters. Thanks to Gemma 4 E2B's highly efficient tokenizer, which packs more text into fewer tokens (averaging ~4.2 characters per token), LiteRT-LM achieves an end-to-end generation speed of ~27.3 characters per second, roughly 300 words per minute, which is twice the speed of normal human speech (~150 wpm). For a voice assistant, that means the model is generating text faster than a human can speak it back.