The Gemma 4 works fine on my laptop, but putting it on a NAS has changed the way I use it

Ever since I used an old GTX 1080 to build a local AI hosting workstation last year, I’ve been benchmarking LLM final assignments on all kinds of hardware, from gaming systems to tiny single-board PCs and outdated cell phones. As you’d expect, devices with NPU and VRAM loaded graphics cards work fine with my local LLM and can handle even large MoE models with the right tweaks.

That said, I’ve come across a lot of setups that work well with AI workloads, even if they have technically weaker hardware. My network attached storage is one of the devices I initially wrote off as too low power for my LLM tasks. But once I reworked my end-to-end tasks and set up proper automation, my NAS became an essential part of my personal hosting arsenal.

The lightweight Gemma 4 templates can run on virtually any hardware

While bringing clear thinking capabilities to the table

When you think of sub-9B models, you usually think of fringe LLMs with very low thinking capabilities. After playing around with the hallucinatory content created by the 0.5B-4B models in the Pi Agent Toolkit experiment this summer, I can say you’re not too far off the mark. That said, the Gemma 4 series is an exception to this notion. Despite their small footprint, the Gemma-4-E2B and Gemma-4-E4B have a surprisingly large knowledge base, and it’s all thanks to their unique architecture.

On paper, the Gemma-4-E2B has a total of 5.1V, while its E4B counterpart reduces this number to 8V. However, effective settings (like those used for real computing tasks) are 2.3V for Gemma-4-E2B and 4.5V for Gemma-4-E4B. Additional settings are stored in special token embedding tables (called Layer Embedding) and help with quick lookups and simple conversions. Thus, the two LLMs can access larger knowledge pools without requiring the same amount of RAM/VRAM real estate as comparable reasoning models. Both models support vision capabilities, making them great for analyzing images and camera footage.

Here, my Raspberry Pi 5 achieves 5.8-6.5 tokens per second on the Gemma-4-E2B, while my i5‐1235U NAS achieves over 10.5 tps using this LLM for simple summary tasks. I’ve experimented with Gemma-4-E4B on my NAS, which is quite slow at 4-5tps. It’s obviously too slow for active inference tasks, where I throw a call to the front end and expect an immediate response. So instead I started changing my workloads…

The slow speed of the NAS forced me to change my LLM workloads

Instead of expecting instant results, I leave my final assignments overnight

Since I have an instance of Qwen3.6-35B-A3B running on an RTX 3080 Ti (via MOE capture) and call for coding applications and quick summary tasks, I decided to dedicate the Gemma 4 samples running on the NAS to performance applications that I can use to unlock additional functionality. Take my Paperless-ngx pipeline for example. More specifically, Paperless-GPT and Paperless AI partner services that can handle LLM heavy tasks in this document organization application.

Unlike my Perplexica searches, where I want my locally linked clunkers to produce answers immediately, I can let Paperless-GPT OCR scans and Paperless AI tagging tasks run in the background. The types of documents I typically archive in Paperless-ngx are accounts, academic records, financial statements and invoices that may need to be reviewed months later rather than being analyzed immediately after uploading.

The same applies to web pages, PDFs, and YouTube videos that I want to summarize and automatically tag after I’ve added my LLMs to Karakeep. Since I use this container to archive media for later review, I don’t mind the Gemma-4-E2B taking a few minutes to run at 10tps. Similarly, I often throw random ideas into the Blinko container and when I’m free to analyze them, a very lightweight instance of Gemma 4 running on my NAS ticks them off.

I also have embedding templates on my NAS that I use with my main LLM

So far, I’ve only mentioned the Gemma 4 LLM that runs my storage server, but I have two other NAS-powered models that play a crucial role in my home lab. Specifically, I’ll be hosting nomic-embed-text-v1.5 and nomic-embed-vision-v1.5 on my NAS, which are embedded models that help with large MoE models running on my RTX 3080 Ti. For the uninitiated, these embedding templates convert regular text documents and images into vector mappings, allowing applications with advanced search generation capabilities to use them to add more context to my inference tasks.

For example, my Open Notebook container uses digital vectors generated by nomic-embed-text-v1.5 to retrieve my research notes that match my queries before feeding them to TM models, further reducing the chance of my LLMs hallucinating.