My local LLM’s real problem was a two-year blind spot, not a hallucination

I went looking up hallucinations at my local LLM and found something even worse. My setup is the proudest thing in my home lab: samples offered by Lemonade, Crush as a terminal-based tool, nothing coming out of my network. But I’ve always wondered how informative these native models are, so I put together a test suite to check.

He didn’t come up with anything. Out of 13 questions to beat him, he came up with something. In the other twelve it was honest and it was a real problem.

My local LLM wasn’t lying; two years behind

The problem turned out to be loyalty

My test script asks the same questions to the same model in two ways: directly to Lemonade and through a self-driving AI search engine. One system calls twice, and if he doesn’t know, he tells him to say so. The 13 questions include the latest releases, the facts that have changed since the exercise, the actual version numbers, the permanent knowledge control group, and a dummy list: the Firewalla router, the Proxmox subcommand, and the Crush flag that never existed.

Without a search, he gave up 11 of 13. When asked about the software running on the model, which answers a question about the latest release of Lemonade Server, he said he didn’t know about the project. Charm was asked what Crush was, the harness I was typing; same answer. Most of the time, fakes also worked: it rejected the fabricated Firewalla router and the fabricated Crush flag every time.

He once invented the data directly. when I asked about square meter The subcommand I built explained that the helper would read the VM configuration and write a human-readable description in Markdown, HTML, or plain text. Believable, well-formed, fictional. It was worse than when I asked for a fact about the release of the iPhone 18 Pro and he told me he didn’t know because his knowledge was June 2024.

It’s not like I’m asking difficult questions either. The current release number for Proxmox and Lemonade, which version of Caddy to use, and even what the current name of the Perplexica container was. These are the type of questions you ask on the job, and if the model can’t answer, you can open the browser yourself.

It wasn’t difficult to deploy the fix on its own

The Perplexica is now Vane and was an easy part to install

The self-hosted answer is Perplexica, changed to Vane in March 2026, so the old guides no longer show the name of the Docker image that matches. It now includes SearxNG, a privacy-focused metasearch engine that performs real-time searches, so a second container is not needed.

I created Debian LXC on a Proxmox node, installed Docker and ran:

docker run -d --name vane -p 3000:3000 -v vane-data:/home/vane/data --restart unless-stopped itzcrazykns1337/vane:latest

Port 3000 brings up the setup wizard, and Vane has a local Lemonade provider, so adding my server means putting in its address. One catch: a URL is required /api/v1 at the end, or you’re logged into Lemonade’s web interface and the wizard keeps the broken connection anyway.

I wanted to use the search inside Crush that works. Community Perplexica has MCP servers, but they didn’t work: Vane wants the current API to provide a UUID provider, and the old one to have a template key that takes simple names. So I wrote about 60 lines that open one in Python web_search tool. It ran into another problem: MCP moved to version 2.x of the Python SDK and changed its name. FastMCP class, so your new setup code will produce a version that you can’t import, and Crush will only report a closed connection. Downgrading to 2.0 fixes it.

Free search engines don’t talk to you

When everything was linked, my first search yielded nothing. The answer is not bad – no, because SearxNG’s default engines have rejected it. DuckDuckGo served CAPTCHA with 180 second delay and Brave speed limit error. The new self-hosted metasearch instance looks exactly like a scraper, because it is.

So I looked at the list of engines that open SearxNG /config and checked each one by hand. Eight engines, six ways to say no. Mojeek SearxNG returned a 403 to the honest user agent, but returned a 200 to the browser. Yes, a 403 was returned for both. Regardless, the bold rate is limited. Google and Startpage did not go quietly.

Exactly one engine worked: Bing. This is the part worth remembering. A self-hosted, private, independent search stack, and the only thing standing in between is Microsoft. Bing limits your speed, stopping you for three minutes after a few queries, and Wayne rewrites each query for about three searches, so you’ll hit that wall quickly.

Underneath the blocked engines were three other unrelated problems, and each presented the same thing: a request that never returned. Lemonade automatically loaded my model into Vulcan instead of ROCm, and the trap I hit on the Proxmox node yielded 10.7 tokens per second versus 33 and 0.2GB of VRAM versus 6.7GB. It barely touched the GPU. Even at full speed, queries timed out because the model was a reasoning model with no output cap, and one response passed 15,000 tokens. This is all an environment variable that is returned every time I open a new terminal.

Search closes the blind spot, but only when everything matches

The model cannot access the tool by itself

I asked the same question three ways: no search, via Vane as written, and finally via Vane with “use web search”. Bottom line, I did it all: the correct answer was from a model who didn’t know there were two years of releases.

The middle row bothers me. Provided a working search engine, which he chose not to use, and then apologized for not being able to access the website. It’s worse than no search at all, which at least made an assumption that I could test. The sample was searched for 7 out of 13 questions; said to use the web search feature, 12. The query does not include a search; it just makes the model more likely to use it, and you can’t predict which questions will benefit.

Four models, four different failure modes

When I gave SearxNG my Brave API key, searching was no longer difficult. Each query returned 26 to 55 sources in about twelve seconds, and the results outnumbered Bing: the Proxmox wiki download page and the 8 to 9 upgrade guide, not the marketing homepage.

So I drove the suite properly and hit a wall. Wayne’s research agent needs a template to issue the framework tool calls, and none of my native templates made it so that the pipeline could use it. This last post is almost comical: he did the search, got the sources, and said he couldn’t find it online.

One good answer came from a sample Coder, in the Perplexica question itself. He searched and found the old picture name still referenced everywhere and said he couldn’t identify the current one. Which is correct: the name change was recent as both names are alive and the sure answer would be wrong.

Faithful beats useless, but current beats both

I set out to prove that my native model was making things up, but it wasn’t. He didn’t know the last two years had happened. When the search is run, it closes the gap, and the 9.2 answer with ten sources proves that the idea works. But it only fires when you ask; free engines often do not talk to you and four different local models could not control the research agent. It’s a slimmer promise than “self-hosting”, even if the Web UI for Vane works.