(I rewrote this post in October 2026. The version I put up in April described a memory design I’ve since torn out and a Mac mini I’ve since replaced, so rather than bolt on another update section I started over.)
Here’s the short version for anyone thinking about building one of these: the language model is the easy part. Getting a 14B model to write a decent paragraph about my day took an afternoon. Getting it to read the right paragraph of my email, and noticing when it quietly stopped doing that, has taken most of the year.
FinkBot is my personal assistant. It runs on a few machines in my basement and reads more or less everything I do (email, calendars, Teams, Slack, notes, git repos, Home Assistant, my meeting recordings). From that it writes me a morning and evening briefing, preps me before meetings, tells me when something at home or in the homelab needs attention, and answers questions. I talk to it through Discord, through an iPhone app I built for it, and (more and more) through Claude Code, which can query its memory directly.
This post is mostly about the LLM side, since that’s what people ask me about. If you run a homelab and you’ve thought about pointing a local model at your own data, these are the things I wish someone had told me first.
The hardware
Inference lives on a Mac mini, an M4 Pro with 48GB of unified memory, which replaced a 16GB M4 in September. A Mac sounds like an odd choice for a homelab inference box, but unified memory means the GPU gets most of that 48GB, and it’s small and quiet enough to sit on a shelf and never be thought about. (Honestly, the 16GB mini ran everything except good transcription. More on that below.)
The workhorse is qwen2.5:14b through Ollama, with qwen2.5:7b for light jobs and qwen2.5:32b there for when I want something smarter and can wait for it. I did try the 72B. It’s 47GB of weights, which in a 48GB machine leaves macOS nowhere to live, so it’s gone. I also tried Qwen3 back in April and rolled it back the same day. It thinks out loud by default, which added one to three minutes to every call, and three flows blew through their timeouts within two hours.
Ollama’s settings cost me more time than its models did, and the bad ones look fine right up until they don’t.
One is OLLAMA_NUM_PARALLEL=2, which lets two requests run at once so a long batch job doesn’t starve the morning briefing. I originally set it with launchctl setenv, which works until the next reboot and then quietly goes back to 1. Put it in the LaunchDaemon plist instead.
The other is the one I’d tell you about if you only gave me thirty seconds. Ollama’s OpenAI-compatible endpoint (/v1/chat/completions) silently ignores parameters it doesn’t understand, and that includes num_ctx. So every request ran with Ollama’s default context window of 2,048 tokens, and when a prompt is longer than that, Ollama keeps the end and drops the beginning, with no error. My meeting summaries put the meeting title, date and attendees at the top of each chunk. A 22,000 character chunk is about 4,150 tokens, and 2,050 of them made it through, so for months every chunk summary was written by a model that had no idea which meeting it was reading. The summaries still looked perfectly reasonable, which is exactly why nobody (read: me) noticed. Switching to Ollama’s native /api/chat with num_ctx set to 8192 fixed it. Whatever you build, check what context length your requests actually run with.
Memory
Everything FinkBot ingests ends up in ChromaDB. As I write this that’s 292,646 documents, and about 266,000 of them are twenty years of my own email.
The April post spent a lot of words on a two-tier memory design: Chroma for raw recall plus MemU, a separate store for “distilled” long-term facts. I removed MemU in August. When I finally looked at what it held, about 400 of its 2,176 rows were MemU’s own demo persona (“The user’s name is Alex”, 105 times), another 836 were things that were true on one particular day stored as permanent facts, and its writes had been failing for a while, every one of them reporting success. Nothing in it was unique. The distilled knowledge now lives in a plain text file I can add to from Discord, which is much less impressive and works much better.
The bigger lesson was that I had no real idea whether memory was working. So I built a fixed set of test questions with known answers (who is this person, what did we decide about that project) and scored them with plain string matching instead of asking another LLM to grade them. Each question also lists wrong answers that must not appear, and that part turned out to matter most, because a model that doesn’t know something will happily give you a confident, plausible, wrong answer. I run the set three times for every change, since one run mostly measures luck.
That test set found the actual problem. Chroma embeds with all-MiniLM-L6-v2, which only really sees roughly the first thousand characters of each document. So FinkBot now pulls the top 40 matches from Chroma and reranks them with bge-m3 on the mini, which reads the whole document. The test set went from 73% to 93% (14 of 15). Each query got about three seconds slower, which is fine for a briefing and a little noticeable when I’m standing there holding my phone.
It also found that the question endpoint answered from vector search alone. It never looked at the Neo4j entity graph or at that facts file, even though other parts of FinkBot already used both. (In my defense, every piece already existed. I just hadn’t connected them.) Wiring them in fixed four complaints I’d been treating as unrelated.
I also seriously thought about deleting the email archive. Most of it is noise (receipts, newsletters, mailing lists) and it crowds useful things out of search results. I tested it first. With the archive, one of my test questions got the right technical answer three times out of three. Without it, the model answered three times out of three with something plausible, confident and wrong. Take the evidence away and the model fills the gap by inventing, so the archive stays.
Meetings
Meeting notes are the newest piece and lean on the LLM side the most. A small Mac app I built records my calls and sends the audio to Whisper running on the mini’s GPU.
It used to transcribe with Whisper’s base.en, which in the first minute of a real meeting invented a sentence nobody said (“Where’s the money?”) and later looped an entire passage. large-v3 on the mini matched Microsoft’s own Teams transcript word for word across four speakers. Transcription is the one job that genuinely justifies the 48GB machine.
Speaker labels were the other problem. Every diarization model I tried was guessing, and guessing badly. What finally worked isn’t clever at all: the recorder keeps my microphone and the call audio as two separate streams, so anything on the mic is me and anything on the call is everyone else. (Unless I’m on speakers instead of a headset, in which case my mic hears the call too. There’s echo detection for that, and on some days it sensibly gives up and skips the labels.)
The summary runs over the transcript one chunk at a time, with each chunk’s topics feeding into the next, on qwen2.5:14b.
If you’re building one
Build the test set before you build the clever thing. Every improvement I’m confident in came with a number attached, and at least two of my “improvements” made answers worse in ways I’d never have caught without one. (My favorite: giving the model a 10,000 character block of facts about a person made it answer less about them, because ten kilobytes of evidence buried a three word answer.)
Watch for jobs that succeed while doing nothing. Prefect will happily show a green run for a flow that hit an error, swallowed it and logged one line. MemU’s writes were one of these, and a UniFi monitor that stopped checking the network was another. A run that did real work leaves evidence in its log (counts, names, something). Read the logs, not the status.
A local model costs nothing per call, but the mini can only do so much at once. Running the same extraction three times and keeping what the passes agree on is free on an idle box, and it still roughly doubles the wait.
And keep your data. Storage is cheap, and as the archive test showed, the model’s alternative to evidence is invention.
If you’ve got a homelab, a spare machine with 16GB of memory or more, and a high tolerance for reading logs, this is very doable. Next for me is probably letting it act rather than suggest (with my approval on anything that touches the real world), which is mostly a UX problem, since nobody wants to wait thirty seconds for a chat reply 🙂