LokalBot Blog

Recall vs. restraint: which local models know when not to save a memory

2026-10-23T00:00:00Z

Why restraint matters

On Mac, LokalBot's Personal Context learns facts, preferences, goals, and principles from what you type and say, so that agents you allow can tailor their work to you. A missed memory costs little: the next conversation usually brings it up again. A wrong one is worse. It quietly ends up in the context every agent reads, and nobody notices until an answer is off.

So the test asks for both recall and restraint. Three kinds of cases make the difference visible:

Who said it. “I live in Valencia” should be saved. “My colleague said, ‘I live in Valencia’” should not.
Already known. A new preference should be saved. One already in the known-memories list should be skipped.
Policy. A job title mentioned alongside a salary should be kept, and the salary left out.

The test

The benchmark has 120 synthetic cases arranged as 60 contrast pairs. The main comparison uses the 96 English cases: 52 contain one useful memory to extract, and in 44 the correct output is an empty list. A separate 24-case Serbian slice is reported on its own.

Every model received the same application prompt with its native chat template, an enforced JSON schema, temperature 0, seed 0, thinking disabled, and a single pass, with no model-specific prompt tuning. The GGUF models ran one after another through llama.cpp on the same Ryzen 7 9700X server. This measures accuracy, not speed. A recalled memory only counts when it also cites the right source line.

Results

Results on the 96 English cases
Model	Useful memories recalled	Unwanted proposals	Empty cases correct
Qwen3.5 4B, Q4_K_M	30/52 (57.7%)	3	42/44 (95.5%)
Qwen3.5 9B, Q4_K_M	37/52 (71.2%)	5	40/44 (90.9%)
Hy-Memory 30B-A3B, official Q4_K_M	52/52 (100%)	76	0/44 (0%)
LFM2.5 24B-A2B, QAD Q4_0	40/52 (76.9%)	53	10/44 (22.7%)
MemT-1.0-Nano, NF4 experts / BF16 dense	51/52 (98.1%)	6	39/44 (88.6%)

The 96 English cases. Higher is better in the first two columns, lower in the third.

What stood out

Perfect recall can hide the real problem. Hy-Memory captured every eligible memory and failed all 44 cases where the right answer was nothing, with 76 unwanted proposals. Scored on recall alone, it would look like the best model here.
The Qwen models held back. Both missed more valid memories, but they handled over 90% of the empty cases and made only three and five unwanted proposals.
LFM2.5 copied from the instructions. In several cases it proposed “I live in Belgrade”, an example from the prompt with no support in the conversation. It also often returned the source sentence instead of the required source-line ID. Accepting exact quotations recovered 9 of its 18 invalid English citations, raising exact source selection from 51 to 56 of 96 cases, but that only repairs formatting. The table keeps the strict scoring.
MemT-1.0-Nano was the best balance. A reader on r/LocalLLaMA suggested it, and it was added on October 25: 51 of 52 useful memories, 6 unwanted proposals, and 39 of 44 empty cases. On the Serbian slice it recalled 13 of 13, made 3 unwanted proposals, and passed 8 of 11 empty cases, but three of its outputs switched language or script.

Caveats

MemT-1.0-Nano ran with experimental NF4 expert offload through Transformers on a Radeon RX 6600 XT; the other models ran as GGUF files in llama.cpp on a CPU. There is no matched run of its Qwen3.6-35B-A3B backbone, so the comparison isn't controlled for hardware or precision.
Each case starts from a fresh state, so MemT's persistent memory feature isn't exercised.
An AI assistant wrote the cases and graded the semantic decisions with the model names visible. There is no independent human validation and no claim of statistical significance.

What it means for LokalBot

These results don't change LokalBot's defaults. What they confirm is that a memory writer has to be judged on what it leaves out as much as on what it finds. On Mac, the suggestions Personal Context learns appear under To review in Settings → Personal Context, where you keep or dismiss them.

Data

Everything needed to check or rerun the comparison is in the repository: the benchmark folder with the cases, prompts, and scorer, the MemT report with every failed case, and the raw responses. Other model results are on the benchmarks page.

LokalBot is on Windows and Linux, in alpha

2026-10-20T00:00:00Z

What works

The Windows and Linux app is written in Rust with GPUI, the UI framework behind the Zed editor. Meetings, transcription, Ask, search, day memory, screen context, dictation, autocomplete, calendar, Agent Mode, and the CLI and MCP server for coding agents all work. Like the Mac app, it runs its models on your computer by default, and it has no account and no telemetry.

A meeting's recap and action items on Linux, with fictional demo data.

What's still missing

The Windows build isn't code-signed yet. SmartScreen asks before the first launch: choose More info → Run anyway.
Wayland support is partial. Autocomplete in other apps needs X11; on Wayland it works in the practice editor in Settings. For dictation on Wayland, bind lokalbot dictate to a shortcut in your desktop's settings.
Meeting audio is the whole computer's. On Windows and Linux LokalBot records everything the computer plays, not one app as on Mac.
Some features are Mac-only for now: selection actions, meeting note export, Personal Context, spoken summaries, and coding-agent sessions in day memory.

The full list is in what works where.

Today on Windows, with fictional demo data.

The models

Transcription: Qwen3-ASR 0.6B through sherpa-onnx, with Parakeet v3, SenseVoice, and Whisper turbo as options.
Speaker separation: pyannote segmentation and TitaNet.
Notes and Ask: Qwen3.5 4B, the same as on Mac.
Search: EmbeddingGemma 2, the same as on Mac.
Autocomplete: LFM2.5 1.2B.

The default download is about 4 GB. llama.cpp runs on a Vulkan GPU or the CPU, and since 0.11.3 you can choose CUDA (experimental) in Settings → Models → llama.cpp on a supported NVIDIA GPU. You can swap any model in Settings.

Install

Windows 10 or 11, x64: run LokalBot-<version>-windows-x86_64-setup.exe from the latest release. It installs for your user, without administrator rights. A portable zip is in the same release.
Linux x86_64: install the .deb (Debian, Ubuntu), the .rpm (Fedora, openSUSE), or the Arch package, or extract the .tar.gz and run ./install.sh. Since 0.11.2 the .deb, and the .rpm on Fedora, add the LokalBot repository, so sudo apt upgrade or sudo dnf upgrade brings later versions.

Linux needs glibc 2.39 or later (Ubuntu 24.04, Debian 13, Fedora 40, or newer), PulseAudio or PipeWire, and a Vulkan or OpenGL driver. See the system requirements.

On Omarchy and other Hyprland desktops

Omarchy already has everything LokalBot uses on Wayland: grim for screen context, wtype and wl-clipboard for dictation, PipeWire for meeting audio, and the Hyprland desktop portal. Install the Arch package with sudo pacman -U, then bind LokalBot's commands in ~/.config/hypr/bindings.lua. These keys are free in Omarchy's defaults:

o.bind("SUPER + ALT + D", "LokalBot dictation", "lokalbot dictate")
o.bind("SUPER + ALT + A", "Ask LokalBot", "lokalbot quick-recall")

The tray icon appears in Omarchy's bar, and the dictation pill and Quick Recall float instead of tiling.

Tell us what breaks

Most testing so far has been on a handful of personal and rented machines, so reports from real hardware help the most. Open an issue with your distribution, desktop, X11 or Wayland, CPU and GPU, and what you tried.

How the Windows and Linux port was built

2026-10-20T00:00:00Z

A first attempt that only looked right

On October 2 a coding agent built a working Windows and Linux app with a native interface and CI on both platforms in one morning. It had its own data model and its own library on disk, so it couldn't open anything recorded on a Mac, and nothing would show when it drifted from how the Mac app behaves. That pull request was closed.

Specs before code

The second attempt spent its first hour on specs. Four agents read the Mac app's 150,000 lines of Swift without changing anything and wrote down the library format, the processing pipeline with every model prompt, every screen with its exact text, and the command-line tools and privacy rules. The core data types were written to match the Mac app's files, and every agent tested against a real library recorded by the Mac app. Six implementation agents then each took one part of the code.

How the work was checked

The port landed as one large pull request, so the checks mattered more than reading every line of the diff:

CI runs the app on a virtual display on Linux and on Windows and takes a screenshot of every page.
The live transcript is tested by playing a scripted conversation into virtual audio devices. That caught two bugs the unit tests had missed.
Autocomplete is tested by typing into Notepad and checking that the suggestion appears after the cursor.
End-to-end runs use the real models from an empty library: they download the models, process a short meeting, and ask a question about it.
Meetings recorded on a Mac are played on Linux, with their timestamps required to line up exactly.
Afterwards, the Mac and Windows/Linux apps were compared feature by feature, and the gaps were closed in five separate pull requests.

Read the full write-up

The full story, including the build machines, the rules for agents working side by side, and the decisions that were left to a person, is on Field Notes: Porting LokalBot to Linux and Windows with agents. The result is described in LokalBot is on Windows and Linux, in alpha.

Autocomplete that knows your meetings

2026-10-13T00:00:00Z

A suggestion that comes from a meeting earlier that day. Tab accepts it word by word.

Fast enough to stay out of the way

Getting autocomplete to be useful without being intrusive or laggy took a lot of experiments with different models and quantizations. The best balance of suggestion quality and latency on a regular MacBook turned out to be the Gemma 4 E2B base model at 6-bit (Q6_K), running in-process through llama.cpp. On an M4 Max a suggestion takes about 50 ms.

Context from your own work

The text above the field. On by default, read through macOS accessibility. Private browser windows are never read.
Your meetings, if you opt in. When your draft names something distinctive, LokalBot can add up to two facts from your own meeting notes to the model's context.

Autocomplete lives in the same app as meetings, dictation, and screen recall, which share one memory budget for their models.

How it compares

Cotypist and Cotabby are the closest alternatives. As described on their sites on October 13, 2026:

Cotypist, Cotabby, and LokalBot autocomplete compared
Feature	Cotypist	Cotabby	LokalBot
License and price	Proprietary; free for 100 words a day, paid for unlimited	AGPL-3.0, free (beta)	GPLv3, free, no limits
Model	Local (Gemma 4 E4B recommended)	Local GGUF through llama.cpp, or Apple Intelligence	Gemma 4 E2B base Q6_K, in-process llama.cpp
Knows what was said in your meetings	No	No	Yes, opt-in
Text above the field	Not documented on its site	OCR of the window	Accessibility text, never from private windows
Also in the app	Autocomplete only	Emoji, macros, autocorrect	Meetings, dictation, screen recall

A head-to-head with Cotabby used identical inputs and the same Gemma 4 E2B weights over 782 next-word checkpoints. LokalBot predicted the exact next word 32.7% of the time (256 of 782) and Cotabby 30.7% (240 of 782), a difference of 2.0 points with a 95% interval of 0.0 to 4.0. In other words, the two are comparable on accuracy; the work now is on the experience around the suggestions.

Try it

Turn on Autocomplete in Settings → Writing. It needs the Accessibility and Input Monitoring permissions and a one-time model download. Tab accepts a word. Password fields and apps you exclude are never read.

Search your meetings from Raycast

2026-10-02T00:00:00Z

Quick Recall in Raycast, with fictional demo data.

Two commands

Quick Recall. Type a word and see every meeting where it came up, from titles, summaries, or the transcript itself. Enter reveals the meeting's files in Finder and ⌘Enter copies the matching snippet.
Recent Meetings. Lists your last 20 meetings.

Local, like the app

The extension makes no network requests. It reads your local LokalBot library, and only after you allow external agents to read your meeting library in LokalBot's privacy settings.

It needs the LokalBot app installed, which on Mac means Apple Silicon and macOS 15 or later.

Install

Get LokalBot in the Raycast Store. It's the first LokalBot extension for Raycast, so tell us what's missing or annoying.

Five local models, 6.8 GB of weights

2026-09-30T00:00:00Z

Update, October 2026: two defaults have changed since this post. Search now uses EmbeddingGemma 2, and autocomplete uses Gemma 4 E2B; see Autocomplete that knows your meetings. Current numbers are on the benchmarks page.

The stack in 0.9.2

Transcription: Qwen3-ASR 1.7B, MLX 8-bit.
Who spoke when: Nemotron 3, Core ML.
Notes and action items: Qwen3.5 4B, Q4_K_M in llama.cpp.
Search embeddings: Harrier 0.6B, Q8_0.
Autocomplete in any app: LFM2.5 1.2B, Q4_K_M.
Screen text: Apple Vision OCR, opt-in.

What changed, in numbers

Measured on an M4 Max with 48 GB of memory:

A 26-minute meeting goes to finished notes in 33 seconds with the model already loaded, decoding at about 85 tokens a second.
Speaker error dropped from 43.4% to 14.6% diarization error rate on the AMI corpus. That's against the previous pyannote setup, so it says more about that configuration than about pyannote.
Autocomplete latency at the 95th percentile fell from 1.83 s with Gemma 4 E4B to 0.49 s.

A model that didn't make it

MiniCPM5 2B was tested for notes. It was smaller and faster, but it got stuck repeating itself on one summary and assigned action items to the wrong person. Qwen3.5 4B stayed the default: saving a few seconds isn't worth getting wrong who agreed to do what.

Also in the app

A read-only MCP server and CLI, off by default, let Claude Code or any other MCP client pull context from your meetings.

There are no numbers yet from 16 GB Macs or other M-series chips. If you run LokalBot on one, especially an M5 or M6, share what you get. The 0.9.2 release notes have the rest.

A whole workday on local models

2026-07-21T00:00:00Z

Update, October 2026: several default models have changed since this post; the current ones are on the benchmarks page.

What it does

Meetings. Detects Zoom, Teams, Meet, and other calls and records you and the other side on two synchronized tracks, with no bot joining the call. Transcription and the summary run on your Mac as soon as the meeting ends.
Dictation. Hold ⌥ Space, talk, and release. It's transcribed locally and pasted at the cursor, and the audio is deleted afterwards.
Autocomplete. Ghost-text suggestions in almost any Mac text field; Tab accepts. Opt-in.
Recall. Ask “what did we decide about this?” and get the answer from your own library, with the source meeting cited and the exact moment ready to replay. Full-text and semantic search cover everything.
Day timeline. An optional private timeline of where your time went, with a daily digest.

New in 0.5

Dreaming. LokalBot can compile a finished day overnight into a morning brief and structured work memory.
Faster autocomplete. New installs get the 0.73 GB LFM2.5 1.2B model, and accepting suggestions is more reliable across native apps, Chromium, and input methods.
Answers in Quick Recall. Quick Recall can answer right in its compact window instead of always opening the full Ask view.
Scheduled day digests, with a prompt you can customize.

The models in daily use

One setup, on a 48 GB M4 Max, all through the bundled llama.cpp with full Metal offload:

Models in daily use
Job	Model	Quantization	Size	Speed
Transcription	Granite Speech 4.1 2B	Q4_K_M + F16 projector	2.3 GB	Faster than real time
Summaries and chat	Qwen3.5 4B	Q4_K_M	2.7 GB	About 100 tokens/s
Autocomplete	Gemma 4 E4B	UD-Q5_K_XL	6.7 GB	About 78 tokens/s
Embeddings	Qwen3-Embedding 0.6B	Q8_0	0.6 GB	—
Diarization	pyannote community-1	Core ML	0.1 GB	—

The full list of changes is in the 0.5.0 release notes.

Introducing LokalBot

2026-07-01T00:00:00Z

Update, October 2026: this is the original announcement. The default models have changed since (see Five local models, 6.8 GB of weights), the app now checks for updates (see the privacy policy for every network use), and LokalBot also runs on Windows and Linux.

One app instead of several

Recording meetings, autocompleting text, and keeping track of the day usually means several separate apps, each with its own subscription and its own cloud. LokalBot does all of it in one Mac app, with no third-party inference cost. It's free and open source under the GPLv3.

The model stack

Summaries, chat, and autocomplete run on a bundled llama.cpp: in-process for autocomplete's low latency, llama-server otherwise. Any of them can point at your own GGUF file, Ollama, an OpenAI-compatible endpoint, or Apple Intelligence.
Transcription uses Granite Speech 4.1, Parakeet, Whisper, or Qwen3-ASR through Core ML and MLX on Apple Silicon. Parakeet runs at about 190 times real time.
Semantic search uses Qwen3-Embedding 0.6B on a second llama-server, with vectors in SQLite and brute-force cosine similarity. At personal scale, brute force is instant, and it adds no dependencies.
Speaker separation is optional: pyannote, through FluidAudio, splits the other side of a call into individual speakers.
An in-app Hugging Face browser searches and downloads GGUF models, with a note on whether each one fits your Mac.

Launch defaults

Transcription: IBM Granite Speech 4.1 2B, Q4.
Summaries: Qwen 3.6 35B-A3B, Q4_K_M.
Autocomplete: Gemma 4 E4B, Q5 XL.

These run well on an M4 Max with 48 GB of memory. With 16 to 24 GB, choose one of the lighter alternatives in the app's model settings.

Private by design

After the one-time model download, LokalBot works offline. Optional screenshots are sealed with AES-GCM and deleted automatically. LokalBot needs an Apple Silicon Mac with macOS 15 or later.

Get it from GitHub, and join the launch discussion on r/LocalLLaMA, especially if you have better local speech or small, fast autocomplete models to suggest.