Wispr Alternative Local LLM

Wispr Alternative Local LLM is the open-source macOS voice tool behind the current GitHub repo sebmer-com/wispr-alternatve-local-llm. It is built as a local-first alternative to cloud voice apps like Wispr Flow. The goal is simple: hold a shortcut, speak, and get useful text or an agent action without opening another app.

The app is more than a plain dictation recorder. It combines local speech recognition, configurable output routes, local command generation, Markdown note capture, Bluetooth keyboard output, and a visible Hermes Agent handoff. It is meant to sit close to the Mac desktop and help with real work.

What it does

• Records while a hotkey is held and transcribes with FluidAudio and CoreML on macOS.
• Uses the multilingual FluidAudio v3 model by default with a German language hint.
• Supports model and language overrides for other setups.
• Applies fast text replacements after speech recognition, before paste, dump, or command work.
• Pastes normal dictation into the active app through the clipboard.
• Can restore the old clipboard after paste, so dictation does not destroy what was already copied.
• Supports a two-step command mode: first speak the information, then speak what the app should do with it.
• Sends command work to an OpenAI-compatible local or hosted LLM endpoint.
• Keeps MLX and Azure provider paths available as fallback options.
• Loads prompt templates from config instead of hard-coding all LLM behavior in Swift.
• Selects local skills from skills/*/SKILL.md and can attach registered tool output.
• Dumps raw notes or command results into a Markdown daily note.
• Supports a continuous note mode with terminal commands like go, stop, status, help, and quit.
• Supports optional Bluetooth keyboard output through an ESP32 device.
• Talks to the ESP32 directly over the KBD1 serial protocol, without launching a separate keyboard process.
• Opens a real Hermes Agent terminal session for agent work and brings the final answer back to the original app or clipboard.

Why this is different

Most voice tools stop at transcription or hide the important parts in a cloud product. This project is different because the useful loop is visible and configurable. The user can choose where text goes, how command generation works, what model endpoint is used, and whether the output should be pasted, written to notes, or typed through a Bluetooth keyboard.

It is also different from a simple Whisper wrapper. It is not only “speech to text.” It is a full desktop control layer: dictation, command rewriting, note capture, skill context, tool output, optional hardware typing, and agent handoff.

Recent work

The recent work added the features that make the app feel like a real product instead of a script. Headset handling improved, so recording works better with more audio devices. Bluetooth keyboard support was added with direct Swift serial handling, chunked UTF-8 transfer, CRC checks, connection status checks, and a direct test command.

The command LLM layer was rebuilt around OpenAI-compatible chat completions. Setup can now ask for provider details, model name, base URL, and API key environment variable. API keys live in an .env file, not in JSON config. If command generation is turned off or unavailable, the app falls back to skill output or the original transcript.

The app also gained a guided setup flow, config inspection and repair commands, audio input routing, safer Bluetooth defaults, and a GitHub installer that can clone, update, build, link the command, reset state, and open setup. The recorder was made more stable, and the test suite now covers installer behavior, config setup, audio routing, Bluetooth protocol behavior, command LLM toggles, text replacements, Markdown dumping, continuous recording, and Hermes shortcut behavior.

The Hermes mode changed a lot. Instead of running a hidden command and hoping for the best, the app opens or reuses a real Hermes terminal session, visibly pastes the prompt there, tracks the run with a unique ID, and exports the answer from that same session. That makes agent work easier to trust because the user can see what happened.

Value for users

The value is speed and control. A user can speak instead of type, rewrite spoken notes with an LLM, save thoughts to a daily note, send text to another computer through Bluetooth, or hand a task to Hermes without building a whole workflow every time.

The app is useful for people who want the speed of a voice assistant but still want local control, visible behavior, and simple config files. It is especially useful for builders who switch between chat, coding, notes, browser work, and agent sessions all day.

Current direction

The current direction is to make the open-source Wispr-style alternative easier to install, easier to configure, and safer to run every day. The main work is reliability: clean setup, clear hotkeys, stable recording, predictable output, and less hidden state.

Public boundary

This page explains the public repo and product behavior. It does not include private recordings, daily notes, API keys, local config values, personal prompts, or private Hermes session contents.

July command-context update

The July work improved the command path without changing the local-first dictation flow. Command prompts now preserve the speaker’s language instead of blindly treating the instruction language as the reply language. This matters when someone dictates an instruction in one language but is replying inside a document, thread, or app that uses another.

Optional screen context can now be attached to a voice command when the user has enabled it and granted macOS permission. The image is used only for that command and removed after the result is generated. That gives an LLM more useful context for a real desktop task while keeping the feature explicit and bounded.

The result is a more practical command loop: local speech capture first, then language-aware and context-aware help only when the user asks for it.