Agent-Cli | Bas Nijholt

Frontier-level dictation on your iPhone keyboard, self-hosted

Tue, 22 Sep 2026 00:00:00 +0000

On a recent month-long trip visiting family in Europe, I did a lot of work from my phone, using my mobile coding workflow. It still boggles my mind that real, productive work from a phone is possible now. Most of that work is talking to coding agents, and the built-in iOS dictation is garbage for that. I had an iOS Shortcut that sent recordings to Agent CLI, my local AI toolbox, running at home instead. The problem was that the recording screen takes over the whole display, so I could not see the thing I was commenting on. The transcript then landed in my clipboard, and I had to paste it myself.

So I threw my principles overboard and installed Wispr Flow, a voice keyboard for iOS. I loved it instantly. iOS does not let keyboards use the microphone, so the keyboard’s button hands off to the Wispr Flow app, which keeps recording in the background. I was less happy about that, since it meant a closed-source app always had the microphone open and sent my voice to their servers. I was on the free plan, so I was probably the product. I also hit the free weekly word limit very quickly, and started looking for an alternative.

That is how I found Diction, which has since become my favorite app when I’m away from my laptop. It works the same way, background recording included. The difference is that it has a self-hosted mode with an open-source gateway, so my voice goes to a machine I own instead of their servers.

I run the gateway on my home machine in the U.S., reachable only over my private network. The gateway streams audio to Agent CLI, which runs Alibaba’s Qwen/Qwen3-ASR-1.7B-hf on an RTX 3090 behind an OpenAI-compatible transcription endpoint. There is no LLM cleanup step, so Diction’s Writing Style and Tones features do not work with this setup. I don’t miss them: Qwen’s raw transcription is already good enough for me.

1. Why Qwen? #

For the longest time I used Faster-Whisper with Whisper large-v3. Whisper is still the default choice for most people, but many newer models beat it. I tried some of them, including NVIDIA’s Parakeet. Parakeet is fast, but it cannot take custom instructions. Those instructions are how Diction’s My Words feature biases a transcription toward specialized names.

At the start of September, I looked at the Hugging Face Open ASR Leaderboard, which tests speech-to-text models on the same recordings and ranks them. The score is the word error rate (WER): the share of words a model gets wrong, counting words it swaps, drops, or makes up. Lower is better, and a WER of 4% means about one wrong word in every 25. The test recordings range from audiobooks and podcasts to meetings and earnings calls, so the average says more than any single clean benchmark.

These are the averages on the public English test sets as of September 19, 2026:

Model	WER	Can I run it at home?
Zoom Scribe v2 Pro	3.6%	No, paid API (#1 overall)
ElevenLabs Scribe v2	4.0%	No, paid API
Qwen3-ASR 1.7B	4.3%	Yes, Apache-2.0
AssemblyAI Universal-3.5 Pro	4.3%	No, paid API
NVIDIA Parakeet TDT 0.6B v3	4.9%	Yes
OpenAI Whisper large-v3	5.8%	Yes

I picked the best model on the board whose weights you can download: Qwen3-ASR 1.7B. It lands within a point of the best paid API, which is what I mean by frontier-level. Since I maintain Agent CLI, I told an agent to add a Qwen backend, and shortly afterward it had landed and was running on my server.

Agent CLI’s transcription server loads a model on the first request and unloads it after an idle timeout, so the model only takes VRAM while I use it. The same process serves an OpenAI-compatible API, which the Diction gateway talks to, and the Wyoming protocol that Home Assistant uses for local voice.

2. The recipe #

You need an NVIDIA GPU with the NVIDIA Container Toolkit, Docker Compose 2.30 or newer, and a private route from your iPhone to the server. I use Headscale, but Tailscale or plain WireGuard works just as well. Do not expose the gateway to the public internet.

Write the Compose file #

Save the following as compose.yaml in an empty directory. Agent CLI’s CI publishes a CUDA image with the Transformers backend and Qwen as the default model, so there is nothing to build. The file points Diction’s gateway at Agent CLI’s transcription port. WHISPER_TTL raises the idle timeout from the default five minutes to a day, so dictation rarely waits for a reload:

services:
 asr:
 image: ghcr.io/basnijholt/agent-cli-whisper:0.112.0-cuda-transformers
 environment:
 WHISPER_TTL: "86400"
 volumes:
 - model-cache:/home/whisper/.cache
 gpus: all
 restart: unless-stopped

 gateway:
 image: dictionlabs/gateway:v14.0
 depends_on:
 - asr
 environment:
 CUSTOM_BACKEND_URL: http://asr:10301
 CUSTOM_BACKEND_MODEL: Qwen/Qwen3-ASR-1.7B-hf
 CUSTOM_BACKEND_CANONICAL_ID: Qwen/Qwen3-ASR-1.7B-hf
 CUSTOM_BACKEND_NEEDS_WAV: "true"
 ports:
 - "8080:8080"
 restart: unless-stopped

volumes:
 model-cache:

Start it and connect the app #

Start both services:

docker compose up -d

The first transcription downloads Qwen into the model-cache volume, so that request can take a few minutes. docker compose logs -f shows the progress.

On the iPhone, go to Settings → General → Keyboard → Keyboards → Add New Keyboard → Diction. Tap Diction in the keyboard list, enable Allow Full Access, and grant microphone access when prompted. Then open Diction, go to Preferences → Mode → Self-Hosted, enter http://<private-server-ip>:8080, and tap Test connection.

On my machine, Qwen3-ASR 1.7B uses about 4.7 GiB of GPU memory and roughly 3 GiB of system memory while loaded.

3. The bar test #

A day later, I met a friend in a bar in the Netherlands and wanted to let him try it.

He was already on my tailnet for some of my other self-hosted services, so his phone only needed an ACL change to reach the gateway. I opened the terminal app on my phone, connected to an agent, and dictated that change. In less than a minute, Diction was connected and working on his phone. His speech crossed the Atlantic, was transcribed by Qwen on my home GPU, and appeared back in the app with barely any delay.

That moment sold the whole setup to me. I was in a bar without a laptop, and I used dictation to grant access to the dictation service.

Auto-installing extras at runtime so users never have to think about dependencies

Wed, 11 Feb 2026 00:00:00 +0000

Table of contents

1. From one command to a dependency maze #

agent-cli started as a simple thing. Voice in, text out. It connected to a Wyoming Whisper server for speech-to-text and a Piper server for text-to-speech, basically a local, open-source Siri triggered from terminal hotkeys. I wrote about the early version in my local AI journey post.

And then, as tends to happen with side projects I use every day, I kept adding things.

First a RAG proxy with ChromaDB, so I could chat with my documents. Then a long-term memory system, so conversations could remember context across sessions. Server commands that run Kokoro TTS or Whisper locally, optimized for whatever hardware you have—Apple Silicon gets MLX, NVIDIA gets CTranslate2. Voice activity detection with Silero VAD. Audio speed adjustment. An agent orchestration system. systemd/launchd service management for running background daemons.

Each of these features pulled in wildly different dependency trees. torch and transformers for GPU-accelerated Whisper. sounddevice and numpy for microphone access. chromadb and onnxruntime for vector storage. kokoro with its entire spacy pipeline for neural TTS.

Today the project has 13 optional extras. The tool became much more useful, but dependency UX got progressively worse with every feature.

2. Why “just use extras” breaks down #

Python’s optional-dependencies in pyproject.toml is the standard mechanism for this, and it’s the right packaging primitive. You define groups, users install with pip install agent-cli[rag]. On paper, this is perfectly reasonable.

In practice, it falls apart when you have 13 of them:

[project.optional-dependencies]
audio = ["numpy", "sounddevice>=0.4.6", "agent-cli[wyoming]"]
llm = ["pydantic-ai-slim[openai,google,duckduckgo,vertexai]>=0.1.1"]
rag = ["pydantic-ai-slim[openai,google]>=0.1.1", "chromadb>=0.4.22", ...]
memory = ["pydantic-ai-slim[openai,google]>=0.1.1", "chromadb>=0.4.22", ...]
vad = ["onnxruntime>=1.16.0"]
faster-whisper = ["faster-whisper>=1.0.0"]
mlx-whisper = ["mlx-whisper>=0.4.0; sys_platform == 'darwin' and platform_machine == 'arm64'"]
kokoro = ["kokoro>=0.9.0", "soundfile>=0.12.0", ...]
piper = ["piper-tts>=1.2.0"]
# ... and more

The UX problem is obvious. If I want to transcribe voice to text, do I need audio? llm? vad? All three? What if I want the Whisper server to run locally—is that faster-whisper or mlx-whisper? Do I also need server for that?

Expecting users to study a dependency matrix before running a CLI command is a terrible experience. I wanted three things:

A lightweight base install that’s fast and doesn’t pull in heavy ML dependencies.
Commands that “just work” when first used.
A clear opt-out for people who don’t want automatic installs.

3. The decorator #

The core idea is simple. Each CLI command declares what it needs, and the decorator handles the rest:

@app.command("transcribe", rich_help_panel="Voice Commands")
@requires_extras("audio", "llm")
def transcribe(...):
 ...

That’s it from the command author’s perspective. One line, and you never think about dependency management in that command again.

Here are a few more examples from the codebase to show how this scales:

# Continuous transcription needs audio, voice activity detection, and an LLM
@requires_extras("audio", "vad", "llm")
def transcribe_live(...):
 ...

# TTS server needs the server framework + one of two TTS backends + Wyoming protocol
@requires_extras("server", "piper|kokoro", "wyoming")
def start_tts_server(...):
 ...

# Whisper server: any of three ASR backends will do
@requires_extras("server", "faster-whisper|mlx-whisper|whisper-transformers", "wyoming")
def start_whisper_server(...):
 ...

Notice the pipe syntax: "piper|kokoro" means “you need either Piper or Kokoro—any one of these satisfies the requirement.” On Apple Silicon, if you already have mlx-whisper installed, it won’t try to install faster-whisper instead.

The decorator itself is straightforward:

def requires_extras(*extras: str) -> Callable[[F], F]:
 def decorator(func: F) -> F:
 @functools.wraps(func)
 def wrapper(*args, **kwargs):
 if _check_and_install_extras(extras):
 raise typer.Exit(1)
 return func(*args, **kwargs)
 return wrapper
 return decorator

The real complexity lives in _check_and_install_extras, which handles several installation scenarios. But the interface for command authors is always a single line.

4. Checking without importing #

You don’t want to import torch just to check if it’s available—that takes seconds and loads hundreds of megabytes into memory. Instead, I use importlib.util.find_spec, which checks whether a module exists on the import path without executing any of its code. No heavy imports, no GPU initialization, no side effects. The check is essentially free.

5. The metadata bridge #

There’s a gap between what pyproject.toml knows and what the runtime check needs. pyproject.toml says the rag extra requires chromadb>=0.4.22—that’s the pip package name. But at runtime, I need to check Python import names. Sometimes these are the same (chromadb → import chromadb), but often they’re not: pydantic-ai-slim is imported as pydantic_ai, piper-tts as piper.

To bridge this, I generate an _extras.json file that maps each extra to its description and the Python import names needed for the runtime check:

{
 "audio": ["Audio recording/playback", ["sounddevice"]],
 "rag": ["RAG proxy (ChromaDB, embeddings)", ["chromadb", "pydantic_ai"]],
 "memory": ["Long-term memory proxy", ["chromadb", "yaml", "pydantic_ai"]],
 "vad": ["Voice Activity Detection (Silero VAD via ONNX)", ["onnxruntime"]],
 "faster-whisper": ["Whisper ASR via CTranslate2", ["faster_whisper"]]
}

This file is generated by a script that parses pyproject.toml and applies a mapping table for the package-name-to-import-name conversions. A pre-commit hook ensures it stays in sync—if you add an extra to pyproject.toml and forget to update the metadata, the commit fails.

I wish Python packaging had a standard way to discover a package’s top-level import name from its distribution name. It doesn’t, so the mapping table is the least-bad option I found.

6. Three environments, three strategies #

The hard part is that people run agent-cli in several different ways, and each needs a different installation strategy.

A) `uv tool install` — the persistent environment #

When installed via uv tool install agent-cli, the tool lives in a persistent environment under ~/.local/share/uv/tools/agent-cli/. This environment has a uv-receipt.toml file that tracks which extras are installed.

When auto-install triggers, the system reads the current extras from that receipt, merges in the new ones, and reinstalls:

def install_extras_impl(extras, *, quiet=False):
 if is_uv_tool_install():
 current_extras = _get_current_uv_tool_extras()
 new_extras = sorted(set(current_extras) | set(extras))
 return _install_via_uv_tool(new_extras, quiet=quiet)

So if you already had audio installed and now run a command that needs llm, it becomes uv tool install agent-cli[audio,llm] --force.

The key insight: uv tool upgrade reads uv-receipt.toml and preserves the extras list. Once auto-install adds an extra, it persists across future upgrades. You never have to install it again.

(Full implementation)

B) `uvx` — the ephemeral environment #

uvx is uv’s tool for running packages in temporary, disposable environments. Running uvx agent-cli transcribe creates a fresh environment, installs agent-cli, runs the command, and throws the environment away.

The problem: there’s nowhere to persist extras. No uv-receipt.toml to update, no virtualenv that survives the invocation.

The solution is one of my favorite tricks in this system: re-execute the entire command with the extras baked into the uvx invocation itself:

def _maybe_reexec_with_uvx(extras):
 if os.environ.get(_REEXEC_MARKER) or not _is_uvx_cache():
 return
 uvx_path = shutil.which("uvx")
 if not uvx_path:
 return
 extras_str = ",".join(extras)
 cmd = [uvx_path, "--python", "3.13",
 f"agent-cli[{extras_str}]", *sys.argv[1:]]
 _maybe_exec_with_marker(cmd, f"Re-running with extras: {extras_str}")

So uvx agent-cli transcribe transparently becomes uvx agent-cli[audio,llm] transcribe. The user sees a brief “Re-running with extras: audio, llm” message, and then the command works.

Environment detection is straightforward—uvx environments live under the uv cache directory:

def _is_uvx_cache():
 prefix_str = Path(sys.prefix).resolve().as_posix()
 return "/cache/uv/" in prefix_str or "/archive-v" in prefix_str

(Full implementation)

C) Regular virtualenv, `pipx`, or system install #

For plain pip install agent-cli, pipx, or virtualenv installs, the system falls back to direct installation. Each extra has a pre-generated requirements file with pinned versions (e.g., _requirements/audio.txt), and the install command adapts to whatever’s available:

def _install_cmd():
 in_venv = sys.prefix != sys.base_prefix
 if shutil.which("uv"):
 cmd = ["uv", "pip", "install", "--python", sys.executable]
 if not in_venv:
 cmd.append("--system")
 return cmd
 cmd = [sys.executable, "-m", "pip", "install"]
 if not in_venv:
 cmd.append("--user")
 return cmd

Prefers uv pip when available (fast), falls back to regular pip. Uses --system or --user as appropriate when not in a virtualenv. This is also the path that pipx installations take—pipx creates a regular virtualenv under the hood, so the auto-installer detects it as a venv and installs directly into it.

(Full implementation)

7. Preventing infinite re-execution loops #

There’s an obvious footgun here: what if the install succeeds but the package still isn’t importable for some reason? You’d get an infinite loop of install → re-exec → check → install → re-exec → …

The fix is a simple environment variable marker:

_REEXEC_MARKER = "_AGENT_CLI_REEXEC"

def _maybe_exec_with_marker(cmd, message):
 if os.environ.get(_REEXEC_MARKER):
 return # Already re-executed once, don't loop
 new_env = os.environ.copy()
 new_env[_REEXEC_MARKER] = "1"
 os.execvpe(cmd[0], cmd, new_env)

os.execvpe replaces the current process entirely—it never returns. The marker ensures we only do this once. If the second run still can’t find the packages, it falls back to a helpful error message instead of looping forever.

8. What users actually see #

With auto-install enabled (the default), a first run looks like this:

Auto-installing missing extras: audio, llm
Running: uv tool install agent-cli[audio,llm] --force --python 3.13 -q
Installation complete!
Re-running with installed extras...

And then the command runs normally. Next time, it runs instantly—no install needed.

With auto-install disabled, the user gets a clear message telling them exactly what to do:

This command requires the 'audio' extra (Audio recording/playback).

Install with:
 uv tool install -p 3.13 "agent-cli\[audio]"
 # or
 agent-cli install-extras audio

If you prefer explicit control, there’s also a manual install-extras command:

agent-cli install-extras rag memory # Install specific extras
agent-cli install-extras --all # Install everything
agent-cli install-extras --list # Show available extras with descriptions

You can opt out via environment variable or config file:

export AGENT_CLI_NO_AUTO_INSTALL=1

# ~/.config/agent-cli/config.toml
[settings]
auto_install_extras = false

9. Keeping everything in sync #

The runtime system depends on two generated artifacts:

_extras.json: maps extras to descriptions and import names (for the find_spec check)
_requirements/*.txt: pinned dependency files for each extra (for the pip/uv install)

_extras.json is generated by sync_extras.py (run manually or in CI). The requirements files are generated by sync_requirements.py, which runs as a pre-commit hook whenever pyproject.toml or uv.lock changes. A check script enforces that _extras.json stays consistent with pyproject.toml—if you add an extra but forget to update the metadata, the commit fails with a clear error.

This means the runtime system never drifts from the source of truth. pyproject.toml is the canonical definition, everything else is derived.

10. Why uv made this practical #

I want to be clear: this system would be much harder without uv.

uv tool install with persistent extras in uv-receipt.toml—extras survive upgrades automatically
uvx for ephemeral execution with extras specified inline
uv pip install that’s fast enough for runtime auto-install to feel acceptable
uv export for generating pinned requirements files from uv.lock

With traditional pip, auto-installing at runtime would take 30-60 seconds and feel terrible. With uv, it typically takes a second. That’s the difference between “annoying” and “barely noticeable.”

11. Tradeoffs #

This approach works well, but it’s still a tradeoff:

There’s metadata to maintain. The package-name-to-import-name mapping requires a lookup table with special cases like piper-tts → piper and pydantic-ai-slim → pydantic_ai. I wish pyproject.toml had a field for this.
The check only verifies importability, not version compatibility. find_spec tells you a module exists, not whether it’s the right version. In practice this hasn’t been a problem because uv’s pinned requirements handle versions, but it’s a theoretical gap.
Auto-install can fail in restricted environments. Corporate machines, read-only containers, air-gapped systems—the opt-out exists for a reason.
Re-exec is a CLI pattern, not a library pattern. os.execvpe replaces the process, which means any shell hooks or signal handlers set up before the decorator runs are lost. Fine for CLI tools, would be problematic for libraries.

12. Not dependency injection #

I originally called this “dependency injection” in my head, but that’s the wrong term entirely. Dependency injection is about providing object dependencies through constructors or function parameters. This is something different.

The best names I’ve come up with:

runtime optional dependency resolution — accurate but a mouthful
on-demand extras installation — emphasizes the user experience
lazy extras — short, probably what I’d use in conversation

If there’s a standard term for this pattern, I haven’t found it.

13. Conclusion #

The underlying principle is simple: the tool should know what it needs, and it should get it. Users shouldn’t have to study a matrix of 13 optional extras to figure out which pip command to run. They should just type the command they want, and it should work.

uv’s speed and tooling model made this practical. A decorator made it ergonomic for command authors. Pre-commit hooks made it maintainable. And three environment-detection strategies made it work regardless of how the user installed the tool.

If you’re building a Python CLI with optional features that keep growing, consider this pattern. Your users will thank you, mostly by never having to ask “which extras do I need?”

The code discussed here is part of agent-cli, an open-source suite of AI-powered command-line tools. The implementation lives primarily in agent_cli/core/deps.py.

Parallel agentic coding made trivial with `agent-cli dev`

Tue, 13 Jan 2026 00:00:00 +0000

Table of contents

1. The “just use worktrees” problem #

Last week, I was chatting with a friend about my agentic coding workflow. I was sharing some of my best practices, and when the topic of parallel development came up, I said:

“Just use git worktrees with quickly installable environments.”

He stared at me blankly.

And then it hit me: for most developers who just git add, git commit, git push and don’t touch their environment setup once it works, this is not trivial. What seems obvious to me involves:

Understanding what git worktrees even are
Knowing the git worktree add command and its flags
Remembering to fetch before creating
Creating a sibling directory structure
Running your package manager’s install command
Copying over .env files
Setting up direnv or manually activating your virtual environment
Opening a new terminal tab
Navigating to the worktree
Finally launching your coding agent

That’s a lot of friction for something you want to do frequently when using agentic tools.

So I built agent-cli dev.

2. Why parallel development matters for agentic coding #

In my previous post on agentic coding, I described my workflow of running 5-6 Claude Code sessions simultaneously, each working on a different feature. This is how I can easily work on multiple features in parallel while waiting for an agent to process and generate code.

The usual way: Start feature A → wait for Claude to finish → review → move to feature B → repeat.

The better way: Start feature A in tab 1 → switch to tab 2, start feature B while A is working → switch to tab 3, start feature C → circle back to tab 1, review what Claude did → give feedback, move to next tab → repeat every 10-15 minutes.

This parallel approach allows me to go much faster. But it only works if each tab has its own completely isolated and working environment.

That’s where git worktrees come in. A worktree gives you a separate working directory for a branch, sharing the same git history but with independent file states.

The problem is the setup. Every time you want to start a new feature:

# The manual way (don't do this)
git fetch origin
git worktree add ../my-project-worktrees/cool-feature -b cool-feature origin/main
cd ../my-project-worktrees/cool-feature
uv sync --all-extras # or npm install, or poetry install, or...
cp ../.env .env
# Now open a new terminal tab...
# Navigate there...
# Finally start claude

After the 100th(?) time doing this manually, I knew I needed to automate it.

3. One command to rule them all #

Here’s what I built:

agent-cli dev new --agent

That’s it. One command. Here’s what it looks like:

→ Generated branch name: cool-crane
→ Creating worktree for branch 'cool-crane'...
→ Running: git fetch origin
→ Running: git worktree add -b cool-crane ~/Work/project-worktrees/cool-crane origin/main
✓ Created worktree at ~/Work/project-worktrees/cool-crane
✓ Copied env file(s): .env.example, .envrc
→ Detected Python project with uv
→ Running: uv sync --all-extras
→ Running: direnv allow
✓ direnv: allowed existing .envrc
✓ Started claude in new iterm2 tab

╭─────────────────────────────────────────────── Success ───────────────────────────────────────────────╮
│ Dev environment created: ~/Work/project-worktrees/cool-crane │
│ Branch: cool-crane │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────╯

4 seconds later, I have a new terminal tab with Claude Code already running in a fresh, isolated environment. Ready to work on my next feature.

What happens under the hood:

Generates a random branch name like clever-fox (you can just set the name yourself of course)
Creates the git worktree in a sibling directory
Detects the project type and runs the appropriate setup (uv sync, npm install, etc.)
Copies .env files from the main repo
Sets up direnv so the environment activates automatically
Opens a new terminal tab (works with tmux, zellij, iTerm2, and more)
Launches your coding agent in that tab

The Docker-style random names (clever-fox, happy-bear, swift-owl) there just for those too lazy to pick a name. Just agent-cli dev new --agent and start coding.

4. It knows your stack #

The command auto-detects your project type:

If it sees…	It runs…
`uv.lock`	`uv sync --all-extras`
`pixi.lock`	`pixi install`
`poetry.lock`	`poetry install`
`requirements.yaml`	`unidep install -e .`
`package-lock.json`	`npm install`
`Cargo.toml`	`cargo build`
`go.mod`	`go mod download`

For my projects using unidep, it even detects monorepos and adjusts the command accordingly.

It also generates the right .envrc for direnv. For a Python project with uv, that’s source .venv/bin/activate. For a conda/unidep project, it’s the full micromamba activation. For Nix projects, it’s use flake.

This means when you cd into the worktree later, your environment is automatically activated. No more source .venv/bin/activate.

5. Terminal magic #

The real magic is opening a new terminal tab with the agent already running.

The command detects which terminal you’re in and uses the appropriate API—whether that’s iTerm2, Kitty, Warp, or GNOME Terminal. It even works with terminal multiplexers like tmux and zellij (my current favorite), opening a new window/tab in your existing session.

6. Spawning agents with prompts #

The --prompt option transforms how I spawn parallel agents. Instead of just opening a blank Claude session, I can give the new agent its task immediately:

agent-cli dev new fix-auth-bug --agent --prompt "Fix the login validation bug in auth.py. The issue is that empty passwords pass validation."

The agent opens with the prompt already loaded and starts working immediately. No copy-pasting, no context switching, no “let me explain what I need you to do.”

For longer, more detailed prompts (which you should use for complex tasks), there’s --prompt-file:

agent-cli dev new new-feature --agent --prompt-file .claude/spawn-prompt.md

This is particularly powerful when you’re in a Claude session and discover a feature that needs work but is unrelated to your current task. Instead of context-switching or noting it for later, you can spawn a new agent right there:

Write a detailed prompt to a file (Claude can do this)
Run agent-cli dev new unrelated-feature --agent --prompt-file .claude/spawn-prompt.md
A new terminal tab opens with a fresh Claude session already working on the task
Continue with your original work

The spawned agent works independently in its own branch while you stay focused.

7. Teaching Claude to spawn agents #

Here’s where it gets really interesting: you can teach Claude Code itself how to spawn parallel agents.

agent-cli dev install-skill

This installs a Claude Code skill that teaches Claude how to use agent-cli dev. The skill includes detailed examples and prompt templates following Anthropic’s prompt engineering guide.

With the skill installed, you can say things like:

“Work on auth, payments, and notifications in parallel”
“This feature is unrelated to what we’re doing—spawn a new agent for it”
“Split this refactoring by module and work on them simultaneously”

Claude will write well-structured prompts (using proper XML tags, clear context, explicit requirements) and spawn the agents for you. Each spawned agent writes a report to .claude/REPORT.md when done, so you can easily review what happened.

One of my favorite patterns: I ask Claude to look at the latest 10 open GitHub issues and spawn an agent for each one. Ten independent parallel environments, set up in seconds, each working on a different issue. That’s the kind of leverage this enables.

8. My actual workflow #

Here’s how I use this in practice. I’m working on agent-cli itself, and I want to add three features:

# Tab 1: New RAG feature
agent-cli dev new rag-improvements --agent --prompt "Improve RAG chunking strategy for better retrieval accuracy"

# Tab 2: Fix a bug in the transcribe command
agent-cli dev new fix-transcribe-vad --agent --prompt "Fix VAD detection cutting off words at sentence boundaries"

# Tab 3: Refactor the config system
agent-cli dev new config-refactor --agent --prompt "Refactor config.py to use a single Config class with lazy loading"

Three commands, three isolated environments, three Claude Code sessions already working on their tasks.

I cycle through the tabs every 10-15 minutes:

Review what Claude did
Give voice feedback using my transcription workflow
Move to the next tab

When a feature is done, I have Claude open a PR from that branch. When the PR is merged:

agent-cli dev clean --merged

This uses the GitHub CLI to check which branches have merged PRs, then removes those worktrees and deletes the branches. No more zombie worktrees cluttering my disk.

9. Managing environments #

A few other commands I use regularly:

# List all worktrees
agent-cli dev list

# Navigate to a worktree
cd "$(agent-cli dev path clever-fox)"

# Remove a worktree
agent-cli dev rm clever-fox

# Force removal + delete branch
agent-cli dev rm clever-fox --force --delete-branch

# Check what's available on your system
agent-cli dev doctor

The doctor command is handy for seeing which terminals, editors, and coding agents are installed and detected.

10. Getting started #

agent-cli dev is part of the agent-cli package:

# With uv (recommended)
uv tool install agent-cli

Then in any git repository:

# Check what's available
agent-cli dev doctor

# Create your first parallel environment with a task
agent-cli dev new my-feature --agent --prompt "Add user profile page with avatar upload"

# Or install the skill so Claude can spawn agents for you
agent-cli dev install-skill

Supported coding agents: Claude Code, Codex, Gemini, Aider, GitHub Copilot, Continue, OpenCode, and Cursor Agent. Use --with-agent gemini to pick a specific one, or set a default in your config.

Supported terminals: tmux, zellij, iTerm2, Terminal.app, Kitty, Warp, and GNOME Terminal.

Supported editors: Cursor, VS Code, Zed, PyCharm (and other JetBrains IDEs), Neovim, Vim, Emacs, Sublime Text, and nano.

Conclusion #

The barrier between “I want to work on this feature” and “I’m actually working on it in an isolated environment” should be zero.

Before:

2-5 minutes of manual setup per feature
Often skipped because “it’s not worth it for a small change”
Mental overhead remembering all the steps

After:

One command, few seconds
No excuse not to work in isolation
More parallel features, faster iteration

If you’re using agentic coding tools and not yet working in parallel, you’re leaving productivity on the table. And if the setup friction was stopping you—well, now it’s gone.

Try it out and let me know what you think! The code is open-source at github.com/basnijholt/agent-cli.

Links #

SOTA RAG & Memory without the database: Files, Git, and simple folders

Wed, 10 Dec 2025 00:00:00 +0000

Table of contents

1. The amnesia problem #

If you’ve read my posts on agentic coding or my local AI journey, you know I am all-in on using AI to control my computer and write software. But one critical piece was missing: Context.

LLMs are powerful but stateless. Every time I start a new chat, the model forgets who I am, what I’m working on, and that I prefer functional Python code over object-oriented styles. Worse, it also forgets the thing I just told it five minutes ago in another window.

The standard solution to this is RAG (Retrieval-Augmented Generation) and Memory. But when I looked at the available tools, I hit a wall.

Most solutions fell into two buckets:

SaaS / Web UIs: Upload your files to a cloud service or use a specific web chat interface that locks you in.
Developer Libraries: “Just use LangChain/LlamaIndex!” they say. Great, now I have to write a Python script every time I want to ask a question about a PDF?

I didn’t want a library. I didn’t want a web UI. I wanted a folder. I wanted to drop a file into ~/my-docs, and have my AI instantly know about it. And I wanted my AI’s “memory” to be a file I can edit, not a hidden vector in a database I can’t inspect.

So I decided to build my own file-based solution.

2. Learning from failure: AI Journal #

This isn’t my first attempt at building a memory system.

A few months ago, I built AI Journal—an ambitious local-first journaling app where an AI would read your entries and build a “living self-model” of who you are. It had sophisticated features: a structured “persona core,” typed “claim atoms” with evidence links, four organizational layers (L1→L4), recency-aware scoring with decay functions, and Git-based time travel to “ask questions of your younger self.”

The architecture seemed solid. The problem was simpler: local models aren’t good enough yet.

I designed the whole thing to run entirely on Ollama with my dual RTX 3090s. But even with a 48GB of VRAM, the models couldn’t reliably extract structured claims, handle the multi-step reconciliation, or maintain coherence across long conversations. Whether the design was right, I still don’t know—but the execution definitely hit the wall of current local LLM capabilities.

The lesson: I tried to build the entire system at once without validating the core concepts and individual ideas first. This time, I took a different approach: tackle it problem by problem. Instead of designing a complete system upfront, I would validate each piece—chunking, retrieval, re-ranking, memory reconciliation—before combining them. The experience and UX would be my own, but the algorithms would be adapted from established projects.

3. How I studied SOTA: clone, read, re-implement #

My approach was more methodical this time. I cloned five major frameworks into the same workspace as agent-cli:

LlamaIndex: The OG RAG framework
LangChain: The kitchen-sink approach
Letta (formerly MemGPT): Stateful agents with long-term memory
Mem0: Focused specifically on memory extraction
PydanticAI: Clean, typed agent framework

I didn’t read through all this code myself—that would take months. Instead, I used AI coding agents to search the codebases and write summaries of the technical implementation details:

Gemini CLI (Gemini 3.0 Pro)
Claude Code (Opus 4.5)
Codex CLI (GPT-5.1-Codex-Max)

I fed those summaries into ChatGPT 5 Pro together with my high level plan of what I wanted to create, and it created an implementation plan. Then I took that plan back to the coding agents for implementation. I personally found Opus 4.5 the best coding model, but GPT‑5.1 Codex Max was the best at reviewing the code and actually identifying critical bugs.

For validation, I used clip-files to copy the agent-cli source code plus the architecture documents (like memory.md and rag.md) into ChatGPT 5 Pro to validate the approach.

The AI did the searching and implementing; I did the understanding and decision-making.

4. The dependency problem: why ONNX over PyTorch #

One thing became immediately clear when going through these frameworks: they’re massive.

Installing LlamaIndex with all its dependencies pulls in PyTorch. PyTorch alone is 2-3GB. Add transformers, sentence-transformers, and the kitchen sink, and you’re looking at an 8GB+ environment just to ask questions about some PDFs.

This felt excessive for a CLI tool.

One of my core design goals was minimal dependencies. I wanted agent-cli to install in seconds, not minutes. I wanted it to work on my Mac, my NixOS server, and my friend’s laptop without downloading half the internet.

The solution: ONNX Runtime.

ONNX Runtime is ~200MB and runs inference on basically anything—CPU, CUDA, even Apple Silicon. For embeddings, I use embeddinggemma running locally via llama.cpp. For re-ranking, I use a cross-encoder exported to ONNX (Xenova/ms-marco-MiniLM-L-6-v2).

The result: pip install agent-cli[rag] takes about 30 seconds and adds ~300MB. The key insight: embeddings and re-ranking are cheap operations. You don’t need a GPU for them—CPU is perfectly fine. PyTorch with CUDA support can easily reach 2-3GB+; ONNX Runtime is ~200MB and runs anywhere. Since I’m not training models, just running inference on small encoders, ONNX is the obvious choice.

5. RAG: just let me grep it (kind of) #

My requirement for RAG was simple: Files on disk are the source of truth.

I didn’t want to manage an ingestion pipeline or trigger APIs. I implemented a RAG proxy that uses OS-level file watchers (watchfiles). Here is the workflow:

I drop a PDF, Markdown, or Python file into ~/rag_docs.
The system detects the Create event.
It chunks the text and embeds it into a local ChromaDB instance.
It’s immediately searchable.

When moving or deleting files, the system updates the index accordingly.

The architecture: OpenAI compatibility #

The “secret sauce” isn’t the retrieval itself; it’s how I expose it. Instead of building a custom chat app, I built an OpenAI-compatible proxy.

flowchart LR Client["Client
(Cursor, Open WebUI)"] --> Retrieve subgraph Proxy ["RAG Proxy"] direction TB Retrieve --> Augment["Augment & Forward"] Chroma[("ChromaDB")] -.-> Retrieve Watcher["File Watcher"] --> Chroma end Augment --> Upstream["Upstream LLM"]

# Point any OpenAI-compatible tool at your local proxy
export OPENAI_BASE_URL=http://localhost:8000/v1

That’s it. Any tool that supports custom OpenAI endpoints—Cursor, Cline, Open WebUI, LibreChat—instantly gains the ability to “read” my document folder.

Two-stage retrieval: the SOTA approach #

I looked at LlamaIndex here. Standard “semantic search” (using just embeddings/bi-encoders) is often disappointing. It finds text that looks similar but isn’t actually relevant.

A common pattern in modern RAG systems is Two-Stage Retrieval.

Fast Stage: Retrieve 3x candidates using standard embeddings (text-embedding-3-small or local models like embeddinggemma).
Smart Stage: Use a Cross-Encoder to re-rank the results.

A cross-encoder looks at the query and the document together and outputs a relevance score. It’s computationally heavier, but since I’m running this locally on my RTX 3090 (or even CPU via ONNX), the 50ms latency penalty is worth it for results that are actually useful.

This isn’t a new idea, but implementing it cleanly with ONNX avoided the huge dependency tax.

6. Memory: files over databases #

This is where my approach diverges most from standard tools like Mem0. Most memory systems treat memories as database rows. If the AI remembers “User likes pizza,” that’s a vector in a DB. If it hallucinates and records “User hates pizza,” good luck fixing it. You need to write a script to query the DB and delete the row.

I prefer data I can inspect.

My memory system stores every fact as a Markdown file with YAML front matter.

---
id: "550e8400-e29b-41d4-a716-446655440000"
role: "memory"
created_at: "2025-12-15T10:00:00Z"
conversation_id: "default"
---
The user prefers using 'uv' for Python package management.

Git integration: versioning memory 🕰️ #

Because memories are just files, I added a feature that I haven’t seen anywhere else: Automatic Git Versioning.

Every time the AI learns a new fact, updates an old one, or deletes a contradiction, the system automatically commits the change to a local git repository.

This leads to incredible capabilities:

Inspectability: I can git log and see exactly when the AI learned something.
Correction: If the AI learns something wrong, I open the markdown file in VS Code, edit it, and save.
Reversion: Agent hallucinated wildly yesterday? git reset --hard HEAD~1.

The reconciliation loop #

The logic for storing memories isn’t just “append to file.” Digging into Mem0 (and Letta) showed me that memory systems need active reconciliation. When I say “Actually, I switched to nix instead of uv,” the system:

Retrieves existing memories about package managers.
Decides (using a lightweight LLM) whether to ADD, UPDATE, or DELETE.
Executes the file operation (moving the old fact to a deleted/ folder for audit trails).

This keeps the memory bank clean and contradictory-free, unlike simple vector stores that just accumulate conflicting junk over time.

The tiny detail that broke everything #

After implementing the memory system—mostly modeled after Mem0—I noticed it didn’t work as well. After a lot of debugging, I found a tiny difference: I was asking the model for structured data via a tool call, whereas Mem0 uses freeform JSON that the LLM returns directly.

Since I use PydanticAI, fixing this was a one-line change:

agent = Agent(
 model=model_cfg,
 system_prompt=UPDATE_MEMORY_PROMPT,
- output_type=list[MemoryDecision],
+ output_type=PromptedOutput(list[MemoryDecision]), # JSON mode instead of tool calls
 retries=3,
)

This is exactly why reading the source code matters—not just the algorithms, but the tiny details that make them work.

7. Why building this was necessary #

You might ask: “Bas, why write thousands of lines of code for this? Why not just use ChatGPT’s memory or install LangChain?”

Three reasons: Privacy, Integration, and Dependencies.

Privacy #

I want to feed my AI my financial records, my health data, and my private journals. I am not comfortable sending that to OpenAI’s long-term storage or some VC-backed startup’s vector cloud. With agent-cli, the vectors live in ~/.cache/agent-cli/chroma, the files live in ~/documents, and the only thing leaving my machine is the specific context needed for a specific query (if I use a cloud model) or nothing at all (if I use Ollama/llama.cpp).

Integration #

By building this as an API proxy, I solved the “fragmented tools” problem. Any tool that supports custom OpenAI-compatible endpoints works out of the box:

agent-cli chat in the terminal
Cursor or Cline for agentic coding
Open WebUI, LibreChat, or Lobe Chat in the browser

Because my system speaks “OpenAI,” all of these tools share the same memory and the same document index. If I tell the terminal agent “I’m working on Project X,” and then switch to Cursor, Cursor knows about Project X. I can try a new chat UI tomorrow without losing anything—just point it to localhost:8000.

Dependencies #

To me, it just doesn’t feel right that asking questions about a PDF requires downloading 8GB of PyTorch. By choosing ONNX Runtime over the PyTorch ecosystem, I kept the entire install lightweight and fast. This matters when you want to run the system on a cheap VPS or your friend’s laptop.

8. Try it yourself #

The entire system is open-source and available in agent-cli.

Try Memory #

Get an LLM that remembers you using Ollama.

# 1. Pull models and start proxy
ollama pull embeddinggemma:300m && ollama pull gpt-oss:20b
uvx -p 3.13 --from "agent-cli[memory]" agent-cli memory proxy \
 --memory-path ./my-memories \
 --openai-base-url http://localhost:11434/v1 \
 --embedding-model embeddinggemma:300m

# 2. Chat from the terminal (env vars point to proxy)
export OPENAI_BASE_URL=http://localhost:8100/v1 OPENAI_API_KEY=dummy

# Tell it something about yourself
uvx openai api chat.completions.create -m gpt-oss:20b -g user "My name is Alice and I love hiking"

# Later, ask if it remembers
uvx openai api chat.completions.create -m gpt-oss:20b -g user "What's my name?"

Try RAG #

Chat with your documents.

# 1. Start the RAG proxy (runs in foreground)
uvx -p 3.13 --from "agent-cli[rag]" agent-cli rag-proxy \
 --docs-folder ./my-docs \
 --openai-base-url http://localhost:11434/v1 \
 --embedding-model embeddinggemma:300m

# 2. Chat with your docs
OPENAI_BASE_URL=http://localhost:8000/v1 OPENAI_API_KEY=dummy \
 uvx openai api chat.completions.create \
 -m gpt-oss:20b \
 -g user "What does my documentation say about X?"

9. What I learned #

This journey reinforced a few beliefs:

Validate components first. AI Journal failed not because the ideas were wrong, but because I tried to build a complete system before validating individual components. This time I tackled problems one at a time—chunking, retrieval, re-ranking, memory reconciliation—and only combined them once each piece worked.
Study implementations, not just papers. Reading about two-stage retrieval is one thing. Reading how LlamaIndex actually implements it—the edge cases, the defaults, the performance tradeoffs—is far more valuable. Clone the repos. Read (or make AI read) the code.
Dependencies matter. A 30-second install versus a 10-minute download changes whether someone actually tries your tool. I am happy with the ONNX Runtime over PyTorch choice.
Files are the ultimate API. By stripping away the complex databases and proprietary interfaces, I ended up with a system that is robust, version-controllable, and surprisingly simple to reason about.
Use the best model for each job. I used 4 different AI models during development, each with different strengths. Gemini 3.0 Pro, Opus 4.5, GPT-5.1-Codex Max. I used all of them during the implementation phase, but each one has its own strengths. Fore example, Opus 4.5 was the best coding model, but GPT-5.1 Codex Max was the best at reviewing the code and actually identifying critical bugs (use the built-in /review feature).

Coding from my phone 📱

Mon, 20 Oct 2025 00:00:00 +0000

I am dictating this post at ten kilometers altitude on a flight home from Mexico, Blink Shell open on my iPhone, while an agentic assistant on my NixOS machine writes an initial draft for this post. Admittedly, I’m a little addicted to agentic coding: when an idea pops up, I want to jump on it right away—even if that means doing it from a cramped seat in the sky. This is the workflow I’ve arrived at after trying many alternatives, it extends my agentic coding write-up and the self-hosted AI obsession in my local AI journey. It’s my backup plan for those moments when a computer simply isn’t nearby—the rest of the time I’m still at a keyboard like any other developer. This is in no way a replacement for a proper computer/laptop workflow but gets the job done when needed. It’s a personal, mostly open-source stack built around agent-cli—my local-first voice layer. Here’s how it works: I speak into my phone, the audio goes to FasterWhisper for transcription, Ollama cleans up the text, and it lands on my clipboard ready to paste. My iOS Shortcut handles this through a small HTTP interface that agent-cli exposes. The coding agent itself still uses the best proprietary model I can access.

Table of contents

1. Why Phone Coding Works Now #

For years I used iSH (a full Alpine Linux emulator) with SSH to hop into servers from my phone, but dropped connections were a constant frustration (Mosh fixes this, more on that later). Because coding on a phone keyboard is terrible, I kept it to tiny configuration tweaks or one-off fixes of a few characters. Agentic tools changed that: with a CLI coding agent, I don’t need to type the code—I describe the change, review the patch, and run it. That made meaningful work on the phone possible for the first time, for those moments when a computer isn’t around.

I tried VS Code in the browser, bounced between iSH and Terminus for SSH, and even lived inside a handful of in-browser terminal clients. I also spent time with mobile companions like Happy and Omnara, both designed to mirror Claude Code sessions on the phone, but they still felt like extra steps between me and my actual terminal. That friction pushed me toward a phone‑ready, self‑hosted workflow that still gives me raw SSH access to my own machine when the computer is out of reach.

This post is based on the way I develop software today. Your mileage may vary, but if you also care about privacy, open tooling, and reproducible environments, I think there are useful pieces here.

2. Constraints and Trade-offs #

This setup grew organically over time. What emerged are less strict requirements and more happy accidents that turned into core benefits:

Single trust boundary: Only the model provider (OpenAI) sees code context; audio and automation stay local, and I avoid any additional third‑party relays.
Resilient sessions: Connections should survive sleep and spotty networks.
Voice-friendly: Dictation should be accurate enough that I can trust it.
Reproducible config: The entire stack must live in my dotfiles and NixOS configuration.

To give you a sense of what I tried, here is the short comparison that convinced me to roll my own:

Approach	Pros	Why I moved on
VS Code in the browser	Familiar editor UI	Needs a steady connection and still lives outside my dotfiles comfort zone
iSH / Terminus SSH	Works without extra infrastructure	Laggy, no Mosh, and awkward keybindings
In-browser terminals	Instant access from anywhere	Poor copy/paste ergonomics and flaky mobile keyboards
Happy (Claude Code companion)	Push notifications, encrypted mobile UI for Claude Code	Requires wrapping every session with a separate CLI and still abstracts away my shell, and was quite buggy for me
Omnara (agent command center)	Centralizes Claude Code/Codex sessions with terminal replay	Proxies via their servers; I already trust OpenAI for the model and don’t want another third‑party handling my code

The stack below gives me the resilience of Mosh, the ergonomics of Zellij, and full control over the AI layer.

3. Layer 1: WireGuard From the Router #

WireGuard is a modern VPN that lets me securely access my home network from anywhere. I terminate WireGuard on my router so every device in the house (and on the road) can dial home with the same config. (If you want something easier to set up without opening ports, Tailscale is a more user-friendly option.)

Server: ASUS XT8 router with WireGuard enabled via the router UI.
Client: The WireGuard iOS app with On-Demand rules so the tunnel flips on whenever I am off trusted Wi-Fi.
DNS: All mobile sessions resolve through my home DNS, so git.nijho.lt and internal services resolve instantly.

This gives Blink a local-LAN address for my desktop nixos.

4. Layer 2: Blink Shell + Mosh for Durable Sessions #

What is Mosh? It’s like SSH, but it stays connected when the network changes or the phone sleeps, and it feels faster on bad connections.

Blink Shell is my daily driver on iOS because it pairs beautifully with Mosh and has great keyboard ergonomics.

I launch sessions using mosh bas@nixos -- zellij attach --create phone (Zellij is my terminal multiplexer—more on that in the next section).
Mosh smooths over spotty LTE and keeps my session alive when the phone sleeps.

Mosh keeps the session responsive across flaky networks and sleep.

5. Layer 3: Zellij Layouts #

I use Zellij as my terminal multiplexer for mobile work.

I stick to the defaults and use a couple of predefined layouts (e.g., a “phone” layout with editor + shell panes).

Ergonomics matter even more on a glass keyboard, so I lean on a few shell helpers:

zoxide means I can jump between repos with z foo instead of pecking long paths.
Single-character aliases like p for pytest and gs for git status live in configs/shell/10_aliases.sh, keeping command entry to a minimum.
Autocompletion plugins—zsh-autosuggestions and zsh-syntax-highlighting—are wired up in configs/shell/70_zsh_plugins.sh, so I get inline hints and colour cues while typing.

I break these tricks down in more depth in Terminal Ninja, explain how I sync them with my Dotfiles, and even package binaries like zoxide with Dotbins.

6. Layer 4: `agent-cli` Server #

agent-cli runs as a small server on my NixOS machine. In this workflow I only use transcribe from iOS Shortcuts: the phone records audio, the server transcribes and cleans it up, and I paste the text. (Agent-CLI has other features like autocorrect and chat, but I don’t use them for this workflow.) On this machine I run a long‑lived agent-cli user service (systemd) so Shortcuts can POST audio to it and get cleaned text back—ready to paste into Codex CLI. The service is defined in my dotfiles—see: configs/nixos/modules/user.nix (agent-cli service).

For more background on why I built it and how I use it in practice, see Local AI Journey (§3) and Agentic Coding.

The models and services run on the same box:

FasterWhisper via faster-whisper-server for high‑accuracy streaming transcription.
Ollama for on‑device rewrite/cleanup before sending prompts to the coding agent.

My iOS Shortcut dictation workflow is slower than Apple’s built-in dictation, but the accuracy is much better—and that matters more.

For the actual coding agent, I use a fork of Codex CLI with OpenAI’s gpt-5-codex-high model (here no open-source solution matches the frontier).

7. Layer 5: The iOS Shortcut Pipeline #

My iPhone’s Action Button runs a Shortcut that records audio and sends my voice to agent-cli on my NixOS machine for transcription and cleanup.

For the full recipe, see the iOS Shortcut Guide: agent-cli/iOS_Shortcut_Guide.md.

In short: I press the Action Button, the Shortcut records a snippet, sends it to agent-cli, and copies the cleaned text to my clipboard so I can paste it into Codex CLI.

The whole loop finishes fast enough that I can capture intent by voice and paste it into Codex CLI without typing.

8. Workflow in Practice #

Here’s the typical flow:

WireGuard auto-connects when I’m off home WiFi
Open Blink—my Mosh session is still alive from last time, Zellij panes intact
Dictate the change, paste into Codex CLI, review the agent’s response
Check diffs in an adjacent Zellij pane or different Blink tab with separate Mosh session
The agent commits and pushes to a branch, then opens a PR via gh for me to review and merge

On the phone, I aim for the smallest practical edits and initial implementations. Even on personal repos, I still open a PR—sometimes prompting alone gets me to a merge‑ready result. For open‑source with real users, I finish on the computer with a careful review and any final edits.

9. Conclusion and Further Reading #

This phone setup extends my Agentic Coding workflow to mobile. I connect to the same Zellij session on nixos, so when an idea hits, I pick up exactly where I left off—same environment, same context, no setup. Everything runs on my hardware (voice processing and automation) except the coding model itself. It’s the most effective mobile workflow I’ve had so far. It’s mostly open‑source not out of dogma, but because those tools are the best options for my needs. The one exception is the coding model: there’s no true open equivalent right now, and it makes no sense to buy 20× H100s just to self‑host a frontier model even if such a high quality model would be available open-source.

If you’re curious to dive deeper, here are a few related posts:

Agentic Coding
Dotfiles: A Practical, Cross-Platform Terminal Setup
Local AI Journey: RTX 3090 Edition
Terminal Ninja
Using LLMs Effectively (already outdated at this point!)

I’d love to hear what self-hosted tricks you’re using for mobile development—reach out if you adapt this stack or build something wild on top of it.

Agent-Cli | Bas Nijholt

Frontier-level dictation on your iPhone keyboard, self-hosted

1. Why Qwen? #

2. The recipe #

Write the Compose file #

Start it and connect the app #

3. The bar test #

Auto-installing extras at runtime so users never have to think about dependencies

1. From one command to a dependency maze #

2. Why “just use extras” breaks down #

3. The decorator #

4. Checking without importing #

5. The metadata bridge #

6. Three environments, three strategies #

A) uv tool install — the persistent environment #

B) uvx — the ephemeral environment #

C) Regular virtualenv, pipx, or system install #

7. Preventing infinite re-execution loops #

8. What users actually see #

9. Keeping everything in sync #

10. Why uv made this practical #

11. Tradeoffs #

12. Not dependency injection #

13. Conclusion #

Parallel agentic coding made trivial with `agent-cli dev`

1. The “just use worktrees” problem #

2. Why parallel development matters for agentic coding #

3. One command to rule them all #

4. It knows your stack #

5. Terminal magic #

6. Spawning agents with prompts #

7. Teaching Claude to spawn agents #

8. My actual workflow #

9. Managing environments #

10. Getting started #

Conclusion #

Links #

SOTA RAG & Memory without the database: Files, Git, and simple folders

1. The amnesia problem #

2. Learning from failure: AI Journal #

3. How I studied SOTA: clone, read, re-implement #

4. The dependency problem: why ONNX over PyTorch #

5. RAG: just let me grep it (kind of) #

The architecture: OpenAI compatibility #

Two-stage retrieval: the SOTA approach #

6. Memory: files over databases #

Git integration: versioning memory 🕰️ #

The reconciliation loop #

The tiny detail that broke everything #

7. Why building this was necessary #

Privacy #

Integration #

Dependencies #

8. Try it yourself #

Try Memory #

Try RAG #

9. What I learned #

Coding from my phone 📱

1. Why Phone Coding Works Now #

2. Constraints and Trade-offs #

3. Layer 1: WireGuard From the Router #

4. Layer 2: Blink Shell + Mosh for Durable Sessions #

5. Layer 3: Zellij Layouts #

6. Layer 4: agent-cli Server #

7. Layer 5: The iOS Shortcut Pipeline #

8. Workflow in Practice #

9. Conclusion and Further Reading #

A) `uv tool install` — the persistent environment #

B) `uvx` — the ephemeral environment #

C) Regular virtualenv, `pipx`, or system install #

6. Layer 4: `agent-cli` Server #