Model-Reviews | Bas Nijholt

Self-hosting AI does not save money, and I do it anyway

Fri, 02 Oct 2026 00:00:00 +0000

I have had this debate many times, so I am finally writing it down.

Whenever I say that self-hosting AI does not save money, people hear that I am against self-hosting. That is not the point at all. I am a massive fan of open-weight models. Qwen3.8 27B runs on two RTX 3090s at home (and 15 more models), and my phone dictation goes to Qwen3-ASR on the same machine. I think open-source AI is the best thing since sliced bread.

I don’t pretend it saves money. I do it for fun, for sovereignty, and for privacy, which I come back to at the end.

All benchmark numbers and prices below come from Artificial Analysis and OpenRouter as of September 30, 2026. I did the math for the hardware I own, and for the counterarguments I hear most. All assumptions are listed at the end.

Benchmarks are not everything, and a high score does not reliably predict how a model does on real work. They are still the best we have: all models are benchmaxed, tuned to do well on the popular benchmarks, so the scores at least work as a reference frame for comparing them with each other.

Update, October 2, 2026: after the discussion on r/LocalLLaMA, I added section 5 on Macs, DGX Sparks, and AMD AI Max machines.

Table of contents

1. The model I love, and the one that beats it on price #

My favorite local model right now is Qwen3.8 27B. It came out in August and it is Apache-2.0. People often say models like this run on a single gaming GPU, but that is only true after quantizing them. At full precision, Qwen3.8 27B needs about 54 GB of memory. Quantized to about 4 bits per weight, it fits on one 24 GB card, at some cost in quality. I still split it across both of my 3090s, because the second card leaves room for a larger context window.

I compare it with OpenAI’s GPT-6 Luna, the cheap tier released on September 22. Both models let you choose how long they think, but the settings do not mean the same thing for both. Luna’s token use grows more than 20 times from its lowest setting to its highest, while Qwen’s barely changes. Qwen on low already writes more tokens than Luna on xhigh.

Average output tokens per Intelligence Index task at each reasoning setting. Qwen3.8 27B has no high or max setting. Hover for scores.

So the name of a setting says little on its own. I compare both models at xhigh, the highest setting Qwen offers, and count the tokens each one actually uses. The extra tokens Qwen needs are part of what I am measuring. There, Qwen scores 33.7 on the Artificial Analysis Intelligence Index and Luna scores 34.6. For reference, Claude Opus 4.6, a frontier model from February, scores 31.9. That is an amazing feat in itself. When Opus 4.6 was the best model we had, I never thought that within the same year I could run essentially that level of intelligence in my own house.

Artificial Analysis also publishes how many tokens each model used to run the index, and what that cost. So the question is simple: what does it cost to run the whole benchmark once?¹

Cost of one run of the Artificial Analysis Intelligence Index. The two bars for hardware at home are electricity only: measured on my 3090s, estimated for the RTX PRO 6000.

Luna runs the whole benchmark for $67. Qwen at full precision, through the cheapest provider that does not keep your data (ZDR), costs $619. Two things cause that gap: providers charge almost four times as much per output token for Qwen, and Qwen generates almost three times as many tokens to get the same work done.

The second bar is my own machine: the electricity alone for running Qwen on my GPUs costs more than Luna’s entire bill.

The third bar is what it takes to match the API’s full precision at home: an RTX PRO 6000, a workstation card with 96 GB of memory, enough for the full model.² Its electricity alone costs twice Luna’s bill, and one run takes more than seven weeks. The card by itself sells for $10,000 to $20,000, so with a computer around it, you are paying for a pretty nice car. Buy a few of them to run agents in parallel, and you are building a small data center of your own.

The bars for hardware at home are electricity only, so they are the lowest these runs can cost: the cards come on top, and how much depends on how busy you keep them, which is what section 4 is about.

What I care about is the cheapest way to get answers of Qwen’s quality, from whichever model gives them.

2. “But my GPUs are already paid for” #

This is the argument I hear most, so assume the hardware is free and only the electricity counts.

I measured my own machine. It runs Qwen in vLLM, split across both cards, at about 107 tokens per second.³ The cards are capped at 270 W each,⁴ and the whole machine draws about 700 W. It then needs four weeks, running day and night, to get through the benchmark once. I also give my quantized copy the score Artificial Analysis measured through Alibaba’s API. I have not rerun the benchmark on it, and anything lost to quantization makes my machine look worse.

Electricity for one benchmark run of Qwen3.8 27B on my two 3090s, by electricity price. The dashed line is what Luna charges for the same work, with OpenAI’s hardware and margin included. Hover for values.

Above 14.5 cents per kWh, my electricity alone costs more than Luna’s API bill. Below it, I save a few dollars per run and wait four weeks instead of a few hours.

Those four weeks matter more than the few dollars. An API takes hundreds of requests at once, so even one benchmark run can finish in an hour or two. My setup works on one conversation at a time: when I sent it eight requests at once, seven of them waited in line.

Wall-clock time for one benchmark run. The API times use Artificial Analysis’s measured time per task for Luna; the parallel case assumes 100 requests at once. Log scale.

This is one more reason my coding agents run on APIs. When I run several agents in parallel, each of them should be as fast as if it were alone.

3. Why datacenters are so much better at this #

When my 3090s generate a token, they read all of the model’s weights from memory to produce one token for one conversation. Most of the chip’s compute sits idle while it waits for memory. A datacenter GPU reads the same weights once and produces a token for hundreds of conversations at the same time. This is called batching, and it is where most of the efficiency comes from.

Artificial Analysis measures this with AgentPerf, which replays real coding-agent sessions against datacenter hardware. It counts how many agents a system can serve while each one still gets at least 60 tokens per second.

Coding agents served per kW of accelerator power, each getting at least 60 tokens per second, from AgentPerf. The datacenter systems all run DeepSeek V4 Pro. The 3090 bar is my machine running Qwen3.8 27B, a much smaller model, with GPU power only, so the comparison favors my machine.

The rack of 36 GB300s serves 20 times more agents per kW than the 8-GPU H200 server. Part of that is newer chips, and part is software: NVIDIA tuned the B300 and GB300 setups itself, while Artificial Analysis configured the H200 one. The B300 and the GB300 share both the chip generation and the software. Most of the gap between those two bars comes from the rack itself: 36 GPUs on one fast interconnect keep more than a thousand agents in flight at once. My two 3090s serve one conversation at a time.

4. “But I have solar panels” #

Solar power is not free. With net metering, every kWh my GPUs burn is a kWh that does not get credited at the retail price. Without net metering, it is a kWh I do not sell, unless the surplus would go to waste anyway, which is the free-power case below. And the GPUs also run at night.

But fine, say the electricity is free and the only cost is the hardware. I was lucky and bought my two 3090s for about $750 each,⁵ and I write them off over three years. The GPUs only save money while they do work I would otherwise pay an API for, and while they sit idle they save nothing. So the question is how many hours a day they have to be busy before they pay for themselves.⁶

To make this fair to open models, imagine an open model that is exactly as efficient as Luna: the same quality from the same number of tokens, sold at Luna’s price.

How many hours a day my two $750 RTX 3090s have to be busy, every day for three years, before they cost less than a ZDR API. The host PC is not counted.

With free power, the GPUs pay for themselves if they are busy 4 hours a day, every day for three years. At 18 cents per kWh, they need 8.5. That uses what I paid for the cards; at today’s price of about $1,500 each, the 8.5 hours become 14.

And API prices keep falling. GPT-6 Luna costs 58% less per output token than GPT-5.6 Luna did in July. If the price halves again, no amount of use pays off at 18 cents per kWh. If it halves twice, the API costs less than my electricity alone.

5. “But my Mac, DGX Spark, or AI Max does save money” #

Several readers told me their machines do save them money: a MacBook Pro with an M5 Max, an NVIDIA DGX Spark, and a computer with AMD’s Ryzen AI Max+ 395. If you would have bought the machine anyway, that is true, for the reason in section 2: their electricity costs little compared with the API. Counting the hardware is a different story.

With 128 GB of memory, all three can hold a better model than my 3090s: Qwen3.8-Flash-Next, a sparse model with 180 billion parameters, of which only 6 billion work on each token. It scores 39.8, close to GPT-6 Luna on its highest setting (38.1), although at the Q2 a reader uses on their Mac it probably loses a few points.⁷ So how many years would each machine have to run nonstop before it pays for itself?

Years of nonstop generation before each machine pays for itself, compared with the most efficient API model at a similar score. Each machine runs the best model it can hold.

Even with free power, the 128 GB machines need six to ten years. The AMD machine draws only about 120 W, but it is slow too: one benchmark run of Qwen3.8 27B takes it four and a half months, and almost as much energy as my 3090s. The RTX PRO 6000 runs the same model about three times as fast but costs about twice as much, so it needs six to nine years. My 3090s pay off in under two years with free power, which is why the 3090 is still the value king, but at 18 cents per kWh they never do: the electricity for one run already costs more than Luna charges for it.

Ten years is a long time in AI. Better and more efficient open models will run on the same machines, but the frontier moves at the same pace, and API prices keep falling. Hardware gets more efficient too, in datacenters and at home, so a machine bought today also ends up competing with next year’s machines. These numbers only show how far apart the two sides start.

6. The bar keeps moving #

Section 4’s eight and a half hours a day would be easy for me: I run agents for longer than that, just not on Qwen. When Claude Opus 4.6 came out in February, I was perfectly happy with it. I thought it was all I would ever need, and I could not have imagined how much better models would get in half a year. Qwen3.8 27B now scores about the same as Opus 4.6, and I would no longer accept it for coding. Until GPT-6 Astra came out at the start of September, my go-to was GPT-5.6 Sol, which scores ten points higher than Qwen. I then used Astra until Claude Opus 5.5 came out less than three weeks later. Now I don’t even accept what was considered the best model a month ago.

Intelligence Index of the best model available on each date, of the models I used as my go-to, and of Qwen’s 27B models, which fit on a single 3090. Scores use the xhigh reasoning setting where a model has it.

The models that fit on my 3090s keep improving, but they stay about half a year behind the frontier, and my standard moves with the frontier.

7. A small company has it worse #

For me, this is a hobby, and a hobby is allowed to be inefficient. It gets worse once you try to use self-hosted models seriously, say for a team of ten developers.

If you stay fully self-hosted and want fast responses at peak, you have to buy hardware for the busiest hour of the year, not for the average one. The rest of the time, it sits idle.

An illustrative week of coding agents for a ten-person team. The hardware has to cover the busiest hour of the year, but a typical week uses only a fraction of it.

In this made-up but realistic week, the team uses 17% of what it paid for, so every token costs about six times more than it would at full load. You also need a spare GPU for when one dies, and someone who gets paged when it does. You could queue work or send the peaks to an API, but then you are paying for an API anyway.

An API provider has the opposite situation. It serves thousands of customers across every time zone, so its load curve is much flatter than yours. It fills the nights with discounted batch jobs (Luna’s batch tier is half price) and training runs. And more concurrent requests mean bigger batches, which is where the efficiency from section 3 comes from.

8. “They lose money on inference, so prices will go up” #

Another argument I hear is that API prices are subsidized by investors and will go up once those investors want their money back. I don’t think that holds for inference.

AgentPerf also reports how many tokens each GPU serves. On the GB300 rack, one GPU serving DeepSeek V4 Pro handles about 4 million output tokens and 530 million input tokens per hour. Most of that input is conversation history that coding agents send again with every step. Providers keep it in a cache and charge less than 1% of the normal input price for it.

At DeepSeek V4 Pro’s list prices, that GPU brings in at least $5.70 per hour, even if every input token is billed at the cache price. If 5% of the input misses the cache, it brings in about $17. Renting a B200, the closest GPU with a published rental price, costs $3.50 to $5.90 per hour from smaller cloud providers, and that price already includes their profit.

So the tokens pay for the hardware that serves them, even in the worst case. This leaves out costs like staff and training, so it does not tell you whether a lab makes money overall.

The money goes to training, research, free users, and flat-rate subscriptions for heavy users like me. When I wrote that I used $10,000 worth of API tokens for $200, that was list price, not what those tokens cost to serve.

And prices go down, not up. This is what the same level of intelligence has cost this year:

Output price per million tokens for models scoring between 30 and 35 on the Intelligence Index, by release date, as listed by Artificial Analysis. Log scale.

From Claude Opus 4.6 in February to GPT-6 Luna in September, the output price for this level of intelligence dropped by a factor of 50. Labs are in a race to the bottom on price.

9. “But they sell your data” #

Some people say APIs are cheap because the provider trains on your data or sells it. That is why I only used prices from providers that OpenRouter lists as zero data retention (ZDR).

Model	Provider	ZDR	Output ($ per million tokens)
GPT-6 Luna	OpenAI	no	0.50
GPT-6 Luna	Azure	yes	0.50
Qwen3.8 27B	AkashML (FP8)	yes	1.78
Qwen3.8 27B	DeepInfra (full precision)	yes	1.88
Qwen3.8 27B	Alibaba	no	2.55
Qwen3.8 27B	Cloudflare	no	3.20

Azure serves GPT-6 Luna with zero data retention at the same price as OpenAI. For Qwen3.8 27B, the cheapest endpoints are all ZDR, and the endpoints without ZDR cost more. The :free tier is where you pay with your data.

10. Where self-hosting does win: the same model #

If you compare running Qwen3.8 27B yourself with paying for Qwen3.8 27B through an API, self-hosting wins.

DeepInfra, the cheapest full-precision ZDR provider, charges $619 for one benchmark run, while my electricity costs $83. That compares the full model with my quantized copy, so the gap overstates my advantage by whatever quantization costs in quality. My machine pays for itself if it is busy 2.4 hours a day, or 1.5 hours with free power (the first group in the chart in section 4).

Providers charge a lot for a dense 27B model, because every token runs through all 27 billion parameters. Sparse open models, which use only a small part of their weights for each token, can be much cheaper to serve. If you specifically need Qwen and use it heavily, self-hosting it pays off. But you don’t need Qwen to get answers of Qwen’s quality. Luna gives you those for $67, with zero data retention.

11. So why do I do it? #

First, it is fun. I like knowing how the whole stack works, from the drivers in my NixOS configuration to how the layers are split between my two GPUs.

Second, nobody can take it away. Artificial Analysis already lists GPT-5.6 Luna as deprecated, less than three months after its release. My Qwen weights will still be on my disk in ten years, and no company can change their license or close their build system on me.

Third, some data should not leave my house. I would never send 200 GB of email, my messages, and my location history to an API, zero data retention or not. That is what my local AI projects are for.

Those are good reasons. Saving money is not one of them.

Appendix: all assumptions #

Show all assumptions, and which side each one favors

Assumptions that favor self-hosting

The host PC around the GPUs is not counted, and my 3090s count at the $750 each I paid, not the $1,500 they sell for today.
Quantized local copies get the score Artificial Analysis measured through an API, with no penalty for quantization.
The payback chart assumes each machine is busy 24 hours a day, with no idle time.
API prices stay where they are, although GPT-6 Luna’s output price is 58% lower than GPT-5.6 Luna’s, two and a half months earlier.
My time to set up and maintain the machine is not counted.
Every machine also gets a case with free electricity.

Assumptions that favor the API

The example electricity price is 18 cents per kWh, and some people pay less.
My machine serves one request at a time; batching several requests could raise its total throughput.
The hardware has no resale value at the end.
Waste heat that replaces heating in winter is not counted.
API outages, rate limits, and deprecated models are not counted.

Inputs

Benchmark: the Artificial Analysis Intelligence Index on September 30, 2026, at the xhigh reasoning setting, or the highest setting where noted. Its token counts per model are the unit of work.
API prices: the cheapest zero-data-retention endpoints on OpenRouter, with cached input billed at the cache price.
My machine: two RTX 3090s capped at 270 W, running Qwen3.8 27B in vLLM with AutoRound INT4 and DFlash2. Measured: 107 tokens per second, 1,600 tokens per second of prefill, and 540 W for both GPUs. Estimated: about 700 W for the whole machine under load and 150 W idle.
Break-even: the cards are written off over three years; busy hours save what the API would charge minus the electricity, and idle hours cost the idle power.
Other machines: prices, power, and speeds as in the footnotes. The M5 Max and RTX PRO 6000 numbers come from readers, and the DGX Spark and AMD Ryzen AI Max+ 395 speeds are estimated from Artificial Analysis measurements.
Datacenter: AgentPerf measures accelerator power only, and the B200 rental price stands in for the GB300, which has no published rental price.
Small company: the peak-load week is made up.

For each model, Artificial Analysis publishes how many input and output tokens the whole index took, and how many of the input tokens were read from a cache. I multiplied those by each provider’s prices, with cached input at the cache price. For Qwen that is 198 million output tokens and 820 million input tokens that miss the cache. For the bars at home, the same tokens divided by the machine’s speed give the run time, with the uncached input processed at the 1,600 tokens per second I measured on my 3090s and an estimated 2,000 on the RTX PRO 6000, and the run time times the machine’s power and the price per kWh gives the electricity. ↩︎
The RTX PRO 6000 has the same memory bandwidth as an RTX 5090, and at full precision every token reads three times as many bytes as in Q4_K_M. Scaling Artificial Analysis’s 5090 measurement gives about 48 tokens per second. I assume 600 W for the whole machine. ↩︎
I measured the model as I normally run it: vLLM with an AutoRound INT4 quant and DFlash2 speculative decoding, split across both cards. Single 1,500-token answers came out at 105 to 109 tokens per second, and reading a 22,000-token prompt ran at 1,600 tokens per second. Sending 2, 4, or 8 requests at once did not raise the total above about 110 tokens per second. The two GPUs drew 540 W together, each at its 270 W power limit; the 700 W adds my estimate for the rest of the machine, since I could not measure the CPU. The setup follows the recipes from club-3090, a community project that tunes LLM serving for RTX 3090s and publishes measured numbers, so I think it is about as fast as these cards get. Its benchmarks for this configuration at a similar power limit match mine: about 110 tokens per second for prose, and up to about 195 for code, where speculative decoding guesses more tokens right. Most of Qwen’s output on the benchmark is reasoning, which runs at the prose speed. ↩︎
The stock limit is 350 W. My cards sit in a normal desktop case with normal fans, and I worry that running both at full power for hours would shorten their life. The cap is a few lines in my NixOS configuration, and measurements from Puget Systems and r/LocalLLaMA put a 3090 at about 95% of its speed at 270 W. That is 77% of the power for 95% of the speed, so the cap also lowers my electricity cost per token. ↩︎
The 3090 is still the value king for VRAM per dollar, and $750 badly understates what mine are worth. That is what I paid more than a year ago; today a used 3090 sells for about $1,500. Even at that price it costs $62.50 per GB of VRAM, the same as an RTX 5090 at its $2,000 launch price, which is not what a 5090 sells for today. The right number for this calculation is what I could sell my cards for, and at $1,500 each the break-even points rise by about two thirds: at 18 cents per kWh, the same-model case moves from 2.4 to 4.0 hours per day, and the Luna-efficient case from 8.5 to 14.2 hours per day. ↩︎
While the cards are busy, they save what the API would have charged for the same work, minus the electricity at 700 W. While they are idle, the machine still draws about 150 W, of which the two GPUs with the model loaded take 85 W. The break-even point is the number of busy hours per day at which those savings cover the price of the cards, written off over three years, plus the idle power. ↩︎
The M5 Max numbers come from the reader: Qwen3.8-Flash-Next at Q2 (about 2.7 bits per weight), about 45 tokens per second, 1,250 tokens per second of prefill, and about 90 W, on a $7,000 laptop. The RTX PRO 6000 numbers come from another reader, u/TheAILegend, who measured Qwen3.8-Flash-Next at NVFP4 on theirs. For a single request, it stays at about 150 tokens per second from 4,000 to 200,000 tokens of context, with about 10,000 tokens per second of prefill and 266 W for the GPU plus about 160 W for the rest of the PC. With 32 short requests in parallel, the card reaches 590 tokens per second and pays off in about two years, which works for short batch jobs. At the long contexts of this benchmark, parallel requests are slower than one, so the chart uses the single-request speed. I use $15,000, the middle of the card’s price range. The DGX Spark and the AMD machine run Qwen3.8-Flash-Next at about 4.5 bits. For their speed, I took Artificial Analysis’s measurements of Qwen3.8 27B on each machine and doubled them, because the owner of the AMD machine finds Qwen3.8-Flash-Next about twice as fast as Qwen3.8 27B. That gives about 55 tokens per second on the DGX Spark ($6,950, about 150 W) and 47 on the AMD Ryzen AI Max+ 395 ($4,000, the price Artificial Analysis lists, about 120 W). All of them are compared with GPT-6 Luna on its highest setting, which runs the whole benchmark for $122. The 3090s run Qwen3.8 27B, as in the rest of this post, and are compared with Luna on xhigh. Artificial Analysis measured Qwen3.8 27B on the AMD machine at 23 tokens per second, so one benchmark run takes it 134 days and about 390 kWh, against about 460 kWh on my 3090s. Running Qwen3.8-Flash-Next on the 3090s with CPU expert offload, which club-3090 measured at 37 tokens per second, does not change their result: one run then takes 82 days, and at 18 cents per kWh its electricity costs more than Luna’s $122 unless the whole machine draws less than about 350 W. ↩︎

Exploring the Jev hype in MindRoom

Wed, 23 Sep 2026 00:00:00 +0000

Last week, every AI newsletter I get was full of Jev, a new model from TypeSafe. So were several subreddits. I am a permanent lurker on r/LocalLLaMA, and even though it is about local models, every other post seemed to be about Jev. There were local clones like mini-jev and von, CLI wrappers like jev-cli, and even someone on Hacker News claiming they had built the same thing a year earlier.

I tried Jev in MindRoom, my open-source agent platform on Matrix, and less than two days later it was making three decisions there.

Update 2026-10-07: OpenAI released the Decisions API today, its own take on the same idea, and MindRoom now supports it as a third backend; see below.

What a System One model is #

The name comes from Daniel Kahneman’s Thinking, Fast and Slow, where System 1 is fast, intuitive thinking and System 2 is slow and deliberate. A System One model does not generate text. You send it a state, for example a chat log, plus typed questions, and it returns probabilities:

Type	Question	Answer
Noul	Yes or no	Probability of yes
Choice	Pick one of up to 255 options	Chosen option plus the full distribution
Score	Rate against a rubric of 2 to 10 levels	Probability-weighted value

TypeSafe claims 70 to 500 ms end to end and $0.042 per million input tokens, with output tokens free. They also write “we can’t prove it isn’t subsidized.” Either way, it is incredibly cheap and incredibly fast.

Adaptive participation #

In MindRoom, a conversation can have several humans, several agents, or both. When I talk to a single agent, it simply replies, and I don’t need to tag it. Once another person joins and writes something, the agent cannot tell whether the message was meant for it or for the other person. Nothing decided that, so the agent stayed quiet unless explicitly tagged. That confused several users.

So I recently added adaptive participation: an agent that has already replied in a thread decides on its own whether to answer an untagged message. I first used an LLM as the judge, GPT-5.6 Luna on low reasoning. A yes-or-no question like “should this agent respond?” is exactly what Jev is for, so it became the second backend. Compared with Luna, it is about ten times cheaper and ten times faster, and if I believe the benchmarks, it is also smarter.

“Thanks” should not stop the agent #

The next thing I used Jev for was interruptions. When I send a message while an agent is still replying, MindRoom injects a notice at the next tool call telling the agent to stop and wrap up, because there is a new message. But often I, and other users, would watch the agent start working and then write “looks good” or “thanks.” The agent would stop early and continue in the next turn, which is both wasteful and counterintuitive.

Now a judgment decides whether the new message needs the interruption, which the docs call mid-turn coalescing. If it does not, the agent reacts with 👀, finishes its reply, and handles the message afterwards.

This is also where I learned a lesson about evals. My first implementation worked at the code level but not functionally. My evals, based on real attempts inside MindRoom that had all failed plus synthetic cases from GPT-6 Astra, passed 5 out of 40 with Jev. The fix changed the question and gave the judge more context, and it went to 40 out of 40.

	Before: 5/40	After: 40/40
Question	“May the active task finish before the queued human messages are handled in a later turn?”	“Does any newly queued message require an immediate change to, pause of, or stop of the active task?”
When unclear	“Also false when context is incomplete, the relationship is unclear…”	“Mere relevance to the task does not make praise or thanks an interruption.”
Context	The request and the new messages	The same, plus the preceding conversation

The old question treated anything unclear as a reason to interrupt, and a bare “thanks” without the conversation around it always looks unclear.

Picking the responder #

The third decision is routing. When a message in a room with many agents does not mention anyone, MindRoom’s router picks who should answer. That used to be a free-text LLM prompt, “choose the most appropriate agent.” It is now a Jev Choice over the eligible agents, plus a no_fit option, as described in the router docs. A confident no_fit asks the user to mention an agent instead of guessing, and anything uncertain falls back to the existing LLM router.

One config line to switch #

I built a very simple abstraction layer so I can swap Jev for an LLM without changing any code. Every decision is one question with criteria for true and false:

MID_TURN_QUESTION = JudgmentQuestion(
 id="interrupt_current_turn",
 instructions="Does any newly queued message require an immediate change to, pause of, or stop of the active task?",
 when_true="A new message requests stopping or pausing the active task, corrects it, ...",
 when_false="The messages acknowledge progress, express thanks, ask to continue unchanged, ...",
)

Jev receives it as a Noul question and returns a probability, which MindRoom compares with a threshold. An LLM receives the same question and context and must return {"decision": true} or {"decision": false}. Switching is a configuration change:

agents:
 helper:
 mid_turn:
 defer_reaction: "👀"
 judgment:
 provider: typesafe  # or: provider: llm, model: fast
 threshold: 0.8

Both backends log their decisions and latency, so I can compare them on real traffic. The code is in src/mindroom/judgment/, and the backends are documented under participation judgment backends.

A third backend: OpenAI Decisions #

On October 7, OpenAI released its own version of the idea, the Decisions API, currently in public beta. The question types match Jev’s: a predicate is a Noul, and choice and score keep their names. It serves a single model, gpt-6-luna, and OpenAI says it returns answers “about 10x faster than the Responses API.” Input costs $0.10 per million tokens and output is free, so it lists at about 2.4 times Jev’s price.

Thanks to the abstraction layer, adding it took one PR, merged the same day, and it works for all three decisions, including the router’s Choice:

judgment:
 provider: openai_decisions
 threshold: 0.8

From MindRoom’s side, only the wire format differs. The endpoint accepts only user messages, so MindRoom sends each chat message as its own user message, prefixed with its role, like assistant: .... Decisions can also answer with a refusal, which MindRoom treats as an abstention, so the usual fallback applies.

I tested it against the live API through MindRoom’s own code, with the same questions Jev gets. For participation, it gave 0.99 to replying to a direct question and 0.0 while two humans were talking to each other. Mid-turn, “Thanks, looks great so far!” got 0.0 for interrupting and “Stop, keep MySQL…” got 1.0. The router sent a pandas bug to code and a meeting request to calendar, both with confidence 1.0. A nonsense request got only 0.62, below the 0.8 threshold, so the normal LLM router took over. Each call took 130 to 280 ms. I have not compared it with Jev on real traffic yet, but both log the same outcome fields, so that only takes a config change.

Named after Jevons #

I only read why TypeSafe named it Jev after building all this: they named it after William Stanley Jevons, who noticed that as steam engines burned coal more efficiently, demand for coal went up instead of down. This is known as the Jevons paradox. That was literally my experience. Most of what I built was perfectly possible with an LLM. I just never built it, because I am somewhat cost- and latency-minded. Once I implemented Jev for one decision, I came up with use case after use case.

One I already built is a plugin, Response Audit JEV. Right after an agent replies, Jev checks the answer against the request and the tool calls the agent actually made, for example whether things are properly cited. If a check flags a problem, the plugin posts one follow-up in the thread that tags the agent and asks for a correction. Because Jev is so fast and cheap, running this after every reply costs almost nothing.

Right now I am full of ideas. For example, a model router. In MindRoom, an agent can switch to a different model mid-conversation with a tool call, and I can do the same with !model. Agents rarely do that without being told, though. So I default to a very expensive model, and I only save money when I remember to switch to a cheap one for something simple. Jev could run on every message and decide whether it needs the capable model or not. That works in both directions, up to a capable model or down to a cheap one. The perfect moment to switch is when the prompt cache has expired. A prompt cache belongs to one model, so switching normally means sending the entire context again at full price. Once the cache has expired, I pay that price anyway, so switching costs nothing extra.

Gemini 3 Pro: First Impressions

Wed, 19 Nov 2025 00:00:00 +0000

I’m a huge fan of agentic coding. I have the maximum subscriptions for both Claude and ChatGPT Pro, and since writing my agentic coding post, I mostly moved from Claude Code to Codex CLI with GPT-5 for my work that requires more complex reasoning. Using Claude Code often feels frustrating in comparison now.

I also frequently pit them against each other, drafting parallel plans with each tool (I’ve done this at least 50 times). The verdict is consistent. Claude Code, without exception, says that GPT‑5’s solution is superior, admitting it is more elegant and robust. Claude also misses bugs during PR reviews that GPT-5 catches immediately.

That said, Claude Code is significantly faster, which makes it my go-to for simple refactors. I also prefer its writing style for conversational answers and documentation.

But looking at the charts on Artificial Analysis, the new Gemini 3 Pro looks truly impressive. It’s at the top of nearly every benchmark. I’m excited for Gemini to potentially take over some of GPT‑5’s workload, so when Gemini 3 Pro Preview came out yesterday, I had to try it.

The good: raw power #

First off, the model is quite powerful. Last night, I built a complex new feature for my agent-cli project in just two hours: a RAG (Retrieval-Augmented Generation) proxy server.

This tool acts as a middleware that sits between any OpenAI-compatible client and server (like Ollama or vLLM). The server watches a specific folder, automatically indexes any PDF, Markdown, or text files into a local ChromaDB vector store, and injects relevant context into your prompts transparently. You can see the full implementation in PR #79.

It even handles dynamic re-indexing when I drop new files in! The speed and code quality for this task were genuinely impressive, and it handled the complexity of the RAG integration surprisingly well.

The bad: panic attacks and infinite loops #

However, the experience wasn’t without its “personality quirks.” Twice, Gemini got stuck in a loop saying “Should I? I should” over and over again. Or the following also happened:

It also has a bad habit of amending commits multiple times and force-pushing them. For me the Git history is a critical source of truth, and I don’t like rewriting it because I might just have reviewed those commits already.

The interface friction #

I also found the Gemini CLI interface frustrating. Basic tasks like selecting text or scrolling up are surprisingly difficult. I’m not sure why they chose this design or why they feel the need to deviate from standard CLI behavior. It feels like unnecessary friction in an otherwise promising tool.

Proofreading this post #

I also asked Gemini to proofread this very blog post. It made some suggestions—some I liked, others I didn’t, so I reverted the ones I didn’t want. But in every single subsequent iteration, it would forcefully re-apply its rejected changes, adding emojis I had removed or rewriting sentences I had explicitly restored.

The “main branch” incident #

Another scary moment: I asked if the PR was ready to be merged into main, and it went ahead and checked out the main branch and merged everything locally. If I hadn’t stopped it, I’m pretty sure it would have pushed the changes to the main branch directly as well. I tend to use “yolo mode” for agentic coding because I feel like Git has my back, but this feels like a dangerous path to go down.

And indeed, while writing this post, it actually happened. Gemini merged my pull request without my permission.

My prompt was > should the README reflect this? and before that I asked to review the PR.

When I asked why it merged it, it instantly force-pushed to main again!

Artifacts and existential crises #

Also, when it commits, it just adds all files, including debugging logs and build artifacts. When it realizes its mistake, it tries to fix it. But if I try to help at the same time (like removing the files myself while it’s thinking), it gets extremely confused.

It spirals into an existential crisis, claiming that I committed the file but now “it is gone.” “What is going on?” it asks, spending a lot of tokens (and time) looking at git reflogs and questioning its own existence. Even when I stop it and explicitly say that I deleted the file and to please continue, it completely ignores me and continues its panic attack. I experienced this pattern of ignoring instructions multiple times.

Conclusion #

Gemini 3 Pro shows massive potential. The raw coding ability is there, and building a clean RAG proxy in two hours is great! But the agentic behavior needs some serious work: the loops, the force pushes, ignoring instructions, and the confusion when reality doesn’t match its internal state.

This is just day one, so I’m confident many of these issues will get ironed out. I don’t love the interface, but given that gemini-cli is open source with ≈400 contributors, I expect rapid improvements, especially now that the model is finally good with agentic workflows!

On agentic coding 🤖

Mon, 25 Aug 2025 00:00:00 +0000

Table of contents

1. From skepticism to productivity #

About eight months ago, I wrote about my disastrous experience with “vibe coding"—where I let AI generate code without careful review. What started as a fun 3-hour prototype turned into a 15-hour debugging nightmare. I concluded that while AI was fantastic for quick prototyping, it was dangerous without human oversight. I even declared I’d “never invest in, build upon, or use such products.”

Fast forward to today, and something remarkable has changed. Not my core principle—I still never merge code I don’t understand—but the tooling and my approach have evolved through two distinct phases.

First, let’s look at the evidence of AI’s overall impact on my productivity:

This graph tells an interesting story. After maintaining a steady pace for years (2016-2023), my productivity exploded with the introduction of AI tools. In 2024-2025 alone, I’ve published 14 new Python packages and made over 400 releases. That’s more new packages than in the previous 6 years combined!

But this transformation didn’t happen overnight—it came in two distinct phases that fundamentally changed how I write code.

2. But don’t you love programming? #

I’ve heard this objection countless times: “I love programming too much to let AI do it for me.”

Sure, I get it. It’s like preferring analog photography over digital, or writing letters by hand instead of typing. Some people genuinely enjoy the manual process, and that’s valid.

But here’s the thing: I absolutely love programming. It’s my biggest passion in life. As I mentioned in my local AI journey, I bought a gaming PC to play games but have only used it for coding and local AI experiments. My idea of a fun weekend? Writing code until 3 AM. I’ve been doing this for over 10 years, and I’ve never had as much fun programming as I do now.

What I’ve discovered is that I don’t just love the act of typing code—I love building and creating things. There’s a difference.

I’m ridiculously particular about code quality. I literally cannot look at poorly formatted code without feeling physical discomfort. Every operator, every indentation, every naming convention matters to me. I enforce strict linting rules and will spend time refactoring code just because the style bothers me. This isn’t about being pretentious—it’s about caring deeply about my craft.

The revelation: AI doesn’t take away my ability to care about these details. Instead, it gives me insane leverage to create more while maintaining my personal standards. I still review every line (although I am more lenient in certain cases), still enforce my style guides, still refactor when something bothers me. But now I can build 10x faster.

It turns out that what I truly love isn’t the mechanical act of typing—it’s the creative act of bringing ideas to life. And with AI, I can bring more ideas to life than ever before.

The power to explore #

AI has unlocked something incredible: the ability to explore ideas that weren’t worth the effort before. It’s now trivial to write 2,000 lines of code to solve a simple problem. Before AI, spending 5+ hours on a utility script wasn’t justifiable. Now? I can build it in 10 minutes.

Here are just some of the “wouldn’t have been worth it” projects I’ve built recently:

Financial independence tracker: A full website visualizing my path to FIRE
Tuitorial: Terminal-based presentation software
Stream Deck home control: Complete tool to control my entire house from a Stream Deck
Dotbins: Automatic binary dependency syncing across all my machines
Agent-CLI: A local, open-source Siri that actually works

Each of these would have taken weeks of manual coding. The effort-to-reward ratio just wasn’t there. But with AI? I can explore every random idea that pops into my head at 2 AM.

The Matty story: from midnight idea to working code #

Here is the perfect example of this power: One night around midnight, I was lying in bed (slightly intoxicated) when I had an idea—I needed a way for my AI agents to communicate via Matrix. So I grabbed my iPhone, SSHed into my machine, and started building.

One hour later, at 1 AM, I had created Matty—a fully functional terminal-based Matrix client. Built entirely from my phone. In bed. (Poor sleep hygiene, I know.)

When I woke up the next morning and tested it… it actually worked! I spent a few more hours adding tests and cleaning it up, but the core functionality was built in that hour between midnight and 1 AM.

This is impossible without AI. Nobody’s writing a Matrix client from their phone in bed at midnight. But with Claude Code? It’s just another Tuesday night idea that becomes reality.

clip-files UI: when ideas strike at random moments #

What happens more often than you’d think: I get an idea at a completely random moment and NEED to implement it immediately.

Case in point: I was sitting in the bathtub during my copy-paste AI era when I realized I needed to access my codebases from anywhere. I’d already built clip-files—a CLI tool that copies entire codebases to your clipboard—but I wanted it accessible from my phone.

So I got out of the bathtub, built a web UI for it in 20 minutes, and got back in. The UI lets me:

Browse all my GitHub repositories
Select any branch
Click one button to copy all source code to clipboard
Paste it into my self-hosted AI chat from anywhere

Ironically, I rarely use this tool now—Claude Code can explore codebases directly. But back in the copy-paste era, this 20-minute bathtub interruption made mobile AI coding possible.

The barrier between “wouldn’t it be cool if…” and “here’s a working prototype” has essentially disappeared.

The 800-rule marathon: 80 AI agents at once #

Another thing that would be literally impossible without AI: I decided to enable ALL 800+ Ruff linting rules on a 20,000-line codebase.

For context, Ruff is a extremely fast Python linter that implements rules from over a dozen different tools. Most projects enable maybe 50-100 rules. I wanted all 800+.

The result? ≈2,500 violations across the entire codebase.

Fixing these by hand would take 20 hours if I could fix one per 30 seconds… But with Claude Code’s agent spawning capabilities, I did something insane:

Opened 8 terminal windows with Claude Code
Created a work orchestration document that one AI managed
Each session spawned 10 parallel sub-agents using /agents
Total: ~80 AI agents working on my code simultaneously
Two hours later: 100% compliance with all 800 rules

The orchestrator agent would assign work like:

“Session 1: Fix all E501 line-too-long violations in src/core/”
“Session 2: Handle all B008 function-call-in-default-argument issues”
“Session 3: Fix all RUF100 unused-noqa violations”

Each session’s sub-agents would tackle different files in parallel. When conflicts arose, the orchestrator would resolve them. This is the kind of code quality improvement that simply wouldn’t happen without AI—not because it’s technically difficult, but because the human effort required is unrealistic.

1,400 commits, rewritten in minutes for $2.50 #

Here’s another “impossible without AI” story: I had 1,400 commits with terrible messages (in a not-yet-released and private project).

Since I commit so frequently (using them as snapshots in feature branches), my commit messages were all over the place:

Some were just a period “.”
Some were decent descriptions
Some were just merge commits
Zero consistency in style

I wanted them all to follow a consistent convention with prefixes like fix:, feat:, test:, ci:, etc.

Obviously, rewriting 1,400 commit messages by hand is not going to happen. But with AI?

I had Claude write me an agent using the Agno framework that would:

Read each commit’s full diff using git show
Analyze the changes to understand what was done
Return structured JSON with three fields:
- commit_message: The improved message (or empty to keep original)
- keep: Boolean flag whether to keep the original
- reasoning: Why the AI made that decision

Then the magic happened: I spawned 1,400 AI agents simultaneously using the DeepSeek API.

Ten minutes and $2.50 later, all my commit messages were rewritten with consistent, meaningful descriptions. The entire Git history now looks more professional and searchable.

The barrier between “wouldn’t it be cool if…” and “here’s a working prototype” has essentially disappeared.

3. The two-phase AI revolution #

My AI coding journey wasn’t a single leap—it happened in two distinct phases, each with its own breakthrough moment.

Phase 1: copy-paste era (March 2023 - July 2025) #

When GPT-4 launched in March 2023, it changed everything. I built tools like clip-files to efficiently copy code context into a ChatGPT-like web interface. For over two years, this workflow was:

Manual but effective: Copy code → paste into AI → review suggestions → implement manually
Limited by context: Could only share what fit in a single message
No validation: AI couldn’t test its suggestions
Still valuable: Helped with ideation, code review, and refactoring

This phase was already productive—I was shipping more code than before. But I was still the one running tests, debugging errors, and validating everything.

Phase 2: the Claude Code revolution (July 2025) #

Just two months ago, in July 2025, everything changed again. Frustrated with Cursor becoming painfully slow, I tried Claude Code based on the buzz online. That first night, I spent $70 on API tokens. Not by accident—I just kept adding $10-$20 increments like a gambling addict. The experience was so mind-blowing that within two weeks, I’d spent nearly $200 and immediately upgraded to the $200/month Pro plan.

To put this in perspective: Using a command-line tool ccusage to track token consumption, I calculated that I used $10,000 worth of API tokens in my first month. Thankfully, the Pro plan capped my cost at $200!

What made Claude Code different? #

The shift from copy-paste to agentic AI was like upgrading from a bicycle to a spaceship:

Before (copy-paste with ChatGPT/Claude Web)

Generate code based on prompts
No access to your codebase
Can’t run tests or see errors
You debug everything manually

After (Claude Code)

Reads and searches your entire codebase
Executes commands and runs tests
Sees error messages and debugs itself
Iterates until tests pass
Uses tools to explore and validate

The difference is profound. I went from carefully copying snippets with clip-files to having an assistant that can explore my entire project, run my test suite, fix failures, and even commit the changes.

4. My current agentic workflow #

Here’s how I actually work with Claude Code on a typical project:

Parallel development: 6 features at once #

One of Claude Code’s superpowers is enabling truly parallel development. Here’s my setup that would make any developer from 10 years ago think I’m insane:

I use Zellij (a terminal multiplexer like tmux) with a custom layout for my projects:

5 main tabs: Each split into two panes—Claude Code on one side, terminal on the other
Using independent copies of same repository: Each tab is a separate Git worktree with its own environment, own environment variables, own deployment
Each tab = different feature: Using Git worktrees to work on 6 features simultaneously
Monitoring tabs: ccusage for tracking Claude usage, htop for CPU, nvtop for GPU
Voice-driven orchestration: I cycle through tabs, review code while speaking, move to next

My workflow looks like this:

Start feature A in tab 1, give Claude instructions
Switch to tab 2, start feature B while A is working
Continue through all 5 tabs, starting different features
Circle back to tab 1, review what Claude did, give feedback via voice
Repeat the cycle every 10-15 minutes

This parallel workflow is how I built a complex project (that I haven’t released yet) in one month that would have taken 9+ months before. The key is Git worktrees—each feature gets its own working directory, so there’s no context switching overhead.

Starting a new package #

# I describe what I want to build
"I need a Python package that manages CLI tool binaries within Git repositories,
automatically downloading the correct binary for the user's platform, here <insert path> is a repo with boilerplate skeleton to copy (use same conventions)"

# Claude Code then:
# 1. Creates the project structure
# 2. Implements core functionality
# 3. Writes comprehensive tests
# 4. Generates documentation
# 5. Sets up CI/CD workflows

The key difference: iteration and validation #

Unlike my vibe coding disaster, Claude Code doesn’t just dump code and leave. Here’s a real interaction pattern:

Claude writes initial implementation
Claude runs the tests → Several fail
Claude reads the error messages and fixes the code
Claude runs tests again → More pass, some still fail
Claude debugs the remaining issues
Claude validates all tests pass
I review the final code and understand it (tell it to make changes if needed)
Claude creates a proper Git commit

This iterative, validated approach is what makes agentic coding so powerful. It’s not about blindly accepting generated code—it’s about having an assistant that can explore, test, and refine solutions.

5. The evidence: explosive productivity #

Let me share some concrete data from my PyPI packages analysis:

Package creation by year #

2016-2022 (7 years pre-AI): ~2 packages/year average
2023 (GPT-4 launch year): 6 packages
2024 (full year with copy-paste AI): 7 packages
2025 until 3 months ago (6 months): 7 packages
since 3 months (2 with Claude Code, 1 with Codex CLI): wrote about 400k line of code (although this includes many iterations)!

The acceleration with AI is clear!

Recent packages built with agentic AI #

Here are some packages I’ve built recently with Claude Code:

matty: Terminal-based AI chat with file integration
agent-cli: Local-first AI-powered CLI agents (built in days, not weeks!)

Each of these would have taken me weeks or months to build alone. With Claude Code, I’m shipping production-ready packages in days.

6. Why this isn’t “vibe coding” (but with nuance) #

You might wonder: isn’t this just vibe coding with extra steps? The answer is nuanced and depends on the context—similar to my philosophy on dependencies.

My context-driven standards #

Just like I have different standards for dependencies in libraries versus applications, I apply different levels of scrutiny based on what I’m building:

For critical libraries (e.g., Adaptive, pipefunc, unidep)

Maximum scrutiny - These are packages others depend on:

Every single line reviewed and understood
100% test coverage with careful test review
Architecture decisions carefully considered
Documentation must be comprehensive
My reputation is on the line

For experimental CLIs and personal tools

Pragmatic approach - Isolated tools with no downstream dependencies:

Core architecture must be fully understood
Implementation details can be AI-generated if tests pass
Test generation can be more automated
Focus on functionality over perfection
Similar to my relaxed dependency stance for applications

The internal dependency graph principle #

The key insight: My scrutiny level correlates with how foundational the code is within the project:

Core/foundational code (what everything else depends on): Maximum scrutiny, every line matters
- Data models, core algorithms, API interfaces
- Authentication, database operations, state management
- These are the “roots” that everything else builds upon
Peripheral/leaf code (nothing depends on it): Can be more AI-delegated
- Plotting functions, display utilities, CLI formatters
- Test helpers, documentation generators
- These are the “leaves” that don’t affect other code
Work projects: Always maximum scrutiny for foundational code, regardless of project type

This isn’t about being lazy—it’s about focusing human attention where it matters most. I can trust AI more with a plotting function that nothing depends on than with a core data structure that the entire system uses.

Building constraints around AI #

The secret to productive agentic coding isn’t just the AI—it’s the constraints and guardrails I build around it:

Automated quality gates

Ruff with strictest rules: Catches style issues, complexity problems, and common bugs
MyPy in strict mode: Enforces type safety across the entire codebase
Pre-commit hooks: Automatically format and validate code before commits
Comprehensive test suites: AI must make tests pass, not just write code

Project-specific guidance

Every project gets a CLAUDE.md file with explicit rules:

No defensive programming: Don’t wrap things in try-except unless necessary
Functional over classes: Prefer simple functions in Python
No backward compatibility: For new projects, embrace breaking changes
Be ruthless: Aggressively remove unused code

Custom commands and workflows

I’ve built specific commands that inject context and constraints:

Anti-cruft reviews: Remove over-engineering and defensive code
Safe commit practices: Never use git add ., always selective staging
Initialize understanding: Load project context and current work state

Local virtual environments: stop the hallucinations

Here’s a critical tip that eliminated a lot of my AI frustrations: keep a virtual environment with all dependencies installed locally.

Instead of letting Claude hallucinate how libraries work, I tell it explicitly in my CLAUDE.md:

### Step 1: understand the context

- **READ THE SOURCE CODE**: This library has a `.venv` folder with all dependencies installed.
 So read the source code when in doubt.
- **Never guess API behavior**: If unsure, inspect the actual implementation in `.venv/lib/python*/site-packages/`

This simple addition transforms Claude from guessing about library APIs to actually reading them:

Before: “I think your custom AsyncProcessor.batch() method takes a list…”
After: Reads /path/to/.venv/lib/python3.11/site-packages/my_internal_lib/processor.py and knows it actually takes an iterator

When Claude can read the actual source of your own packages, internal company libraries, or niche dependencies, it stops making assumptions and starts working with facts. This is especially powerful for:

Your own libraries that Claude has never seen before
Internal company packages that aren’t public
Niche libraries with barely any GitHub stars or documentation
Modified versions of popular libraries with custom patches
Edge cases where even good documentation doesn’t cover everything

These constraints transform AI from a loose cannon into a precision tool.

7. Common pitfalls I’ve noticed #

After a few months of using agentic AI tools, I’ve noticed consistent patterns that require vigilance:

Note from the future: I now realize this is very model dependent! GPT-5 has different pitfalls than Claude Opus 4.1

The “defensive programming” trap #

# AI loves this:
try:
 result = some_function()
except Exception:
 pass # Silently suppress errors 😱

# But we need this:
result = some_function() # Let it fail loudly if something's wrong

Claude tends to wrap everything in try-except blocks, suppressing errors that should bubble up. This is why my CLAUDE.md explicitly forbids unnecessary error handling.

The “backwards compatibility” obsession #

Claude constantly adds backwards compatibility for features that were literally just introduced in the same session:

“Maintaining compatibility with the old version” (that never existed)
Fallback mechanisms for code paths that were just created
Multiple ways to do the same thing “for flexibility”

The “over-engineering” disease #

Implements factory patterns for simple object creation
Adds abstraction layers that serve no purpose
“Production ready” code for experimental scripts

The git commit sins #

Despite explicit instructions in CLAUDE.md:

Creates your_module_v2.py alongside your_module.py instead of updating
Still tries git add -A or git add . regularly despite explicit bans
Loves to “helpfully” revert debugging changes from other files
Commits .env files if not watched carefully

The “helpful” anti-patterns #

Loves defensive programming: Validates things that can’t be wrong
Gladly reads .env: Will expose secrets if not careful
Reverts unrelated changes: “Cleans up” debugging code from other features

The “mission accomplished” hallucination #

This is a dangerous pitfall. Claude Code will sometimes claim complete success when it hasn’t actually fixed anything:

Claims victory: “I’ve fixed all the issues!”
Ask for proof: “Show me the test output”
Backpedals: “Oh, let me actually run the tests…”
Still claims success: “Yes, I can see in the logs it works!”
Demand the actual log: “Show me the exact log file”
Finally admits: “Actually, there are no log files. The tests don’t pass.”

Always demand proof. Never accept “it’s done” without seeing actual test output.

The nuclear option #

I’ve had Claude Code literally try to delete all project files when “cleaning up”:

# Claude's "helpful" cleanup:
"Task complete! Let me remove the temporary files..."
rm -rf src/ # 😱

This is why frequent git commits are non-negotiable. I commit after every small success.

The debugging debris #

During problem-solving, Claude Code leaves a trail of attempts:

Debug logging statements everywhere
Failed attempt code commented out
Multiple approaches tried in parallel
Temporary test files and scripts
Extra imports and unused functions

Before merging, always ask: “Review your changes and remove all debugging artifacts and failed attempts.” Getting to a high code coverage (approaching 100%) helps ensure no dead code remains, as long as your tests use public APIs.

Why this happens #

These patterns emerge because AI is trained on public code that often:

Maintains backward compatibility for years
Uses defensive programming for public APIs
Includes extensive error handling for user input
Follows “enterprise” patterns even for simple scripts
Contains debugging code from development

This is why constraints are essential—without them, Claude defaults to these “safe” but overcomplicated patterns.

8. Critical success factors: what actually makes this work #

After two months of intense usage, here are the non-negotiable practices that make agentic coding actually productive:

Teach it your test commands (day 1 priority) #

This is absolutely crucial. Claude Code needs to know how to run your tests and how to activate the environment:

# To run tests, use: 
uv run pytest tests/ -xvs"

# For coverage:
uv run pytest --cov=src --cov-report=term-missing

# To run specific test
uv run pytest tests/test_module.py::test_function

Without this, it’s just guessing whether code works. With it, it becomes genuinely useful.

Git commits are your safety net #

I commit obsessively when using Claude Code:

After every successful (even partial) feature implementation
Before letting it attempt any major refactoring
Whenever tests pass
Before any “cleanup” operation

But here’s the key: I develop every feature in its own branch, even as a solo developer. My workflow:

Create a feature branch for each new feature or fix
Make frequent commits as snapshots (sometimes just a period for the message or tell Claude to commit)
Open a pull request to review the full diff myself
Merge only after reviewing all changes

This has saved me from disaster multiple times. Git is your undo button when Claude goes nuclear (e.g., breaks or removes your code).

Maintain healthy skepticism #

Never trust, always verify:

“I fixed it!” → “Show me the test output”
“It’s working now!” → “Run the tests again with -xvs”
“The logs show success!” → “Cat the actual log file”
“It’s production ready!” → “Did you run the tests?”

Think of Claude Code as an enthusiastic junior developer who sometimes exaggerates their accomplishments.

Force it to clean up after itself #

After any debugging session, always:

"Review all your changes and remove:
- Debug print statements
- Commented out code
- Failed attempt implementations
- Temporary test files
- Unused imports"

My custom /anti-cruft command automates this, but you can do it manually too.

Code coverage is your friend #

High test coverage (90%+) ensures:

The code Claude wrote actually runs
No dead code from failed attempts
All paths are exercised
You can refactor confidently

9. My secret weapon: voice-to-code workflow #

One of my biggest productivity multipliers isn’t Claude Code itself—it’s how I communicate with it. Using my agent-cli tool, I’ve developed a voice-first workflow that’s transformed how I write prompts.

The problem with typing prompts #

Effective agentic coding requires precise, detailed instructions. A good prompt isn’t 20 words—it’s often 200-500 words explaining exactly what you want, what to avoid, and how to approach the problem. Most people don’t want to type that much.

My voice workflow #

Here’s my actual workflow:

Start recording with agent-cli transcribe (hotkey triggered)
Review Claude’s changes while speaking my thoughts aloud
Speak for minutes about what I want, what’s wrong, what to fix
Paste the transcription directly into Claude Code

This is powered by OpenAI’s Whisper model running locally on my RTX 3090 at home. It’s more reliable than macOS dictation and gives me complete privacy.

Why this works so well #

Rich prompts: I naturally give more context when speaking
Code review narration: I can review code while explaining issues
Thinking out loud: Speaking helps clarify my own thoughts
Speed: Speaking is 3-4x faster than typing
Precision: I can be incredibly specific without typing fatigue

For example, instead of typing “fix the bug,” I might say:

“I’m looking at line 45 where you’re handling the authentication. The problem is you’re not checking if the token has expired before making the API call. Also, you’ve added a try-except block here that’s suppressing errors—remove that. And while you’re at it, the logging statement on line 52 is using the wrong format. Make sure to follow our project’s logging conventions…”

This level of detail is what makes AI coding actually productive.

10. Additional tips for agentic coding #

Beyond the critical success factors and voice workflow, here are more tips to maximize your productivity:

Set clear constraints #

Create a CLAUDE.md or similar file in your project root with your preferences:

## Project standards

- Never use try/except to suppress errors silently
- Always run pytest before marking task complete
- Use type hints for all functions
- Prefer simple solutions over clever ones

Use the task system #

Claude Code’s todo system is great for complex work:

1. ✅ Implement core functionality
2. ✅ Write comprehensive tests
3. 🔄 Fix failing tests
4. ⏳ Add documentation

Leverage the search capabilities #

Let Claude search for existing patterns:

“Find all API endpoint implementations in this project”
“Show me how we handle authentication elsewhere”
“What testing patterns are we using?”

Review in chunks #

Instead of reviewing 500 lines at once:

Have Claude implement a single feature
Review and understand it
Run tests
Commit(s) and PR
Move to the next feature

11. The tools I’ve tried #

My journey through AI coding assistants has been evolutionary:

Pre-AI era #

Manual coding with IDE autocomplete
Stack Overflow copy-paste
Productivity baseline: 1-2 packages per year

Phase 1: copy-paste tools (March 2023 - July 2025) #

ChatGPT/Claude Web + my clip-files tool
Manual context sharing
Helpful for ideation and code review
Productivity: 3x improvement (from 2 to 6 packages/year)

Cursor (early 2024) #

Great autocomplete
Limited context awareness
Became frustratingly slow
Still led to my vibe coding disaster
Productivity: 5x improvement (but with quality issues)

Phase 2: Claude Code (July 2025 - present) #

Full codebase awareness
Can execute commands and debug
Self-correcting through test iterations
Maintains context across sessions
$70 spent on first night, $10,000 worth in first month
Productivity: 24x improvement over pre-AI baseline with maintained quality

OpenAI’s Codex CLI (the reasoning beast) #

Recently tried OpenAI’s Codex—their new CLI alternative to Claude Code—and I’ll admit, for pure reasoning with their GPT-5-Codex model, it’s impressive:

Solved a race condition in an unfamiliar language that took me hours with Claude Opus
More elegant solutions for complex algorithmic problems
Better at deep reasoning when given the same context

But here’s why I still use Claude Code daily:

Claude Code’s UX is superior: Resume conversations, better interface
Codex’s CLI is fragile: Hit Ctrl-C accidentally? Lose everything, start over (muscle memory keeps betraying me!)
Claude Code handles interruptions: First Ctrl-C clears text, second quits gracefully
No session persistence: Codex forgets everything, Claude Code remembers
Context management: Have to re-provide all context after accidental exits

For a complex race condition bug, Codex with GPT-5 gave me the most elegant solution. But for day-to-day development where I need reliability, good UX, and the ability to resume work?

Why not Copilot or other “cheaper” tools? #

Let me be blunt: You get what you pay for. People often say “I tried Copilot and it wasn’t that useful” or “Gemini in chat didn’t help much.” Of course not—these tools are at least 10x cheaper than Claude Code with Opus.

Your $20/month GitHub Copilot subscription simply cannot provide the same quality or quantity of high-quality tokens as Claude Code. It’s like comparing a bicycle to a Ferrari and wondering why the bicycle isn’t as fast. The economics don’t work out:

Copilot: $20/month for limited autocomplete
Claude Code Pro: $200/month for unlimited* agentic assistance (*capped, but generous)
Actual API value used: $10,000+/month at my usage level

The 10x price difference reflects a 10x (or more) difference in capability. Copilot is autocomplete on steroids. Claude Code is a tireless pair programming partner with perfect memory who can debug, test, and iterate. The ROI is obvious when you’re shipping 10x more code.

12. Is this sustainable? #

You might wonder if this productivity is sustainable or if I’m just building lower-quality software faster. Here’s my honest assessment:

The good #

Faster prototyping: Ideas to working code in hours, not days
Better test coverage: AI never gets lazy about writing tests
More experimentation: Lower cost to try new ideas
Improved documentation: AI loves writing docs (I don’t)
Reduced burnout: Less time on boilerplate, more on interesting problems

The challenges #

Code review fatigue: You must stay vigilant
Dependency on tools: What if Claude Code disappears?
Cost: $200/month Pro plan (but worth every penny given the $10K+ value)
The temptation to “vibe”: Constant discipline required
Addiction potential: I literally couldn’t stop coding, having the desire to deplete all tokens and get my money’s worth!

My verdict #

It’s absolutely sustainable if you maintain discipline. The moment you start accepting code you don’t understand, you’re building a house of cards. But with proper review and testing, agentic AI is a massive force multiplier.

13. The experience multiplier: why seniority matters more than ever #

Here’s a counterintuitive truth about agentic AI tools: they amplify your existing skills exponentially, not linearly. This creates a fascinating paradox where these tools become increasingly valuable as you become more experienced.

The exponential trap for beginners #

For beginners, agentic tools can feel like a superpower initially. They can build a working application in hours! But here’s the danger: without the experience to recognize bad patterns, they’re essentially building at 10x speed in the wrong direction. It’s like giving a Formula 1 car to someone who just got their driver’s license—the speed multiplies both the distance traveled and the potential for catastrophic crashes.

A beginner using Claude Code might:

Accept complex solutions they don’t understand
Build massive technical debt at record speed
Miss architectural problems that will haunt them later
Create code that “works” but is impossible to maintain

They’re forced into a difficult position: either slow down to understand everything (negating the speed benefit) or accumulate technical debt at an exponential rate.

The senior developer advantage #

For experienced developers, it’s a completely different game. When I look at Claude Code’s suggestions, I can instantly recognize:

“Oh, that’s the Factory pattern—makes sense here”
“This error handling is too defensive, let’s simplify”
“That’s going to cause N+1 queries, need to refactor”
“This violates our project’s conventions, fix it”

The review process that might take a beginner hours takes me minutes. I’m not learning what the code does—I’m validating that it does what I already know it should do.

The right way for every level #

That said, everyone can benefit from agentic tools—you just need to adjust your approach:

For beginners

Use AI as a learning accelerator: Ask it to explain every decision
Build small projects first: Master fundamentals before scaling up
Prioritize understanding over speed: Better to build slowly and learn
Pair with mentors: Have experienced developers review AI-generated code

For intermediate developers

Focus on patterns: Use AI to learn architectural patterns and best practices
Experiment safely: Try new approaches in side projects first
Question everything: Don’t accept solutions without understanding the tradeoffs

For senior developers

Leverage for velocity: Use your experience to review and direct quickly
Focus on architecture: Let AI handle implementation while you design systems
Teach the AI: Create project-specific guidelines to improve output quality
Push boundaries: Explore more ambitious projects now that implementation is faster

The real multiplier effect #

The productivity boost isn’t just about writing code faster. For senior developers, agentic tools multiply:

Architecture exploration: Test multiple approaches quickly
Refactoring confidence: Refactor fearlessly with AI helping maintain functionality
Learning velocity: Explore new languages and frameworks with a knowledgeable assistant
Documentation quality: Finally have time (and help) for comprehensive docs
Testing coverage: AI never skips tests, even for “simple” functions

This is why I’ve been able to ship 14 packages in 18 months. It’s not just that I’m writing code faster—I’m able to explore ideas, validate approaches, and iterate on designs at a pace that was previously impossible.

14. Reality check: what AI can and can’t do #

Many people try AI coding and give up disappointed. That’s because they’re treating it like a magic genie that will solve the unsolved problems of the universe. It won’t.

What AI CAN do (the churn work) #

AI excels at tasks you could do yourself but would take hours:

Setting up CI/CD pipelines: Boilerplate YAML configuration
Writing tests: Especially unit tests for existing code
Fixing dependency issues: Adapting to breaking changes in libraries
CRUD operations: Basic create, read, update, delete functionality
Data transformations: Converting between formats, parsing, serialization
Documentation: README files, docstrings, API documentation
Refactoring: Renaming variables, extracting functions, reorganizing code
Bug fixes: Especially those with clear error messages

This is the “churn”—the necessary but time-consuming work that makes up 80% of development.

What AI CAN’T do (the creative work) #

Don’t expect AI to:

Solve novel algorithmic problems: It won’t invent the next PageRank
Make architectural decisions: It can’t decide if you need microservices
Understand business logic: It doesn’t know why your company does things
Create truly innovative solutions: It remixes what it’s seen, not invents
Make judgment calls: Security vs convenience, performance vs maintainability

The sweet spot #

I use AI for tasks where:

I know how to do it but it would take time
The solution exists somewhere in its training data
Success is measurable through tests or clear output
The problem is well-defined with clear constraints

This isn’t about AI doing things you can’t do—it’s about AI doing things you don’t want to spend time doing. Once you understand this, AI becomes incredibly powerful.

15. Looking forward: the future of development #

As I showed in my local AI journey, I’m deeply interested in where AI and development intersect. Agentic coding represents a fundamental shift in how we build software:

From writing code to reviewing and directing
From memorizing APIs to understanding patterns
From debugging alone to collaborative problem-solving
From slower iteration to rapid experimentation

This isn’t about replacing developers—it’s about amplifying our capabilities. I’m still the architect, the decision-maker, and the quality gatekeeper. But now I can focus on the interesting problems while my AI assistant handles the implementation details.

16. Conclusion: principles over process #

My journey from vibe coding skeptic to agentic coding advocate might seem like a complete reversal, but it’s not. My core principle remains unchanged: never commit code you don’t understand.

What’s changed is the tooling has finally caught up to the promise. With agentic AI assistants, I can maintain my standards while dramatically increasing my output. Most of those 32 packages on PyPI have users and are well-tested and documented projects that solve real problems.

The key is treating AI as what it is: an incredibly powerful assistant, not a replacement for thinking. Use it to explore ideas faster, implement solutions quicker, and test more thoroughly. But always, always understand what you’re building.

As I continue building in public and sharing my tools, I’m excited to see where this technology takes us. If you’re interested in trying agentic coding yourself, start small, maintain your standards, and prepare to be amazed by what you can build.

What’s your experience with AI coding assistants? Have you tried moving from generative to agentic tools? I’d love to hear your thoughts!