I replaced thefuck with local Gemma — and a native-first CNF

Hook

The screenshot that started this:

Bash

$ agent reach doctor
zsh: command not found: agent
Did you mean: wget, genv, gron …

agent-reach is an OpenCode skill — not a shell binary — so no nearest-neighbor corrector can guess it. But the irrelevant wget / genv suggestion bugged me enough to ask: can a small local model do better than zsh's built-in suggester for the common case of real-command typos?Spoiler: yes — but only if you don't let the model run on every typo.

Why I didn't just install thefuck

thefuck (97k ⭐) is the obvious answer. I checked its repo first:

Last commit to master: 2024-01-25 ("Fix typos in README.md")
434 open issues, 129 open PRs, 0 merged in 16+ months
Requires Python 3.5+ — EOL since 2020-09
Not archived, but every recent contributor has gone quiet

That's "effectively abandoned." So instead I built a 60-line zsh + 250-line Python replacement that uses local Gemma 4 e4b on Apple Silicon via oMLX. The result handles ~70% of what thefuck did, plus typo correction and arbitrary-buffer explanation — with no Python deps, no per-rule maintenance, and no abandonment risk on someone else's calendar.

Design rule #1: native-first

The naive design fires the model on every CNF event. That's:

~300 ms TTFT for Gemma 4 e4b (warm) on an M3 Max
Cold start: 5–30 s if oMLX hasn't loaded the model yet
Lots of LLM cost for the trivial case where you just typed gti instead of git

So the helper does this instead:

Native first. A tiny Python routine walks $PATH, computes Damerau-Levenshtein distance, applies a priority tie-break (git beats gtr for gti), and returns one suggestion. ~50 ms.
Model fallback only when:
- No native hit (the typo doesn't resolve to any single binary within edit distance 2), OR
- The cmdline has arguments — because kubclt get pods -o wde has 3 typos and nearest-neighbor on kubclt alone is useless without seeing the rest.

Live verification (real Ghostty shell, Powerlevel10k prompt):

Bash

$ gti status
zsh: command not found: gti
  ↪ did you mean: git status ?

That arrow line cost me 50 ms. No model fired.

Design rule #2: model only knows one prompt schema

Every model call goes through the same shape: system prompt enforces "EXACTLY ONE LINE, no markdown, no preamble", temperature 0, max_tokens 80. The Python wrapper strips a small set of common LLM prefixes (corrected:, command:, $ , % ) defensively in case the model still adds them.Why so strict? Because the output goes straight to the user's terminal as a single suggestion line. If the model says "Here's the corrected command:\n\ngit status", that's broken UX. One line is the contract.

The CLI: ~/.bin/zsh-ai-helper

Single Python file. Four subcommands:

Mode	Purpose	Timeout
native <cmd>	Damerau-Levenshtein nearest-PATH binary	n/a (in-process)
cnf <cmdline>	Correct a failed command via Gemma	3.0 s
explain <cmdline>	One-line explanation of the current buffer	4.0 s
fix <cmdline>	Fix the last failed command (reads stderr from stdin)	3.0 s

Configuration via env:

Bash

export ZAI_OMLX_URL="http://127.0.0.1:11234/v1/chat/completions"  # default
export ZAI_MODEL="gemma-4-e4b-it-4bit"                             # default
export OMLX_API_KEY="..."                                          # else: bw get password 'oMLX API'

The Bitwarden fallback was a late addition — first version blocked for 2 s on bw get and made every CNF feel sluggish. Fix: a precmd hook warms the key in the background on the first interactive prompt, then unhooks itself.

The wiring: ~/.zshrc

Three things land:

command_not_found_handler — native first, model fallback as above, silent if nothing useful. Prints to stderr, returns 127 (preserves zsh exit semantics).
Ctrl-G explain widget — captures $BUFFER, asks Gemma "what does this do?", shows result via zle -M below the prompt. Does not modify your buffer. You re-type if you want it.
fk — the thefuck replacement (next section).

Zsh

# Ctrl+G: explain whatever's in your buffer
bindkey '^G' _zai_explain_widget
bindkey -M viins '^G' _zai_explain_widget
bindkey -M vicmd '^G' _zai_explain_widget

The fk story

fk re-runs the last failed command to capture its stderr, then proposes a fix:

Bash

$ git push
fatal: The current branch feature/x has no upstream branch.
[exit 128]

$ fk
🔧 Run: git push --set-upstream origin feature/x  [Enter to run · any other key to cancel]

Five deterministic fast-path rules run first (the high-traffic 80%):

Rule	Trigger	Fix
dry	git git push	collapse repeated leading token
sudo	stderr "permission denied"	prepend sudo
mkdir_p	mkdir "no such file or directory"	add -p
brew_install	stderr "no available formula"	add --cask
git_push_upstream	"no upstream branch"	parse git's own suggested --set-upstream line, or fall back to git symbolic-ref

If no rule matches, the last command + stderr (truncated to 600 chars) goes to Gemma with the prompt:

You fix the most recent failed zsh command. Given the original command line and its stderr output, reply with EXACTLY ONE LINE: the corrected command to run. […] NEVER propose destructive operations like rm -rf, git push --force, kubectl delete, dd if=, or mkfs.*. If you cannot fix it, repeat the original command verbatim.

The model's reply is then run through a destructive-cmd guard — 13 regex patterns covering rm -[rRf], force pushes, kubectl delete, drop database, dd if=, fork bombs, chmod 777, halt/reboot, and writing to /dev/sd[a-z]. If the proposal matches any, it's silently dropped. The guard is defense in depth — the prompt does most of the work — but two layers feel right when an LLM might one day decide rm -rf /tmp/* ; mkdir -p /tmp is a "fix" for a permission error.The execution flow:

precmd hook captures $history[-1] + $? after every command
fk checks rc != 0 (refuses on success) and last cmd doesn't start with fk itself
If the last cmd matches the destructive regex on the shell side, skip replay entirely (let the model try from cmd shape only — usually it stays silent)
Otherwise replay with a 5-s timeout: timeout 5 zsh -c "$last" 2>&1 >/dev/null
Pipe (cmd, stderr) to zsh-ai-helper fix
Print proposal + read -k 1 — Enter to execute, anything else cancels
On confirm: print -s -- "$fix" (push to history) then eval "$fix"

No auto-execute. That matters.

Bench (M3 Max, model warm)

Operation	TTFT
native gti → git	~50 ms
cnf "sudp brew install foo" → sudo brew install foo	~320 ms
cnf "find . -tpe f -nmae '.md'" → find . -type f -name '.md'	~320 ms
cnf "kubclt get pods -o wde" → kubectl get pods -o wide	~240 ms
explain "find . -type f -name '*.md' -exec grep -l TODO {} +"	~470 ms
fk with rule hit (e.g. dry / git push upstream)	~80 ms
fk with Gemma fallback	~350 ms
Cold start (oMLX loads e4b for the first time)	5–30 s

Warmup trick: ~/.bin/zsh-ai-helper cnf "warmup" once after omlx-restart.

Honest limits

Cycling. thefuck lets you ↑/↓ through alternative fixes. Mine offers one. If you don't like it, you cancel and type your own.
Multi-step fixes. fk only handles "the last single command failed." If you need git stash; git pull; git stash pop it won't propose that — by design.
The destructive guard is not airtight. It's pattern-based, not semantic. A creatively-crafted "fix" could still hurt. The proposal-only (no auto-exec) UX is the real safety layer.
You need oMLX running. I run mine as a LaunchAgent on :11234. If it's down, the CLI is silent (curl --max-time 3 is forgiving). The CNF path still works via native suggester only.
The 4B model has a ceiling. gemma-4-e4b-it-4bit is great for one-line transformations. It is not great at multi-line shell wizardry. Use Ctrl+G explain for "what does this pipeline do," not for "rewrite my Kubernetes operator."

What I'd reach for next

Apple Silicon + a 4B local model + a deterministic-first wrapper hits a sweet spot I didn't expect:

Free (zero per-call cost)
Offline-capable (works on a plane)
Fast enough for terminal interactivity (≤500 ms warm, 80 ms when a deterministic rule hits)
Boring enough to forget (silent on no-op, no popups, no auto-exec)

The same architecture could swap the model for gpt-5-nano or gemini-flash-lite via Sub2API and trade local-only for slightly higher accuracy. I left the env hook (ZAI_OMLX_URL, ZAI_MODEL) in place for that day.

Let AI Do It

If you use an AI coding assistant like OpenCode, paste this prompt and let it handle everything:

Bash

I want to set up a local-Gemma-backed zsh terminal helper on my Mac.
It should provide: native-first command-not-found suggestions, Ctrl+G explain,
and an `fk` last-failed-command fixer (replacement for thefuck).
Reference: https://mjshao.fun/blog/replacing-thefuck-with-local-gemma

My setup:
- Local model backend (oMLX / Ollama / LMStudio / Sub2API): [ask me]
- Model name (default: gemma-4-e4b-it-4bit): [ask me]
- API endpoint (default: http://127.0.0.1:11234/v1/chat/completions): [ask me]
- API key source (env var name, Bitwarden item, or none): [ask me]

Steps:
1. Pre-flight: verify python3, curl, zsh ≥ 5.8, the model backend reachable
2. Show me what you found (5-line summary) before writing files
3. Write ~/.bin/zsh-ai-helper (single Python file, 4 subcommands: native/cnf/explain/fix)
4. Append the CNF handler + Ctrl+G widget + fk function to ~/.zshrc
   (idempotent — grep before append; never duplicate)
5. Warm the model with one `cnf "warmup"` call
6. Smoke test: gti → git, sudp ls → sudo ls, mkdir /a/b/c then fk
7. Print final verification before declaring done

(For Chinese readers — same prompt works in Chinese; just swap the language.)

Hook

起因是这张截图：

Bash

$ agent reach doctor
zsh: command not found: agent
Did you mean: wget, genv, gron …

agent-reach 是一个 OpenCode skill，不是 shell binary，所以任何 nearest-neighbor corrector 都猜不到它。但 wget / genv 这种不相干的建议还是把我烦到了，于是我想试一下：对于真实命令拼错这种常见场景，一个小本地模型能不能比 zsh 内置 suggester 更靠谱？剧透：可以，但前提是别让模型在每一次 typo 上都跑一遍。

为什么我没直接装 thefuck

thefuck (97k ⭐) 当然是最明显的答案。我先去看了它的 repo：

Last commit to master: 2024-01-25 ("Fix typos in README.md")
434 open issues, 129 open PRs, 0 merged in 16+ months
Requires Python 3.5+ — EOL since 2020-09
Not archived, but every recent contributor has gone quiet

这基本就是 "effectively abandoned"。所以我换了个思路，写了一个 60-line zsh + 250-line Python 的替代品，在 Apple Silicon 上通过 oMLX 调 local Gemma 4 e4b。结果覆盖了 thefuck 大概 70% 的用法，还多了 typo correction 和 arbitrary-buffer explanation，而且没有 Python deps，不需要维护一堆 per-rule 规则，也不用把未来押在别人的维护日历上。

设计规则 #1：native-first

最天真的设计，是每次 CNF event 都直接打模型。代价是：

~300 ms TTFT for Gemma 4 e4b (warm) on an M3 Max
Cold start: 5–30 s if oMLX hasn't loaded the model yet
你只是把 git 打成了 gti，却要为这种小事付一次 LLM cost

所以这个 helper 实际上这么做：

Native first. 一个很小的 Python routine 会扫 $PATH，算 Damerau-Levenshtein distance，加上 priority tie-break（git 比 gtr 更适合 gti），然后返回一个 suggestion。~50 ms。
Model fallback 只在这两种情况触发：
- 没有 native hit，也就是 typo 在 edit distance 2 内没法解析成某个单一 binary，OR
- cmdline 带 arguments，因为 kubclt get pods -o wde 有 3 个 typo，只看 kubclt 的 nearest-neighbor，完全看不到后面的上下文。

Live verification（真实 Ghostty shell，Powerlevel10k prompt）：

Bash

$ gti status
zsh: command not found: gti
  ↪ did you mean: git status ?

这行 arrow 只花了我 50 ms。模型没有启动。

设计规则 #2：模型只认识一种 prompt schema

每一次 model call 都走同一种 shape：system prompt 强制 "EXACTLY ONE LINE, no markdown, no preamble"，temperature 0，max_tokens 80。Python wrapper 还会防御性地去掉一小组常见 LLM prefix（corrected:, command:, $ , % ），免得模型还是顺手加了前缀。为什么这么严格？因为输出会直接进用户的 terminal，显示成一行 suggestion。如果模型回答 "Here's the corrected command:\n\ngit status"，UX 就坏了。一行，就是 contract。

CLI：~/.bin/zsh-ai-helper

一个 Python 文件，四个 subcommands：

Mode	Purpose	Timeout
native <cmd>	Damerau-Levenshtein nearest-PATH binary	n/a (in-process)
cnf <cmdline>	Correct a failed command via Gemma	3.0 s
explain <cmdline>	One-line explanation of the current buffer	4.0 s
fix <cmdline>	Fix the last failed command (reads stderr from stdin)	3.0 s

Configuration via env:

Bash

export ZAI_OMLX_URL="http://127.0.0.1:11234/v1/chat/completions"  # default
export ZAI_MODEL="gemma-4-e4b-it-4bit"                             # default
export OMLX_API_KEY="..."                                          # else: bw get password 'oMLX API'

Bitwarden fallback 是后面才加的。第一版每次都卡在 bw get 上 2 s，导致每个 CNF 都很迟钝。修法是：用 precmd hook 在第一个 interactive prompt 后台 warm key，然后把自己 unhook 掉。

接线：~/.zshrc

落地就三件事：

command_not_found_handler — native first，model fallback，找不到有用建议就静默。打印到 stderr，返回 127，保留 zsh exit semantics。
Ctrl-G explain widget — 捕获 $BUFFER，问 Gemma "what does this do?"，再用 zle -M 显示在 prompt 下面。不会修改你的 buffer。 想用的话你自己重新输入。
fk — thefuck replacement，下一节讲。

Zsh

# Ctrl+G: explain whatever's in your buffer
bindkey '^G' _zai_explain_widget
bindkey -M viins '^G' _zai_explain_widget
bindkey -M vicmd '^G' _zai_explain_widget

fk 的故事

fk 会重新跑上一个失败命令来捕获 stderr，然后给出一个修复建议：

Bash

$ git push
fatal: The current branch feature/x has no upstream branch.
[exit 128]

$ fk
🔧 Run: git push --set-upstream origin feature/x  [Enter to run · any other key to cancel]

它会先跑五个 deterministic fast-path rules，覆盖高频的 80%：

Rule	Trigger	Fix
dry	git git push	collapse repeated leading token
sudo	stderr "permission denied"	prepend sudo
mkdir_p	mkdir "no such file or directory"	add -p
brew_install	stderr "no available formula"	add --cask
git_push_upstream	"no upstream branch"	parse git's own suggested --set-upstream line, or fall back to git symbolic-ref

如果没有 rule 命中，就把 last command + stderr（截断到 600 chars）丢给 Gemma，prompt 是：

You fix the most recent failed zsh command. Given the original command line and its stderr output, reply with EXACTLY ONE LINE: the corrected command to run. […] NEVER propose destructive operations like rm -rf, git push --force, kubectl delete, dd if=, or mkfs.*. If you cannot fix it, repeat the original command verbatim.

模型的回复还会再过一层 destructive-cmd guard，13 个 regex pattern，覆盖 rm -[rRf]、force pushes、kubectl delete、drop database、dd if=、fork bombs、chmod 777、halt/reboot，以及写 /dev/sd[a-z]。如果 proposal 命中任何一个，就静默丢掉。这个 guard 是 defense in depth，主要工作还是 prompt 在做，但当一个 LLM 某天可能把 rm -rf /tmp/* ; mkdir -p /tmp 当成 permission error 的 "fix" 时，两层保险让我更放心。执行流是这样：

precmd hook captures $history[-1] + $? after every command
fk checks rc != 0 (refuses on success) and last cmd doesn't start with fk itself
If the last cmd matches the destructive regex on the shell side, skip replay entirely (let the model try from cmd shape only — usually it stays silent)
Otherwise replay with a 5-s timeout: timeout 5 zsh -c "$last" 2>&1 >/dev/null
Pipe (cmd, stderr) to zsh-ai-helper fix
Print proposal + read -k 1 — Enter to execute, anything else cancels
On confirm: print -s -- "$fix" (push to history) then eval "$fix"

不自动执行。这点很重要。

Bench (M3 Max, model warm)

Operation	TTFT
native gti → git	~50 ms
cnf "sudp brew install foo" → sudo brew install foo	~320 ms
cnf "find . -tpe f -nmae '.md'" → find . -type f -name '.md'	~320 ms
cnf "kubclt get pods -o wde" → kubectl get pods -o wide	~240 ms
explain "find . -type f -name '*.md' -exec grep -l TODO {} +"	~470 ms
fk with rule hit (e.g. dry / git push upstream)	~80 ms
fk with Gemma fallback	~350 ms
Cold start (oMLX loads e4b for the first time)	5–30 s

Warmup trick: ~/.bin/zsh-ai-helper cnf "warmup" once after omlx-restart.

诚实讲限制

Cycling. thefuck 可以让你用 ↑/↓ 在多个 alternative fixes 之间切。我的只有一个建议。不喜欢就 cancel，然后自己打。
Multi-step fixes. fk 只处理 "the last single command failed"。如果你需要的是 git stash; git pull; git stash pop，它不会这么提，故意的。
The destructive guard is not airtight. 它是 pattern-based，不是 semantic。一个设计得很刁钻的 "fix" 仍然可能造成伤害。真正的安全层是 proposal-only，也就是不 auto-exec。
You need oMLX running. 我自己的 oMLX 是 LaunchAgent，跑在 :11234。如果它挂了，CLI 会静默（curl --max-time 3 比较宽容）。CNF path 仍然会通过 native suggester 工作。
The 4B model has a ceiling. gemma-4-e4b-it-4bit 很适合 one-line transformations，不适合 multi-line shell wizardry。Ctrl+G explain 用来问 "what does this pipeline do" 很好，别拿它去 "rewrite my Kubernetes operator"。

下一步我会怎么做

Apple Silicon + 一个 4B local model + deterministic-first wrapper，打到了一个我没想到的甜点区：

Free (zero per-call cost)
Offline-capable (works on a plane)
Fast enough for terminal interactivity (≤500 ms warm, 80 ms when a deterministic rule hits)
Boring enough to forget (silent on no-op, no popups, no auto-exec)

同一套架构也可以通过 Sub2API 把模型换成 gpt-5-nano 或 gemini-flash-lite，用 local-only 换一点更高的 accuracy。我保留了 env hook（ZAI_OMLX_URL, ZAI_MODEL），等哪天需要再说。

Let AI Do It

如果你用 OpenCode 这种 AI coding assistant，直接把这段 prompt 粘进去，让它替你干完：

Bash

我想在 Mac 上搭一个 local-Gemma-backed zsh terminal helper。
它需要提供：native-first command-not-found suggestions、Ctrl+G explain，
以及一个 `fk` last-failed-command fixer，用来替代 thefuck。
Reference: https://mjshao.fun/blog/replacing-thefuck-with-local-gemma

My setup:
- Local model backend (oMLX / Ollama / LMStudio / Sub2API): [ask me]
- Model name (default: gemma-4-e4b-it-4bit): [ask me]
- API endpoint (default: http://127.0.0.1:11234/v1/chat/completions): [ask me]
- API key source (env var name, Bitwarden item, or none): [ask me]

Steps:
1. Pre-flight: verify python3, curl, zsh ≥ 5.8, the model backend reachable
2. Show me what you found (5-line summary) before writing files
3. Write ~/.bin/zsh-ai-helper (single Python file, 4 subcommands: native/cnf/explain/fix)
4. Append the CNF handler + Ctrl+G widget + fk function to ~/.zshrc
   (idempotent — grep before append; never duplicate)
5. Warm the model with one `cnf "warmup"` call
6. Smoke test: gti → git, sudp ls → sudo ls, mkdir /a/b/c then fk
7. Print final verification before declaring done

给中文读者：上面这段就是中文版，直接用就行。

I replaced thefuck with local Gemma — and a native-first CNF

我用本地 Gemma 替掉了 thefuck —— 还顺手做了 native-first 的命令纠错

Hook

Why I didn't just install thefuck

Design rule #1: native-first

Design rule #2: model only knows one prompt schema

The CLI: ~/.bin/zsh-ai-helper

The wiring: ~/.zshrc

The fk story

Bench (M3 Max, model warm)

Honest limits

What I'd reach for next

Let AI Do It

Hook

为什么我没直接装 thefuck

设计规则 #1：native-first

设计规则 #2：模型只认识一种 prompt schema

CLI：~/.bin/zsh-ai-helper

接线：~/.zshrc

fk 的故事

Bench (M3 Max, model warm)

诚实讲限制

下一步我会怎么做

Let AI Do It

For AI Agents

Recent posts