zsh-ai-autocomplete

AI-powered, context-aware ghost-text autocomplete for zsh, plus a prompt <natural language> → shell-command translator with an approval step. Runs on a local model via Ollama (free, private, no rate limits) or the Gemini API — switchable with one env var. Aware of and able to drive zoxide and fzf.

It replaces the experience of zsh-autosuggestions (history-only) and the old smart-suggestion plugin with something context-aware, while reusing zsh-autosuggestions proven async rendering engine under the hood.

---

What you get

1. AI ghost text as you type — the gray suggestion after your cursor is generated by Gemini using your current folder & its project type, the commands runnable there (real npm/pnpm/make/just/cargo/go targets), git state, files, recent history, and frecent zoxide dirs as context. So in a Node repo npm run → your actual scripts; in a Makefile dir make → your real targets. Accept it with Tab (your existing binding). Falls back to instant local history when AI is empty/offline/rate-limited, so it never feels broken.
   • AI-first, history-aware: the model is told your recent commands so its completions match your habits.
   • Next-command prediction: on an empty prompt it anticipates your likely next command from history + context (after git add → git commit; a fresh shell in a Node repo → yarn run dev; a repo with changes → git status). It declines when nothing is obvious. Tab to accept. Toggle: aizsh predict off.
2. prompt <what you want> — natural language → a real command:

   prompt show the 5 biggest files under my home dir
   prompt jump to my api project and run the tests
   prompt delete every node_modules folder below here
   

   The command is pre-filled on your next prompt line, editable — nothing runs until you press Enter. Aliases: ai, ask.
3. Automatic AI fix for failures — when a command fails (or isn't found), the corrected command appears as grey ghost text on your next prompt:

   $ gti status
   zsh: command not found: gti
   $ git status        ← grey correction; Tab to accept, or just keep typing
   

   Fixes run on the always-warm coder model so they're quick. The correction is plain grey ghost text — accept with Tab, or ignore it and type whatever you want. Handles typos, wrong flags, missing sudo, command not found (incl. brew install …). Nothing runs without you. Skips failures where non-zero is normal (grep, diff, test, …) and aborts (Ctrl-C / exit ≥128). Takes priority over a generic prediction. Toggle: aizsh autofix off.
4. Uses zoxide & fzf when it helps — the model is instructed to emit z <kw> to jump to frecent dirs, and to pipe through fzf when you need to choose (a file to edit, a branch to switch to, a process to kill). When it returns alternatives, you pick between them in an fzf menu.
5. Safety, not censorship — commands are classified none / caution / dangerous (a local regex upgrades the rating even if the model under-rates it). Dangerous commands are allowed when they're what you want: in prompt mode they're flagged red and need an extra y to pre-fill; ghost and prediction can surface them when contextually appropriate (never speculatively). You always review the gray text before accepting — nothing auto-runs.
6. Ctrl-O — force an AI suggestion immediately for the current line (bypasses the typing debounce).

---

Install

Requires zsh, Python 3, and Ollama (free, local). Optional but recommended: fzf, zoxide, zsh-autosuggestions, zsh-syntax-highlighting.

brew install ollama && brew services start ollama
ollama pull qwen2.5-coder:7b      # ghost text (FIM)
ollama pull qwen3.5:9b            # prompt / fix / predict
git clone https://github.com/paiksca/zsh-ai-autocomplete.git ~/.config/ai-zsh

Add to ~/.zshrc — after zsh-autosuggestions, before zsh-syntax-highlighting:

export AIZSH_PROVIDER=ollama
export AIZSH_GHOST_MODEL=qwen2.5-coder:7b
export AIZSH_PROMPT_MODEL=qwen3.5:9b
ZSH_AUTOSUGGEST_USE_ASYNC=1
ZSH_AUTOSUGGEST_STRATEGY=(ai history)
source ~/.config/ai-zsh/ai-zsh.plugin.zsh

(Prefer the cloud? Set AIZSH_PROVIDER=gemini with GEMINI_API_KEY instead.) Open a new shell, then run aizsh doctor to verify.

---

How it works

 your keystrokes
      │
 zsh-autosuggestions (async worker)        prompt()  /  Ctrl-O
      │  strategy = (ai history)                │
      ▼                                         ▼
 _zsh_autosuggest_strategy_ai  ──────►  pure-zsh unix-socket client
      │  (debounce sleep, gating)               │  (falls back to one-shot python)
      ▼                                         ▼
                    backend.py  (daemon)
        warm TLS connection · cache · rate-limit · context (git/ls/zoxide)
                              │
                              ▼
                         Gemini API

• backend.py — stdlib-only Python. Runs as a small per-user daemon on a unix socket ($TMPDIR/ai-zsh-$UID.sock) so it can keep a warm HTTPS connection to Gemini (big latency win), cache ghost results, and rate-limit to stay inside the free tier. Also works one-shot if the daemon is down.
• ai-zsh.plugin.zsh — registers the ai suggestion strategy, the prompt function, the Ctrl-O widget, and the aizsh management command.
• Debounce trick: the AI strategy sleeps briefly first; because zsh-autosuggestions kills the previous async worker on every keystroke, a fetch only survives (and hits the network) once you actually pause typing.
• Request cancellation: when a worker is killed (you typed another key), the daemon aborts the in-flight model generation instead of letting it finish. Without this, a near-zero AIZSH_DEBOUNCE would queue a generation per keystroke and the local model would backlog; with it, you can run AIZSH_DEBOUNCE ~0 and only the latest keystroke actually generates.
• Typing into the ghost never re-queries: if your buffer is just you typing what the suggestion already shows, the remaining ghost is served from cache (prefix-aware) in ~1 ms — the model only runs when you diverge from it.
• Instant statistical layer (best of both worlds): the daemon also keeps a fast model of your habits — frecency (frequency + recency), directory affinity, command sequences (make build → make test), and success rate (failed commands rank lower), seeded from your ~/.zsh_history. On every keystroke it renders an instant guess (~ms) as grey text, and the LLM refines/replaces it when it lands a beat later (no keystroke needed). So you get statistical speed and LLM intelligence. Empty-prompt prediction is instant too.
  • Smart from day one: preloaded with common commands + sequences (git, npm, docker, cargo, …) so a fresh install is useful before it's learned anything.
  • Context-aware, cheaply: boosts commands that fit the current project and suggests its real runnable targets (npm run build, cargo test, make …) — using the already-cached folder context, so it stays instant and complements what the LLM does with the full context.
• Simple, consistent keys: Tab accepts the whole ghost, Ctrl-Right accepts the next word, Esc dismisses (the prompt spinner). Rebind word-accept via AIZSH_WORD_ACCEPT_KEY.

---

Files

File	Purpose
backend.py	Gemini client, daemon, context-builder, safety classifier
ai-zsh.plugin.zsh	zsh integration (strategy, prompt, aizsh, Ctrl-O)

These live in this folder and are sourced from ~/.zshrc.

---

Configuration (env vars, set before sourcing the plugin)

Variable	Default	Meaning
AIZSH_PROVIDER	auto (ollama if running, else gemini)	ollama | gemini | openai
AIZSH_GHOST_MODEL	per-provider*	model for ghost text
AIZSH_PROMPT_MODEL	per-provider*	model for prompt
AIZSH_OLLAMA_URL	http://127.0.0.1:11434	Ollama server
AIZSH_OLLAMA_KEEPALIVE	30m	keep the model warm in RAM
AIZSH_FIM	1 (ollama)	ghost uses fill-in-middle for clean token-level completion
AIZSH_FIM_TOKENS	12	max ghost completion length; latency scales with it (raise for longer completions, lower for snappier)
AIZSH_THINK	0	enable thinking for prompt (qwen3.x) — better on hard tasks, but +25-50s
AIZSH_THINK_TOKENS	1500	token budget when thinking
AIZSH_AUTOFIX	1	suggest a fix (grey ghost text) when a command fails / isn't found
AIZSH_FIX_MODEL	ghost model	model for auto-fix (default = the always-warm coder, for speed)
AIZSH_AUTOFIX_SKIP	grep, diff, test, …	leading commands where non-zero is normal (no fix)
AIZSH_PREDICT	1	predict the next command on an empty prompt
AIZSH_STATS	1	instant statistical ghost (frecency/dir/sequence) shown under the LLM
AIZSH_WORD_ACCEPT_KEY	^[[1;5C	key (Ctrl-Right) to accept the next word of the ghost
AIZSH_STATS_FILE	~/.cache/ai-zsh/stats.json	where the learned command stats persist
AIZSH_STATS_HALFLIFE	7	recency half-life (days) for frecency scoring
AIZSH_STATS_MAX	4000	cap on learned commands (lowest-frecency evicted on save)
AIZSH_BASE_URL	—	OpenAI-compatible server (llama.cpp, LM Studio, vLLM)
AIZSH_API_KEY	—	bearer token for openai provider (if needed)
GEMINI_API_KEY	(from Keychain)	required for gemini provider
AIZSH_COMPACT	on for local, off for gemini	trim context for small models
AIZSH_DEBOUNCE	0.18	seconds idle before an AI fetch fires; can go ~0 (daemon cancels superseded calls)
AIZSH_MIN_LEN	2	min chars before AI ghost text
AIZSH_HIST_N	15	recent history lines sent as context
AIZSH_RATE_MAX	12	max calls / 60s; 0 = unlimited (recommended for local)
AIZSH_FORCE_KEY	^o	key to force an AI suggestion
AIZSH_DEBUG	off	set to 1 to log to $AIZSH_LOG

*Code defaults per provider — ollama: qwen2.5-coder:1.5b (ghost) / qwen2.5-coder:3b (prompt); gemini: gemini-2.0-flash-lite / gemini-2.0-flash. This machine is configured (in ~/.zshrc) for ollama with ghost = qwen2.5-coder:7b (FIM, ~0.5s) and fix/prompt = qwen3.5:9b. Ghost must stay a coder model (FIM); the qwen3.x general models have no FIM. Enable deeper reasoning on hard prompts with export AIZSH_THINK=1 (slower).

Switching providers

# Local (current setup) — free, private, no limits:
export AIZSH_PROVIDER=ollama
export AIZSH_GHOST_MODEL=qwen2.5-coder:1.5b
export AIZSH_PROMPT_MODEL=qwen2.5-coder:7b

# Back to Gemini cloud:
export AIZSH_PROVIDER=gemini
# (uses GEMINI_API_KEY)

# Any OpenAI-compatible local server (llama.cpp / LM Studio / vLLM):
export AIZSH_PROVIDER=openai
export AIZSH_BASE_URL=http://localhost:1234/v1
export AIZSH_GHOST_MODEL=<model-name>

Then aizsh restart. Pull other Ollama models with ollama pull <name> (e.g. qwen2.5-coder:0.5b for max speed, :7b/:14b for quality).

---

aizsh command

aizsh status      # daemon state, models, key, strategy
aizsh doctor      # diagnose model access + quota (recommended if nothing appears)
aizsh restart     # restart the daemon (REQUIRED after editing backend.py or changing models/config)
aizsh stop|start
aizsh log|tail    # view the daemon log
aizsh on|off      # toggle AI ghost text (off = plain history)
aizsh autofix on|off   # toggle automatic fix-on-failure
aizsh predict on|off   # toggle empty-prompt next-command prediction
aizsh test prompt "free up disk space"   # one-off backend call

---

Troubleshooting

First step always: aizsh doctor — shows the provider, whether the model backend is reachable, whether the model is pulled, and a live generation probe.

(Ollama) No ghost text / prompt errors.

• Is the server up? brew services start ollama (it should auto-start at login).
• Is the model pulled? ollama list; if not, ollama pull qwen2.5-coder:1.5b and ollama pull qwen2.5-coder:7b. History-based ghost text still works regardless (the AI layer degrades quietly).

(Gemini) generation: FAILED with 403 or "limit: 0". The API key's Google Cloud project has no quota/access — enable billing or use a key with quota. (This is why this machine defaults to local Ollama instead.)

Suggestions feel slow.

• First call after idle loads the model into RAM (a few seconds); keep_alive (30m) keeps it warm after that. Raise AIZSH_OLLAMA_KEEPALIVE to keep it loaded longer.
• Use a smaller ghost model: export AIZSH_GHOST_MODEL=qwen2.5-coder:0.5b, then aizsh restart.
• For near-instant, per-keystroke updates set export AIZSH_DEBOUNCE=0.02 and export AIZSH_RATE_MAX=0 (local has no quota). The daemon cancels superseded generations, so this stays fast instead of backlogging the model.

Turn the AI off temporarily: aizsh off (back on with aizsh on).

Logs: aizsh tail (set AIZSH_DEBUG=1 first for detail).