diff --git a/.github/workflows/wiki-currency.yml b/.github/workflows/wiki-currency.yml index 9b3578e..37f4d26 100644 --- a/.github/workflows/wiki-currency.yml +++ b/.github/workflows/wiki-currency.yml @@ -1,8 +1,8 @@ # Wiki-currency check — the single home of the "is the wiki current?" rules. # # This workflow is the ONLY place the check logic lives (symbol audit, idiom deny-list, -# provenance). It versions with the content it grades, which is why it lives in -# autolens_assistant and not in the release hub. Two triggers feed it: +# provenance, chat-bundle currency). It versions with the content it grades, which is why +# it lives in autolens_assistant and not in the release hub. Two triggers feed it: # # 1. workflow_call — invoked by PyAutoHands at stack-release time with the new version, # so a release that moves the API is graded against the wiki immediately. PyAutoHands @@ -150,6 +150,20 @@ jobs: run "Provenance (--check-provenance)" --check-provenance run "Citation paths (--check-citations)" --check-citations + # The free-tier chat bundles embed a generated snapshot of the public API + # surface, so they go stale exactly when the wiki does — same check, same + # report. Also catches AGENTS.md → AGENTS_CHAT.md rule drift and dead links. + echo "## Chat bundle currency (chat_bundle.py --check)" >> "$REPORT" + echo '```' >> "$REPORT" + python autoassistant/chat_bundle.py --check >> "$REPORT" 2>&1 + bundle_rc=$? + echo '```' >> "$REPORT" + if [ "$bundle_rc" -ne 0 ]; then + echo "**FAILED** — regenerate with \`make chat-bundle\` (needs the stack installed)." >> "$REPORT" + fail=1 + fi + echo >> "$REPORT" + cat "$REPORT" >> "$GITHUB_STEP_SUMMARY" if [ "$fail" -ne 0 ]; then echo "::error::wiki-currency drift detected — see the job summary / drift-report artifact." diff --git a/AGENTS_CHAT.md b/AGENTS_CHAT.md new file mode 100644 index 0000000..caedad5 --- /dev/null +++ b/AGENTS_CHAT.md @@ -0,0 +1,154 @@ +# AGENTS_CHAT.md — chat-mode instructions for autolens_assistant + +You are the **PyAutoLens Assistant** running in a **browser chat** (claude.ai, ChatGPT, or +similar). You can read, reason, plan and write code — you **cannot** run code, open the +user's files, or edit this repository. + +This file is the chat-mode counterpart of [`AGENTS.md`](AGENTS.md). `AGENTS.md` is canonical +for coding agents that can execute; this file keeps the rules that still apply when nothing +can be run, and drops the ones that assume a shell, a checkout, or write access. Where a rule +below is quoted from `AGENTS.md`, it is reproduced **verbatim** and drift-checked by +`autoassistant/chat_bundle.py --check`. + +Everything here applies whether the repository reached you through a GitHub connector, an +uploaded knowledge pack, or a pasted bundle. + +--- + +## Lead by engaging + +**Never open a reply with what you can't do.** Ask what the user is trying to achieve, ask +them to describe or plot their data, plan the model with them, and draft the script — do that +*first*. You can route to examples, explain lensing, design an analysis, and review pasted +scripts, errors and figures. That is most of the job. + +Raise the handoff to a local coding agent (Claude Code, Codex) when **execution** becomes the +actual blocker — running the fit, inspecting `.fits` files, iterating on results — not as an +opening disclaimer. + +--- + +## Safety invariants that survive with no execution + +### Real data → inspect before fitting + +Before composing a model-fit on real observational data, the data must be looked at: **(a)** +extra galaxies / foreground stars / artefacts (the #1 source of fit bias), and **(b)** the +mask extent — the radius/shape that captures the lensed emission without dragging in noise or +contaminants. Never leave the mask radius as a silent default on real data. + +> **If you can't plot it yourself — no code execution, e.g. a GitHub-connector chat +> — the gate is not waived: ask the user to plot and inspect the data, and to confirm both (a) +> contaminants and (b) the mask extent, before you compose the fit.** These are the questions +> every real-data run must ask, on every harness. + +Give the user a short plotting snippet and ask them to report back what they see. Simulated +data is exempt. + +### Never reconstruct the API from memory + +Older PyAutoLens releases used a different API and are **heavily represented in model training +data**. On a coding-agent harness a code gate blocks stale symbols; **in chat there is no gate, +so this discipline is the only safeguard**: + +> if you +> can't point at a `skills/` (or `dir()`) example for a call, treat it as unverified and say so +> rather than emitting it. + +Check any symbol you are unsure about against `api_surface.md` (the generated public-symbol +list shipped with this bundle). A name there exists; for *call syntax*, mirror the matching +`skills/` example. + +### Plotting is functional — the #1 stale-API error + +Pass `output_path=...`, `output_filename=...`, `output_format="png"` straight to the `aplt.*` +call. + +> **The object-oriented plotters (`aplt.FitImagingPlotter`, `ImagingPlotter`, `TracerPlotter`, +> …) and the `aplt.MatPlot2D` / `aplt.Output` objects have been removed — do not use them. + +Wrong: + +```python +aplt.FitImagingPlotter(fit=fit, mat_plot_2d=aplt.MatPlot2D(...)).subplot_fit_imaging() +``` + +Right: + +```python +aplt.subplot_fit_imaging( + fit=fit, output_path="scripts/scratch/ring/", output_filename="fit", output_format="png" +) +``` + +### Standard imports + +```python +import autofit as af +import autolens as al +import autolens.plot as aplt +``` + +--- + +## How to use this material + +1. **Instructions** — this file. +2. **Skills** (`skills/*.md`) — *procedural*: how to do one task. Lensing skills are + `al_.md`. Read the relevant one end-to-end before writing code; its examples are the + source of truth for API calls. +3. **Wiki** (`wiki/core/`) — *content*: what a Sersic profile is, which searches exist, how + SLaM phases work. + +> **Rule of thumb.** *How do I do X?* → a skill. *What / which / why X?* → the wiki. *Build +> something end-to-end?* → compose skills, citing wiki pages as you go. + +If you have a GitHub connector, fetch pages on demand from the raw URLs listed in +[`llms-chat.txt`](llms-chat.txt). If you are working from an uploaded or pasted bundle, use +only what is in front of you — and say so plainly when the answer would need a page you don't +have, rather than inventing it. + +**Do not bulk-fetch.** `wiki/literature/` and `llms-full.txt` are very large; pulling them in +will crowd out the conversation. Fetch the one page you need. + +--- + +## Modes + +- **Teacher** — *learn*: explain the physics and inference, step through, link to examples. +- **Assistant** — *do*: plan, then draft. Narrate what you're doing and why; give a one-line + plan read-back before diving in. Ask a blocking question only when correctness genuinely + depends on the answer. + +Infer the mode from the opening request (default **assistant**), state it in one line, and +invite correction. Say "Teacher mode" or "Assistant mode" in a prompt to set it explicitly. + +--- + +## Generated script style + +Every script you write uses the PyAutoLens **workspace** style, not banner comments: an +opening docstring (title underlined with `=`, short orientation, `__Contents__`), then each +section introduced by a `"""__Section__"""` docstring carrying the physics/inference framing. +The full spec and a copyable worked example are in `skills/_style.md` — mirror it rather than +reconstructing the format from memory. + +Scripts are written for the user to run locally: committed scripts → `scripts/`, throwaway +plots → `scripts/scratch/`, `search.fit(...)` output → `./output/`. + +--- + +## What is out of scope in chat + +These are `AGENTS.md` rules that need a shell or a checkout. **Do not claim to perform them, +and do not ask the user to run them as a precondition:** + +- the session-start environment/API drift-check (`audit_skill_apis.py --check-version`) +- the executable code gate (`audit_skill_apis.py --code ...`) +- reading or writing `wiki/project/profile.md`, `wiki/project/` journal entries, or `.maintainer` +- the commit cadence, and any git operation +- cloning source repos into `sources/`, or editing PyAuto\* source +- science-project scaffolding (`start-new-project`), which creates and manages a repo + +When one of these is the natural next step, name it as **the point to switch to a local coding +agent** and describe what the user would do there. diff --git a/FREE_TIER_SETUP.md b/FREE_TIER_SETUP.md new file mode 100644 index 0000000..31432e5 --- /dev/null +++ b/FREE_TIER_SETUP.md @@ -0,0 +1,256 @@ +# Setting up the Assistant on a free AI plan + +This page gets the **PyAutoLens Assistant** running inside a browser chat — including on the +**free tiers** of Claude and ChatGPT, with no paid subscription and no local install. + +If you can install software locally and want the assistant to actually *run* fits, skip this +page: a coding agent (Claude Code, Codex) is strictly more capable. See the README's +[AI Coding Agent](README.md#ai-coding-agent-cli) section. This page is for everyone else. + +**Last verified: 2026-08-02.** Plan features and quotas change often, and the free tiers change +most. Everything below describes *observed behaviour at that date*, not a promise about what +any plan includes today. If a step doesn't match what you see, the troubleshooting section at +the end covers the failure modes we know about. + +--- + +## What you get, and what you don't + +In a browser chat the assistant can do most of the thinking work: + +- plan a lens model with you and explain the trade-offs +- write complete, current-API PyAutoLens scripts for you to run +- explain strong-lensing concepts, and route you to the right example +- review a script, an error, or a figure you paste in + +What it **cannot** do without a coding agent: run the fit, read your `.fits` files, inspect +your results folder, or iterate on a live run. When you hit that wall, the assistant will say +so and tell you what to switch to. + +One rule matters more than any other, and it is why this setup exists at all: **older +PyAutoLens releases are heavily represented in AI training data, and their API is out of +date.** An AI answering from memory will confidently write code that no longer works. Loading +this repository is what stops that — it ships the current API surface, generated from the +pinned stack, so the assistant checks itself instead of guessing. + +--- + +## Pick your route + +| Your situation | Route | Setup effort | +|---|---|---| +| Claude, any plan (incl. Free) | **A — GitHub connector** | ~2 min, best results | +| Claude Free, connector not working or not wanted | **B — Project + knowledge pack** | ~5 min | +| ChatGPT Free | **C — paste the bundle** (connectors are paid-only) | ~30 s per chat | +| ChatGPT Plus/Pro, or you want a one-click share | **D — custom GPT** | maintainer builds once | +| Any other chat (Gemini, Copilot, …) | **C — paste the bundle** | ~30 s per chat | + +Routes A and B give the assistant the whole repository. Route C gives it a curated ~6k-token +core plus links it can fetch if it has browsing. All of them enforce the API-currency rule. + +--- + +## Route A — Claude with the GitHub connector + +The GitHub connector is available on **all Claude plans, including Free** +([Anthropic's docs](https://support.claude.com/en/articles/10167454-use-the-github-integration)). +This is the best free setup: the assistant reads the repository directly and always sees +current content. + +1. In Claude, open **Settings → Connectors** and enable **GitHub**. Authorise it and grant + access to public repositories (this repo is public — you do not need to grant access to + anything of your own). +2. *(Recommended)* Create a **Project** — Claude Free allows up to 5 — and put the prompt from + step 3 in its custom instructions, so every chat in that project starts configured. +3. Start a chat with this prompt: + +```text +Use the autolens_assistant repository: https://github.com/PyAutoLabs/autolens_assistant + +Start by reading its front door: +https://raw.githubusercontent.com/PyAutoLabs/autolens_assistant/main/llms.txt + +Follow its read order (AGENTS_CHAT.md → the relevant skill → wiki) and its API rules. +First tell me whether you can actually read llms.txt — if you can't, say so plainly +and don't answer from memory. +``` + +**Naming `llms.txt` explicitly matters.** The connector does not reliably find it on its own, +and answers are markedly better when it is pointed there first. + +The final sentence is a deliberate honesty check. If the assistant can't read the file, you +want to know immediately — not after it writes you a script from a 2023 API. + +--- + +## Route B — Claude Project with the knowledge pack + +Use this when you'd rather not connect GitHub, or the connector is misbehaving. Claude Free +Projects hold roughly 200K tokens of knowledge and — on Free — put it in **full context** +rather than retrieving fragments, so the assistant sees all of it. + +1. Download the files in **[`chat_pack/`](chat_pack/)** (11 files, ~61k tokens total). Easiest + way without git: download the repository ZIP from the green **Code** button on GitHub and + take that folder. +2. In Claude, create a **Project**, then upload every file from `chat_pack/` into its + **knowledge**. +3. Paste this into the project's **custom instructions**: + +```text +You are the PyAutoLens Assistant. Follow 00_instructions.md in your project knowledge +exactly — especially: never write PyAutoLens from memory, check symbols against +01_api_surface.md, and use only the functional plotting API (aplt.subplot_*), never the +removed object-oriented plotters. Lead by engaging with my science goal; raise the +handoff to a local coding agent only when actually running code is the blocker. +``` + +Every chat in that project is now configured. You can add your own papers or data notes to the +same project knowledge. + +> **Note.** Claude Projects can't be shared on Free or Pro (sharing is a Team/Enterprise +> feature), so each person does this once for themselves. It takes about five minutes. + +--- + +## Route C — paste the bundle (works anywhere, incl. ChatGPT Free) + +ChatGPT's connectors are **paid-plan only**, and free ChatGPT has a small context window. So +the free-ChatGPT route is a deliberately compact paste. + +1. Open **[`llms-chat.txt`](llms-chat.txt)** and copy the whole file (~6k tokens — it is sized + to leave room for an actual conversation). +2. Paste it as your first message in a new chat, followed by your question. + +That file is self-contained: chat-mode instructions, the complete generated list of public +PyAuto\* symbols, and a routing table of raw URLs. If your chat has browsing, it can fetch any +skill or wiki page it needs from those URLs. If it doesn't, it still has the rules and the API +surface, and it is instructed to tell you when an answer would need a page it can't reach. + +**Re-paste it in each new chat.** Nothing persists between conversations on a free plan unless +you put it in custom instructions or a project. + +> **ChatGPT tip.** Paste the bundle, then add your question in the *same* message. A first +> message that is only context sometimes gets a "what would you like to do?" reply that wastes +> one of your limited flagship-model turns. + +--- + +## Route D — a custom GPT (one-click for your users) + +Free ChatGPT users **can use** custom GPTs; they just can't create them. So one person with a +Plus/Pro account can build a GPT once and share the link with everyone else — the only +first-class way to hand a configured assistant to free ChatGPT users. + +Build recipe (maintainers — see [Maintaining](#maintaining-the-bundles) for regenerating +inputs first): + +1. **Create a GPT** → *Configure*. +2. **Instructions**: paste the contents of `chat_pack/00_instructions.md`. +3. **Knowledge**: upload all files from `chat_pack/`. They are pre-split into ≤20 topic files + with distinctive headings, because GPT knowledge is retrieved by **RAG chunking** rather + than read whole — one topic per file is what makes retrieval land on the right chunk. +4. **Capabilities**: enable *Web Browsing* (so it can fetch skills and wiki pages beyond the + pack). Code Interpreter is optional and does **not** give it PyAutoLens — the library is not + installed in that sandbox, so it cannot run a fit there. +5. *(Optional)* Add an **Action** against `raw.githubusercontent.com` for always-current + content. Publishing a GPT with an Action requires a privacy-policy URL. +6. Publish **Anyone with the link**, and record the link here. + +**Status: not yet built.** This needs a paid ChatGPT account and a browser, so it is a manual +maintainer step. When it exists, its link goes here and in the README. + +--- + +## First prompts to try + +Once set up, any of these work (the COSMOS-Web Ring data ships with the repository): + +```text +Find the data on the COSMOS-Web ring, give me a short script to plot it in PyAutoLens, +and then, given that I'm a new user, give me an overview of the different ways we can +perform strong lens modeling of this system. +``` + +```text +Teacher mode. + +I'm new to PyAutoLens and want to learn the basic workflow end-to-end. Walk me through +simulating Euclid-like imaging of a simple strong lens, plotting it, and fitting it. +``` + +```text +I have HST imaging of a galaxy-scale lens. Help me plan the model: lens light, mass, and +source. Ask me what you need to know about the data first. +``` + +That last one exercises the behaviour that matters most — on **real data** the assistant is +required to make you look at the image before it composes a fit, and to settle two things with +you: whether there are extra galaxies or artefacts in the frame, and how big the mask should +be. It can't plot your data itself in chat, so it will ask you to. That is the rule working, +not the assistant being unhelpful. + +--- + +## Troubleshooting + +**"I can't access that repository" / it answers from memory anyway.** +The most common cause is a `blob/` URL. `github.com/.../blob/main/llms.txt` is an HTML page +that many chats receive as an empty JavaScript shell. Always use the +`raw.githubusercontent.com` form — every link in `llms.txt` is already in that form. If it +still fails, fall back to Route C. + +**It fetched `llms.txt` but can't follow the links inside it.** +Some consumer chats only fetch URLs *you* pasted, not ones they discovered while reading. Paste +the specific URL you want it to read, or use Route C. + +**It wrote `aplt.FitImagingPlotter(...)` or `aplt.MatPlot2D(...)`.** +That is the stale-API failure. Those classes were removed. Reply: *"Plotting is functional now +— re-check against `01_api_surface.md` and the plotting skill, and rewrite using +`aplt.subplot_fit_imaging(...)`."* If it keeps happening, your context was lost — re-paste the +bundle or check the project knowledge actually uploaded. + +**It ran out of context / got slow and vague.** +Something large was pulled in. `wiki/literature/` and `llms-full.txt` are big enough to crowd +out the conversation on their own. Start a fresh chat and tell it to fetch single pages only. + +**It stopped mid-task, or switched to a weaker model.** +Free plans have usage limits that are unpublished and vary with load — in the region of a few +dozen messages per 5 hours on Claude, and a smaller number of flagship-model turns on ChatGPT +before it falls back to a lighter model. Long modelling sessions are where a paid plan or a +local coding agent genuinely pays for itself. + +**A generated script fails on an API error anyway.** +The bundle pins one specific stack version (stated at the top of `01_api_surface.md`). If your +installed PyAutoLens is newer, there may be genuine drift. Report it as an issue with the error +text — and note this is exactly the failure a coding agent's code gate catches automatically. + +--- + +## Maintaining the bundles + +`llms-chat.txt` and `chat_pack/` are **generated** — do not hand-edit them. + +```bash +make chat-bundle # regenerate both artifacts +make chat-bundle-check # verify the committed copies are current (CI-friendly) +``` + +Regenerate on a machine with the PyAuto\* stack installed, so the API surface is refreshed +from a live `dir()` rather than reused from the committed copy (the script warns loudly when it +falls back). + +The generator (`autoassistant/chat_bundle.py`) enforces four things, each of which fails the +build or the check: + +- **Verbatim-anchor drift** — the rules `AGENTS_CHAT.md` shares with `AGENTS.md` must stay + word-identical. Reword one in `AGENTS.md` and the check fails until both are updated. +- **Dead links** — every rewritten raw URL must resolve to a real path in the repository. +- **Staleness** — the committed artifacts must match a fresh build. +- **Budgets** — the paste tier stays under its token budget, and the pack under the 20-file + GPT knowledge limit. + +It also *warns* when a complete skill belongs to no group in `SKILL_GROUPS`, so newly written +skills don't silently fail to ship. + +Chat-surface smoke tests for each route are in +[`modes/maintainer.md`](modes/maintainer.md#chat-surface-compatibility-smoke-test). diff --git a/Makefile b/Makefile index edbb81a..657937d 100644 --- a/Makefile +++ b/Makefile @@ -1,4 +1,4 @@ -.PHONY: validate-literature-citations audit test +.PHONY: validate-literature-citations audit test chat-bundle chat-bundle-check validate-literature-citations: python -m autoassistant.literature validate-citations @@ -10,3 +10,12 @@ audit: # Assistant tooling test suite (slow: the gate tests import autolens per case). test: python -m pytest autoassistant/tests -q + +# Regenerate the free-tier chat bundles (llms-chat.txt + chat_pack/). +# Run with the stack installed so the API surface is refreshed, not reused. +chat-bundle: + python autoassistant/chat_bundle.py + +# Verify the committed bundles are current and their invariants hold. +chat-bundle-check: + python autoassistant/chat_bundle.py --check diff --git a/README.md b/README.md index d5e21e3..76ea636 100644 --- a/README.md +++ b/README.md @@ -26,10 +26,25 @@ There are two ways to use `autolens_assistant`, choose whichever best suits how Ask questions to a conversational AI assistant such as **ChatGPT** or **Claude** in a desktop browser or web. -This requires you to do two things: - -- Make sure your assistant has a **GitHub connector** enabled so it can read this repository and in your initial prompt give it the URL to this repository (https://github.com/PyAutoLabs/autolens_assistant). -- Make sure your initial prompt points the assistant explicitly at the file [`llms.txt`](llms.txt), which gives it the initial instructions on how `autolens_assistant` works. +**This works on free plans.** The best setup is to give the assistant the repository through a +**GitHub connector** — available on every Claude plan including Free — and to point your first +prompt explicitly at [`llms.txt`](llms.txt), which tells the assistant how `autolens_assistant` +works. Naming that file matters: assistants do not reliably find it on their own. + +No connector (ChatGPT Free, where connectors are paid-only)? Paste +[`llms-chat.txt`](llms-chat.txt) — a self-contained ~6k-token bundle carrying the rules and the +current API surface — as your first message instead. + +Step-by-step instructions for each platform, plus troubleshooting for when an assistant answers +from memory or can't read the repo: **[`FREE_TIER_SETUP.md`](FREE_TIER_SETUP.md)**. + +| Interface | Repo access | Setup | +|---|---|---| +| **Claude** (incl. Free) | GitHub connector, all plans | Enable the connector, use the prompt below | +| **Claude** (incl. Free), no connector | Project knowledge | Upload [`chat_pack/`](chat_pack/) to a Project | +| **ChatGPT Free** | Connectors are paid-only | Paste [`llms-chat.txt`](llms-chat.txt) | +| **ChatGPT Plus/Pro** | Connectors, or a custom GPT | Connector, or build a GPT from `chat_pack/` | +| **Anything else** | Browsing, or nothing | Paste [`llms-chat.txt`](llms-chat.txt) | Here is a good initial prompt which you can copy and paste it ChatGPT or Claude to try it out, noting that data for the COSMOS-Web Ring is included in this repository as an example: @@ -56,7 +71,7 @@ Use the autolens_assistant (www.github.com/PyAutoLabs/autolens_assistant with th first reading its llms.txt file for initial start up. I want to model the F277W and F444W JWST imaging of the COSMOS-Web Ring simultaneously, which are in -the folder dataset/cosmos_web_ring. Model the lens light with a multi-Gaussian expansion (MGE), its mass with a singular +the folder dataset/imaging/cosmos_web_ring. Model the lens light with a multi-Gaussian expansion (MGE), its mass with a singular isothermal ellipsoid plus external shear, and model the source also using an MGE. For speed, run the analysis on my laptop GPU using a JAX optimizer that estimates only the maximum-likelihood solution. Plot the observed image at each wavelength in the left column, its lensed source model in the middle column, and its source on the right column. @@ -91,7 +106,7 @@ Or, if you want to see `autolens_assistant` perform end-to-end lens modeling: ``` I want to model the F277W and F444W JWST imaging of the COSMOS-Web Ring simultaneously, which are in -the folder dataset/cosmos_web_ring. Model the lens light with a multi-Gaussian expansion (MGE), its mass with a singular +the folder dataset/imaging/cosmos_web_ring. Model the lens light with a multi-Gaussian expansion (MGE), its mass with a singular isothermal ellipsoid plus external shear, and model the source also using an MGE. For speed, run the analysis on my laptop GPU using a JAX optimizer that estimates only the maximum-likelihood solution. Plot the observed image at each wavelength in the left column, its lensed source model in the middle column, and its source on the right column. diff --git a/autoassistant/audit_skill_apis.py b/autoassistant/audit_skill_apis.py index 9d02c41..892592b 100644 --- a/autoassistant/audit_skill_apis.py +++ b/autoassistant/audit_skill_apis.py @@ -798,6 +798,83 @@ def write_baseline(root: Path) -> Path: return path +def render_symbol_dump() -> str: + """Render the installed stack's public API surface as Markdown. + + The chat bundle (`autoassistant/chat_bundle.py`) ships this page so a + no-execution harness has the *current* symbol list to check itself against, + rather than trusting prose that has to be hand-maintained. The baseline JSON + records only a hash and a count; this renders the names themselves. + + Raises SystemExit if the stack is not importable — a symbol dump must never + be written from a partial import, or the bundle would advertise a truncated + API surface as complete. + """ + versions: dict[str, str] = {} + for name in VERSIONED_MODULES: + try: + mod = importlib.import_module(name) + except Exception as e: # noqa: BLE001 + sys.exit( + f"cannot dump symbols: {name} not importable ({e!r}). " + f"Activate the venv (source activate.sh) and retry." + ) + versions[name] = str(getattr(mod, "__version__", "(no __version__)")) + + lines: list[str] = [ + "# PyAuto* public API surface (generated)", + "", + "Generated by `python autoassistant/audit_skill_apis.py --dump-symbols`.", + "**Do not hand-edit.** This is the public `dir()` surface of the exact stack", + "the assistant content was validated against — use it to check whether a symbol", + "you are about to write actually exists.", + "", + # Deliberately no generation date: this page is committed and compared + # byte-for-byte by `chat_bundle.py --check`, so a rendered-on date would + # make the check fail every day after it was written. The stack version + # below is the provenance that actually matters — it changes when the + # surface changes, and not otherwise. + "- Stack versions: " + + ", ".join(f"`{k}` {v}" for k, v in sorted(versions.items())), + "", + "> A name appearing here means the symbol exists; it does **not** document its", + "> signature. For call syntax, mirror the matching `skills/` example.", + "", + ] + + total = 0 + for name in BASELINE_MODULES: + mod = importlib.import_module(name) + names = _public_names(mod) + total += len(names) + alias = { + "autolens": "al", + "autolens.plot": "aplt", + "autogalaxy": "ag", + "autoarray": "aa", + "autofit": "af", + }.get(name) + heading = f"## `{name}`" + (f" (alias `{alias}`)" if alias else "") + lines += [heading, "", f"{len(names)} public symbols.", ""] + lines += [", ".join(f"`{n}`" for n in names), ""] + + lines += [f"**Total: {total} public symbols across {len(BASELINE_MODULES)} modules.**", ""] + return "\n".join(lines) + + +def dump_symbols(out: Optional[str]) -> int: + """Write (or print) the public-API-surface Markdown page.""" + text = render_symbol_dump() + if out: + path = Path(out) + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(text, encoding="utf-8") + print(f"wrote {path}") + else: + print(text) + return 0 + + def check_version(root: Path) -> int: """Compare the installed stack's public API surface against the committed baseline. @@ -1414,6 +1491,13 @@ def main() -> int: help="Snapshot the installed stack (versions + API-surface hash) to " "wiki/core/api_audit_baseline.json and exit. Re-pin after a deliberate upgrade.", ) + parser.add_argument( + "--dump-symbols", + action="store_true", + help="Emit the installed stack's public API surface as Markdown (to --out, else " + "stdout) and exit. Shipped in the chat bundle so no-execution harnesses can check " + "a symbol exists; generated, never hand-maintained.", + ) parser.add_argument( "--check-version", action="store_true", @@ -1477,6 +1561,10 @@ def main() -> int: if args.check_install: return check_installation() + # Self-contained: reads the installed stack directly, needs no repo root. + if args.dump_symbols: + return dump_symbols(args.out) + # Code-gate modes are self-contained: they resolve symbols straight against the # installed library, so they need neither `sources.yaml` nor the version baseline. if args.code is not None or args.file is not None: diff --git a/autoassistant/chat_bundle.py b/autoassistant/chat_bundle.py new file mode 100644 index 0000000..cd5cf58 --- /dev/null +++ b/autoassistant/chat_bundle.py @@ -0,0 +1,640 @@ +"""Generate the free-tier chat bundles for autolens_assistant. + +A browser chat (claude.ai Free, ChatGPT Free) has no shell, often no repository +access, and — on ChatGPT Free — a context window measured in low tens of +thousands of tokens. This script turns the repository into two artifacts sized +for those constraints: + +* ``llms-chat.txt`` — the **paste tier**. One self-contained file the user pastes + into a fresh chat. Deliberately small enough to survive a small context window, + so it carries the rules, the generated public-API surface, and a routing table + of absolute raw URLs rather than the reference pages themselves. +* ``chat_pack/`` — the **upload tier**. A handful of merged topic files to attach + to a Claude Project or a custom GPT's knowledge. Fewer than 20 files (a GPT + knowledge limit) and each one topic under a distinctive heading, because GPT + knowledge is retrieved by RAG chunking rather than read whole. + +Both are committed so a user can grab them without running anything. + +Usage:: + + python autoassistant/chat_bundle.py # regenerate both artifacts + python autoassistant/chat_bundle.py --check # verify committed copies are current + +``--check`` is the CI-friendly mode: it regenerates in memory and diffs against +what is committed, exiting non-zero on drift. It also runs the verbatim-anchor +check described under `VERBATIM_ANCHORS` below. + +The API-surface page is produced by ``audit_skill_apis.py --dump-symbols``, which +needs the installed stack. When the stack is absent the committed copy is reused +and a warning is printed, so the bundle stays buildable on a docs-only checkout +without silently advertising a stale API as fresh. +""" + +from __future__ import annotations + +import argparse +import re +import sys +from pathlib import Path +from typing import Iterable, Optional + +RAW_BASE = "https://raw.githubusercontent.com/PyAutoLabs/autolens_assistant/main" +REPO_URL = "https://github.com/PyAutoLabs/autolens_assistant" + +PASTE_REL = Path("llms-chat.txt") +PACK_REL = Path("chat_pack") +API_SURFACE_NAME = "01_api_surface.md" + +# A pasted bundle has to leave room for the actual conversation. ChatGPT Free's +# context is the binding constraint (reported in the low tens of thousands of +# tokens), so the paste tier is budgeted well under it and the reference pages +# are left to fetch-on-demand or the upload tier. +PASTE_TOKEN_BUDGET = 14_000 +GPT_KNOWLEDGE_FILE_LIMIT = 20 + +# --------------------------------------------------------------------------- +# Skill selection +# --------------------------------------------------------------------------- +# Excluded by *role*, not by quality: these need a shell, a checkout, or write +# access, so a chat harness cannot act on them and shipping them only invites it +# to claim it can. Stubs are excluded separately and automatically (see +# `stub_skills`), which keeps this list from drifting as stubs are filled in. +ROLE_EXCLUDED_SKILLS = { + # Setup & maintenance — operate on this repo / the installed stack. + "al_setup_environment", + "al_update_wiki", + "al_audit_skill_apis", + "al_refresh_api_docs", + "al_ingest_paper", + # Project workflow — create and manage repositories. + "start-new-project", + "contribute-upstream", + "init-slam", + # Meta-skills — for authoring skills, not doing science. + "_style", # shipped separately as the script-style page + "_bootstrap_skill", + "README", +} + +# Grouped for the upload tier so related recipes land in one retrievable chunk. +# Order mirrors the workflow: prepare → build → fit → inspect. +SKILL_GROUPS: tuple[tuple[str, str, tuple[str, ...]], ...] = ( + ( + "data_preparation", + "Data preparation", + ("al_prepare_imaging_data", "al_simulate_dataset"), + ), + ( + "model_building", + "Model building", + ( + "al_build_imaging_model", + "al_build_interferometer_model", + "al_custom_profile", + ), + ), + ( + "fitting", + "Fitting", + ( + "al_configure_search", + "al_run_search", + "al_chain_searches", + "al_run_slam_pipeline", + "al_debug_fit_failure", + ), + ), + ( + "results", + "Results & visualisation", + ( + "al_load_results", + "al_plot_tracer", + "al_plot_fit_residuals", + "al_inspect_source_reconstruction", + "al_inspect_results_mcp", + "al_to_notebook", + ), + ), + ( + "advanced", + "Advanced techniques", + ("al_potential_correction",), + ), +) + +# --------------------------------------------------------------------------- +# Verbatim anchors +# --------------------------------------------------------------------------- +# AGENTS_CHAT.md is hand-authored (a regex-stripped AGENTS.md would be unreviewable +# and would break on any rewrap), but the rules that are *identically* correct in +# chat must not quietly diverge from the canonical file. Each anchor below is +# required to appear, whitespace-normalised, in BOTH AGENTS.md and AGENTS_CHAT.md. +# Edit a rule in AGENTS.md and `--check` fails until the chat copy is updated too. +VERBATIM_ANCHORS: tuple[tuple[str, str], ...] = ( + ( + "real-data gate (no-execution branch)", + "If you can't plot it yourself — no code execution, e.g. a GitHub-connector chat " + "— the gate is not waived: ask the user to plot and inspect the data, and to confirm " + "both (a) contaminants and (b) the mask extent, before you compose the fit.", + ), + ( + "standard imports", + "import autofit as af import autolens as al import autolens.plot as aplt", + ), + ( + "never reconstruct from memory", + "if you can't point at a `skills/` (or `dir()`) example for a call, treat it as " + "unverified and say so rather than emitting it.", + ), + ( + "removed object-oriented plotters", + "The object-oriented plotters (`aplt.FitImagingPlotter`, `ImagingPlotter`, " + "`TracerPlotter`, …) and the `aplt.MatPlot2D` / `aplt.Output` objects have been " + "removed — do not use them.", + ), +) + + +def normalise(text: str) -> str: + """Flatten Markdown so anchors compare on wording alone. + + Strips blockquote markers then collapses whitespace, so a rule survives being + rewrapped, re-indented under a list, or quoted with `>` in the chat copy — + the anchor check is about the words, not the layout. + """ + text = re.sub(r"(?m)^[ \t]*>+[ \t]?", "", text) + return re.sub(r"\s+", " ", text).strip() + + +def est_tokens(text: str) -> int: + """Rough token estimate (~4 chars/token). Used for budgeting, not billing.""" + return len(text) // 4 + + +# --------------------------------------------------------------------------- +# Link rewriting +# --------------------------------------------------------------------------- +MD_LINK = re.compile(r"(!?\[[^\]]*\]\()([^)\s]+)(\))") + + +def rewrite_links(text: str, source_rel: Path) -> str: + """Rewrite repo-relative Markdown links to absolute raw URLs. + + A relative link is dead weight in a chat: the harness has no working + directory to resolve it against. Resolving each one against the *source + file's* directory and re-emitting it as a raw URL makes every pointer + fetchable by a connector — and at minimum legible to a user without one. + + Anchors, absolute URLs and mailto: are left alone. + """ + source_dir = source_rel.parent + + def repl(m: re.Match) -> str: + prefix, target, suffix = m.group(1), m.group(2), m.group(3) + if target.startswith(("http://", "https://", "#", "mailto:")): + return m.group(0) + anchor = "" + if "#" in target: + target, _, anchor_part = target.partition("#") + anchor = "#" + anchor_part + if not target: # pure in-page anchor + return m.group(0) + resolved = (source_dir / target).as_posix() + # Normalise ./ and ../ segments without touching the filesystem. + parts: list[str] = [] + for part in resolved.split("/"): + if part in ("", "."): + continue + if part == "..": + if parts: + parts.pop() + continue + parts.append(part) + return f"{prefix}{RAW_BASE}/{'/'.join(parts)}{anchor}{suffix}" + + return MD_LINK.sub(repl, text) + + +# --------------------------------------------------------------------------- +# Repository reads +# --------------------------------------------------------------------------- +def read(root: Path, rel: str | Path) -> str: + path = root / rel + if not path.exists(): + sys.exit(f"chat_bundle: missing required file {rel}") + # Symlinks (e.g. .claude/skills/*) are read through to their target, so a + # bundle never ships a 20-byte path string in place of a skill. + return path.read_text(encoding="utf-8") + + +def stub_skills(root: Path) -> set[str]: + """Skill names marked ``(stub)`` in skills/README.md. + + Parsed rather than hard-coded so filling a stub in automatically promotes it + into the bundle, with no second list to remember to update. + """ + text = read(root, "skills/README.md") + return set(re.findall(r"\[`([^`]+)\.md`\]\([^)]*\)\s*\(stub\)", text)) + + +def selected_skills(root: Path) -> list[str]: + """Every complete, chat-actionable lensing skill, in SKILL_GROUPS order.""" + stubs = stub_skills(root) + out: list[str] = [] + for _, _, names in SKILL_GROUPS: + for name in names: + if name in stubs or name in ROLE_EXCLUDED_SKILLS: + continue + if not (root / "skills" / f"{name}.md").exists(): + sys.exit(f"chat_bundle: SKILL_GROUPS names a missing skill: {name}.md") + out.append(name) + return out + + +def audit_selection(root: Path) -> list[str]: + """Warn about complete `al_*` skills that no group claims. + + Without this, a newly-written skill is silently absent from every bundle. + """ + stubs = stub_skills(root) + grouped = {n for _, _, names in SKILL_GROUPS for n in names} + missing = [] + for path in sorted((root / "skills").glob("al_*.md")): + name = path.stem + if name in stubs or name in ROLE_EXCLUDED_SKILLS or name in grouped: + continue + missing.append(name) + return missing + + +# --------------------------------------------------------------------------- +# API surface +# --------------------------------------------------------------------------- +def api_surface(root: Path) -> tuple[str, bool]: + """Return (markdown, regenerated). Falls back to the committed copy.""" + committed = root / PACK_REL / API_SURFACE_NAME + try: + sys.path.insert(0, str(root / "autoassistant")) + from audit_skill_apis import render_symbol_dump # type: ignore + + return render_symbol_dump(), True + except SystemExit: + pass # stack not importable — handled below + except Exception: # noqa: BLE001 - any import failure means "no stack here" + pass + finally: + if sys.path and sys.path[0] == str(root / "autoassistant"): + sys.path.pop(0) + + if committed.exists(): + print( + "chat_bundle: WARNING - PyAuto* stack not importable; reusing the committed " + f"{PACK_REL / API_SURFACE_NAME}. Regenerate on a machine with the stack " + "installed before releasing.", + file=sys.stderr, + ) + return committed.read_text(encoding="utf-8"), False + + sys.exit( + "chat_bundle: the PyAuto* stack is not importable and no committed API-surface " + f"page exists at {PACK_REL / API_SURFACE_NAME}. Install the stack " + "(source activate.sh) and re-run." + ) + + +# --------------------------------------------------------------------------- +# Artifact construction +# --------------------------------------------------------------------------- +def pinned_stack(root: Path) -> str: + """The stack version these artifacts describe, from the committed baseline. + + Used instead of a generation date. The artifacts are committed and compared + byte-for-byte by ``--check``, so anything that changes on its own (a + rendered-on date) would fail the check every day after it was written. The + pinned version changes exactly when the content it describes changes. + """ + import json + + path = root / "wiki" / "core" / "api_audit_baseline.json" + try: + data = json.loads(path.read_text(encoding="utf-8")) + return str(data["versions"]["autolens"]) + except Exception: # noqa: BLE001 - provenance is best-effort, never fatal + return "unknown" + + +def header(title: str, version: str, extra: Iterable[str] = ()) -> str: + lines = [ + f"# {title}", + "", + f"Generated by `python autoassistant/chat_bundle.py` against PyAutoLens `{version}`.", + f"Source of truth: {REPO_URL} — **do not hand-edit this file.**", + ] + lines.extend(extra) + lines.append("") + return "\n".join(lines) + + +def skill_routing_table(root: Path, names: list[str]) -> str: + """One line per shipped skill: name, raw URL, and its one-line description.""" + index = read(root, "skills/README.md") + lines = ["| Skill | What it does | Fetch |", "|---|---|---|"] + for name in names: + # The index describes each skill as "— " after its link. + m = re.search( + rf"\[`{re.escape(name)}\.md`\]\([^)]*\)\s*(?:\(stub\)\s*)?—\s*(.+?)(?=\n\s*-\s|\n\n|\Z)", + index, + re.S, + ) + desc = normalise(m.group(1)) if m else "" + desc = desc.rstrip(".") + lines.append(f"| `{name}` | {desc} | `{RAW_BASE}/skills/{name}.md` |") + return "\n".join(lines) + + +def build_paste(root: Path, surface: str, names: list[str]) -> str: + """The single pasteable file — rules + API surface + where to fetch the rest.""" + version = pinned_stack(root) + parts = [ + header( + "PyAutoLens Assistant — chat bundle (paste this whole file)", + version, + extra=[ + "", + "You are being given the PyAutoLens Assistant's chat instructions, the exact", + "public API surface of the pinned PyAuto\\* stack, and a routing table for", + "everything else. Read it all before answering.", + "", + "If you can fetch URLs, pull the specific page you need from the tables below.", + "If you cannot, work from what is here and say plainly when an answer would", + "need a page you don't have — do not fill the gap from memory.", + ], + ), + "---", + "", + rewrite_links(read(root, "AGENTS_CHAT.md"), Path("AGENTS_CHAT.md")), + "", + "---", + "", + surface, + "", + "---", + "", + "# Skills — fetch the one that matches the task", + "", + "Each skill is a complete recipe: read it end-to-end before writing code, and mirror", + "its calls rather than recalling the API.", + "", + skill_routing_table(root, names), + "", + "---", + "", + "# Reference pages", + "", + "| Page | Fetch |", + "|---|---|", + f"| Wiki index (start here) | `{RAW_BASE}/wiki/core/index.md` |", + f"| Light-profile catalogue | `{RAW_BASE}/wiki/core/api/light_profile_catalog.md` |", + f"| Mass-profile catalogue | `{RAW_BASE}/wiki/core/api/mass_profile_catalog.md` |", + f"| Non-linear searches | `{RAW_BASE}/wiki/core/api/searches.md` |", + f"| Plotting API | `{RAW_BASE}/wiki/core/api/plotting.md` |", + f"| Analysis objects | `{RAW_BASE}/wiki/core/api/analysis_objects.md` |", + f"| Datasets | `{RAW_BASE}/wiki/core/api/datasets.md` |", + f"| Aggregator (loading results) | `{RAW_BASE}/wiki/core/api/aggregator.md` |", + f"| Configuration | `{RAW_BASE}/wiki/core/api/configuration.md` |", + f"| Generated-script style | `{RAW_BASE}/skills/_style.md` |", + "", + "**Do not fetch** `wiki/literature/` or `llms-full.txt` wholesale — they are very", + "large and will crowd out the conversation. Fetch one page at a time.", + "", + "Runnable end-to-end examples live in the workspace:", + "`https://raw.githubusercontent.com/PyAutoLabs/autolens_workspace/main/llms.txt`", + "", + ] + return "\n".join(parts) + + +def build_pack(root: Path, surface: str, names: list[str]) -> dict[str, str]: + """The upload tier: merged topic files for Project / GPT knowledge.""" + version = pinned_stack(root) + files: dict[str, str] = {} + + files["00_instructions.md"] = "\n".join( + [ + header("PyAutoLens Assistant — chat instructions", version), + "---", + "", + rewrite_links(read(root, "AGENTS_CHAT.md"), Path("AGENTS_CHAT.md")), + ] + ) + + files[API_SURFACE_NAME] = surface + + files["02_skill_index.md"] = "\n".join( + [ + header("PyAutoLens Assistant — skill index", version), + "", + "The skills shipped in this pack, and what each one is for.", + "", + skill_routing_table(root, names), + "", + "Skills not shipped here (they need a shell, a checkout, or write access, or are", + "still stubs) are listed in the full index:", + f"`{RAW_BASE}/skills/README.md`", + "", + ] + ) + + files["03_script_style.md"] = "\n".join( + [ + header("PyAutoLens Assistant — generated script style", version), + "---", + "", + rewrite_links(read(root, "skills/_style.md"), Path("skills/_style.md")), + ] + ) + + files["04_wiki_index.md"] = "\n".join( + [ + header("PyAutoLens reference — wiki index", version), + "---", + "", + rewrite_links(read(root, "wiki/core/index.md"), Path("wiki/core/index.md")), + ] + ) + + api_pages = sorted((root / "wiki/core/api").glob("*.md")) + api_parts = [header("PyAutoLens reference — API catalogues", version)] + for page in api_pages: + rel = page.relative_to(root) + api_parts += ["", "---", "", rewrite_links(page.read_text(encoding="utf-8"), rel)] + files["05_wiki_api_reference.md"] = "\n".join(api_parts) + + stubs = stub_skills(root) + n = 6 + for key, title, group_names in SKILL_GROUPS: + shipped = [ + x for x in group_names if x not in stubs and x not in ROLE_EXCLUDED_SKILLS + ] + if not shipped: + continue + parts = [header(f"PyAutoLens skills — {title}", version)] + for name in shipped: + rel = Path("skills") / f"{name}.md" + parts += ["", "---", "", rewrite_links(read(root, rel), rel)] + files[f"{n:02d}_skills_{key}.md"] = "\n".join(parts) + n += 1 + + return files + + +# --------------------------------------------------------------------------- +# Entry point +# --------------------------------------------------------------------------- +def check_anchors(root: Path) -> list[str]: + canonical = normalise(read(root, "AGENTS.md")) + chat = normalise(read(root, "AGENTS_CHAT.md")) + problems = [] + for label, anchor in VERBATIM_ANCHORS: + want = normalise(anchor) + in_canon = want in canonical + in_chat = want in chat + if not in_canon: + problems.append( + f"anchor {label!r} no longer appears in AGENTS.md — the rule was reworded; " + f"update VERBATIM_ANCHORS and AGENTS_CHAT.md together" + ) + if not in_chat: + problems.append( + f"anchor {label!r} missing from AGENTS_CHAT.md — the chat copy has drifted " + f"from AGENTS.md" + ) + return problems + + +def check_generated_links(root: Path, artifacts: dict[str, str]) -> list[str]: + """Verify every rewritten raw URL points at a path that exists in the repo. + + A rewritten link that resolves nowhere is worse than a relative one: it looks + authoritative and a connector will fetch a 404. Cheap to check here because + every URL we emit is repo-relative by construction. + """ + pattern = re.compile(re.escape(RAW_BASE) + r"/([^)`\s]+)") + problems: list[str] = [] + for label, text in artifacts.items(): + for m in pattern.finditer(text): + rel = m.group(1).split("#")[0] + if not (root / rel).exists(): + problems.append(f"{label} links to {rel}, which does not exist") + return sorted(set(problems)) + + +def main() -> int: + parser = argparse.ArgumentParser(description=__doc__.split("\n")[0]) + parser.add_argument( + "--check", + action="store_true", + help="Verify the committed artifacts match a fresh build and the verbatim anchors " + "still hold; exit non-zero on drift. Writes nothing.", + ) + parser.add_argument("--root", default=None) + args = parser.parse_args() + + root = Path(args.root) if args.root else Path(__file__).resolve().parent.parent + if not (root / "AGENTS.md").exists(): + sys.exit(f"chat_bundle: {root} does not look like the assistant repo root.") + + problems = check_anchors(root) + for p in problems: + print(f"chat_bundle: ANCHOR DRIFT - {p}", file=sys.stderr) + + orphans = audit_selection(root) + if orphans: + print( + "chat_bundle: WARNING - complete skills in no SKILL_GROUPS group (they will " + "not ship): " + ", ".join(orphans), + file=sys.stderr, + ) + + surface, regenerated = api_surface(root) + names = selected_skills(root) + paste = build_paste(root, surface, names) + pack = build_pack(root, surface, names) + + if len(pack) > GPT_KNOWLEDGE_FILE_LIMIT: + sys.exit( + f"chat_bundle: {len(pack)} pack files exceeds the {GPT_KNOWLEDGE_FILE_LIMIT}-file " + "GPT knowledge limit; merge some SKILL_GROUPS." + ) + + artifacts = {str(PASTE_REL): paste, **{str(PACK_REL / k): v for k, v in pack.items()}} + dead_links = check_generated_links(root, artifacts) + for d in dead_links: + print(f"chat_bundle: DEAD LINK - {d}", file=sys.stderr) + + paste_tokens = est_tokens(paste) + over_budget = paste_tokens > PASTE_TOKEN_BUDGET + + if args.check: + failures = list(problems) + dead_links + committed_paste = root / PASTE_REL + if not committed_paste.exists(): + failures.append(f"{PASTE_REL} is missing") + elif committed_paste.read_text(encoding="utf-8") != paste: + # A date-only diff is still drift, but say so precisely. + failures.append(f"{PASTE_REL} is out of date - re-run chat_bundle.py") + for name, text in pack.items(): + path = root / PACK_REL / name + if not path.exists(): + failures.append(f"{PACK_REL / name} is missing") + elif path.read_text(encoding="utf-8") != text: + failures.append(f"{PACK_REL / name} is out of date - re-run chat_bundle.py") + for stale in sorted((root / PACK_REL).glob("*.md")) if (root / PACK_REL).exists() else []: + if stale.name not in pack: + failures.append(f"{PACK_REL / stale.name} is no longer generated - delete it") + if over_budget: + failures.append( + f"{PASTE_REL} is ~{paste_tokens} tokens, over the {PASTE_TOKEN_BUDGET} budget" + ) + if failures: + for f in failures: + print(f"chat_bundle: FAIL - {f}", file=sys.stderr) + return 1 + print(f"chat_bundle: OK - artifacts current (~{paste_tokens} tokens pasteable)") + return 0 + + if problems or dead_links: + return 1 + + (root / PASTE_REL).write_text(paste, encoding="utf-8") + pack_dir = root / PACK_REL + pack_dir.mkdir(exist_ok=True) + for name, text in pack.items(): + (pack_dir / name).write_text(text, encoding="utf-8") + for stale in sorted(pack_dir.glob("*.md")): + if stale.name not in pack: + stale.unlink() + print(f"chat_bundle: removed stale {PACK_REL / stale.name}") + + print(f"wrote {PASTE_REL} (~{paste_tokens} tokens, {len(paste):,} chars)") + total = sum(est_tokens(t) for t in pack.values()) + print(f"wrote {PACK_REL}/ ({len(pack)} files, ~{total:,} tokens total)") + for name, text in sorted(pack.items()): + print(f" {name:34s} ~{est_tokens(text):>6,} tokens") + print(f"skills shipped: {len(names)}") + if not regenerated: + print("API surface: REUSED committed copy (stack not importable)") + if over_budget: + print( + f"WARNING: paste tier ~{paste_tokens} tokens exceeds the {PASTE_TOKEN_BUDGET} " + "budget — trim before release.", + file=sys.stderr, + ) + return 1 + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/autoassistant/tests/test_chat_bundle.py b/autoassistant/tests/test_chat_bundle.py new file mode 100644 index 0000000..8477ea2 --- /dev/null +++ b/autoassistant/tests/test_chat_bundle.py @@ -0,0 +1,138 @@ +"""Unit tests for the free-tier chat bundle generator. + +Stdlib-only and fast: the API-surface page falls back to the committed copy when the +PyAuto* stack is absent, so every test here runs on a docs-only checkout. + +The point of these tests is the *guards*, not the prose. The bundles are generated +content that a user pastes into a chat with no way to tell whether it is current, so +the failure modes that matter are silent ones: a rule that drifted out of the chat +copy, a rewritten link that 404s, a committed artifact nobody regenerated. +""" + +from __future__ import annotations + +from pathlib import Path + +from autoassistant import chat_bundle as cb + +REPO_ROOT = Path(__file__).resolve().parent.parent.parent + + +# --------------------------------------------------------------------------- +# normalise +# --------------------------------------------------------------------------- +def test_normalise_strips_blockquotes_and_wrapping(): + """A rule quoted with `>` and rewrapped must compare equal to the original.""" + canonical = "the gate is not waived: ask the user\nto plot and inspect the data." + quoted = "> the gate is not waived: ask the user to plot\n> and inspect the data." + assert cb.normalise(canonical) == cb.normalise(quoted) + + +def test_normalise_does_not_merge_distinct_wording(): + assert cb.normalise("gate is not waived") != cb.normalise("gate is optional") + + +# --------------------------------------------------------------------------- +# rewrite_links +# --------------------------------------------------------------------------- +def test_rewrite_links_resolves_relative_to_source_dir(): + """`../wiki/x.md` inside skills/ must resolve to wiki/x.md, not skills/../wiki.""" + out = cb.rewrite_links("see [x](../wiki/core/index.md)", Path("skills/al_x.md")) + assert f"{cb.RAW_BASE}/wiki/core/index.md" in out + assert ".." not in out + + +def test_rewrite_links_handles_dot_slash_and_anchors(): + out = cb.rewrite_links("[a](./AGENTS.md#modes)", Path("llms.txt")) + assert f"{cb.RAW_BASE}/AGENTS.md#modes" in out + + +def test_rewrite_links_leaves_absolute_and_pure_anchors_alone(): + for text in ( + "[a](https://example.com/x.md)", + "[a](#section)", + "[a](mailto:x@y.z)", + ): + assert cb.rewrite_links(text, Path("README.md")) == text + + +def test_rewrite_links_rewrites_images_too(): + out = cb.rewrite_links("![f](docs/images/x.png)", Path("README.md")) + assert f"![f]({cb.RAW_BASE}/docs/images/x.png)" in out + + +# --------------------------------------------------------------------------- +# Repository invariants +# --------------------------------------------------------------------------- +def test_verbatim_anchors_hold_in_both_instruction_files(): + """AGENTS_CHAT.md must not drift from the AGENTS.md rules it reproduces.""" + assert cb.check_anchors(REPO_ROOT) == [] + + +def test_every_shipped_skill_exists_and_is_not_a_stub(): + stubs = cb.stub_skills(REPO_ROOT) + names = cb.selected_skills(REPO_ROOT) + assert names, "no skills selected — SKILL_GROUPS or the index parser is broken" + for name in names: + assert (REPO_ROOT / "skills" / f"{name}.md").exists() + assert name not in stubs + + +def test_stub_detection_finds_known_stubs(): + """Parsed from skills/README.md, so a filled-in stub ships automatically.""" + stubs = cb.stub_skills(REPO_ROOT) + assert "al_point_source" in stubs + assert "al_run_search" not in stubs + + +def test_committed_artifacts_have_no_dead_links(): + """Every rewritten raw URL must resolve to a real path in the repo.""" + artifacts = {str(cb.PASTE_REL): (REPO_ROOT / cb.PASTE_REL).read_text(encoding="utf-8")} + for path in sorted((REPO_ROOT / cb.PACK_REL).glob("*.md")): + artifacts[str(path.relative_to(REPO_ROOT))] = path.read_text(encoding="utf-8") + assert cb.check_generated_links(REPO_ROOT, artifacts) == [] + + +def test_paste_tier_fits_a_small_context_window(): + """The paste route exists for harnesses with tiny context; keep it small.""" + text = (REPO_ROOT / cb.PASTE_REL).read_text(encoding="utf-8") + assert cb.est_tokens(text) <= cb.PASTE_TOKEN_BUDGET + + +def test_pack_respects_the_gpt_knowledge_file_limit(): + n = len(list((REPO_ROOT / cb.PACK_REL).glob("*.md"))) + assert 0 < n <= cb.GPT_KNOWLEDGE_FILE_LIMIT + + +def test_paste_tier_carries_the_removed_plotter_warning(): + """The single highest-value rule in the bundle — assert it actually shipped.""" + text = (REPO_ROOT / cb.PASTE_REL).read_text(encoding="utf-8") + assert "FitImagingPlotter" in text + assert "subplot_fit_imaging" in text + + +def test_generated_artifacts_are_date_independent(): + """A build must not embed 'today', or `--check` fails every following day. + + Regression: the first version stamped `date.today()` into every header, so CI + went red one day after the bundles were committed even though nothing had + changed. Provenance is the pinned stack version, which moves only when the + content does. + """ + import datetime as real_datetime + + texts = [(REPO_ROOT / cb.PASTE_REL).read_text(encoding="utf-8")] + texts += [ + p.read_text(encoding="utf-8") for p in (REPO_ROOT / cb.PACK_REL).glob("*.md") + ] + today = real_datetime.date.today().isoformat() + for text in texts: + assert today not in text, "a generation date leaked into a committed artifact" + + assert cb.pinned_stack(REPO_ROOT) != "unknown" + + +def test_no_role_excluded_skill_leaks_into_the_pack(): + """Shell-dependent workflows must not be shipped to a no-execution harness.""" + names = set(cb.selected_skills(REPO_ROOT)) + assert names.isdisjoint(cb.ROLE_EXCLUDED_SKILLS) diff --git a/chat_pack/00_instructions.md b/chat_pack/00_instructions.md new file mode 100644 index 0000000..c4db579 --- /dev/null +++ b/chat_pack/00_instructions.md @@ -0,0 +1,161 @@ +# PyAutoLens Assistant — chat instructions + +Generated by `python autoassistant/chat_bundle.py` against PyAutoLens `2026.7.29.2`. +Source of truth: https://github.com/PyAutoLabs/autolens_assistant — **do not hand-edit this file.** + +--- + +# AGENTS_CHAT.md — chat-mode instructions for autolens_assistant + +You are the **PyAutoLens Assistant** running in a **browser chat** (claude.ai, ChatGPT, or +similar). You can read, reason, plan and write code — you **cannot** run code, open the +user's files, or edit this repository. + +This file is the chat-mode counterpart of [`AGENTS.md`](https://raw.githubusercontent.com/PyAutoLabs/autolens_assistant/main/AGENTS.md). `AGENTS.md` is canonical +for coding agents that can execute; this file keeps the rules that still apply when nothing +can be run, and drops the ones that assume a shell, a checkout, or write access. Where a rule +below is quoted from `AGENTS.md`, it is reproduced **verbatim** and drift-checked by +`autoassistant/chat_bundle.py --check`. + +Everything here applies whether the repository reached you through a GitHub connector, an +uploaded knowledge pack, or a pasted bundle. + +--- + +## Lead by engaging + +**Never open a reply with what you can't do.** Ask what the user is trying to achieve, ask +them to describe or plot their data, plan the model with them, and draft the script — do that +*first*. You can route to examples, explain lensing, design an analysis, and review pasted +scripts, errors and figures. That is most of the job. + +Raise the handoff to a local coding agent (Claude Code, Codex) when **execution** becomes the +actual blocker — running the fit, inspecting `.fits` files, iterating on results — not as an +opening disclaimer. + +--- + +## Safety invariants that survive with no execution + +### Real data → inspect before fitting + +Before composing a model-fit on real observational data, the data must be looked at: **(a)** +extra galaxies / foreground stars / artefacts (the #1 source of fit bias), and **(b)** the +mask extent — the radius/shape that captures the lensed emission without dragging in noise or +contaminants. Never leave the mask radius as a silent default on real data. + +> **If you can't plot it yourself — no code execution, e.g. a GitHub-connector chat +> — the gate is not waived: ask the user to plot and inspect the data, and to confirm both (a) +> contaminants and (b) the mask extent, before you compose the fit.** These are the questions +> every real-data run must ask, on every harness. + +Give the user a short plotting snippet and ask them to report back what they see. Simulated +data is exempt. + +### Never reconstruct the API from memory + +Older PyAutoLens releases used a different API and are **heavily represented in model training +data**. On a coding-agent harness a code gate blocks stale symbols; **in chat there is no gate, +so this discipline is the only safeguard**: + +> if you +> can't point at a `skills/` (or `dir()`) example for a call, treat it as unverified and say so +> rather than emitting it. + +Check any symbol you are unsure about against `api_surface.md` (the generated public-symbol +list shipped with this bundle). A name there exists; for *call syntax*, mirror the matching +`skills/` example. + +### Plotting is functional — the #1 stale-API error + +Pass `output_path=...`, `output_filename=...`, `output_format="png"` straight to the `aplt.*` +call. + +> **The object-oriented plotters (`aplt.FitImagingPlotter`, `ImagingPlotter`, `TracerPlotter`, +> …) and the `aplt.MatPlot2D` / `aplt.Output` objects have been removed — do not use them. + +Wrong: + +```python +aplt.FitImagingPlotter(fit=fit, mat_plot_2d=aplt.MatPlot2D(...)).subplot_fit_imaging() +``` + +Right: + +```python +aplt.subplot_fit_imaging( + fit=fit, output_path="scripts/scratch/ring/", output_filename="fit", output_format="png" +) +``` + +### Standard imports + +```python +import autofit as af +import autolens as al +import autolens.plot as aplt +``` + +--- + +## How to use this material + +1. **Instructions** — this file. +2. **Skills** (`skills/*.md`) — *procedural*: how to do one task. Lensing skills are + `al_.md`. Read the relevant one end-to-end before writing code; its examples are the + source of truth for API calls. +3. **Wiki** (`wiki/core/`) — *content*: what a Sersic profile is, which searches exist, how + SLaM phases work. + +> **Rule of thumb.** *How do I do X?* → a skill. *What / which / why X?* → the wiki. *Build +> something end-to-end?* → compose skills, citing wiki pages as you go. + +If you have a GitHub connector, fetch pages on demand from the raw URLs listed in +[`llms-chat.txt`](https://raw.githubusercontent.com/PyAutoLabs/autolens_assistant/main/llms-chat.txt). If you are working from an uploaded or pasted bundle, use +only what is in front of you — and say so plainly when the answer would need a page you don't +have, rather than inventing it. + +**Do not bulk-fetch.** `wiki/literature/` and `llms-full.txt` are very large; pulling them in +will crowd out the conversation. Fetch the one page you need. + +--- + +## Modes + +- **Teacher** — *learn*: explain the physics and inference, step through, link to examples. +- **Assistant** — *do*: plan, then draft. Narrate what you're doing and why; give a one-line + plan read-back before diving in. Ask a blocking question only when correctness genuinely + depends on the answer. + +Infer the mode from the opening request (default **assistant**), state it in one line, and +invite correction. Say "Teacher mode" or "Assistant mode" in a prompt to set it explicitly. + +--- + +## Generated script style + +Every script you write uses the PyAutoLens **workspace** style, not banner comments: an +opening docstring (title underlined with `=`, short orientation, `__Contents__`), then each +section introduced by a `"""__Section__"""` docstring carrying the physics/inference framing. +The full spec and a copyable worked example are in `skills/_style.md` — mirror it rather than +reconstructing the format from memory. + +Scripts are written for the user to run locally: committed scripts → `scripts/`, throwaway +plots → `scripts/scratch/`, `search.fit(...)` output → `./output/`. + +--- + +## What is out of scope in chat + +These are `AGENTS.md` rules that need a shell or a checkout. **Do not claim to perform them, +and do not ask the user to run them as a precondition:** + +- the session-start environment/API drift-check (`audit_skill_apis.py --check-version`) +- the executable code gate (`audit_skill_apis.py --code ...`) +- reading or writing `wiki/project/profile.md`, `wiki/project/` journal entries, or `.maintainer` +- the commit cadence, and any git operation +- cloning source repos into `sources/`, or editing PyAuto\* source +- science-project scaffolding (`start-new-project`), which creates and manages a repo + +When one of these is the natural next step, name it as **the point to switch to a local coding +agent** and describe what the user would do there. diff --git a/chat_pack/01_api_surface.md b/chat_pack/01_api_surface.md new file mode 100644 index 0000000..82d45fb --- /dev/null +++ b/chat_pack/01_api_surface.md @@ -0,0 +1,49 @@ +# PyAuto* public API surface (generated) + +Generated by `python autoassistant/audit_skill_apis.py --dump-symbols`. +**Do not hand-edit.** This is the public `dir()` surface of the exact stack +the assistant content was validated against — use it to check whether a symbol +you are about to write actually exists. + +- Stack versions: `autoarray` 2026.7.29.2, `autofit` 2026.7.29.2, `autogalaxy` 2026.7.29.2, `autolens` 2026.7.29.2, `autonerves` 2026.7.29.2 + +> A name appearing here means the symbol exists; it does **not** document its +> signature. For call syntax, mirror the matching `skills/` example. + +## `autonerves` + +33 public symbols. + +`Config`, `JSONPriorConfig`, `Path`, `WorkspaceVersionMismatchError`, `cached_property`, `check_version`, `class_path`, `conf`, `csvable`, `default_prior`, `dictable`, `directory_config`, `exc`, `fitsable`, `for_autolens`, `instance`, `is_test_mode`, `jax_wrapper`, `json_prior`, `make_config_for_class`, `output`, `path_for_class`, `setup_colab`, `setup_notebook`, `skip_checks`, `skip_fit_output`, `skip_visualization`, `sys`, `test_mode`, `test_mode_level`, `tools`, `warnings`, `workspace` + +## `autoarray` (alias `aa`) + +121 public symbols. + +`AbstractDataset`, `AbstractFit`, `AbstractImageMesh`, `AbstractInversion`, `AbstractLinearObjFuncList`, `AbstractMesh`, `AbstractPreloads`, `AbstractRegularization`, `AbstractTriangles`, `Array1D`, `Array2D`, `Array2DRGB`, `ArrayIrregular`, `BorderRelocator`, `Circle`, `Convolver`, `DatasetInterface`, `DatasetModel`, `DeriveGrid1D`, `DeriveGrid2D`, `DeriveIndexes2D`, `DeriveMask1D`, `DeriveMask2D`, `FitDataset`, `FitImaging`, `FitInterferometer`, `Geometry2D`, `Grid1D`, `Grid2D`, `Grid2DContour`, `Grid2DIrregular`, `GridsInterface`, `Header`, `Imaging`, `ImagingSparseOperator`, `Interferometer`, `InterferometerSparseOperator`, `InterpolatorDelaunay`, `InterpolatorRectangular`, `Inversion`, `InversionImagingMapping`, `InversionImagingSparse`, `InversionInterferometerMapping`, `InversionInterferometerSparse`, `Layout1D`, `Layout2D`, `LinearObj`, `Mapper`, `Mask1D`, `Mask2D`, `MeshGeometryDelaunay`, `MeshGeometryRectangular`, `OverSampler`, `Pixelization`, `Polygon`, `PreloadsImaging`, `PreloadsInterferometer`, `Region1D`, `Region2D`, `Settings`, `SimulatorImaging`, `SimulatorInterferometer`, `Square`, `TransformerDFT`, `TransformerNUFFT`, `TransformerNUFFTPyNUFFT`, `Triangle`, `VectorYX2D`, `VectorYX2DIrregular`, `Visibilities`, `VisibilitiesNoiseMap`, `Zoom2D`, `abstract_ndarray`, `conf`, `dataset`, `decorators`, `exc`, `fit`, `fitsable`, `fixtures`, `from_dict`, `from_json`, `geometry`, `grid_dec`, `hdu_list_for_output_from`, `header_obj_from`, `image_mesh`, `interp_2d`, `inversion`, `is_test_mode`, `jax_wrapper`, `layout`, `m`, `mask`, `mesh`, `mock`, `ndarray_via_fits_from`, `ndarray_via_hdu_from`, `numba_util`, `operators`, `output_to_fits`, `output_to_json`, `over_sample`, `plot`, `preloads`, `preprocess`, `reg`, `register_parser`, `settings`, `setup_colab`, `setup_notebook`, `skip_checks`, `skip_fit_output`, `skip_visualization`, `structures`, `test_mode_level`, `to_dict`, `type`, `util`, `with_config`, `with_test_mode_segment` + +## `autofit` (alias `af`) + +149 public symbols. + +`Abs`, `AbsoluteWidthModifier`, `AbstractModel`, `AbstractPaths`, `AbstractPriorModel`, `Add`, `AggBase`, `AggregateCSV`, `AggregateFITS`, `AggregateImages`, `Aggregator`, `Analysis`, `AnalysisFactor`, `AnnotationPriorModel`, `Array`, `AutoCorrelationsSettings`, `BFGS`, `BlackJAXNUTS`, `Collection`, `ComparisonAssertion`, `Constant`, `CovarianceInterpolator`, `DatabasePaths`, `DeferredArgument`, `DeferredInstance`, `DiagonalMatrix`, `DirectoryPaths`, `Divide`, `Drawer`, `DynestyDynamic`, `DynestyStatic`, `EPAnalysisFactor`, `EPHistory`, `Emcee`, `FactorGraphModel`, `Fit`, `GaussianPrior`, `GreaterThanLessThanAssertion`, `GreaterThanLessThanEqualAssertion`, `GridList`, `GridSearchAggregator`, `GridSearchResult`, `HierarchicalFactor`, `InitializerBall`, `InitializerParamBounds`, `InitializerParamStartPoints`, `InitializerPrior`, `Instance`, `LBFGS`, `LaplaceOptimiser`, `Latent`, `LinearInterpolator`, `LinearRelationship`, `Log`, `Log10`, `LogGaussianPrior`, `LogUniformPrior`, `Mapper`, `Mod`, `Model`, `ModelInstance`, `ModelMapper`, `ModelObject`, `MultiStartADABelief`, `MultiStartAdam`, `MultiStartGradientConvergence`, `MultiStartLion`, `MultiStartProdigy`, `Multiply`, `Nautilus`, `NonLinearSearch`, `Power`, `Prior`, `PriorVectorized`, `Query`, `RelativeWidthModifier`, `Result`, `ResultsCollection`, `Sample`, `Samples`, `SamplesMCMC`, `SamplesNest`, `SamplesPDF`, `SamplesStored`, `SamplesSummary`, `SearchGridSearch`, `SearchOutput`, `Sensitivity`, `SettingsSearch`, `SplineInterpolator`, `TruncatedGaussianPrior`, `TuplePrior`, `UniformPrior`, `ValueType`, `VisualiseGraph`, `Visualizer`, `WidthModifier`, `Zeus`, `abc`, `aggregator`, `check_version`, `conf`, `database`, `db`, `ex`, `example`, `exc`, `fitsable`, `formatter`, `from_dict`, `from_json`, `graphical`, `hdu_list_for_output_from`, `header_obj_from`, `interpolator`, `is_test_mode`, `jax_wrapper`, `load_from_table`, `m`, `mapper`, `marginalize`, `messages`, `mock`, `ndarray_via_fits_from`, `ndarray_via_hdu_from`, `non_linear`, `output_to_fits`, `output_to_json`, `path_instances_of_class`, `pickle`, `prior`, `register`, `register_parser`, `samples_text`, `save_abc`, `setup_colab`, `setup_notebook`, `skip_checks`, `skip_fit_output`, `skip_visualization`, `test_mode_level`, `text`, `to_dict`, `tools`, `type_`, `util`, `visualise`, `with_config`, `with_test_mode_segment` + +## `autogalaxy` (alias `ag`) + +137 public symbols. + +`AbstractImageMesh`, `AbstractMesh`, `AbstractRegularization`, `AbstractToInversion`, `AdaptImages`, `AnalysisEllipse`, `AnalysisImaging`, `AnalysisInterferometer`, `Array1D`, `Array2D`, `Array2DRGB`, `ArrayIrregular`, `BorderRelocator`, `Clicker`, `Convolver`, `DatasetInterp`, `DatasetModel`, `EllProfile`, `Ellipse`, `EllipseMultipole`, `EllipseMultipoleScaled`, `FitEllipse`, `FitEllipseSummed`, `FitImaging`, `FitInterferometer`, `Galaxies`, `GalaxiesToInversion`, `Galaxy`, `GalaxyModelRow`, `GalaxyModelTable`, `GalaxyTable`, `Grid1D`, `Grid2D`, `Grid2DIrregular`, `Header`, `Imaging`, `Interferometer`, `InterpolatorDelaunay`, `InterpolatorRectangular`, `Inversion`, `Latent`, `LatentGalaxy`, `Layout2D`, `LensCalc`, `LightProfile`, `LightProfileLinearObjFuncList`, `Mapper`, `Mask1D`, `Mask2D`, `OperateImage`, `OperateImageGalaxies`, `OperateImageList`, `OverSampler`, `Pixelization`, `Redshift`, `Region1D`, `Region2D`, `Scribbler`, `Settings`, `ShearYX2D`, `ShearYX2DIrregular`, `SimulatorImaging`, `SimulatorInterferometer`, `TransformerDFT`, `TransformerNUFFT`, `TransformerNUFFTPyNUFFT`, `VectorYX2D`, `VectorYX2DIrregular`, `Visibilities`, `VisibilitiesNoiseMap`, `Zoom2D`, `abstract_fit`, `agg`, `aggregator`, `analysis`, `check_version`, `conf`, `convert`, `cosmo`, `cosmology`, `ellipse`, `exc`, `fitsable`, `from_dict`, `from_json`, `galaxies_from_csv_tables`, `galaxy`, `galaxy_af_models_from_csv_tables`, `galaxy_models_from_csv`, `galaxy_models_to_csv`, `galaxy_name_image_dict_via_result_from`, `galaxy_table_from_csv`, `galaxy_table_to_csv`, `gui`, `hdu_list_for_output_from`, `header_obj_from`, `image_mesh`, `imaging`, `interferometer`, `is_test_mode`, `jax_wrapper`, `lmp`, `lmp_linear`, `lp`, `lp_basis`, `lp_linear`, `lp_linear_operated`, `lp_operated`, `lp_snr`, `m`, `mesh`, `mock`, `model_util`, `mp`, `ndarray_via_fits_from`, `ndarray_via_hdu_from`, `operate`, `output_to_fits`, `output_to_json`, `plot`, `preprocess`, `profiles`, `ps`, `rectangular_edge_pixel_list_from`, `reg`, `register_parser`, `setup_colab`, `setup_notebook`, `skip_checks`, `skip_fit_output`, `skip_visualization`, `sr`, `test_mode_level`, `to_dict`, `util`, `with_config`, `with_test_mode_segment` + +## `autolens` (alias `al`) + +154 public symbols. + +`AbstractFitPositionsImagePair`, `AbstractImageMesh`, `AbstractMesh`, `AbstractRegularization`, `AdaptImages`, `AnalysisImaging`, `AnalysisInterferometer`, `AnalysisPoint`, `AnalysisWeak`, `Array1D`, `Array2D`, `Array2DRGB`, `ArrayIrregular`, `BorderRelocator`, `Circle`, `Clicker`, `Convolver`, `DatasetInterface`, `DatasetModel`, `EllProfile`, `FitFluxes`, `FitFluxesSolved`, `FitImaging`, `FitInterferometer`, `FitPointDataset`, `FitPositionsImagePair`, `FitPositionsImagePairAll`, `FitPositionsImagePairAllSolved`, `FitPositionsImagePairRepeat`, `FitPositionsImagePairRepeatSolved`, `FitPositionsSource`, `FitPositionsSourceSolved`, `FitTimeDelays`, `FitTimeDelaysSolved`, `FitWeak`, `Galaxies`, `Galaxy`, `GalaxyModelRow`, `GalaxyModelTable`, `GalaxyTable`, `Grid1D`, `Grid2D`, `Grid2DIrregular`, `GridsInterface`, `Imaging`, `Interferometer`, `InterpolatorDelaunay`, `InterpolatorRectangular`, `Inversion`, `Latent`, `LatentLens`, `LensCalc`, `LightProfile`, `LightProfileLinearObjFuncList`, `Mapper`, `Mask1D`, `Mask2D`, `OperateImage`, `OverSampler`, `Pixelization`, `PointDataset`, `PointSolver`, `Polygon`, `PositionsLH`, `Redshift`, `Scribbler`, `Settings`, `ShapeSolver`, `SimulatorImaging`, `SimulatorInterferometer`, `SimulatorShearYX`, `SolvedCentre`, `SourceMaxSeparation`, `Square`, `SubhaloSensitivityResult`, `Tracer`, `TracerToInversion`, `TransformerDFT`, `TransformerNUFFT`, `TransformerNUFFTPyNUFFT`, `Triangle`, `VectorYX2D`, `VectorYX2DIrregular`, `Visibilities`, `VisibilitiesNoiseMap`, `VisualizerImaging`, `VisualizerInterferometer`, `WeakDataset`, `Zoom2D`, `agg`, `aggregator`, `analysis`, `check_version`, `conf`, `convert`, `cosmo`, `exc`, `fitsable`, `from_dict`, `from_json`, `galaxies_from_csv_tables`, `galaxy_af_models_from_csv_tables`, `galaxy_models_from_csv`, `galaxy_models_to_csv`, `galaxy_name_image_dict_via_result_from`, `galaxy_table_from_csv`, `galaxy_table_to_csv`, `hdu_list_for_output_from`, `header_obj_from`, `image_mesh`, `imaging`, `interferometer`, `is_test_mode`, `jax_wrapper`, `lens`, `list_from_csv`, `lmp`, `lmp_linear`, `lp`, `lp_basis`, `lp_linear`, `lp_linear_operated`, `lp_operated`, `lp_snr`, `m`, `mesh`, `mock`, `model_util`, `mp`, `ndarray_via_fits_from`, `ndarray_via_hdu_from`, `output_to_csv`, `output_to_fits`, `output_to_json`, `pc`, `point`, `potential_correction`, `preprocess`, `ps`, `rectangular_edge_pixel_list_from`, `reg`, `setup_colab`, `setup_notebook`, `skip_checks`, `skip_fit_output`, `skip_visualization`, `sr`, `subhalo`, `test_mode_level`, `to_dict`, `util`, `weak`, `with_config`, `with_test_mode_segment` + +## `autolens.plot` (alias `aplt`) + +61 public symbols. + +`corner_anesthetic`, `corner_cornerpy`, `fits_array`, `fits_imaging`, `fits_interferometer`, `log_likelihood_vs_iteration`, `output_figure`, `plot_array`, `plot_caustics`, `plot_chi_squared_map`, `plot_convergence_map`, `plot_critical_curves`, `plot_data_vs_model`, `plot_ellipticities`, `plot_grid`, `plot_image_group_zooms`, `plot_noise_map`, `plot_phis`, `plot_positions_overlay`, `plot_residuals`, `plot_shear_profile`, `plot_shear_yx_2d`, `subplot_adapt_images`, `subplot_basis_image`, `subplot_cluster_dataset`, `subplot_detection_fits`, `subplot_detection_imaging`, `subplot_ellipse_errors`, `subplot_fit_combined`, `subplot_fit_combined_log10`, `subplot_fit_dirty_images`, `subplot_fit_ellipse`, `subplot_fit_imaging`, `subplot_fit_imaging_log10`, `subplot_fit_imaging_log10_x1_plane`, `subplot_fit_imaging_of_galaxy`, `subplot_fit_imaging_of_planes`, `subplot_fit_imaging_tracer`, `subplot_fit_imaging_x1_plane`, `subplot_fit_interferometer`, `subplot_fit_interferometer_real_space`, `subplot_fit_interferometer_tracer`, `subplot_fit_point`, `subplot_fit_real_space`, `subplot_fit_weak`, `subplot_galaxies`, `subplot_galaxies_images`, `subplot_galaxy_images`, `subplot_galaxy_light_profiles`, `subplot_galaxy_mass_profiles`, `subplot_imaging_dataset`, `subplot_imaging_dataset_list`, `subplot_interferometer_dirty_images`, `subplot_lensed_images`, `subplot_parameters`, `subplot_point_dataset`, `subplot_sensitivity`, `subplot_sensitivity_figures_of_merit`, `subplot_sensitivity_tracer_images`, `subplot_tracer`, `subplot_weak_dataset` + +**Total: 655 public symbols across 6 modules.** diff --git a/chat_pack/02_skill_index.md b/chat_pack/02_skill_index.md new file mode 100644 index 0000000..a8ead2a --- /dev/null +++ b/chat_pack/02_skill_index.md @@ -0,0 +1,31 @@ +# PyAutoLens Assistant — skill index + +Generated by `python autoassistant/chat_bundle.py` against PyAutoLens `2026.7.29.2`. +Source of truth: https://github.com/PyAutoLabs/autolens_assistant — **do not hand-edit this file.** + + +The skills shipped in this pack, and what each one is for. + +| Skill | What it does | Fetch | +|---|---|---| +| `al_prepare_imaging_data` | load and preprocess FITS imaging, decide masking for real data, measure noise, prepare PSF | `https://raw.githubusercontent.com/PyAutoLabs/autolens_assistant/main/skills/al_prepare_imaging_data.md` | +| `al_simulate_dataset` | synthesise a lens dataset (imaging or interferometer) from a ground-truth model | `https://raw.githubusercontent.com/PyAutoLabs/autolens_assistant/main/skills/al_simulate_dataset.md` | +| `al_build_imaging_model` | compose a `Tracer` from light + mass profiles and wrap it in an `AnalysisImaging` | `https://raw.githubusercontent.com/PyAutoLabs/autolens_assistant/main/skills/al_build_imaging_model.md` | +| `al_build_interferometer_model` | same, but for visibility-plane data | `https://raw.githubusercontent.com/PyAutoLabs/autolens_assistant/main/skills/al_build_interferometer_model.md` | +| `al_custom_profile` | write a new light or mass profile subclass and register it for use in models | `https://raw.githubusercontent.com/PyAutoLabs/autolens_assistant/main/skills/al_custom_profile.md` | +| `al_configure_search` | pick and tune a non-linear search (Nautilus, Dynesty, Emcee, Zeus, …) for your problem | `https://raw.githubusercontent.com/PyAutoLabs/autolens_assistant/main/skills/al_configure_search.md` | +| `al_run_search` | execute `search.fit(model=..., analysis=...)` and monitor convergence | `https://raw.githubusercontent.com/PyAutoLabs/autolens_assistant/main/skills/al_run_search.md` | +| `al_chain_searches` | sequence searches so a later phase inherits priors from an earlier one | `https://raw.githubusercontent.com/PyAutoLabs/autolens_assistant/main/skills/al_chain_searches.md` | +| `al_run_slam_pipeline` | run a Source-Light-Mass pipeline (the canonical automated lensing workflow) | `https://raw.githubusercontent.com/PyAutoLabs/autolens_assistant/main/skills/al_run_slam_pipeline.md` | +| `al_debug_fit_failure` | diagnose a fit that didn't converge or produced unphysical results | `https://raw.githubusercontent.com/PyAutoLabs/autolens_assistant/main/skills/al_debug_fit_failure.md` | +| `al_load_results` | load a completed fit's `Tracer`, `Samples`, dataset and FITS products from its output folder | `https://raw.githubusercontent.com/PyAutoLabs/autolens_assistant/main/skills/al_load_results.md` | +| `al_plot_tracer` | plot ray tracing, critical curves, caustics, magnification maps | `https://raw.githubusercontent.com/PyAutoLabs/autolens_assistant/main/skills/al_plot_tracer.md` | +| `al_plot_fit_residuals` | plot model image, residuals, normalised residuals, chi-squared map | `https://raw.githubusercontent.com/PyAutoLabs/autolens_assistant/main/skills/al_plot_fit_residuals.md` | +| `al_inspect_source_reconstruction` | inspect a pixelised inversion: regularisation, source-plane image, reconstruction diagnostics | `https://raw.githubusercontent.com/PyAutoLabs/autolens_assistant/main/skills/al_inspect_source_reconstruction.md` | +| `al_inspect_results_mcp` | the read-only results-inspector MCP server: browse fits, summaries, result images and bulk subplot/FITS extraction from chat harnesses without code execution (Claude Desktop first) | `https://raw.githubusercontent.com/PyAutoLabs/autolens_assistant/main/skills/al_inspect_results_mcp.md` | +| `al_to_notebook` | convert a generated narrative-docstring script to a Jupyter notebook (docstrings → markdown cells, code → code cells) via the stdlib-only `autoassistant/to_notebook.py` | `https://raw.githubusercontent.com/PyAutoLabs/autolens_assistant/main/skills/al_to_notebook.md` | +| `al_potential_correction` | gravitational imaging: pixelized corrections to the lensing potential (`al.pc`), reconstructed jointly with the source, for substructure whose form you do not want to assume | `https://raw.githubusercontent.com/PyAutoLabs/autolens_assistant/main/skills/al_potential_correction.md` | + +Skills not shipped here (they need a shell, a checkout, or write access, or are +still stubs) are listed in the full index: +`https://raw.githubusercontent.com/PyAutoLabs/autolens_assistant/main/skills/README.md` diff --git a/chat_pack/03_script_style.md b/chat_pack/03_script_style.md new file mode 100644 index 0000000..d7815a1 --- /dev/null +++ b/chat_pack/03_script_style.md @@ -0,0 +1,418 @@ +# PyAutoLens Assistant — generated script style + +Generated by `python autoassistant/chat_bundle.py` against PyAutoLens `2026.7.29.2`. +Source of truth: https://github.com/PyAutoLabs/autolens_assistant — **do not hand-edit this file.** + +--- + +--- +name: _style +description: Writing guide for every workspace skill. Read first before adding or revising a skill. Defines tone (conversational, physics-first, encourages reading), structure (Orient → Ask → Branch → Combine), the four properties every skill must have, the python-first rule, and the source-citation form (project-name + repo-relative path). +--- + +# How to write a workspace skill + +This file is a meta-skill: it does not help a user run a lensing task directly. It is +the writing guide every other skill in this folder is authored against. Read it before +adding a new skill and re-read it before revising one. + +## What a skill is + +A skill is a single Markdown file at `skills/.md`. It guides an AI agent through +one lensing task — composing an imaging model, running a search, inspecting a fit, +simulating a dataset — and in doing so produces or evolves a **Python script** the user +can run. The deliverable is *understanding + a runnable script*, not a chat answer. + +The agent reads the skill when activated (either by the user typing `/al_` or by +the agent matching the skill's frontmatter `description` to the user's request). + +## The five properties every skill must have + +1. **Scientific and statistical context first.** Before showing API calls, set both the + physics and the inference. *Why* are we loading the result, fitting this model, or + running this profile? *What is being inferred*, with what likelihood, what priors, + what search? The API is in service of the science and the inference, not the other + way around. Both framings come before code — neither is optional. + +2. **Encourage reading the wiki.** Every skill should point at relevant `wiki/` + pages for the *what* (what is a Sersic, what is Nautilus, what is a pixelisation). + Skills are procedure; the wiki is content. If a piece of content doesn't exist yet, + draft the wiki page in the same change as the skill. + +3. **Conversational tone, invites questions.** Talk to the user the way a postdoc + collaborator would. Ask what they want before doing it. After explaining a concept, + invite a follow-up. Don't narrate procedures (`Step 1. Step 2.`) when prose works. + +4. **Skills compose.** Each skill should leave breadcrumbs to other skills that build on + it. Mention adjacent skills by name when chaining would unlock something a single + skill can't. + +5. **Records work to the project wiki.** When a skill produces or evolves a non-trivial + script, the agent must offer (default-yes) to add a dated + `wiki/project/YYYY-MM-DD-.md` entry covering (a) *domain motivation* — what + physics question this is in service of, (b) *statistical motivation* — what's being + inferred and how, (c) *implementation choice* — the script produced and the key + decisions. Cross-link every named concept and profile/model into `wiki/core/` and + `wiki/literature/` using `[[wiki-link]]` slugs (e.g. `[[Sersic1968]]`, + `[[NavarroFrenkWhite1996]]`, `[[mass-sheet-degeneracy]]`). Default to **no** only + for typo fixes, throwaway exploration, or repeated re-runs of an existing pipeline. + +## Python-first + +This is the rule that distinguishes the workspace from a tutorial workspace. + +- Every skill's main deliverable is a Python script written *for this user's data*. +- The skill body contains the API recipe inline, in fenced ```python``` blocks. +- The skill should leave the user with a `.py` file in `scripts/` they can re-run and + modify themselves. +- Do **not** just point the user at a pre-existing script in another repo. If you + reference an example (e.g. inside `autolens_workspace`), say so as a citation but + produce the user-specific script in the working directory regardless. + +## Generated script style + +Every Python script the agent saves — whether to `scripts/` or `scripts/scratch/` — follows the +PyAutoLens **workspace** style, not ad-hoc banner comments. It is the same style used by +every script in `autolens_workspace/scripts/` (canonical example: +`autolens_workspace:scripts/imaging/start_here.py`), and it exists for two reasons: it +keeps the science and inference narrative inline with the code, and it makes the script +mechanically convertible to a Jupyter notebook — each top-level `"""..."""` block becomes a +markdown cell and the code between blocks becomes a code cell. + +Two rules. + +The level of detail in a saved script is **mode-invariant**. Teacher and assistant modes — +at any autonomy level — may change the pacing and depth of the surrounding conversation, but they must not +change the completeness of the script artefact. Write docstrings as if the script may become +part of the open-source repository accompanying a paper: preserve the scientific motivation, +what is inferred and how, consequential assumptions and configuration choices, enough context +to reproduce or adapt the analysis, and resolvable source citations. Avoid tutorial padding and +repetition, but do not remove this information merely because the user is experienced or has +asked for concise interaction. + +**1. Title block + `__Contents__` header.** The module opens with a single docstring: a +title underlined with `=`, two or three sentences of orientation, then a `__Contents__` +list with one `- **Name:** one-line summary.` bullet per section that follows. + +```python +""" +Lens Model: HST Imaging +======================= + +Fit a galaxy-scale strong lens observed with HST imaging: load the data, compose an +SIE + external-shear mass model with a Sersic source, and fit it with Nautilus. + +__Contents__ + +- **Imports:** Import the required libraries. +- **Dataset:** Load imaging, apply the mask and over-sampling. +- **Model:** Compose the lens (light + mass) and source galaxies. +- **Search:** Configure the Nautilus non-linear search. +- **Fit:** Run the fit and inspect the result. +- **Plot:** Save the best-fit subplot. +""" +``` + +**2. Per-section narrative docstrings, not banner comments.** Each logical section is +introduced by a `"""__Section__"""` docstring whose name matches a `__Contents__` bullet. +The prose carries the physics and inference framing (property #1) and any source citations +(see below) — *not* `# ---` banner comments and *not* `# source:` lines. + +Do this: + +```python +""" +__Dataset__ + +We load three ingredients for lens modeling: the image (CCD counts), a per-pixel +noise-map, and the PSF. `pixel_scales` converts pixels to arcseconds — set it correctly +for your instrument (HST/ACS ~ 0.05"). Loading is handled by `al.Imaging.from_fits` +(`PyAutoArray:autoarray/dataset/imaging/dataset.py`). +""" +dataset = al.Imaging.from_fits( + data_path=DATASET_PATH / "data.fits", + noise_map_path=DATASET_PATH / "noise_map.fits", + psf_path=DATASET_PATH / "psf.fits", + pixel_scales=PIXEL_SCALES, +) +``` + +Not this: + +```python +# --------------------------------------------------------------------------- +# 1. Load imaging +# --------------------------------------------------------------------------- +# source: PyAutoArray:autoarray/dataset/imaging/dataset.py (Imaging.from_fits) +dataset = al.Imaging.from_fits(...) +``` + +Short clarifying `#` comments *inside* a code block are still fine (e.g. annotating a +single prior). What changes is that section structure and citations live in the docstring, +not in comment banners. + +## Source citations + +Code references inside a skill must use the **project name + path relative to that +project's repo root**, resolvable via [`../sources.yaml`](https://raw.githubusercontent.com/PyAutoLabs/autolens_assistant/main/sources.yaml). + +Good: + +> See `PyAutoFit:autofit/non_linear/search/nest/nautilus/` for the search's default +> settings, and `wiki/core/api/searches.md#nautilus` for when to pick it. + +Bad: + +> See `/Users/other/autolens/fit/autofit/non_linear/search/nest/nautilus.py`. + +The reason: this workspace is meant to be cloned to other machines. Absolute local paths +break the moment anyone else opens it. + +The same `:` form is used in **generated scripts**, but there it belongs +inside the section docstring prose (see "Generated script style" above) — woven into the +sentence that explains what the call does, never as a standalone `# source:` comment +banner. + +## Adaptive depth + +Adaptive depth governs the conversation and teaching around a script; it does not reduce the +publication-quality docstring detail required by "Generated script style" above. + +Users arrive with different backgrounds. The same skill needs to serve all of them: + +- **The lensing newcomer.** Knows Python, maybe some astronomy, but hasn't worked with + strong lensing before. Doesn't yet know what a `Tracer`, a deflection angle, or a + caustic is. Frame the physics each time a new concept appears; lean heavily on the + wiki. +- **The PyAuto\* newcomer.** Knows the science fluently — Bartelmann, Treu, the lens + equation, magnification — but new to the API. Map straight from science question to + object; skip the physics lecture. +- **The returning user.** Has used PyAutoLens before. Just wants to load a fit and + inspect the residuals. Quick API recall, no lecture. + +Pick depth from cues in the user's question. *"I'm new to lensing"* → newcomer. *"How +do I get the caustics?"* → already knows lensing. *"Load `output/.../abc/`"* → returning +user. If ambiguous, ask one disambiguating question; never default to the longest +explanation. + +Read `wiki/project/profile.md` if it exists — that's the persistent record of the user's +level and goal, built up over sessions. If it disagrees with what the user just said, +trust the user and update the profile. + +### Resource routing by audience + +The three external resources cover different audiences. Match the user's level to +the source, and pull the URL from +[`wiki/core/external/skill_citation_map.md`](https://raw.githubusercontent.com/PyAutoLabs/autolens_assistant/main/wiki/core/external/skill_citation_map.md): + +| Audience | Lead resource | Secondary | +|----------|---------------|-----------| +| Lensing newcomer | **HowToLens** notebook — *surfaced before the code block, not after* | RTD `overview_1_start_here` | +| PyAutoLens newcomer (lensing-fluent) | **RTD** `overview_2_new_user_guide` + `overview_3_features` | Workspace example script | +| Returning PyAutoLens user | **Workspace** script for the science case | RTD API reference | + +Never dump all three on the user unprejudiced — pick one to lead, optionally cite a +second. + +### Newcomer mode + +When the user signals they're new to lensing — *"I'm new to lensing"*, *"I've never +done this before"*, *"can you explain what a caustic is?"* — the agent shifts into a +more pedagogical shape. The conversation arc still applies; what changes is the +depth, ordering, and pacing. + +1. **Lead with the HowToLens notebook.** Before any code block, surface the tutorial + notebook URL from + [`wiki/core/external/skill_citation_map.md`](https://raw.githubusercontent.com/PyAutoLabs/autolens_assistant/main/wiki/core/external/skill_citation_map.md). + The notebook is the primary path; the skill-produced script is the follow-up + artefact, not the lead. +2. **One concept at a time.** Don't stack three concepts in one branch — pick the + one most central to the user's question, frame it, then offer to go deeper. + *"Let's get the deflection angle clear first; once that lands we can move on to + the lens equation"* beats firing all three simultaneously. +3. **Physics framing → statistical framing → code.** Property 1 already requires + both framings; for newcomers each framing gets at least one short paragraph and + at least one `wiki/core/concepts/` link before any code. +4. **Check-in beat after each concept.** End with an explicit invitation: *"does + that make sense, or want me to unpack X further?"*. Don't barrel into the next + branch. +5. **Encourage running HowToLens themselves.** A newcomer who runs the linked + notebook alongside the script learns far faster than one who only reads + citations. Mention this once per session: *"the notebook is short — if you run + it as you read along, it'll click faster."* + +Newcomer mode is a default for the lensing-newcomer audience, not a separate state. +As soon as the user shows they've absorbed a concept, drop the check-in beats and +move on. + +## The conversation arc — Orient → Ask → Branch → Combine + +Structure every skill as a conversation, not a checklist. + +**Orient.** When the skill activates, give a short opening: what this task is +scientifically, what the user is about to do, the most relevant wiki page, and one +concrete data example tailored to what they mentioned (HST, JWST, Euclid, ALMA, JVLA…). +Two short paragraphs at most. + +**Ask.** Before writing code, ask what the user wants out of the task. *"Want to fit the +mass model first or the source first?"* The answer chooses the branch and lets the skill +calibrate depth. Skip this step only when the user has already told you. + +**Branch.** Each sub-task lives in its own narrative branch. A branch has four parts: + +- Physics framing (one or two sentences, scaled to the user's depth). +- The Python recipe — actual code, in a fenced block, that the agent should adapt and + save to `scripts/`. When the recipe is a full saved script (not a one-off fragment), + write it in the **Generated script style** above: title + `__Contents__` header and + `"""__Section__"""` narrative sections rather than banner comments. +- The wiki page that teaches this in depth, plus the source-code citation + (`:`). +- An invitation to dig deeper. + +**Combine.** End the skill (or the chosen branch) with a short note on what else the +user could do, especially with other skills. *"Once you have the fit running, feed the +output into `al_load_results` and `al_plot_fit_residuals`."* + +A slim agent-facing procedural checklist at the very bottom of the file is fine — but +the user-facing content above should read like a conversation arc, not a recipe. + +## Voice rules + +**Do** + +- Speak in second person. The user is the protagonist. +- Invite questions explicitly (*"ask if you want me to explain how this works"*). +- Tie at least one concrete example to the user's data when their data type is known. +- Point at the wiki by relative path every time you teach a concept. +- For newcomers, surface the relevant HowToLens notebook before the code block, not + after. See "Newcomer mode" in Adaptive depth above. +- When a script produces plot files, quote the absolute path and offer to open it + with the platform's opener. See "Plot output and path announcement" below. + +**Don't** + +- Don't open with a numbered procedure. +- Don't dump a wall of links — one or two per concept, chosen for relevance. +- Don't present code as the deliverable on its own — the deliverable is understanding + + a saved script. +- Don't build a skill's *default* prose around a "just run this for me" tone — the standing + deliverable is understanding plus a runnable script, so frame the science and cite the wiki + rather than assuming black-box automation. This governs how the skill reads by default; it + does **not** override an explicit user opt-out. When the user asks to one-shot it (see + [`../modes/assistant.md`](https://raw.githubusercontent.com/PyAutoLabs/autolens_assistant/main/modes/assistant.md) "Opt-out — silent execution"), honour that + and run it — the two are not in conflict. + +## Frontmatter + +Every skill file starts with YAML frontmatter: + +```markdown +--- +name: +description: +--- +``` + +The `description` is what the agent uses to decide when the skill applies. Write it so +a future agent that has only read the description (not the body) can decide from it +alone. Mention the kind of task, the kind of input, and what the skill should NOT be +used for. + +## When a skill needs new wiki content + +If you cannot point at a wiki page that explains a concept your new skill uses, draft +the wiki page in the same change. The wiki page should follow the wiki frontmatter +format (see `wiki/README.md`) and cite source code by `:`. + +The reverse is also true: don't write a wiki page nobody references. The wiki exists to +back up the skills. + +## Plot output and path announcement + +Skills that produce visualisations save them through the function-style +`autolens.plot` API — every entry point takes `output_path` / `output_filename` +/ `output_format` kwargs directly (see +[`wiki/core/api/plotting.md`](https://raw.githubusercontent.com/PyAutoLabs/autolens_assistant/main/wiki/core/api/plotting.md)). Three rules: + +1. **Pass `output_path` / `output_filename` / `output_format` directly to + each plot function.** Every `autolens.plot` `plot_*` and `subplot_*` helper accepts + these kwargs, e.g. `aplt.subplot_imaging_dataset(dataset=…, + output_path="scripts/scratch//", output_filename=…, + output_format="png")`. Never rely on interactive display — the user is + often running the script from a terminal where `plt.show()` flashes and + vanishes. The `` slug is usually the dataset name; for general + exploration any short slug works. +2. **`print(...)` each plot's path** at the end of the Python recipe so the + absolute location lands in stdout. Use + `print(f"Saved to: {PLOT_DIR.resolve()}")` once per branch (sufficient + because each `aplt.*` call writes deterministically inside `PLOT_DIR`); for + single-figure calls it's fine to print the exact `.png` path instead. +3. **The agent quotes the path back** to the user after running the script + and offers to open it — one offer per plot run, not nagging. Use the + platform's opener: `open ` on macOS, `xdg-open ` on Linux, + `explorer.exe` (or `wslview`) from WSL. + +The full convention — committed Python lives in `scripts/`; throwaway plots and data +dumps go to the gitignored `scripts/scratch/` — is in `AGENTS.md` +"Conventions". Skills here are the application of that rule. + +## Output folder announcement + +A running fit is not a black box. `search.fit(...)` writes to +`output////` **on the fly**, using the highest-likelihood +model found so far, so the folder is worth opening the moment the search starts — not +when it terminates hours later. Users new to the stack rarely know this and sit watching +a silent terminal. Three rules: + +1. **Announce the folder at launch, not at the end.** Quote the absolute path once the + fit is running, and say `model.results` and the `image/` subplots refresh as the + search goes — there is nothing to wait for. +2. **Point at the workspace's own layout prose; don't restate it.** The annotated tree of + `files/`, `image/`, `model.info`, `model.results`, `search.summary` and the + `` resume behaviour is `__Output Folder Layout__` in + [`autolens_workspace/scripts/imaging/modeling.py`](https://github.com/PyAutoLabs/autolens_workspace/blob/main/scripts/imaging/modeling.py) + (the same section is in the `interferometer`, `point_source`, `group`, `cluster` and + `weak` `modeling.py` scripts). Link it once per fit; never copy the tree into a skill, + where it would rot. +3. **Name what to open first.** `model.results` for the human-readable fit summary and + `image/fit.png` for data / model image / residuals — then + [`al_load_results`](https://raw.githubusercontent.com/PyAutoLabs/autolens_assistant/main/skills/al_load_results.md) for the programmatic read. + +Depth follows "Adaptive depth" above. For either **newcomer** audience, and whenever +[`modes/teacher.md`](https://raw.githubusercontent.com/PyAutoLabs/autolens_assistant/main/modes/teacher.md) is active, walk all three rules — reading the +output folder *is* part of the workflow being taught. For a returning user, rule 1 alone +(one line quoting the path) is enough. + +## External resource citation + +Every `al_*` skill ends with a single `## Further reading` block above the agent +checklist (if present). The block is generated from one row of +[`wiki/core/external/skill_citation_map.md`](https://raw.githubusercontent.com/PyAutoLabs/autolens_assistant/main/wiki/core/external/skill_citation_map.md) +and follows this shape: + +```markdown +## Further reading + +- **Student / new to lensing** — [HowToLens: ](): one + line on what the tutorial teaches. +- **General reference** — [RTD: ](): canonical PyAutoLens + documentation page. +- **Experienced PyAutoLens user** — [workspace/lens: