Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/docs-prep-example-workflow-fix.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@cdot65/prisma-airs-cli": patch
---

Rewrite `docs/cli/examples/README.md` to document the real CLI-docs pipeline: YAML sidecars (`docs/cli/examples/<area>.yaml`) feed `pnpm docs:gen`, which writes the auto-generated pages under `docs/cli/**`. Editing those pages by hand fails `pnpm docs:check`.
230 changes: 138 additions & 92 deletions docs/cli/examples/README.md
Original file line number Diff line number Diff line change
@@ -1,118 +1,164 @@
# CLI Docs — Live Examples Template
# CLI Docs — Live Examples Workflow

This directory holds the fixture and example conventions used by every page
under `docs/cli/runtime/`, `docs/cli/redteam/`, and `docs/cli/model-security/`.
Every `airs` command must render at least one real input + real output on its
docs page. This directory drives that rendering.

Project rule: **every command must have a real input + real output**. No
fabricated payloads. When a live capture is blocked by an upstream API, CLI,
or SDK defect, document it with a `!!! bug` admonition and a tracking-issue
link instead of inventing output.
**Doc pages under `docs/cli/runtime/`, `docs/cli/redteam/`, and
`docs/cli/model-security/` are AUTO-GENERATED. Never edit them by hand —
your edit will be overwritten by `pnpm docs:gen` and will fail
`pnpm docs:check` in CI.**

## Capture environment
## How the pipeline works

All `airs` invocations during capture **must** run from
`~/development/cdot65/prisma-airs-cli`. The repo-local `.env` (or
`.env.cdot65` / `.env.dus` for alternate tenants) supplies
`PANW_AI_SEC_API_KEY`, `PANW_MGMT_*`, and `PANW_DLP_ENDPOINT`. Running from
anywhere else fails with missing-credential errors.
1. `scripts/gen-cli-docs.ts` walks the live `airs` program tree, joins each
command with examples from the YAML sidecars in this directory, and writes
`docs/cli/<area>/<page>.md`.
2. `scripts/check-cli-docs.ts` (`pnpm docs:check`, runs in CI) regenerates
into a tempdir, byte-compares against the committed `.md`, and fails on
drift. It also fails when a leaf command has no examples AND is not on
`.missing-allowlist`.
3. The allowlist is the burndown list — every entry should disappear as
examples land, except for entries explicitly blocked by an upstream / CLI /
SDK defect (see below).

## Directory layout
## Files in this directory

```
docs/cli/examples/
├── README.md # this file
├── REDACTION.md # redaction rules
├── dlp/
├── dictionaries/ # fixtures for write ops on dlp dictionaries
├── patterns/
├── profiles/
│ └── filtering-profiles/
├── runtime/ # fixtures for runtime/* write ops
├── redteam/ # fixtures for redteam/* write ops
└── model-security/ # fixtures for model-security/* write ops
├── REDACTION.md # redaction rules (apply before pasting output)
├── .missing-allowlist # commands awaiting examples (burn to zero)
├── runtime.yaml # sidecar for `airs runtime ...` commands
├── redteam.yaml # sidecar for `airs redteam ...` commands
├── model-security.yaml # sidecar for `airs model-security ...` commands
── dlp/{dictionaries,patterns,profiles,filtering-profiles}/
├── runtime/ # body-file fixtures for runtime/* write ops
├── redteam/ # body-file fixtures for redteam/* write ops
└── model-security/ # body-file fixtures for model-security/* write ops
```

Fixture files live under `docs/cli/examples/<area>/<command>.json` and follow
the redaction rules in [REDACTION.md](REDACTION.md).

## Example block format (the contract)

Replaces each `!!! warning "Example needed"` placeholder. For **list/get**
commands, show Pretty + JSON + YAML:
Sidecar files key by **full command path** (no `airs` prefix):

````markdown
#### Examples

**Input**
```bash
airs runtime dlp dictionaries list
```yaml
"runtime dlp dictionaries list":
examples:
- note: Pretty output (default)
input: airs runtime dlp dictionaries list
output: |
<live capture, redacted>
```

**Output (pretty / default)**
```text
<live capture, redacted>
```
The renderer emits each entry as italic note + bash codefence (input) + text
codefence (output). There is no `!!! bug` admonition path — see "Blocked
commands" below for how to handle defects.

## Per-command authoring workflow

1. `cd ~/development/cdot65/prisma-airs-cli` (required — `.env` auto-loads
`PANW_AI_SEC_API_KEY`, `PANW_MGMT_*`, `PANW_DLP_ENDPOINT`).
2. For any write op needing `--body-file`, drop a minimal valid fixture under
`docs/cli/examples/<area>/<command>.json`. Redact per
[REDACTION.md](REDACTION.md) before committing.
3. Live-capture each command:
- **list / get** → run three times: pretty (default), `--output json`,
`--output yaml`. One sidecar entry per format.
- **create / update / replace / patch / delete** → JSON only.
4. Redact every capture per [REDACTION.md](REDACTION.md).
5. Append entries to the right sidecar (`runtime.yaml` / `redteam.yaml` /
`model-security.yaml`).
6. Remove the command path from `.missing-allowlist`.
7. `pnpm docs:gen` — regenerates `docs/cli/**/*.md`.
8. `pnpm format` — Biome fixes JSON fixture formatting.
9. `pnpm docs:check` — must pass before commit.
10. Add `.changeset/<slug>.md` (`patch`) describing what got examples.
11. Open PR.

## Format contract (sidecar entries per command)

**list / get** — three entries (pretty / JSON / YAML):

**JSON**
```bash
airs runtime dlp dictionaries list --output json
```
```json
<live capture, redacted>
```yaml
"runtime dlp dictionaries list":
examples:
- note: Pretty output (default)
input: airs runtime dlp dictionaries list
output: |
ID NAME CATEGORY
00000000-0000-0000-0000-000000000001 example-dict custom
- note: JSON output
input: airs runtime dlp dictionaries list --output json
output: |
[
{
"id": "00000000-0000-0000-0000-000000000001",
"name": "example-dict",
"category": "custom"
}
]
- note: YAML output
input: airs runtime dlp dictionaries list --output yaml
output: |
- id: 00000000-0000-0000-0000-000000000001
name: example-dict
category: custom
```

**YAML**
```bash
airs runtime dlp dictionaries list --output yaml
```
**create / update / replace / patch / delete** — JSON only, plus a
reference to the fixture if one is used:

```yaml
<live capture, redacted>
"runtime dlp patterns create":
examples:
- note: Create from body fixture (see docs/cli/examples/dlp/patterns/create.json)
input: airs runtime dlp patterns create --body-file docs/cli/examples/dlp/patterns/create.json --output json
output: |
{
"id": "00000000-0000-0000-0000-000000000001",
"name": "example-pattern",
"state": "created"
}
```
````

For **write ops** (`create` / `update` / `replace` / `patch` / `delete`),
only the JSON capture appears (plus the body/fixture if applicable).
## Blocked commands (upstream / CLI / SDK defect)

## Bug admonition variant
The renderer has no `!!! bug` admonition path — every sidecar entry produces
input + output codefences. So when live capture is impossible:

When live capture is blocked by an upstream API, CLI, or SDK defect, replace
the example block with a `!!! bug` admonition that links the tracking issue.
Never fabricate output.
1. **File a tracking issue first.** Upstream API → label `upstream-bug` in
`cdot65/prisma-airs-cli`. CLI defect → label `bug` in
`cdot65/prisma-airs-cli`. SDK defect → label `bug` in
`cdot65/prisma-airs-sdk`. Search existing issues first
(#80 already covers DLP `patterns get` / `profiles get` 400).
2. **Leave the entry on `.missing-allowlist`** with a comment line above
linking the tracking issue:

````markdown
#### Examples
```
# Blocked by upstream — see https://github.com/cdot65/prisma-airs-cli/issues/80
runtime dlp patterns get
```

**Input**
```bash
airs runtime dlp patterns get <id>
```
3. The doc page will render the default `!!! warning "Example needed"`
placeholder. That is correct until the bug is fixed — better than
fabricating output.
4. Reference the tracking issue from your changeset so reviewers know why
the entry survives the burndown.

!!! bug "Upstream API returns 400"
The DLP `/v2/api/data-patterns/{id}` endpoint currently returns HTTP 400 for all
pattern UUIDs. Tracking: [cdot65/prisma-airs-cli#80](https://github.com/cdot65/prisma-airs-cli/issues/80).
Example will be backfilled once the upstream API is fixed.
## Capture environment

**Workaround**
```bash
airs runtime dlp patterns list --output json | jq '.[] | select(.id=="...")'
```
````

If no tracking issue exists yet, file one before adding the admonition:

- Upstream API defect → label `upstream-bug`
- CLI defect → label `bug` (this repo)
- SDK defect → label `bug` in `cdot65/prisma-airs-sdk`

## Per-page workflow

1. List every command on the page; confirm the placeholder is present.
2. Build fixtures under `docs/cli/examples/<area>/` for any write op needing
`--body-file`.
3. Live capture: run each command up to three times (pretty / JSON / YAML)
from `~/development/cdot65/prisma-airs-cli`; save outputs locally.
4. Redact per [REDACTION.md](REDACTION.md).
5. Detect bugs; file tracking issue + add `!!! bug` admonition if needed.
6. Swap the placeholder for the example block above.
7. Add a `.changeset/<slug>.md` (`patch`) with user-facing description.
8. Open the PR; post the same summary as a PR comment.
All `airs` invocations during capture **must** run from
`~/development/cdot65/prisma-airs-cli`. The repo-local `.env` (and
`.env.cdot65` / `.env.dus` for alternate tenants) supplies the management +
runtime credentials. Running from anywhere else fails with
"missing credentials".

## Checklist before opening the PR

- [ ] Sidecar entries added; YAML loads cleanly (try `pnpm docs:gen`).
- [ ] `.missing-allowlist` updated (entries removed for completed commands;
comment + entry kept for blocked commands with tracking issue link).
- [ ] Body fixtures redacted per REDACTION.md.
- [ ] `pnpm docs:gen` run; regenerated `.md` committed.
- [ ] `pnpm format` clean.
- [ ] `pnpm docs:check` clean.
- [ ] `.changeset/<slug>.md` present (`patch`).
- [ ] No edits to `.md` files outside the autogenerated set.
Loading