Skip to content

automation.md

Latest commit

 

History

History
168 lines (115 loc) · 5.31 KB

automation.md

File metadata and controls

168 lines (115 loc) · 5.31 KB
Raw
title Automation
nav_order 14
permalink /automation/

Automation

{: .no_toc }

Stable JSON contracts, agent recipes, and patterns for keeping the local mirror warm without manual ceremony. {: .fs-6 .fw-300 }

  1. TOC {:toc}

JSON output is first class

Every command supports --json (or the global --format json). The resulting payload is pretty-printed with stable field names so you can pipe it directly into jq or feed it to an agent as structured context.

gitcrawl sync owner/repo --json | jq '{run_id, inserted, updated}'
gitcrawl clusters owner/repo --json --sort size --min-size 5 \
  | jq '.clusters[] | {id, members: .member_count, latest: .latest_thread_number}'

For the full per-command JSON shapes, see the individual feature pages and the Commands reference.

Exit codes

  • 0 — success
  • non-zero — usage error, command not implemented, runtime error

Stderr always carries a human-readable error message; stdout is reserved for the requested output (text or JSON) so you can pipe stdout to jq without losing diagnostics.

Keeping the mirror fresh

Three patterns, in increasing order of automation:

On-demand staleness check

Use --sync-if-stale on gitcrawl search:

gitcrawl search issues "manifest cache" \
  -R owner/repo \
  --sync-if-stale 5m \
  --json number,title,url

Best for ad-hoc agent tools that should bound staleness but minimize sync calls.

Periodic background refresh

Run gitcrawl refresh owner/repo on a cron, systemd timer, or launchd agent every few minutes per repo.

# Every 5 minutes, refresh the active repos.
*/5 * * * * $HOME/bin/gitcrawl refresh openclaw/gitcrawl --json > /tmp/gitcrawl.openclaw.json 2>&1

For multiple repos, loop in a small shell script — gitcrawl is happy to run sequentially against a shared SQLite file.

Agent recipes

"Look up an issue without burning quota"

octopool gh api repos/owner/repo/issues/123

Octopool owns pooled GitHub reads. Use gitcrawl search first when local discovery is enough.

"Find candidates, hydrate them, summarize"

NUMS=$(gitcrawl search issues "checksum mismatch" -R owner/repo \
        --json number --limit 30 \
        | jq -r '[.[].number] | join(",")')

gitcrawl sync owner/repo --numbers "$NUMS" --include-comments --with pr-details

gitcrawl cluster-detail owner/repo --id "$(gitcrawl clusters owner/repo --json \
        | jq '.clusters[0].id')"

Search is local; the targeted sync brings exactly the rows you need; cluster-detail returns the structured triage view.

"Find duplicates around a new bug report"

NUM=789
gitcrawl sync owner/repo --numbers "$NUM" --include-comments
gitcrawl embed owner/repo --number "$NUM"
gitcrawl neighbors owner/repo --number "$NUM" --limit 10 --json

"Triage a cluster end to end"

ID=42

# Read.
gitcrawl cluster-detail owner/repo --id "$ID" --body-chars 600 --json

# Decide canonical, then close locally.
gitcrawl set-cluster-canonical owner/repo --id "$ID" --number 123
gitcrawl close-cluster owner/repo --id "$ID" --reason "consolidated under #123"

# Comment upstream via real gh.
gh issue comment 456 -R owner/repo --body "Duplicate of #123"

Shared GitHub cache

octopool login
octopool gh api repos/openclaw/openclaw/pulls/123

Gitcrawl no longer maintains a runtime gh command cache. Use Octopool for org-authenticated pooled GitHub reads, and keep using gitcrawl for local mirror/search/cluster automation.

Multi-repo automation

A single gitcrawl.db can hold many repositories. Loop in shell:

for repo in openclaw/gitcrawl steipete/repo-a octocat/repo-b; do
  gitcrawl refresh "$repo" --json | jq '{repo: "'"$repo"'", sync: .sync, embed: .embed}'
done

Or maintain a small script that reads a list of repos from a file and runs them on a schedule.

Output formats

Format When to use
text (default) Humans at a terminal
json (or --json) Pipelines, scripts, agents
log Internal logging output; structured key/value pairs

Force a format globally with --format json or per-command with --json. The log format is mostly used internally and is subject to change.

CI integration

Run gitcrawl in CI to validate a portable store's freshness, sanity-check cluster shapes, or produce a triage report:

- name: Refresh and snapshot clusters
  run: |
    gitcrawl init --portable-store $PORTABLE_STORE_URL
    gitcrawl refresh openclaw/gitcrawl --json > sync.json
    gitcrawl clusters openclaw/gitcrawl --json --sort size --min-size 5 > clusters.json
  env:
    GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
    OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}

- uses: actions/upload-artifact@v4
  with: { name: triage, path: "*.json" }

The artifact gives reviewers a structured view of what changed and how the cluster graph looks today.

Best practices

  • Set both tokens in a single place. Either env or [env] in config.toml. Mixing sources tends to confuse doctor reports.
  • Bound the staleness window. --sync-if-stale on every agent-driven search is cheaper than a hot cron loop.
  • Re-cluster after a backfill. A large --state all sync should be followed by gitcrawl refresh --no-sync (or just gitcrawl embed && gitcrawl cluster) so the durable graph reflects the new content.