Skip to content

Add CodeRabbit configuration for automated PR reviews#56

Merged
adalton merged 6 commits into
flightctl:mainfrom
adalton:andalton/coderabbit-config
May 26, 2026
Merged

Add CodeRabbit configuration for automated PR reviews#56
adalton merged 6 commits into
flightctl:mainfrom
adalton:andalton/coderabbit-config

Conversation

@adalton

@adalton adalton commented May 26, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Adds .coderabbit.yaml tuned for this Ansible collection's content (Python modules, module_utils, plugins, Ansible YAML, shell scripts, CI workflows)
  • Balances security review (prodsec-skills) with design quality review (idempotency, check_mode, argument specs, FQCN, test quality)
  • Includes Ansible-specific pre-merge gates: hardcoded secrets, injection vectors, sensitive data in logs, idempotency violations, AI attribution
  • Enables auto-labeling of PRs by area (modules, module-utils, plugins, tests, ci, docs, collection-metadata)
  • Enables yamllint, markdownlint, gitleaks, semgrep, actionlint, trivy, and osvScanner

Test plan

  • Comment @coderabbitai configuration on this PR to validate the config schema
  • Verify CodeRabbit activates on this PR and produces a review summary
  • Confirm labels are suggested based on file paths changed
  • Open a follow-up test PR touching a module file to verify path_instructions fire correctly

🤖 Generated with Claude Code

Configuration: CodeRabbit Automated Review Setup (updated)

Affected Areas

  • plugins/modules/
  • plugins/module_utils/
  • plugins/connection/
  • plugins/inventory/
  • plugins/doc_fragments/
  • tests/unit/, tests/integration/
  • demo/
  • .github/workflows/ (CI)
  • changelogs/
  • Collection metadata: galaxy.yml, meta/runtime.yml
  • Root: .coderabbit.yaml

What changed

  • Adds a complete .coderabbit.yaml to enable CodeRabbit automated PR reviews for the flightctl.core Ansible Collection.
  • Enables auto-review and incremental reviews scoped by path filters, base-branch rules, and ignored PR title keywords.
  • Adds automatic, path-based labeling for: modules, module-utils, plugins, tests, ci, docs, collection-metadata.
  • Enables linters/scanners: yamllint, markdownlint, gitleaks, semgrep, actionlint, ast-grep (essential rules), trivy, osvScanner; hadolint and checkov are disabled.
  • Introduces detailed path_instructions and review rubrics covering:
    • Ansible modules: idempotency, check_mode, argument_spec completeness, return/exit consistency, documentation/return-shape constraints, FQCN expectations.
    • module_utils: public API minimization, typed signatures, safe exception patterns, TLS/credential handling, safe deserialization.
    • connection/inventory plugins: credential protection, TLS validation, graceful failure modes.
    • doc_fragments, demo playbooks: FQCN usage, no_log guidance, no hardcoded secrets.
    • Tests: AAA patterns, mocking of external calls, idempotency/check_mode assertions, fixture handling.
    • CI workflows: pinned actions, least-privilege tokens, actionlint and supply-chain checks.
    • Shell scripts, Markdown/RST docs, changelog fragment categorization, dependency pinning/CVE/license scanning, crypto filename/algorithm restrictions, and AI-attribution trailer rules.
  • Configures pre-merge hard gates (error-level) for hardcoded secrets, weak crypto, injection vectors, and sensitive data in logs; warning-level gates for Ansible idempotency risk and AI attribution requirements.
  • Adds knowledge_base guidance scoped to AGENTS/CLAUDE/CONTRIBUTING patterns with automatic issue/PR/learnings scoping.
  • Replaced a brace-expansion glob with 10 explicit glob entries using YAML anchors to ensure minimatch-compatible patterns (no brace-expansion globs remain).
  • Disables CodeRabbit automatic PR approval behavior in a commit (commit metadata includes an "Assisted-by" attribution).

Module API surface

  • No runtime or code changes in modules. The configuration enforces review checks that will surface comments about argument_spec completeness, return-shape consistency, idempotency, check_mode support, and FQCN usage but does not alter module APIs.

Shared utilities

  • module_utils will be subject to stricter review guidance (API surface, typing, TLS/credential handling, safe deserialization). Violations will be flagged for review — no automated code changes introduced by this PR.

Plugin behavior

  • Connection and inventory plugins are only evaluated by reviewers/automated checks for credential handling, TLS validation, and failure modes; no behavioral changes are introduced.

Test coverage & demos

  • Tests and demo playbooks will be linted and reviewed for proper mocking, idempotency/check_mode assertions, absence of hardcoded secrets, and use of FQCN; maintainers may be asked to update tests/demos to meet expectations.

CI configuration

  • GitHub workflows under .github/workflows/ will be scanned for pinning, secret handling, least-privilege tokens, and supply-chain issues by enabled scanners; the config itself does not change CI behavior.

Collection metadata

  • galaxy.yml and meta/runtime.yml are included in path filters and will be validated for versioning, dependency pinning, and CVE/license guidance via the configured tools.

Backward compatibility

  • Configuration-only change. No code, API, or runtime behavior is modified. New automated review gates are prospective and may require future code/documentation updates to satisfy stricter checks but do not introduce breaking changes.

Review Change Stack

Assisted-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@adalton adalton self-assigned this May 26, 2026
@coderabbitai

coderabbitai Bot commented May 26, 2026

Copy link
Copy Markdown

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

Adds a complete .coderabbit.yaml configuring global review behavior and labels, detailed path-scoped rubrics for Ansible components, enables specific scanners, defines pre-merge hard gates and warnings, and enables a knowledge base for guideline tracking.

Changes

CodeRabbit Configuration for flightctl-ansible

Layer / File(s) Summary
Global review settings and automation
.coderabbit.yaml 1–89
Sets language (en-US), enables early_access, configures assertive review tone, auto_review (including incremental drafts), base-branch scope, PR title exclusions, path_filters, and initial labeling taxonomy.
Path-scoped review rubrics
.coderabbit.yaml 90–360
Large path_instructions map with detailed rubrics for Ansible modules (idempotency, check_mode, docs/return shapes), module_utils patterns, connection/inventory plugins (TLS/credential handling), documentation fragments, Python security/quality (eval/exec/yaml.load safety, secret handling, constant-time comparisons), tests and integration YAML expectations, demo playbooks, Ansible YAML conventions (FQCN, naming, no_log), CI/workflow constraints, shell safety, dependency/supply-chain guidance, crypto filename rules, changelog fragment checks, and Markdown/RST validations.
Security scanner toolchain
.coderabbit.yaml 364–390
Enables gitleaks, semgrep, actionlint, ast-grep, yamllint, markdownlint, trivy, osvScanner; explicitly disables hadolint and checkov.
Pre-merge checks and severities
.coderabbit.yaml 391–448
Defines pre_merge_checks with error-mode gates for hardcoded secrets, weak crypto/non-constant-time comparisons, injection vectors, and sensitive-data in logs; warning-mode checks for potential Ansible idempotency violations and AI attribution trailer compliance.
Knowledge base and learning capture
.coderabbit.yaml 449–463
Enables knowledge_base guideline tracking scoped to AGENTS/CLAUDE/CONTRIBUTING file patterns, with automatic scoping for issues, pull requests, and learnings.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Poem

🔧 A config blooms in YAML light,
Scan rules wake and set to right,
Rubrics guard each module's claim,
Gates catch secrets, warn the same,
Learnings logged — the guide takes flight.

🚥 Pre-merge checks | ✅ 11
✅ Passed checks (11 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'Add CodeRabbit configuration for automated PR reviews' directly and clearly summarizes the main change: adding a .coderabbit.yaml configuration file for automated PR reviews.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
No-Hardcoded-Secrets ✅ Passed .coderabbit.yaml contains no hardcoded secrets, tokens, passwords, API keys, or credentials; all references to security terms are in instructional text (review guidelines), not actual values.
No-Weak-Crypto ✅ Passed .coderabbit.yaml contains no crypto implementations or weak algorithms. It's a configuration file that defines rules to detect and flag weak crypto issues in reviewed code.
No-Injection-Vectors ✅ Passed No injection vectors found: yaml.load() uses SafeLoader; shell/command tasks only in controlled test/demo environments using safe utilities.
No-Sensitive-Data-In-Logs ✅ Passed PR adds only .coderabbit.yaml configuration with no executable code containing logging that exposes sensitive data. File documents security best practices for protecting credentials.
Ansible-Idempotency ✅ Passed PR #56 adds .coderabbit.yaml configuration, not modules. The ansible-idempotency check targets module implementations, not config files.
Ai-Attribution ✅ Passed AI-assisted commit properly attributed with "Assisted-by: Claude Opus 4.6" trailer; no improper Co-Authored-By usage for AI tools detected.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.coderabbit.yaml:
- Around line 324-332: The no-hardcoded-secrets rule is too broad because its
“base64 strings >32 chars” check is producing false positives; update the rule
named "no-hardcoded-secrets" to narrow or relax that check: either change mode
from "error" to "warning" for the base64 detection, remove the blanket base64
>32 chars condition, or limit base64 checks to only run when the key name
matches sensitive identifiers (api_key, secret, token, password, private_key)
and exclude known benign keys like certificate-authority-data and request
payloads; ensure the rule still enforces hard literals for
api_key/secret/token/password while adding an allowlist for fixture keys to
avoid noisy failures.
- Around line 135-136: The YAML uses brace-expanded globs like
"plugins/{connection,inventory}/**/*.py" and several others (e.g.
"tests/integration/**/*.{yaml,yml}", "demo/**/*.{yaml,yml}", "**/*.{yaml,yml}",
"**/{requirements*.txt,pyproject.toml,galaxy.yml,.pre-commit-config.yaml}",
"**/*{crypt,cipher,sign,hash,tls,ssl,cert,key,token}*", "**/*.{md,rst}") which
may not be supported by CodeRabbit’s minimatch; replace each brace-expanded
pattern with explicit separate glob entries for each alternative (e.g., split
"plugins/{connection,inventory}/**/*.py" into two entries
"plugins/connection/**/*.py" and "plugins/inventory/**/*.py") and do the same
for all listed patterns so the matcher definitely covers each path, or confirm
brace support and add a comment indicating confirmation.
- Around line 340-347: The no-injection-vectors custom check is too broad
(pre_merge_checks.custom_checks.no-injection-vectors) and should only hard-fail
on true unsafe patterns (e.g., explicit shell=True patterns, dynamic command
construction from untrusted inputs, use of eval/exec/pickle.loads/yaml.load
without SafeLoader, os.system with interpolated variables) while allowing
known-good Ansible command/shell usages in controlled test/demo code; change the
rule to (1) narrow its regexes to match those risky constructs specifically, (2)
change mode to "warning" by default and only set "error" when the risky-pattern
regex matches, and (3) add an allowlist exemption for patterns or paths matching
the integration/demo test namespaces (e.g., integration/demo targets) so
ansible.builtin.command and ansible.builtin.shell usages in those controlled
locations do not block merges.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: f32290f4-e560-4b55-959d-3e0441ca95db

📥 Commits

Reviewing files that changed from the base of the PR and between ee90556 and bdefcc9.

📒 Files selected for processing (1)
  • .coderabbit.yaml

Comment thread .coderabbit.yaml Outdated
Comment thread .coderabbit.yaml
Comment thread .coderabbit.yaml
adalton added 2 commits May 25, 2026 21:01
- Replace brace-expansion globs with explicit entries and YAML anchors
  for minimatch compatibility (keep crypto pattern with comment)
- Narrow no-hardcoded-secrets to require credential context for
  base64 flagging, exempting test fixtures and cert data
- Narrow no-injection-vectors to target unsafe Python patterns only,
  exempting legitimate Ansible command/shell usage in tests and demos

Assisted-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Assisted-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
.coderabbit.yaml (1)

305-318: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Brace expansion on line 309 is unverified — consider splitting or confirming support.

The comment on lines 306-308 asserts that "minimatch (used by CodeRabbit) supports brace expansion by default," but a previous review's web search found no explicit CodeRabbit documentation confirming this. The minimatch library itself has optional brace expansion behavior that may or may not be enabled in CodeRabbit's implementation.

Security impact: if this pattern silently fails to match, cryptography-related files like flightctl_certificate_management.py won't receive the specialized crypto security review instructions.

Given that all other brace-expanded patterns were split in this PR, consider either:

  1. Splitting this pattern into explicit entries for consistency and certainty, or
  2. Testing this specific pattern to confirm it matches as expected before merge.
♻️ Option to split into explicit patterns
     # ── Cryptography-related files ───────────────────────────────
-    # Note: minimatch (used by CodeRabbit) supports brace expansion
-    # by default. This pattern matches files with crypto-related
-    # keywords in their names (e.g., flightctl_certificate_management.py).
-    - path: "**/*{crypt,cipher,sign,hash,tls,ssl,cert,key,token}*"
+    - path: "**/*crypt*"
+      instructions: &crypto-instructions |
+        Cryptographic security:
+        - Banned: MD5, SHA1, DES, RC4, 3DES, Blowfish, ECB mode
+        - Symmetric: AES-256-GCM or ChaCha20-Poly1305
+        - Signing: Ed25519 or ECDSA P-256+
+        - No custom crypto; use vetted libraries only
+        - Constant-time comparison for all secret/token data
+        - Certificate validation: verify full chain by default
+
+    - path: "**/*cipher*"
+      instructions: *crypto-instructions
+
+    - path: "**/*sign*"
+      instructions: *crypto-instructions
+
+    - path: "**/*hash*"
+      instructions: *crypto-instructions
+
+    - path: "**/*tls*"
+      instructions: *crypto-instructions
+
+    - path: "**/*ssl*"
+      instructions: *crypto-instructions
+
+    - path: "**/*cert*"
+      instructions: *crypto-instructions
+
+    - path: "**/*key*"
+      instructions: *crypto-instructions
+
+    - path: "**/*token*"
       instructions: |
         Cryptographic security:
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.coderabbit.yaml around lines 305 - 318, The brace-expanded glob pattern
under the "Cryptography-related files" block (the path entry
"**/*{crypt,cipher,sign,hash,tls,ssl,cert,key,token}*") may not be supported by
CodeRabbit's minimatch settings; to fix, replace the single brace-expanded path
with explicit separate path entries (one per keyword) or add/verify a test that
demonstrates the existing brace expansion matches files like
"flightctl_certificate_management.py"; update the YAML so targets use either
multiple lines of "- path: \"**/*crypt*\"" etc. or include a documented test
confirming brace expansion is enabled.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In @.coderabbit.yaml:
- Around line 305-318: The brace-expanded glob pattern under the
"Cryptography-related files" block (the path entry
"**/*{crypt,cipher,sign,hash,tls,ssl,cert,key,token}*") may not be supported by
CodeRabbit's minimatch settings; to fix, replace the single brace-expanded path
with explicit separate path entries (one per keyword) or add/verify a test that
demonstrates the existing brace expansion matches files like
"flightctl_certificate_management.py"; update the YAML so targets use either
multiple lines of "- path: \"**/*crypt*\"" etc. or include a documented test
confirming brace expansion is enabled.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: 5def929f-3b7b-4a0e-8617-cdf627975e01

📥 Commits

Reviewing files that changed from the base of the PR and between bdefcc9 and 6ff6946.

📒 Files selected for processing (1)
  • .coderabbit.yaml

@adalton adalton requested a review from amir-yogev-gh May 26, 2026 01:09

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
.coderabbit.yaml (1)

99-100: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Fix Ansible module rubric: make required/default/choices and msg optional per contract

In .coderabbit.yaml (lines 99-100), the rubric incorrectly implies msg is a mandatory return key and that every argument must specify required, default, and choices—both conflict with Ansible’s module conventions. In particular: choices is optional, and required: true must not be combined with default. (docs.ansible.com)

Proposed fix
-        - Return consistent result dicts: changed, msg, and resource-specific keys
-        - Argument spec: define type, required, default, choices for every parameter
+        - Return consistent result dicts: changed plus resource-specific keys; include msg when it adds useful user-facing context
+        - Argument spec: define the applicable attributes for each parameter (e.g., type, required, default, choices, no_log, aliases); avoid required+default
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.coderabbit.yaml around lines 99 - 100, Update the Ansible module rubric
entries so that the return key "msg" is optional and the Argument spec does not
require every parameter to declare "required", "default", or "choices";
specifically modify the checklist items that mention "Return consistent result
dicts: changed, msg, and resource-specific keys" and "Argument spec: define
type, required, default, choices for every parameter" to state that "msg" is
optional and that "required" must not be combined with "default" and "choices"
is optional per Ansible conventions (referencing the keys "msg", "required",
"default", and "choices" in the rubric text to locate and change those lines).
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In @.coderabbit.yaml:
- Around line 99-100: Update the Ansible module rubric entries so that the
return key "msg" is optional and the Argument spec does not require every
parameter to declare "required", "default", or "choices"; specifically modify
the checklist items that mention "Return consistent result dicts: changed, msg,
and resource-specific keys" and "Argument spec: define type, required, default,
choices for every parameter" to state that "msg" is optional and that "required"
must not be combined with "default" and "choices" is optional per Ansible
conventions (referencing the keys "msg", "required", "default", and "choices" in
the rubric text to locate and change those lines).

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: 96fb3e4a-8eed-422b-beef-6aa12f890170

📥 Commits

Reviewing files that changed from the base of the PR and between 6ff6946 and 569913a.

📒 Files selected for processing (1)
  • .coderabbit.yaml

Assisted-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@adalton

adalton commented May 26, 2026

Copy link
Copy Markdown
Contributor Author

Addressed in 360b7ff. Split the last brace-expansion pattern (**/*{crypt,cipher,...,token}*) into 10 explicit entries using YAML anchors (&crypto-instructions / *crypto-instructions), consistent with how the other patterns were split. No brace-expansion globs remain in the config.

adalton added 2 commits May 25, 2026 21:30
Assisted-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Assisted-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@adalton adalton merged commit dea4246 into flightctl:main May 26, 2026
8 checks passed
@adalton adalton deleted the andalton/coderabbit-config branch May 26, 2026 15:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants