Skip to content

Commit f58fdbc

Browse files
sellakumaranclaude
andauthored
Fix blueprint secret gap, improve --show-secret and setup all reliability (#407)
* Add --show-secret to setup blueprint and display secret at creation time (fixes #406) - Print the blueprint client secret to the terminal at creation time with a 'copy this now' warning so customers are not left without the value - Add 'setup blueprint --show-secret': reads a365.generated.config.json, decrypts via SecretProtectionHelper, and prints the plaintext; exits 1 with instructions if no secret is found - On Windows, --show-secret requires the same machine and user account that ran setup (DPAPI constraint); plaintext on Linux/macOS Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Improve --show-secret UX and add ownership checks/tests - Clarify --show-secret help text and error guidance - Refactor config loading and error handling for --show-secret - Remove DPAPI info log; simplify user-facing output - Add ownership check before client secret creation with warnings - Refine post-creation instructions for secret retrieval - Skip clientAppId validation for --show-secret (offline support) - Lower authentication log level to Debug - Add unit tests for ownership and --show-secret scenarios - Extend test infrastructure for file and mock handling * Address Copilot review comments on PR #407 - Fix [displayed to terminal] appearing in console output (LogDebug instead of LogInformation) - Change IsApplicationOwnerAsync to return bool? so Graph/token errors return null (indeterminate) instead of false, eliminating false-positive ownership warnings - Update callers: owner-assignment fallback uses == true; pre-creation warning uses == false - Update tests: error-path assertions now expect null; mock setups use Task.FromResult<bool?> Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
1 parent a36dc90 commit f58fdbc

19 files changed

Lines changed: 605 additions & 88 deletions

.claude/agents/pr-code-reviewer.md

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -207,6 +207,31 @@ For each changed file, analyze:
207207
- Run `Grep` for the old string in `src/Tests/**/*.cs`
208208
- If test files assert `Contains("old string")` and the old string is gone, the test will silently pass with the wrong message — flag HIGH
209209

210+
**D. `catch (SomeException ex)` sets `context.ExitCode = <literal>` → verify literal matches exception class**
211+
When the diff adds or modifies a catch block that sets `context.ExitCode` to a literal integer:
212+
- Use `Grep` or `Read` to find the exception class definition (look for `class <ExceptionType>` in `Exceptions/` or `Commands/`)
213+
- Check whether the class has an `ExitCode` property override (e.g., `public override int ExitCode => 2;`)
214+
- If the literal does not match the property's return value, flag **HIGH** — the command will report a wrong exit code to scripts and CI pipelines that check `$LASTEXITCODE`
215+
- **Fix pattern**: replace `context.ExitCode = 1;` with `context.ExitCode = ex.ExitCode;` so the exit code is always authoritative from the exception definition
216+
- **Real example (PR #406, Comments 5 & 6)**: `ConfigFileNotFoundException.ExitCode => 2`, but two catch blocks in `AllSubcommand.cs` hardcoded `context.ExitCode = 1`. Scripts expecting exit code 2 on configuration errors received 1 instead.
217+
218+
**D2. Cross-method "catch-returns-null / caller-sets-ExitCode" pattern — same exit-code problem, harder to spot**
219+
Rule 9-D catches *inline* catch blocks. This variant spans two methods and is easy to miss:
220+
```csharp
221+
// Helper:
222+
catch (ConfigFileNotFoundException) { logger.Log...; return null; }
223+
// Caller:
224+
var config = await LoadConfigAsync(...);
225+
if (config == null) { context.ExitCode = 1; return; } // ← wrong exit code
226+
```
227+
Detection:
228+
- When a diff adds a private helper method that catches a typed exception, logs guidance, and **returns null**, search the file for every call site of that helper
229+
- At each call site, find the null-check guard and read the hardcoded `context.ExitCode` literal
230+
- Look up the exception class's `ExitCode` override as in Rule 9-D
231+
- If the literals differ, flag **HIGH**
232+
- Also check whether the PR already fixed this pattern in a *sibling* command (e.g., `AllSubcommand.cs` was fixed to use `ex.ExitCode`). If so, grep all files in `Commands/` for the same helper+null-check pattern — any remaining hardcoded literal is a miss.
233+
- **Fix pattern**: Change `return null;` to `throw;` in the helper; add `catch (ExceptionType ex) { context.ExitCode = ex.ExitCode; }` before the outer `catch (Exception ex)` at each call site. This matches the AllSubcommand.cs pattern.
234+
210235
### Step 3: Generate Findings
211236

212237
For each issue found, provide:
@@ -880,6 +905,25 @@ When a string property represents a discrete set of values (e.g., `"obo"`, `"s2s
880905
```
881906
- **Real example (PR #391, Comments 7 & 8)**: `SetupContext.AuthMode` and `NonDwBlueprintSetupOrchestrator.effectiveMode` both used `?.ToLowerInvariant()`, allowing `""` to disable OBO without any visible error.
882907

908+
### 29. `args.Contains("--flag")` Misses `--flag=true` Form in System.CommandLine Preflight
909+
910+
`System.CommandLine` normalises boolean options: a user can pass `--show-secret` (bare) **or** `--show-secret=true` / `--show-secret false`. Raw `args.Contains("--show-secret")` only matches the bare form; the `=true` variant slips through and re-enables any middleware that was meant to be skipped.
911+
912+
- **Pattern to catch**: `args.Contains("--some-flag")` used in a middleware or startup guard (e.g., `isShowSecret`, `isHelpOrVersion`) — especially when it's a boolean `Option<bool>` in the command definition.
913+
- **Severity**: `medium` — wrong branch taken when the option is supplied as `--flag=true`; can silently re-enable Graph/network calls on commands that should work offline.
914+
- **Check**: Read `Program.cs` (or wherever the preflight guard lives) for every `args.Contains("--")` expression in the diff. If it guards a middleware skip, verify both forms are handled.
915+
- **Fix**:
916+
```csharp
917+
// Wrong — misses --show-secret=true
918+
var isShowSecret = args.Contains("--show-secret");
919+
920+
// Correct — handles bare and =value forms
921+
var isShowSecret = args.Any(a =>
922+
a.Equals("--show-secret", StringComparison.Ordinal)
923+
|| a.StartsWith("--show-secret=", StringComparison.Ordinal));
924+
```
925+
- **Real example (PR #406, Comment 1)**: `isShowSecret = args.Contains("--show-secret")` in `Program.cs` would not skip the Graph preflight when the user passed `--show-secret=true`, accidentally triggering an online call on an offline-only command.
926+
883927
## Example Invocation
884928

885929
When you receive a request like "Review PR #253", you should:

.claude/skills/review-staged/SKILL.md

Lines changed: 37 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -1,36 +1,38 @@
11
---
22
name: review-staged
3-
description: Generate structured code review for staged files (git staged changes) using Claude Code agents. Provides feedback before committing to catch issues early.
3+
description: Generate structured code review for staged files AND all branch commits since main, using Claude Code agents. Provides full PR-equivalent coverage before pushing.
44
allowed-tools: Bash(git:*), Bash(dotnet:*), Bash(cd:*), Read, Write
55
---
66

77
# Review Staged Files Skill
88

9-
Generate AI-powered code review comments for your staged files (git staged changes) before committing. Catch issues early in the development process using the same rigorous review standards as PR reviews.
9+
Generate AI-powered code review comments covering **both** your currently staged changes and all commits on the branch since it diverged from `main`. This gives the same full-branch view that Copilot and other PR reviewers see — not just what you're about to commit.
1010

1111
## Usage
1212

1313
```bash
14-
/review-staged # Review all staged files
14+
/review-staged # Review staged changes + full branch diff against main
1515
/review-staged --verbose # Show detailed analysis
1616
```
1717

1818
Examples:
19-
- `/review-staged` - Review all currently staged files
19+
- `/review-staged` - Review staged changes and all branch commits not yet on main
2020
- `/review-staged --verbose` - Show detailed analysis with full context
2121

2222
## What this skill does
2323

2424
1. **Checks for staged files** using `git diff --staged --name-only`
2525
2. **Fetches staged changes** using `git diff --staged`
26-
3. **Performs architectural review**: Questions design decisions, checks for scope creep, validates use cases
27-
4. **Analyzes changes** for security, testing, design patterns, and code quality issues
28-
5. **Differentiates contexts**: CLI code vs GitHub Actions code (different standards)
29-
6. **Creates actionable feedback**: Specific refactoring suggestions based on file names and patterns
30-
7. **Verifies documentation completeness** — detects user-visible surface changes (new CLI flags, renamed public types, payload contract changes, etc.) and verifies `CHANGELOG.md` + relevant `README.md`/`design.md` are updated in the same diff. For renames, greps all `*.md` files for stale references. See the Documentation Completeness section in `.claude/agents/pr-code-reviewer.md` for detection rules and severity calibration.
31-
8. **Runs the test suite and measures per-test timing** — flags any test taking > 1 second as a performance regression
32-
9. **Generates structured review document** saved to a markdown file
33-
10. **Shows summary** of all issues found organized by severity
26+
3. **Fetches full branch diff** using `git diff $(git merge-base HEAD origin/main)...HEAD` — covers all commits on the branch, not just staged files. This prevents issues in prior commits from going unreviewed.
27+
4. **Combines both diffs** for review: staged diff = what you're about to add; branch diff = full PR picture that Copilot will see
28+
5. **Performs architectural review**: Questions design decisions, checks for scope creep, validates use cases
29+
6. **Analyzes changes** for security, testing, design patterns, and code quality issues
30+
7. **Differentiates contexts**: CLI code vs GitHub Actions code (different standards)
31+
8. **Creates actionable feedback**: Specific refactoring suggestions based on file names and patterns
32+
9. **Verifies documentation completeness** — detects user-visible surface changes (new CLI flags, renamed public types, payload contract changes, etc.) and verifies `CHANGELOG.md` + relevant `README.md`/`design.md` are updated in the same diff. For renames, greps all `*.md` files for stale references. See the Documentation Completeness section in `.claude/agents/pr-code-reviewer.md` for detection rules and severity calibration.
33+
10. **Runs the test suite and measures per-test timing** — flags any test taking > 1 second as a performance regression
34+
11. **Generates structured review document** saved to a markdown file
35+
12. **Shows summary** of all issues found organized by severity
3436

3537
## Engineering Review Principles
3638

@@ -102,8 +104,8 @@ Generated review is saved to:
102104
```
103105

104106
The review includes:
105-
- **Summary**: Overview of changes and key concerns
106-
- **Critical Issues**: Blocking issues that must be fixed
107+
- **Summary**: Overview of changes and key concerns — includes both staged and branch coverage
108+
- **Critical Issues**: Blocking issues that must be fixed (labeled `[staged]` or `[branch]`)
107109
- **High Priority**: Important issues that should be addressed
108110
- **Medium Priority**: Issues that improve code quality
109111
- **Low Priority**: Suggestions for enhancement
@@ -117,8 +119,17 @@ The skill uses **Claude Code directly** for semantic code analysis (same as revi
117119
2. Claude Code reads `.github/copilot-instructions.md` for coding standards
118120
3. Claude Code gets staged files: `git diff --staged --name-only`
119121
4. Claude Code gets staged changes: `git diff --staged`
120-
5. **Claude Code reads the complete current content of every staged file** (not just diff lines) to enable full-file semantic analysis.
121-
5a. **If any staged file is a test file** (path contains `.Tests.` or filename ends in `Tests.cs`), Claude Code MUST also invoke the `pr-test-analyzer` agent as a subagent to review test coverage quality. Pass the staged diff and list of staged test files as context. The `pr-test-analyzer` agent focuses on:
122+
5. **Claude Code gets the full branch diff against main:**
123+
```bash
124+
git diff $(git merge-base HEAD origin/main)...HEAD --name-only # files changed on branch
125+
git diff $(git merge-base HEAD origin/main)...HEAD # full branch diff
126+
```
127+
This covers ALL commits on the branch since it diverged from `origin/main` — including prior commits that are no longer in the staged diff. The union of staged files and branch-diff files is the full review surface. This is the same view Copilot and PR reviewers see.
128+
129+
**When there are no staged files**, skip step 4 and use only the branch diff. When both exist, label findings clearly — `[staged]` for changes only in the staged diff, `[branch]` for changes from prior commits, so the developer knows which issues are still ahead of them vs. already committed.
130+
131+
6. **Claude Code reads the complete current content of every file that appears in either diff** (staged or branch) to enable full-file semantic analysis.
132+
6a. **If any file in the combined diff is a test file** (path contains `.Tests.` or filename ends in `Tests.cs`), Claude Code MUST also invoke the `pr-test-analyzer` agent as a subagent to review test coverage quality. Pass the combined diff and list of test files as context. The `pr-test-analyzer` agent focuses on:
122133
- NSubstitute predicate precision (predicates on mutable reference types evaluated at assertion time vs. call time)
123134
- Missing test scenarios for new orchestration code paths (e.g., 409 idempotent path, tri-state verification results)
124135
- Assertions that track implementation rather than requirements
@@ -128,15 +139,14 @@ The skill uses **Claude Code directly** for semantic code analysis (same as revi
128139
- Parallel code structures that should be consolidated (e.g., a method building the same spec list as a shared helper)
129140
- Unused or dead code that was already there but not touched by the diff
130141
- Missing calls to shared helpers — where the diff adds a new use but existing code still has the old duplicate pattern
131-
For each file path returned in step 3, Claude Code must `Read` the full file before performing analysis.
132-
6. Claude Code performs semantic analysis using its own capabilities
133-
7. Claude Code identifies specific issues with line numbers and code references
134-
8. **Claude Code runs the full test suite with per-test timing:**
142+
7. Claude Code performs semantic analysis using its own capabilities
143+
8. Claude Code identifies specific issues with line numbers and code references
144+
9. **Claude Code runs the full test suite with per-test timing:**
135145
```bash
136146
cd src && dotnet test tests.proj --configuration Release --logger "console;verbosity=normal" 2>&1
137147
```
138148
Parse the output for lines matching `[X s]` or `[X,XXX ms]` patterns. Extract test class name, method name, and duration. Flag any test method taking **> 1 second**. Group findings by test class and include the measured times in the review.
139-
8. Claude Code writes markdown file to `.codereviews/claude-staged-<timestamp>.md`
149+
10. Claude Code writes markdown file to `.codereviews/claude-staged-<timestamp>.md`
140150

141151
**Test timing output format** (from `dotnet test --logger "console;verbosity=normal"`):
142152
```
@@ -157,8 +167,8 @@ Any line showing `[X s]` where X ≥ 1 is a slow test. Report all such tests in
157167
1. **Stage your changes**: `git add <files>`
158168

159169
2. **Review staged files**: `/review-staged`
160-
- Analyzes all staged changes
161-
- Generates review document
170+
- Analyzes staged changes AND all commits on the branch since `main`
171+
- Generates review document with `[staged]` / `[branch]` labels
162172
- Shows summary of issues
163173

164174
3. **Address issues**: Fix any blocking or high-priority issues
@@ -169,15 +179,15 @@ Any line showing `[X s]` where X ≥ 1 is a slow test. Report all such tests in
169179

170180
## When to Use
171181

172-
- **Before committing**: Catch issues early
173-
- **Before creating a PR**: Ensure quality before sharing
174-
- **After addressing PR comments**: Verify fixes are correct
182+
- **Before any commit**: Catch issues before they land in the branch history
183+
- **Before creating a PR**: Get the same full-branch view that Copilot will see — no surprises
184+
- **After addressing PR comments**: Verify fixes across the entire branch, not just the latest staged diff
175185
- **During code cleanup**: Validate refactoring changes
176186
- **When learning**: Get feedback on coding patterns
177187

178188
## Requirements
179189

180-
- Git repository with staged changes
190+
- Git repository (staged changes optional — branch diff is always produced)
181191
- Repository must follow Agent365 DevTools coding standards
182192
- `.claude/agents/pr-code-reviewer.md` must exist (for review guidelines)
183193
- `.github/copilot-instructions.md` must exist (for coding standards)

CHANGELOG.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,8 @@ Agents provisioned before this release need `Agent365.Observability.OtelWrite` g
2323
**Option B — CLI** (`a365 setup admin`) has been removed in this release. Use Option A above, or copy the PowerShell instructions printed in the `a365 setup all` summary output.
2424

2525
### Added
26+
- `setup blueprint --show-secret` — displays the blueprint client secret stored in `a365.generated.config.json` in plaintext without re-running any setup steps. On Windows, decryption requires the same machine and user account that ran setup (DPAPI). When no secret is found, the command prints instructions to run `a365 setup blueprint --agent-name <name>`.
27+
- Blueprint client secret is now printed to the terminal at creation time with a "copy this value now" warning. Use `a365 setup blueprint --show-secret` to retrieve it afterwards.
2628
- Version check: stable-channel users now see an informational notice when a newer preview release exists above the current stable version, without triggering the update-required banner.
2729
- `setup requirements` Global Administrator path: when the well-known CLI client app is not found in a new tenant, Global Admins are prompted to create the app and grant admin consent automatically (enter an app ID or type `C` to create).
2830
- `--authmode obo|s2s|both` option on `setup all` — controls how the agent identity service principal receives permissions:
@@ -40,6 +42,7 @@ Agents provisioned before this release need `Agent365.Observability.OtelWrite` g
4042
- Defensive fallback when the server rejects the new request with a known contract-mismatch signature — the CLI logs `"Automated messaging endpoint registration is not available for this tenant yet. You'll need to configure it manually."` and directs the user to the Teams Developer Portal. Same user-facing path is reused when registration fails because the signed-in user is not a blueprint owner.
4143

4244
### Fixed
45+
- Error messages for commands run without required configuration no longer expose internal file paths. `setup all`, `cleanup`, and `create-instance` without `--agent-name` now show actionable guidance with the exact command to run. `develop addpermissions` and `develop gettoken` without `--app-id` now prompt for the application ID directly.
4346
- `setup all` no longer inherits stale resource IDs when the user switches tenants between runs (`az logout` + `az login` to a different tenant). The CLI detects the tenant change before loading configuration, silently backs up files from the previous run, and prompts the user to re-run with `--agent-name` for a clean setup in the new tenant.
4447
- `setup permissions bot` no longer emits "Bot API permissions configured successfully" when any S2S app-role assignment fails; shows a warning with retry instructions instead.
4548
- Consent-required message "You are running as a non-admin user and cannot grant admin consent" replaced with "An administrator must grant tenant-wide consent to proceed" — the message fires when tenant-wide consent for S2S scopes has not yet been granted, not when the caller lacks admin rights.

0 commit comments

Comments
 (0)