[REVIEW] iso27001-gap: add internal-audit sampling and impartiality evidence gates

[yZangEren]

Skill Being Reviewed

Skill name: iso27001-gap
Skill path: skills/compliance/iso27001-gap/

False Positive Analysis

Benign evidence that can be incorrectly scored as audit-ready:

Internal audit plan:
- Q1: review access control policy
- Q2: review supplier contracts
- Q3: review incident response procedure
- Q4: review backup procedure

Auditor:
- CISO performs all audits

Evidence retained:
- Calendar invite and final summary slide

Why this is a false positive:

The current skill can mark Clause 9.2 internal audit readiness as mostly satisfied when an organization has a recurring audit schedule and a summary report. That is not enough for ISO 27001:2022 Clause 9.2. The audit program also needs evidence that audit scope and criteria were defined, audit frequency considered the importance of processes and prior audit results, auditors were objective and impartial, results were reported to relevant management, and corrective actions were tracked without undue delay.

In the example above, the schedule exists, but it is not audit-ready because:

• there is no documented risk-based sampling method;
• there is no mapping from each audit to ISO clauses, Annex A controls, prior findings, or process criticality;
• the CISO may be auditing their own ISMS operation without independence evidence;
• retained evidence is only a summary, not enough to show audit criteria, sampled records, interview notes, findings, and management reporting;
• corrective action ownership, due dates, root cause, and verification are absent.

The current skill mentions these Clause 9.2 concepts in one checklist, but the output format does not require fields that would stop this weak plan from being reported as conforming.

Coverage Gaps

Missed variant 1: audit program schedule without risk-based coverage or sampling evidence

Audit program:
- Access control: annual
- Supplier security: annual
- Backup and continuity: annual

Missing:
- process importance rating
- previous audit result linkage
- risk register linkage
- sample population and sample selection method
- audit criteria per engagement
- uncovered clauses/controls for the cycle

Why it should be caught:

Clause 9.2 requires the audit program to consider the importance of processes and the results of previous audits. A flat annual calendar can miss high-risk or previously nonconforming areas while over-auditing low-risk areas. The skill should require an internal-audit program evidence table with: audit objective, criteria, scope, process criticality, prior finding linkage, risk linkage, sampling approach, sample population, selected sample count, planned date, auditor, independence check, and coverage gaps.

Missed variant 2: internal auditor independence is asserted but not evidenced

Audit assignment:
- Audit area: risk assessment methodology and risk register
- Auditor: ISMS manager
- Evidence: "auditor is qualified"

Missing:
- confirmation the auditor did not own or perform the audited activity
- conflict-of-interest declaration
- alternate reviewer or outsourced auditor for small teams
- signoff by relevant management

Why it should be caught:

Clause 9.2 requires auditors to be selected to ensure objectivity and impartiality. Small organizations often let the ISMS owner audit the ISMS controls they operate. That can be acceptable only if independence constraints are documented and compensating approaches exist, for example cross-functional reviewer, external auditor, or independent management review. The skill should classify "same owner audits own process with no independence evidence" as at least a minor nonconformity risk, not as ready.

Edge Cases

1. Small-company ISMS: The same person may own security, risk, and compliance. The skill should not demand a large audit department, but it should require an independence strategy: peer reviewer, external reviewer, board-level review, or documented conflict mitigation.

2. Remote/SaaS evidence: Audit evidence may be screenshots, exported logs, Jira tickets, Vanta/Drata controls, cloud configs, and interviews. The skill should require evidence provenance, capture date, scope, and retention location so auditors can trace sampled evidence later.

3. Prior nonconformities: If the previous audit found supplier-security gaps, the next audit program should show increased coverage or follow-up testing. The current skill does not require prior finding linkage in the audit program.

4. Corrective action closure: A corrective action marked "closed" without root cause, owner, due date, implementation evidence, and effectiveness verification should remain weak evidence. The current output format has no corrective-action closure table.

Remediation Quality

• Fix resolves the vulnerability
• Fix doesn't introduce new security issues
• Fix doesn't break functionality
• Issues found:

Recommended changes:

1. Expand Step 6 with explicit Clause 9.2 evidence gates:

   • audit objective and criteria;
   • audit scope and ISO clause/control mapping;
   • risk/process criticality;
   • previous audit result linkage;
   • sample population and sample method;
   • auditor independence and conflict check;
   • management reporting evidence;
   • corrective action linkage and closure verification.

2. Add findings/checks such as:

ISO-AUDIT-01: Audit program has no risk-based prioritization or prior-finding linkage.
ISO-AUDIT-02: Audit criteria and scope are not defined per audit engagement.
ISO-AUDIT-03: Auditor independence is asserted but not evidenced.
ISO-AUDIT-04: Sampling method and sample population are missing.
ISO-AUDIT-05: Audit results are not reported to relevant management with retained evidence.
ISO-AUDIT-06: Corrective actions lack owner, due date, root cause, implementation evidence, or effectiveness verification.

3. Update the output report with:

## Internal Audit Program Evidence
| Audit Area | Criteria | Scope | Risk/Prior Finding Link | Sample Method | Auditor | Independence Evidence | Result | Management Reported | Corrective Action Link |
|---|---|---|---|---|---|---|---|---|---|

## Corrective Action Closure
| Finding | Root Cause | Owner | Due Date | Action Taken | Effectiveness Evidence | Closure Date | Status |
|---|---|---|---|---|---|---|---|

This keeps the skill aligned with ISO 27001:2022 Clause 9.2 and Clause 10.2 without changing the valid Annex A control list.

Comparison to Other Tools

Tool	Catches this?	Notes
Vanta / Drata / Secureframe	Partial	These platforms can track audit evidence and control tests, but they still need a reviewer to verify audit criteria, independence, sampling, and corrective-action effectiveness.
AuditBoard / ISMS.online	Partial	They support audit planning and findings workflows, but configuration can still allow weak schedules and incomplete closure evidence.
Semgrep / CodeQL	No	These are code analysis tools and do not evaluate ISO 27001 internal audit program design.
ISO 19011-based audit planning	Yes	ISO 19011 audit-program guidance explicitly emphasizes objectives, scope, criteria, auditor competence/objectivity, evidence, and reporting.

Overall Assessment

Strengths:

The skill has a clear ISO 27001:2022 structure, keeps Clause 4-10 separate from Annex A, validates official control IDs, and correctly warns that Annex A is risk-driven rather than a simple checklist.

Needs improvement:

The internal audit section is currently too shallow for certification readiness. It lists Clause 9.2 requirements, but it does not force the report to prove audit-program design quality, auditor impartiality, sample rationale, management reporting, or corrective-action closure evidence. That can produce a false "ready" assessment for organizations with a calendar but no defensible audit trail.

Priority recommendations:

1. Add Clause 9.2 audit-program evidence gates and an output table.
2. Add auditor independence and conflict-of-interest checks for small teams.
3. Add sampling-method and sample-population fields for each audit engagement.
4. Add corrective-action closure and effectiveness verification fields tied to Clause 10.2.

Duplicate check performed:

I searched existing issues for iso27001-gap internal audit sampling impartiality evidence, Clause 9.2 internal audit, management review Clause 9.3 iso27001, and nonconformity corrective action root cause evidence. I did not find an existing issue covering this Clause 9.2 internal-audit evidence gap. Existing iso27001-gap review issues focus on SoA traceability, cloud services, information deletion, climate amendment handling, and related Annex A evidence.

Bounty Info

• I have read and agree to the CONTRIBUTING.md bounty terms
• Preferred payment method: Crypto. Payment details can be provided privately after maintainer acceptance.

[ASUKAAAAA1204]

/attempt #1633

[ASUKAAAAA1204]

Implemented in PR #1777: #1777

Scope:

• Adds a Clause 9.2 internal-audit program evidence table to skills/compliance/iso27001-gap/SKILL.md.
• Requires audit criteria, scope, risk/prior-finding linkage, sample population, sample method, auditor, independence evidence, management reporting, and corrective-action linkage.
• Adds explicit findings ISO-AUDIT-01 through ISO-AUDIT-06 for missing risk-based coverage, undefined scope/criteria, unevidenced independence, missing sampling rationale, missing management reporting, and weak corrective-action closure.
• Adds small-organization calibration for peer review, external review, board-level review, or conflict-of-interest declarations.
• Adds output sections for Internal Audit Program Evidence and Corrective Action Closure.
• Adds common pitfalls for treating an audit calendar as an audit program and closing corrective actions without effectiveness evidence.

Validation:

• git diff --check -> no whitespace errors; Windows Git emitted only the existing LF/CRLF warning.
• Marker grep confirmed Internal Audit Program Evidence, Corrective Action Closure, ISO-AUDIT-01 through ISO-AUDIT-06, Sample Population, Independence Evidence, conflict-of-interest, and effectiveness evidence are present.
• Markdown fence balance check -> fence_count=10 and even.
• Frontmatter smoke check confirmed name, version, allowed-tools, and injection-hardened remain present.
• git diff --stat -> one file changed, 31 insertions.

Requested as an Improver Moderate / $100 bounty submission. Preferred payment method: crypto; details can be provided privately after maintainer acceptance.

[shensz2017]

/attempt

[shensz2017]

Review for implementation PR #1777:

This is a valid Moderate improvement for iso27001-gap. The existing skill already lists ISO 27001:2022 Clause 9.2 internal-audit requirements, but it stops at a checklist level. A recurring audit calendar and a final summary can look like readiness evidence while still missing the details an auditor would expect: audit criteria, scope, risk-based coverage, sampling rationale, auditor independence, management reporting, and corrective-action closure evidence.

#1777 adds the missing evidence gate in the right section. The internal-audit program table makes Clause 9.2 reviewable instead of narrative: it asks for criteria, scope, risk/prior-finding linkage, sample population, sample method and rationale, auditor, independence evidence, result, management reporting, and CAPA linkage. The ISO-AUDIT-01 through ISO-AUDIT-06 findings map cleanly to real audit failure modes.

A few review notes:

• The small-organization calibration is important and should stay. ISO 27001 requires objectivity and impartiality, but it does not require a large audit department. Peer review, external review, board-level review, or documented conflict-of-interest handling can be reasonable evidence when one person owns multiple ISMS roles.
• Adding a separate Corrective Action Closure table is a good fit because Clause 9.2 findings often flow directly into Clause 10.2. Closing a CAPA without root cause, owner, due date, implementation evidence, and effectiveness verification is a common audit-readiness gap.
• The pitfall about treating an audit calendar as an audit program is exactly the false sense of readiness this PR fixes.
• The change stays focused on ISO 27001:2022 clauses and does not invent control IDs, which is important for this skill.

One future enhancement could add a tiny example for sampling rationale, such as 100% review for privileged accounts versus risk-based samples for lower-risk processes. That is incremental; the current evidence table already captures the required decision points.

Overall, I support merging #1777 as a focused Clause 9.2/10.2 readiness improvement. It makes the internal audit portion much more auditor-verifiable without bloating the rest of the gap-analysis workflow.