Add regex pattern support for branch protection rules

[mistercrunch]

Managing branch protection across Apache projects has become increasingly complex as projects adopt more sophisticated branching strategies. Consider the challenges faced by large Apache projects:

Current Pain Points:

• Release Branch Explosion: Projects like Kafka, Spark, and Tomcat maintain dozens of active release branches (release/3.1.x, release/3.2.x, release/4.0.x) that each require identical protection rules
• Feature Branch Proliferation: With hundreds of contributors, feature branches (feature/user-auth, feature/payment-gateway) need consistent protection without manual configuration for each branch
• Maintenance Overhead: Every new release branch requires manual .asf.yaml updates, creating bottlenecks during release cycles
• Configuration Drift: Different protection rules accidentally applied to similar branches, creating security gaps
• Hotfix Urgency: Critical security hotfixes (hotfix/CVE-2024-1234) need immediate protection rules without waiting for manual configuration

This implementation introduces regex pattern support for protected_branches, enabling rule-based branch protection that scales automatically with project growth.

1. Pattern-Based Rules

github:
  protected_branches:
    # Exact branches (existing behavior, unchanged)
    main:
      required_signatures: true

    # NEW: Pattern-based protection
    all-release-branches:
      pattern: "release/.*"
      required_signatures: true
      required_linear_history: true

    feature-branches:
      pattern: "feature/.*"
      required_pull_request_reviews:
        required_approving_review_count: 1

2. Smart Precedence System

• Exact branch names always override pattern matches (backwards compatibility)
• First matching pattern wins when multiple patterns match the same branch
• Clear warnings show exactly what happened and why

3. Production-Grade Security

• ReDoS Protection: 1000-character pattern limit prevents regex denial-of-service attacks
• Validation at Parse Time: Invalid regex patterns fail immediately with helpful error messages
• Pattern Compilation Caching: Optimized performance for repeated operations

4. Comprehensive Observability

Branch Protection Changes:

=== Branches Protected by Pattern Rules ===

release/v3.1.0 (via pattern rule "release-branches" (release/v.*)):
  - Set required signatures to True
  - Set required linear history to True

feature/user-auth (via pattern rule "feature-branches" (feature/.*)):
  - Set required approving review count to 1

• Automatic Protection: New release branches inherit protection immediately

• Reduced Toil: No manual .asf.yaml updates for routine branch operations

• Consistent Security: Identical protection across all similar branches

• Faster Releases: Remove YAML maintenance from release checklists

• Default Secure: New branches matching patterns get protection automatically

• Zero Gaps: No window where new branches are unprotected

• Consistent Enforcement: Same rules applied uniformly across branch families

• Audit Trail: Clear logging shows which rules applied to which branches

• Backwards Compatible: All existing configurations work unchanged

• Gradual Adoption: Projects can migrate at their own pace

• Reduced Support Load: Fewer tickets about branch protection configuration

• Scalable: Supports projects with hundreds of branches without performance impact

• 24 new test cases covering all pattern functionality

• Edge case coverage: Invalid regex, no matches, conflicts, precedence, performance

• Integration scenarios: Real-world project configurations

• Backwards compatibility: All existing tests pass unchanged (33 total tests pass)

• Security hardened: ReDoS protection and input validation

• Error handling: Graceful failure with actionable messages

• Performance optimized: Single API query, compiled patterns

• Observability: Detailed logging and warning system

• asfyaml/validators.py - Added BranchPattern validator with security constraints

• asfyaml/feature/github/__init__.py - Updated schema to support optional pattern field

• asfyaml/feature/github/branch_protection.py - Core pattern matching and resolution logic

• tests/github_branch_protection_patterns.py - Comprehensive test suite (24 test cases)

• README.md - Complete documentation with examples and migration guidance

[rusackas]

LGTM, the docs look great! Looking forward to leveraging this.

[sadpandajoe]

This will be helpful such that people won't try to push into our release branches

[mistercrunch]

Yeah that's a not-so-uncommon pattern on Superset:

• someone opens a PR against a release branch (they want to fix 5.0 for instance, don't know we always target master and cherry-pick) 🤦
• a reviewer reviews the PR, don't notice it's not targeting master, super easy to miss 🤦
• gets merged 🤦
• breaks the workflow/pattern in release branch 🤦
• release manager has to force push on a release branch 🤦

[potiuk]

Shall we merge it ? (after rebasing and reviewing - it's been a long time since it was touched)

It seems we have a long-standing PRs here that we could possibly speed up. Is there a way we can get some more volunteers to join to review and merge those?

Similary like it happened in https://github.com/apache/infrastructure-tooling (and with AI agents driving largely most of the creation/review process - ASF could definietly benfit ffrom more eyes and hands being allowed to review and merge the changes.

GitHub - responding to the recent wave of AI traffic - significantly sped up release processes and adding new features to GitHub - so we shoudl also allow our PMCs to make use of those features faster - specifically features that are aimed to harden security.

@Humbedooh @dave2wave @killerbees - I think we have enough willing (and capable and trusted) volunteers - myself, @ppkarwasz @raboof that could not only help to speed it up but also review and check upon each other's work (And agent submissions).

With infrastructure-actions - for example we are now - after all the volunteer time and agentic investments - down to "0 backlog" in terms of responding to the needs of the ASF PMCs. Basically what we have now - and number (and time) of volunteers workign there - and agentic help they get allowed to achieve that.

I think similar improvements can be achieved if the trusted volunteers are also allowed to revieew and merge changes here.

Of course it might not be desireable, but it seems that more gating currently means potentially more delays in getting the features in the hands of ASF PMCs.

Up to you to decide - but I think we need to speed up the review process for new GH features to be added - and you have pool of capable and trusted voiunteers who are wiling to help - if you choose so.

[ppkarwasz]

I think we should close this PR and concentrate on improving the recently added GitHub Rulesets support:

• Rulesets allow branch name patterns, which is what this PR introduces,
• Moreover they are public (e.g. an HTML overview is available at https://github.com/<owner>/<repo>/rules).

[dave2wave]

@ppkarwasz and @potiuk what would the rulesets look like to support the features offered in this PR?

[ppkarwasz]

The branch protection rules in the example above should map to these rulesets:

meta:
  environments:
    - github_rulesets

github:
  # Clear any legacy Branch Protection entries; they are replaced by Rulesets.
  protected_branches: ~

  rulesets:
    - name: "default-branch"
      type: branch
      branches:
        includes:
          - "~DEFAULT_BRANCH"
      required_signatures: true

    - name: "all-release-branches"
      type: branch
      branches:
        includes:
          - "release/*"
      required_signatures: true
      required_linear_history: true

    - name: "feature-branches"
      type: branch
      branches:
        includes:
          - "feature/*"
      required_pull_request_reviews:
        required_approving_review_count: 1

@mistercrunch: does this cover your use case?

[potiuk]

I am for it. This is very good idea.