Skip to content
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
d2baec6
feat: update recommendations with evidence-based priorities from 2025…
natifridman May 3, 2026
fab0433
style: apply black formatting to new and modified files
natifridman May 3, 2026
5d78cd8
fix: address CodeRabbit review findings on PR #382
natifridman May 3, 2026
0d5f6cd
fix: complete backward-compat migration for renamed attribute IDs and…
natifridman May 3, 2026
4da6982
fix: address PR #382 review findings — harden assessors and add missi…
natifridman May 3, 2026
a7b00a4
feat: add industry source fetching to research updater (RSS, sitemaps…
natifridman May 3, 2026
8a6cba7
fix: resolve CodeQL incomplete URL substring sanitization alerts
natifridman May 3, 2026
56f7689
fix: address remaining PR #382 review findings
natifridman May 3, 2026
ee3b0a6
fix: address final PR #382 review findings — scoring gates, pattern s…
natifridman May 3, 2026
3bb1622
fix: address remaining open PR #382 review comments
natifridman May 3, 2026
0ea562b
fix: address final open PR #382 review comments
natifridman May 3, 2026
2400911
fix: align docs/attributes.md tier model with default-weights.yaml
natifridman May 3, 2026
dcde1ab
fix: tighten single-file verification, fix CodeQL findings, align docs
natifridman May 4, 2026
ae7f693
fix: black formatting for test_research_updater.py
natifridman May 4, 2026
9a9e9a6
fix: use HTTPS for ArXiv API, validate sitemap origins, remove placeh…
natifridman May 4, 2026
978a4a8
fix: require PR triggers for ci_quality_gates, runner for test_execut…
natifridman May 4, 2026
c316c9b
fix: couple gate detection to PR-triggered workflows, add GET fallbac…
natifridman May 4, 2026
9be9fec
fix: guarantee all sampler patterns get slots, use recursive glob for…
natifridman May 4, 2026
e007586
fix: broaden design_intent keywords, remove orphaned SemanticNamingAs…
natifridman May 5, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 18 additions & 16 deletions .agentready-config.example.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,37 +12,39 @@
# Tier 4 (Advanced) attributes default to 1% each
weights:
# Tier 1 (Essential) - 50% total
claude_md_file: 0.10 # CLAUDE.md configuration files
readme_structure: 0.10 # README structure and content
type_annotations: 0.10 # Type hints in code
standard_layout: 0.10 # Standard project layout
lock_files: 0.10 # Dependency lock files
test_execution: 0.10 # Test execution & coverage (highest priority)
type_annotations: 0.08 # Type hints in code
claude_md_file: 0.07 # CLAUDE.md/AGENTS.md configuration files
ci_quality_gates: 0.05 # CI quality gates (lint + type-check + tests)
single_file_verification: 0.05 # Single-file lint/type-check commands
readme_structure: 0.05 # README structure and content
standard_layout: 0.05 # Standard project layout
lock_files: 0.05 # Dependency lock files
dependency_security: 0.05 # Dependency security & vulnerability scanning

# Tier 2 (Critical) - 30% total
test_coverage: 0.03 # Test coverage requirements
precommit_hooks: 0.03 # Pre-commit hooks & linting
# Tier 2 (Critical) - 27% total
deterministic_enforcement: 0.03 # Hooks & lint rules (deterministic > advisory)
conventional_commits: 0.03 # Conventional commit messages
gitignore_completeness: 0.03 # Comprehensive .gitignore
one_command_setup: 0.03 # One-command build/setup
concise_documentation: 0.03 # Concise structured docs
inline_documentation: 0.03 # Inline code documentation
file_size_limits: 0.03 # File size constraints
dependency_freshness: 0.03 # Dependency updates & security
separation_concerns: 0.03 # Separation of concerns
pattern_references: 0.03 # Pattern references for common changes

# Tier 3 (Important) - 15% total
# Tier 3 (Important) - 14% total
design_intent: 0.02 # Design intent documentation
cyclomatic_complexity: 0.03 # Complexity thresholds
structured_logging: 0.03 # Structured logging
openapi_specs: 0.03 # OpenAPI/Swagger specs
architecture_decisions: 0.03 # Architecture Decision Records
semantic_naming: 0.03 # Semantic file naming

# Tier 4 (Advanced) - 5% total
security_scanning: 0.01 # Security scanning automation
performance_benchmarks: 0.01 # Performance benchmarks
# Tier 4 (Advanced) - 4% total
code_smells: 0.01 # Code smell elimination
issue_pr_templates: 0.01 # GitHub templates
container_setup: 0.01 # Container/virtualization
progressive_disclosure: 0.01 # Progressive disclosure (path-scoped rules)

# Exclude specific attributes from assessment
excluded_attributes: []
Expand Down Expand Up @@ -81,9 +83,9 @@ report_theme: default
# shadow: "rgba(0, 0, 0, 0.5)"

# Example: Increase weight for CLAUDE.md and tests
# This increases CLAUDE.md from 10% to 15% and test_coverage from 3% to 5%
# This increases CLAUDE.md from 7% to 15% and test_execution from 10% to 15%
# Other attributes are automatically rescaled to maintain sum of 1.0
#
# weights:
# claude_md_file: 0.15
# test_coverage: 0.05
# test_execution: 0.15
2 changes: 1 addition & 1 deletion CONTAINER.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ podman run --rm \
-v $(pwd):/repo:ro \
-v $(pwd)/agentready-reports:/reports \
ghcr.io/ambient-code/agentready:latest \
assess /repo --output-dir /reports -e type_annotations -e test_coverage
assess /repo --output-dir /reports -e type_annotations -e test_execution
```

### Save Output Files
Expand Down
16 changes: 8 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -172,12 +172,12 @@ Evaluated across 13 categories:

Attributes are weighted by importance:

- **Tier 1 (Essential)**: 50% of total score - CLAUDE.md, README, types, layouts, lock files
- **Tier 2 (Critical)**: 30% of total score - Tests, commits, build setup
- **Tier 3 (Important)**: 15% of total score - Complexity, logging, API docs
- **Tier 4 (Advanced)**: 5% of total score - Security scanning, performance benchmarks
- **Tier 1 (Essential)**: 55% of total score - Test execution, types, CLAUDE.md, CI gates, verification, README, layout, lock files, dependency security
- **Tier 2 (Critical)**: 27% of total score - Enforcement, commits, build setup, patterns
- **Tier 3 (Important)**: 14% of total score - Design intent, complexity, logging, API docs
- **Tier 4 (Advanced)**: 4% of total score - Code smells, templates, containers, progressive disclosure

Missing essential attributes (especially CLAUDE.md at 10% weight) has 10x the impact of missing advanced features.
Missing essential attributes (especially test execution at 10% weight) has 10x the impact of missing advanced features.

### Interactive HTML Reports

Expand All @@ -195,13 +195,13 @@ Create `.agentready-config.yaml` to customize weights:

```yaml
weights:
claude_md_file: 0.15 # Increase importance (default: 0.10)
test_coverage: 0.05 # Increase importance (default: 0.03)
claude_md_file: 0.15 # Increase importance (default: 0.07)
test_execution: 0.15 # Increase importance (default: 0.10)
conventional_commits: 0.01 # Decrease importance (default: 0.03)
# Other attributes use defaults, rescaled to sum to 1.0

excluded_attributes:
- performance_benchmarks # Skip this attribute
- container_setup # Skip this attribute

output_dir: ./custom-reports
```
Expand Down
32 changes: 16 additions & 16 deletions docs/attributes.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,23 +50,23 @@ Attributes are organized into four weighted tiers:

| Tier | Weight | Focus | Attribute Count |
|------|--------|-------|-----------------|
| **Tier 1: Essential** | 50% | Fundamentals enabling basic AI functionality | 5 attributes |
| **Tier 2: Critical** | 30% | Major quality improvements and safety nets | 6 attributes |
| **Tier 3: Important** | 15% | Significant improvements in specific areas | 9 attributes |
| **Tier 4: Advanced** | 5% | Refinement and optimization | 5 attributes |
| **Tier 1: Essential** | 55% | Fundamentals enabling basic AI functionality | 9 attributes |
| **Tier 2: Critical** | 27% | Major quality improvements and safety nets | 9 attributes |
| **Tier 3: Important** | 14% | Significant improvements in specific areas | 5 attributes |
| **Tier 4: Advanced** | 4% | Refinement and optimization | 4 attributes |
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Comment thread
coderabbitai[bot] marked this conversation as resolved.

**Impact**: Missing a Tier 1 attribute (10% weight) has **10x the impact** of missing a Tier 4 attribute (1% weight).

---

## Tier 1: Essential Attributes

*Fundamentals that enable basic AI agent functionality — 50% of total score*
*Fundamentals that enable basic AI agent functionality — 55% of total score*

### 1. CLAUDE.md Configuration File

**ID**: `claude_md_file`
**Weight**: 10%
**Weight**: 7%
**Category**: Context Window Optimization
**Status**: ✅ Implemented

Expand Down Expand Up @@ -631,14 +631,10 @@ go mod tidy

---

## Tier 2: Critical Attributes

*Major quality improvements and safety nets — 30% of total score*
### 6. Test Execution & Coverage

### 6. Test Coverage

**ID**: `test_coverage`
**Weight**: 5%
**ID**: `test_execution`
**Weight**: 10%
**Category**: Testing & CI/CD
**Status**: ✅ Implemented
Comment thread
coderabbitai[bot] marked this conversation as resolved.

Expand Down Expand Up @@ -690,10 +686,14 @@ go tool cover -html=coverage.out

---

### 7. Pre-commit Hooks & CI/CD Linting
## Tier 2: Critical Attributes

**ID**: `precommit_hooks`
**Weight**: 5%
*Major quality improvements and safety nets — 27% of total score*

### 7. Deterministic Enforcement (Hooks & Lint Rules)

**ID**: `deterministic_enforcement`
**Weight**: 3%
**Category**: Testing & CI/CD
**Status**: ✅ Implemented

Expand Down
114 changes: 101 additions & 13 deletions scripts/research_config.yaml
Original file line number Diff line number Diff line change
@@ -1,40 +1,128 @@
# Research Update Configuration
# Controls weekly automated updates to agent-ready-codebase-attributes.md
# Controls automated updates to RESEARCH_REPORT.md

update_settings:
# Maximum number of attribute updates per weekly run
# Keeps PRs reviewable and focused
# Maximum number of attribute updates per run
max_updates_per_run: 5

# Minimum Claude-rated relevance score (0.0-1.0)
# Higher = more selective about what gets included
min_citation_quality_score: 0.7

# Only include research from last N months
search_recency_months: 12

# Tier 1 attributes get priority in updates
# These are processed first before other attributes
# Maximum total results returned from all sources combined
max_total_results: 15

# Priority attributes — processed first (use section numbers from report)
# Updated in v2.0.0 to reflect evidence-based priorities
priority_attributes:
- "1.1" # CLAUDE.md Configuration Files
- "2.1" # README Structure
- "3.3" # Type Annotations (Static Typing)
- "5.1" # Test Coverage Requirements
- "7.1" # Conventional Commit Messages
- "5.1" # Test Execution & Coverage (highest priority)
- "16.1" # Single-File Verification
- "16.2" # CI Quality Gates
- "1.1" # CLAUDE.md/AGENTS.md Configuration Files
- "3.3" # Type Annotations
- "17.1" # Pattern References

# Search domain preferences
search_domains:
# Prioritized sources (high authority)

# RSS/Atom feed sources — structured, date-sorted, richest data
rss_feeds:
- name: "GitHub Blog"
domain: github.blog
feed_url: "https://github.blog/feed/"
format: rss
max_results: 5
relevance_keywords: ["copilot", "AI", "agent", "code", "LLM", "developer"]

- name: "Martin Fowler"
domain: martinfowler.com
feed_url: "https://martinfowler.com/feed.atom"
format: atom
max_results: 5
relevance_keywords: ["AI", "agent", "coding", "development", "testing", "refactoring"]

- name: "Microsoft Research"
domain: microsoft.com
feed_url: "https://microsoft.com/research/feed/"
format: rss
max_results: 5
relevance_keywords: ["AI", "agent", "code", "LLM", "software", "developer"]

# Sitemap-based sources — discover URLs, fetch metadata from HTML
sitemap_sources:
- name: "Anthropic Engineering"
domain: anthropic.com
sitemap_url: "https://www.anthropic.com/sitemap.xml"
path_filters: ["/engineering/", "/research/"]
max_results: 5
relevance_keywords: ["agent", "claude", "code", "tool", "context", "harness"]

- name: "OpenAI Research"
domain: openai.com
sitemap_url: "https://openai.com/sitemap.xml/research/"
path_filters: ["/index/"]
max_results: 5
relevance_keywords: ["agent", "code", "model", "developer"]

- name: "Cursor Blog"
domain: cursor.com
sitemap_url: "https://cursor.com/sitemap.xml"
path_filters: ["/blog/"]
max_results: 5
relevance_keywords: ["agent", "AI", "code", "editor", "model"]

- name: "Factory AI"
domain: factory.ai
sitemap_url: "https://factory.ai/sitemap.xml"
path_filters: ["/news/"]
max_results: 5
relevance_keywords: ["agent", "code", "droid", "benchmark", "review"]

- name: "Google Research"
domain: research.google
sitemap_url: "https://research.google/sitemap.xml"
path_filters: ["/pubs/"]
max_results: 5
relevance_keywords: ["AI", "agent", "code", "LLM", "software"]

# Curated static URLs — content rarely changes, checked periodically
curated_sources:
- name: "OpenSSF Best Practices"
domain: best.openssf.org
urls:
- "https://best.openssf.org/Concise-Guide-for-Developing-More-Secure-Software"
- "https://best.openssf.org/npm-best-practices"
source_label: "OpenSSF"

- name: "AGENTS.md Specification"
domain: agents.md
urls:
- "https://agents.md"
source_label: "AGENTS.md"

- name: "Claude Code Documentation"
domain: docs.anthropic.com
urls:
- "https://docs.anthropic.com/en/docs/claude-code/overview"
source_label: "Anthropic Docs"

# Prioritized domains — used for result ranking and citation filtering
prioritized:
- anthropic.com
- code.claude.com
- microsoft.com/research
- research.google
- arxiv.org
- dl.acm.org
- openai.com/research
- github.blog
- martinfowler.com
- thoughtworks.com
- agents.md
- cursor.com
- factory.ai
- best.openssf.org

# Blocked domains (low quality, spam)
blocked:
Expand Down
Loading
Loading