Detect outdated Orca agent skills and prompt per-skill updates#8747
Detect outdated Orca agent skills and prompt per-skill updates#8747tebayoso wants to merge 13 commits into
Conversation
App updates do not refresh globally installed agent skills, so agents can keep stale workflow docs after an Orca upgrade. Compare installed SKILL.md hashes against app-bundled references, surface one UpdateCard-style prompt per outdated skill, and show amber Outdated badges in Settings.
tebayoso
force-pushed
the
feat/outdated-skills-update-prompts
branch
from
abfa3e8 to
7c524ac
Compare
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds bundled Orca skill references and a freshness engine that hashes managed 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
@coderabbitai review |
✅ Action performedReview finished.
|
|
@coderabbitai help |
ChatThere are 3 ways to chat with CodeRabbit:
CodeRabbit commands
Other keywords and placeholders
CodeRabbit configuration file (
|
|
@coderabbitai review |
✅ Action performedReview finished.
|
Only count global/home installs for freshness, key the renderer cache by discovery target, avoid Escape stealing focus from dialogs/inputs, and cover the repo-local false positive with a regression test.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 6
🧹 Nitpick comments (1)
src/renderer/src/components/skills/OutdatedSkillUpdateHost.tsx (1)
42-47: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueSimplify the conditional logic.
The nested conditional block can be flattened for better readability.
♻️ Proposed refactor
- if (loading || promptQueue.length === 0) { - if (!loading && promptQueue.length === 0) { - setActiveSkillName(null) - } - return - } + if (loading) { + return + } + if (promptQueue.length === 0) { + setActiveSkillName(null) + return + }
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: 7b04a667-2926-4249-8063-b7dc785b0cd8
📒 Files selected for processing (27)
config/electron-builder.config.cjssrc/main/ipc/skills.test.tssrc/main/ipc/skills.tssrc/main/runtime/rpc/methods/skills.tssrc/main/skills/bundled-skills-root.tssrc/main/skills/freshness.test.tssrc/main/skills/freshness.tssrc/preload/api-types.tssrc/preload/index.tssrc/renderer/src/App.tsxsrc/renderer/src/components/settings/AgentSkillSetupPanel.tsxsrc/renderer/src/components/settings/CliSection.tsxsrc/renderer/src/components/settings/ComputerUseSkillSetupPanel.tsxsrc/renderer/src/components/settings/EphemeralVmsPane.tsxsrc/renderer/src/components/settings/OrchestrationSetupCard.tsxsrc/renderer/src/components/settings/Settings.tsxsrc/renderer/src/components/settings/SettingsSidebar.test.tsxsrc/renderer/src/components/settings/SettingsSidebar.tsxsrc/renderer/src/components/skills/OutdatedSkillUpdateDialog.tsxsrc/renderer/src/components/skills/OutdatedSkillUpdateHost.tsxsrc/renderer/src/components/skills/outdated-skill-reminder.test.tssrc/renderer/src/components/skills/outdated-skill-reminder.tssrc/renderer/src/hooks/useOrcaSkillFreshness.tssrc/renderer/src/lib/settings-navigation-types.tssrc/renderer/src/web/web-preload-api.tssrc/shared/orca-managed-skills.tssrc/shared/skill-freshness.ts
|
@coderabbitai review Please re-review the latest commit (a620dcc) which hardens freshness detection after the prior review attempt failed mid-rebase. |
|
The PR description image ("outdated-skills-settings-and-card") shows a macOS desktop app settings panel titled around "Orca CLI". Key details visible:
This matches the PR's described UI additions (Outdated badges, status pills, and the UpdateCard-style bottom-right prompt).
✅ Action performedReview finished.
|
Reject oversized SKILL.md prefixes, treat unreadable installs as unknown, scope freshness only to home installs, key the renderer cache by discovery target, drop global Escape dismiss, and use hash-based i18n keys.
|
@coderabbitai review Addressed the prior review comments:
|
|
The PR description image shows a macOS desktop app settings panel. The left sidebar lists sections including "Orchestration" with an amber "Outdated" badge, "Computer Use" with a green "Installed" badge, "Voice" with a gray "Not installed" badge, and under "SET UP", "General" also shows an amber "Outdated" badge. The main panel shows "Orca CLI" with a "Shell command" field (
✅ Action performedReview finished.
|
|
Sorry about the noise, my automated PR review tool decided to fight CodeRabbit in a loop, won't happen again. I apologize. |
Serialize forced refreshes per discovery target, cover packaged/dev reference root resolution, and keep the prior CodeRabbit correctness fixes covered by regression tests.
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
src/renderer/src/components/skills/OutdatedSkillUpdateDialog.tsx (1)
54-61: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winGive each skill prompt a unique heading ID.
Multiple outdated skills render multiple cards, but each uses
outdated-skill-update-heading. Duplicate IDs makearia-labelledbyambiguous for assistive technology.Proposed fix
const { skill, onDismiss, onUpdate } = props + const headingId = `outdated-skill-update-heading-${skill.skillName}` <Card className="gap-0 py-0" role="complementary" - aria-labelledby="outdated-skill-update-heading" + aria-labelledby={headingId} > - <h3 id="outdated-skill-update-heading" className="text-sm font-semibold"> + <h3 id={headingId} className="text-sm font-semibold">
🧹 Nitpick comments (1)
src/main/skills/freshness.ts (1)
22-30: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winUse Node path utilities instead of parsing separators manually.
Replace
split(/[\\/]/)withnode:pathhelpers so path semantics remain centralized and platform-safe.As per coding guidelines, use
path.joinor Electron/Node path utilities for file paths; never assume/or\.Source: Coding guidelines
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: a5e9f722-ed63-4938-808c-7dd67ed573dd
📒 Files selected for processing (6)
src/main/skills/freshness.test.tssrc/main/skills/freshness.tssrc/renderer/src/components/settings/AgentSkillSetupPanel.tsxsrc/renderer/src/components/settings/SettingsSidebar.tsxsrc/renderer/src/components/skills/OutdatedSkillUpdateDialog.tsxsrc/renderer/src/hooks/useOrcaSkillFreshness.ts
🚧 Files skipped from review as they are similar to previous changes (3)
- src/renderer/src/components/settings/SettingsSidebar.tsx
- src/renderer/src/components/settings/AgentSkillSetupPanel.tsx
- src/renderer/src/hooks/useOrcaSkillFreshness.ts
There was a problem hiding this comment.
Actionable comments posted: 1
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: 239d8c79-4c78-44a2-9be5-850557b59b75
📒 Files selected for processing (2)
src/main/skills/bundled-skills-root.test.tssrc/renderer/src/hooks/useOrcaSkillFreshness.ts
🚧 Files skipped from review as they are similar to previous changes (1)
- src/renderer/src/hooks/useOrcaSkillFreshness.ts
Loop-read skill files to EOF with an oversize guard, clean temp dirs in tests, give each outdated-skill card a unique heading id, and simplify the prompt queue effect.
There was a problem hiding this comment.
🧹 Nitpick comments (2)
src/renderer/src/components/skills/OutdatedSkillUpdateDialog.tsx (2)
23-23: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueUse
useId()for accessibility IDs.React provides the
useIdhook specifically to generate stable, unique IDs for accessibility attributes. UsinguseId()is the idiomatic way to handle ARIA ID linking and avoids potential collisions if multiple instances of this component ever render simultaneously.♻️ Proposed refactor
- const headingId = `outdated-skill-update-heading-${skill.skillName}` + const headingId = React.useId()(Note: Ensure
ReactoruseIdis imported as needed).
55-55: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueConsider a more appropriate ARIA role.
The
complementaryrole creates a document landmark (similar to an<aside>), which is typically reserved for supporting sections in the main document flow like a sidebar. For a floating notification or card,role="region",role="status", orrole="dialog"is generally more semantically appropriate.♻️ Proposed refactor
- <Card className="gap-0 py-0" role="complementary" aria-labelledby={headingId}> + <Card className="gap-0 py-0" role="region" aria-labelledby={headingId}>
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: c7f77721-1d09-4e6d-ba28-e259b27f916d
📒 Files selected for processing (5)
src/main/skills/bundled-skills-root.test.tssrc/main/skills/freshness.test.tssrc/main/skills/freshness.tssrc/renderer/src/components/skills/OutdatedSkillUpdateDialog.tsxsrc/renderer/src/components/skills/OutdatedSkillUpdateHost.tsx
🚧 Files skipped from review as they are similar to previous changes (4)
- src/main/skills/bundled-skills-root.test.ts
- src/main/skills/freshness.test.ts
- src/main/skills/freshness.ts
- src/renderer/src/components/skills/OutdatedSkillUpdateHost.tsx
Flag outdated when any home provider copy diverges, skip repo walks on the freshness path, coalesce forced renderer rescans, suppress re-prompts after an update attempt for the current app reference hash, and avoid Settings "Checking..." flicker from freshness loading.
Mark update-attempted only when Settings opens the update terminal (not the card click), skip project/cwd roots on freshness discovery, key the cache by host/WSL runtime only, and drop the unused freshness-changed notifier.
Record update-attempted from the card Update action for all managed skills, mark Settings terminals only after prerequisites succeed, and skip freshness IPC when project runtime requires repair.
There was a problem hiding this comment.
Actionable comments posted: 1
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Run ID: f2c8f281-bad1-4d89-ae36-3dddb8044f62
📒 Files selected for processing (20)
src/main/ipc/skills.test.tssrc/main/ipc/skills.tssrc/main/runtime/rpc/methods/skills.tssrc/main/skills/discovery.tssrc/main/skills/freshness.test.tssrc/main/skills/freshness.tssrc/main/skills/skill-discovery-sources.tssrc/renderer/src/components/settings/CliSection.tsxsrc/renderer/src/components/settings/ComputerUseSkillSetupPanel.tsxsrc/renderer/src/components/settings/EphemeralVmsPane.tsxsrc/renderer/src/components/settings/OrchestrationSetupCard.tsxsrc/renderer/src/components/settings/Settings.tsxsrc/renderer/src/components/settings/cli-section-platform-copy.tssrc/renderer/src/components/skills/OutdatedSkillUpdateHost.tsxsrc/renderer/src/components/skills/mark-outdated-skill-update-attempt.tssrc/renderer/src/components/skills/outdated-skill-reminder.test.tssrc/renderer/src/components/skills/outdated-skill-reminder.tssrc/renderer/src/hooks/useOrcaSkillFreshness.tssrc/shared/orca-managed-skills.tssrc/shared/skill-freshness.ts
🚧 Files skipped from review as they are similar to previous changes (11)
- src/renderer/src/components/skills/outdated-skill-reminder.test.ts
- src/renderer/src/components/settings/OrchestrationSetupCard.tsx
- src/renderer/src/components/settings/ComputerUseSkillSetupPanel.tsx
- src/shared/skill-freshness.ts
- src/renderer/src/components/settings/EphemeralVmsPane.tsx
- src/main/ipc/skills.test.ts
- src/main/skills/freshness.test.ts
- src/main/ipc/skills.ts
- src/renderer/src/components/skills/OutdatedSkillUpdateHost.tsx
- src/main/skills/freshness.ts
- src/renderer/src/components/settings/Settings.tsx
Reset hook result/loading from the new target's cache on host/WSL switches so panels never flash the previous runtime's outdated/installed badges. Also use useId and role=region for the floating prompt a11y nits.
Apply dual-review consensus: Settings/pills honor dismiss+attempt flags, web checkFreshness propagates errors instead of caching empty success, match installs by directory basename only, and keep card Update as session-snooze only while terminals record the persistent attempt.
Use a generated hash catalog instead of shipping skills/ into the app bundle. Limit outdated detection to the four skills with Settings update rails, probe only managed home basenames, and leave discovery sources unchanged. Tests use temp dirs only.
The previous cleanup incorrectly reset discovery sources and tests to an outdated origin/main tip, dropping multi-agent home roots and the stablyai#8256/stablyai#8503 symlink coverage cases. Those belong to orchestration agent coverage, not the outdated-skills packaging change.
Record update-attempt suppression when the setup terminal closes instead of on open, stack the skill card above UpdateCard and StarNag, make hash catalog verify resilient to oxfmt quote rewrites, and cover the freshness hook for target switches, coalescing, and repair-required.
2 participants
Summary
After an Orca app update, globally installed agent skills can lag behind the skill docs that ship with the new build. Agents then keep outdated workflow guidance until someone runs
npx skills update …by hand.This change adds offline freshness checks against a generated hash catalog (not a packaged
skills/tree) and surfaces outdated skills in three places:Scope (intentional)
Only skills with an in-app update surface:
orca-cliorchestrationcomputer-useorca-per-workspace-envLinear and emulator skills keep existing install flows and are not prompted.
Only installed skills whose home copies diverge from the catalog hash are prompted—missing skills keep the install flow; current skills stay quiet.
Design
skills/: hashes insrc/shared/orca-skill-reference-hashes.generated.tsviapnpm run generate:orca-skill-reference-hashes(verified in lint).skillName + expectedHash.bottom-[220px]/bottom-[430px]).Related
npx skills updatelagScreenshots
Testing
pnpm devagainst outdated global installsupstream/main; forkmainsyncedpnpm lint/pnpm build(rely on PR CI)AI Review Report
Addressed multi-pass Claude / OpenCode / CodeRabbit review:
resources/orca-skillsSecurity Audit
skills:checkFreshnessonly reads localSKILL.mdunder known home skill paths vs compile-time catalog hashes; no network, no shell, no writeslocalStorageNotes