Skip to content

feat: per-connector schedules (targeted cron pause/resume/delete) - #438

Open
Himanshu Kumar (himanshu231204) wants to merge 5 commits into
langchain-ai:mainfrom
himanshu231204:feature/per-connector-schedules
Open

feat: per-connector schedules (targeted cron pause/resume/delete)#438
Himanshu Kumar (himanshu231204) wants to merge 5 commits into
langchain-ai:mainfrom
himanshu231204:feature/per-connector-schedules

Conversation

@himanshu231204

Copy link
Copy Markdown
Contributor

Summary

Implements per-connector schedules so openwiki cron pause|resume|delete <source> targets a single connector instead of all-or-nothing. Each connector gets its own independent launchd job, schedule state, and pause/resume/delete lifecycle.

Closes #399


Motivation

Today OpenWiki keeps a single unified schedule (config.ingestionSchedule) backed by one OS-level job. openwiki cron pause|resume|delete is all-or-nothing — you can't pause just one connector.

This matters in practice:

  • Temporarily silence one noisy or rate-limited source (e.g. pause Slack ingestion) while keeping the rest running
  • Stop an expensive connector without disabling the whole pipeline
  • Give different connectors different cadences (e.g. git-repo hourly, web-search daily)

There's also an inconsistency: openwiki ingest <source|all> is per-source (on-demand), so users reasonably expect cron to be too.


What Changed

Data Model & Migration (src/onboarding.ts)

  • Preserved sourceInstances[].schedule — the field already existed in types but was always deleted during normalization. Now it's preserved as the source of truth.
  • Migrated global schedule to per-source — when config.ingestionSchedule exists, it's copied to every source instance that doesn't already have its own schedule. Migration is transparent with no manual steps required.

Command Parser (src/commands.ts)

  • CronTarget type widened from ConnectorId | "all" to IngestionTarget (includes source instance IDs)
  • cron pause|resume|delete now accept <source|source-instance|all> instead of just all
  • Help text and examples updated

Schedule Operations (src/schedules.ts)

  • listConnectorSchedules — returns one entry per source instance with its own schedule state, instead of a single "all" entry
  • pauseConnectorSchedules — pauses individual instances by ID, connector type, or all
  • resumeConnectorSchedules — resumes individual instances with per-instance launchd reinstall
  • deleteConnectorSchedules — removes schedules from individual instances with per-instance plist cleanup
  • New helper functionsresolveScheduleTarget, getLaunchAgentLabelForInstance, getLaunchAgentPathForInstance, unloadLaunchAgentForInstance, removeLaunchAgentPlistForInstance, isLaunchAgentLoadedForInstance, cleanupGlobalPlist
  • ScheduleTarget updated to IngestionTarget to match the CLI type chain

Launchd Management (src/schedules.ts)

  • Per-connector plist files — labels changed from com.openwiki.ingestion to com.openwiki.ingestion.<source-instance-id>
  • Targeted ingestioncreateLaunchAgentPlist now runs ingest <connectorId> --scheduled instead of ingest all --scheduled
  • Global plist cleanup — automatically removes the old com.openwiki.ingestion.plist on migration

Ingestion Scheduling (src/ingestion.ts)

  • resolveIngestionSourceInstances now checks sourceConfig.schedule.pausedAt per-source instead of the global config.ingestionSchedule.pausedAt
  • A source instance without a schedule is excluded from scheduled runs
  • A source instance with a paused schedule is excluded; others remain active

Power Schedule (src/schedules.ts)

  • getPowerWindowForConfiguredSchedules iterates all sourceInstances[].schedule instead of the single global schedule
  • Power window calculated from union of all active per-source schedules
  • Paused instances excluded from wake window calculation

CLI Dispatch (src/cli.tsx)

  • runCronCommand validates source-instance targets exist in config before dispatching
  • formatScheduleMutationResult resolves source instance IDs to display names
  • resolveSourceInstanceLabel helper for human-readable output

Credentials (src/credentials.tsx)

  • Pass sourceInstanceId to installConnectorSchedule during onboarding setup

Acceptance Criteria

Criterion Status
openwiki cron pause web-search pauses only that connector; others keep running
cron list shows per-connector state
all still applies to every connector
Existing configs migrate with no manual steps
All tests pass
Launchd jobs are per-connector (com.openwiki.ingestion.<source-instance-id>)

Tests

Updated (test/commands.test.ts)

  • 4 existing tests updated from "rejected" to "accepted" for per-source targets
  • 3 new tests added: connector ID acceptance, invalid target rejection

New (test/schedules.test.ts)

  • 12 tests covering list/pause/resume/delete per-source operations
  • Tests for "all" target, connector ID target, source instance ID target
  • Tests for skipping instances without schedules

New (test/ingestion-scheduling.test.ts)

  • 5 tests covering per-source scheduledOnly filtering
  • Tests for: no schedule → excluded, paused → excluded, active → included, no connectedAt → excluded

Updated (test/onboarding.test.ts)

  • Tests for schedule migration from global to per-source
  • Tests for preserving existing per-source schedules
  • Tests for preserving schedule field during normalization

Migration Path

Existing single-schedule configs are handled automatically:

  1. User has config.ingestionSchedule (old format)
  2. On first config read, normalizeOnboardingConfig copies the global schedule to every source instance
  3. Per-source schedules become the source of truth
  4. Old global plist (com.openwiki.ingestion.plist) is cleaned up on first resume/install
  5. No manual intervention required

Files Changed

File Lines Description
src/schedules.ts +372/-108 Core schedule operations rewrite
src/commands.ts +37/-18 Command parser per-source support
src/cli.tsx +37/-14 CLI dispatch and output formatting
src/onboarding.ts +18/-9 Data model migration
src/ingestion.ts +14/-6 Per-source pause check
src/credentials.tsx +1 Source instance ID for install
test/schedules.test.ts +440 New per-connector tests
test/ingestion-scheduling.test.ts +345 New filtering tests
test/onboarding.test.ts +160 Migration tests
test/commands.test.ts +57/-13 Updated cron tests

…d promise rejection crash

Any tool error during agent execution crashed the entire run via
unhandled promise rejection in the langchain tool-call transformer.

Root cause: when wrapToolCall middleware is present, tool errors get
wrapped in MiddlewareError by the composition layer. ToolNode.#handleError
then re-throws them as fatal because handleToolErrors !== true. The
createToolCallTransformer.rejects output promises that nobody catches,
leading to UnhandledPromiseRejection -> process crash.

Fix: catch tool execution errors inside the wrapToolCall middleware and
convert them to ToolMessages before they reach the composition layer.
GraphInterrupt and AbortError are re-thrown as control flow. Also add
defensive error handling to addFrontmatterWarning so validation failures
don't crash the run.

Closes langchain-ai#426
Implement per-connector schedules so openwiki cron pause|resume|delete
<source> targets a single connector, not just all.

Changes:
- Data model: preserve sourceInstances[].schedule during normalization,
  migrate global ingestionSchedule to per-source instances
- Command parser: accept <source|source-instance|all> for cron commands
- Schedule operations: rewrite list/pause/resume/delete to operate on
  individual source instances with IngestionTarget support
- Launchd management: per-connector plist files with
  com.openwiki.ingestion.<source-instance-id> labels
- Ingestion scheduling: check per-source schedule.pausedAt instead of
  global config.ingestionSchedule
- Power schedule: iterate all active source schedules for wake window
- CLI dispatch: validate source-instance targets, show per-source results
- Tests: update existing cron tests, add per-connector schedule tests

Closes langchain-ai#399
@himanshu231204

Copy link
Copy Markdown
Contributor Author

Colin Francis (@colifran) could you please review and approve the PR, Thanks

@corridor-security corridor-security Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The per-connector schedule feature interpolates an unvalidated sourceInstanceId into filesystem paths; while a path traversal is technically possible via a crafted onboarding.json, exploitation is limited to the local user's own files with no privilege escalation, making this a low-severity hardening gap rather than a critical vulnerability.

Comment thread src/schedules.ts
@@ -182,10 +188,13 @@ export async function installConnectorSchedule({
);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The sourceInstanceId read from ~/.openwiki/onboarding.json reaches two path.join() calls in src/schedules.ts with no sanitization: the log path at line 182 and the plist filename at lines 1070-1074. A crafted ID like ../../.zshrc causes writes to resolve outside the intended directories when openwiki cron resume is run, overwriting arbitrary user-owned files with plist XML or log content.

// 1) Per-instance log path in the generated plist
logPath: path.join(logsDir, `ingestion.${sourceInstanceId}.schedule.log`),
// 2) Per-instance launchd plist filename
function getLaunchAgentPathForInstance(instanceId: string): string {
  return path.join(
    getLaunchAgentsDir(),
    `${getLaunchAgentLabelForInstance(instanceId)}.plist`,
  );
}

Remediation: Validate sourceInstanceId against a strict allowlist (e.g. /^[a-z][a-z0-9-]{0,63}$/) in normalizeOnboardingConfig and at the top of installConnectorSchedule, or perform a path.resolve-and-startsWith containment check on every constructed path before writing.

Attack Path
  1. Attacker writes crafted ~/.openwiki/onboarding.json with "id": "../../.zshrc".
  2. Victim runs openwiki cron resume.
  3. cli.tsx calls resumeConnectorSchedulesinstallConnectorSchedule with the unsanitized ID.
  4. getLaunchAgentPathForInstance builds ~/Library/LaunchAgents/com.openwiki.ingestion.../../.zshrc.plist → resolves to ~/.zshrc.plist.
  5. writeFile writes plist XML to that path; launchd later appends log output to the traversed log path.

For more details, see the finding in Corridor.

Provide feedback: Reply with whether this is a valid vulnerability or false positive to help improve Corridor's accuracy.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: per-connector schedules (targeted cron pause/resume/delete)

1 participant