Skip to content

chore: Try concurrent integration tests #848

@jirispilka

Description

@jirispilka

Problem

Integration tests are slow to iterate quickly.

tests/integration/suite.ts sets concurrent: false on the outer describe
("Make all tests sequential to prevent state interference"). Vitest already runs
the three integration files in parallel, but inside each file ~10–15 it()s
that each start a real Actor run execute serially, dominating wall time.

Proposed solution

Two-line change. Audit found only one nested describe with cross-it() state.

// suite.ts:257
describe(suiteName, { concurrent: true, ... }, () => { ... });

// suite.ts:1879 — beforeAll captures datasetId/runId reused by 4 it()s
describe('rag-web-browser run reads via storage tools', { concurrent: false }, () => { ... });

Everything else is already per-test: client is created in beforeEach,
stdio spawns one subprocess per client, streamable-http supports concurrent
sessions.

Plan

  • Apply the two-line change on a branch
  • Measure wall time before/after locally
  • Run CI 3× to check for flakes
  • Only merge if all 3 are green

Risks — why this needs testing, not just merging

  • Not certain it works.
  • Flakes. Any test that implicitly assumed exclusive server access will surface.
  • Runner memory. Each concurrent stdio test spawns a Node subprocess; only
    enable if CI runner has headroom.
  • Heavy Actors. Most call-actor tests use apify/rag-web-browser. We need to increase RAM at our accounts

Metadata

Metadata

Assignees

No one assigned

    Labels

    t-aiIssues owned by the AI team.

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions