Skip to content

fix(test): stop tests from registering xctest as a real login item - #134

Merged
kazukinakai merged 1 commit into
mainfrom
fix/smappservice-test-isolation
Aug 6, 2026
Merged

fix(test): stop tests from registering xctest as a real login item#134
kazukinakai merged 1 commit into
mainfrom
fix/smappservice-test-isolation

Conversation

@kazukinakai

Copy link
Copy Markdown
Member

Incident

While running swift test repeatedly across worktrees today, testBootstrapWithStoredTrueAndServiceNotFoundRegistersInstead (pre-existing, unrelated to today's feature work) called AppSettings.bootstrap() against the real SMAppService.mainApp. For the .notFound status a bare swift test process reliably reports, bootstrap() calls setLaunchAtStartup(true)SMAppService.mainApp.register() — for real. That registered the xctest test-runner binary itself as a macOS login item. The test's own existing comment already said as much ("this only exercises whatever status the real API reports... .notFound (confirmed empirically)") — it just didn't treat the side effect as a defect.

Confirmed via sfltool dumpbtm: entry #74, Bundle Identifier: com.apple.dt.xctest.tool, Disposition: [enabled, allowed, not notified].

Cleanup already done (outside this diff)

Unregistered through the same code path — a one-off test called SMAppService.mainApp.unregister() directly — and confirmed removed:

  • sfltool dumpbtm re-read: Disposition flipped to [disabled, allowed, not notified]
  • Live API check: SMAppService.mainApp.status.rawValue == 0 (.notRegistered)

(A second, unrelated CmdIME.app entry also appears in dumpbtm — that's the user's real installed app's legitimate "Launch at Login" registration; not touched.)

Fix

  • toggleLaunchAtStartup.swift: new LoginItemService protocol seam over SMAppService.mainApp (status/register()/unregister()). SMAppService conforms for free via an extension. setLaunchAtStartup(_:service:) now takes an injectable service, defaulting to the real SMAppService.mainApp — production behavior unchanged.
  • AppSettings: gains an injectable loginItemService (same default), used everywhere it previously reached for SMAppService.mainApp directly (init, bootstrap(), the $launchAtStartup sink).
  • FakeLoginItemService (test-only): in-memory spy, so tests never touch the real registry.
  • AppSettingsTests: the offending test now injects a fake with .notFound status and asserts registerCallCount == 1 instead of relying on (and mutating) real system state. Also added testBootstrapWithStoredTrueAndRequiresApprovalFollowsOSStateInstead — the .requiresApproval branch was previously unreachable from a real test process and had zero coverage; the fake makes it directly testable.
  • SMAppServiceTestIsolationTests (new): regression guard that scans every test file's source for the literal string SMAppService.mainApp and fails if found (excluding the fake and itself), so this class of bug can't be silently reintroduced.

Acceptance

$ swift build   # Build complete
$ swift test    # 53 tests, 1 skip (pre-existing diagnostic dump test), 0 failures

Re-checked sfltool dumpbtm after this full test run: the xctest entry's Disposition/Generation are unchanged from the cleaned-up state — confirms no new real registration occurs.

Incident: testBootstrapWithStoredTrueAndServiceNotFoundRegistersInstead
called AppSettings.bootstrap() on the real SMAppService.mainApp, which
for the .notFound status a bare `swift test` process reports calls
setLaunchAtStartup(true) -> SMAppService.mainApp.register() for real.
That registered the xctest test-runner binary itself as a macOS login
item (confirmed via `sfltool dumpbtm`: entry #74, Bundle Identifier
com.apple.dt.xctest.tool, Disposition enabled). The test's own comment
already flagged this as hitting the real API; it just didn't treat
that as a defect. Unregistered via the same code path (a one-off test
calling SMAppService.mainApp.unregister()), confirmed removed by
re-reading dumpbtm (status flipped to disabled) and by the live API
(SMAppService.mainApp.status.rawValue == 0, .notRegistered).

Fix: introduce a LoginItemService protocol seam over SMAppService.mainApp
(toggleLaunchAtStartup.swift) and inject it through AppSettings instead
of reaching for the singleton directly. Production code is unaffected —
the default parameter value is still SMAppService.mainApp.

Tests now use FakeLoginItemService, an in-memory spy, so no test can
reach the real OS-level registry. This also made the .requiresApproval
branch of bootstrap() reachable and tested for the first time (it was
previously unreachable from a real swift test process and had zero
coverage — added testBootstrapWithStoredTrueAndRequiresApprovalFollowsOSStateInstead).

Added SMAppServiceTestIsolationTests as a regression guard: it scans
every test file's source for the literal string "SMAppService.mainApp"
and fails if found (excluding the fake and itself), so this class of
bug can't be silently reintroduced.
kazukinakai added a commit that referenced this pull request Aug 6, 2026
Ported from fix/smappservice-test-isolation (PR #134): this branch's
AppSettingsTests.swift still had the pre-fix version, which calls the
real SMAppService.mainApp.register() and re-registered the xctest
test-runner as a login item after this branch's own test runs.
See PR #134 for the full incident writeup.
kazukinakai added a commit that referenced this pull request Aug 6, 2026
Ported from fix/smappservice-test-isolation (PR #134): this branch's
AppSettingsTests.swift still had the pre-fix version, which calls the
real SMAppService.mainApp.register() and re-registered the xctest
test-runner as a login item after this branch's own test runs.
See PR #134 for the full incident writeup.
@kazukinakai
kazukinakai merged commit a68028e into main Aug 6, 2026
18 of 20 checks passed
@kazukinakai
kazukinakai deleted the fix/smappservice-test-isolation branch August 6, 2026 15:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant