Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,7 @@ swift build
# Produces .build/debug/Parley and .build/debug/audio-capture-helper-xpc

swift test --filter TranscriberTests -Xswiftc -F/Library/Developer/CommandLineTools/Library/Developer/Frameworks/ -Xlinker -rpath -Xlinker /Library/Developer/CommandLineTools/Library/Developer/Frameworks/ -Xlinker -rpath -Xlinker /Library/Developer/CommandLineTools/Library/Developer/usr/lib/
# 874 tests across 99 suites (Config, ConfigManager, EngineID, WavFileWriter, AppState, FilenameUtils, CalendarEventPicker, PermissionManager, AudioDeviceEnumerator, InputLevelMonitor, RecordingSentinel, LaunchAgentManager, DiscoverSegments, SegmentNaming, SpeakerAssignment, SpeakerBoundarySplitTests, SpeakerReconciler, TranscriptMerger, ChunkSession, ChunkRecovery, AudioConverter, VadSpeechMap, ChunkRotator, ChunkProcessor, CLIParser, RecordingTimer, PathDisplay, OpenAISummaryProvider, LMStudioSummaryProvider, MeetingSummarizer, TokenRatioCache, EchoDeduplicator, etc.)
# 881 tests across 100 suites (Config, ConfigManager, EngineID, WavFileWriter, AppState, FilenameUtils, CalendarEventPicker, PermissionManager, AudioDeviceEnumerator, InputLevelMonitor, RecordingSentinel, LaunchAgentManager, DiscoverSegments, SegmentNaming, SpeakerAssignment, SpeakerBoundarySplitTests, SpeakerReconciler, TranscriptMerger, ChunkSession, ChunkRecovery, AudioConverter, VadSpeechMap, ChunkRotator, ChunkProcessor, CLIParser, RecordingTimer, PathDisplay, OpenAISummaryProvider, LMStudioSummaryProvider, MeetingSummarizer, TokenRatioCache, EchoDeduplicator, etc.)
# Uses Swift Testing, not XCTest -- no Xcode installed, only CommandLineTools
# Test path: SwiftTests/TranscriberTests/ (not Tests/ -- case collision with Python tests/ on APFS)
```
Expand Down Expand Up @@ -165,12 +165,12 @@ fault outright.
- [docs/development-process.md](docs/development-process.md) -- How work gets from idea to release; when to bump MINOR vs PATCH
- [docs/pipeline.md](docs/pipeline.md) -- End-to-end pipeline: recording → transcription → echo dedup → summary
- [docs/parameters.md](docs/parameters.md) -- All tunable parameters with config keys and defaults
- [docs/gotchas.md](docs/gotchas.md) -- 64 platform-specific gotchas
- [docs/gotchas.md](docs/gotchas.md) -- 66 platform-specific gotchas
- [docs/mic-capture-design.md](docs/mic-capture-design.md) -- Mic capture API choice (AVCaptureSession + Core Audio HAL) + auto-follow-default direction + when to revisit AVAudioEngine
- [docs/benchmarks/](docs/benchmarks/) -- Dated benchmark reports

## Key Gotchas
See [docs/gotchas.md](docs/gotchas.md) -- 64 platform-specific gotchas (macOS APIs, ScreenCaptureKit, XPC, audio formats, TCC, Liquid Glass, engine quirks). New items are appended there.
See [docs/gotchas.md](docs/gotchas.md) -- 66 platform-specific gotchas (macOS APIs, ScreenCaptureKit, XPC, audio formats, TCC, Liquid Glass, engine quirks). New items are appended there.

## Debugging
See [docs/pipeline.md](docs/pipeline.md#debugging) for full unified logging reference.
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
![macOS](https://img.shields.io/badge/macOS-15%2B-black?logo=apple)
![Apple Silicon](https://img.shields.io/badge/Apple%20Silicon-M1–M5-black?logo=apple)
![Swift](https://img.shields.io/badge/Swift-5.9%2B-orange?logo=swift)
![Tests](https://img.shields.io/badge/tests-874%20passing-brightgreen)
![Tests](https://img.shields.io/badge/tests-881%20passing-brightgreen)
![Cloud](https://img.shields.io/badge/cloud-none-success)
![License](https://img.shields.io/badge/license-AGPL%203.0-blue)

Expand Down Expand Up @@ -201,7 +201,7 @@ TranscriberApp/ SwiftUI menu-bar app (MenuBarExtra + Settings), XPC clien
TranscriberCore/ engines, diarization, echo-dedup, speaker reconciliation, summaries, transcript I/O
AudioCaptureHelper/ XPC audio-capture service (ScreenCaptureKit, dual-stream)
AudioCaptureProtocol/ shared @objc XPC protocol
SwiftTests/ 874 tests across 99 suites
SwiftTests/ 881 tests across 100 suites
tools/engine-benchmark/ WER + speed benchmark harness
docs/ architecture, pipeline, parameters, gotchas
```
Expand Down
56 changes: 56 additions & 0 deletions SwiftTests/TranscriberTests/PadRatioMonitorTests.swift
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,62 @@ import Testing
#expect(firedAtSecond <= 45, "detection should stay prompt, fired at \(firedAtSecond)s")
}

// MARK: - THE 2026-08-11 FALSE POSITIVE (device-observed)

/// The first real-world firing of this detector was WRONG, and the recording was healthy.
///
/// A call recorded on speaker: capture started at 11:59:44, the call audio began ~60s later.
/// Measured zero-sample ratio on the system track — 97.6% in 0-30s, 2.8% from 60-300s, 4.2%
/// overall. Nothing was wrong with it. But the monitor judged at exactly t=30s and saw 27s of
/// padding in 30s (ratio 0.911), clearing both the ratio threshold and the 15s absolute floor.
///
/// The cause is a property of the Core Audio tap that SCK does not share: the tap delivers no
/// buffers at all while the output device is idle. Before the call connects there is genuinely
/// nothing to capture, so the padder fills wall clock and every one of those frames counts as
/// "fabricated". Starting a recording before joining the call is the single most common way to
/// use this app, so this fires on the ordinary case.
///
/// The distinction that matters: padding BEFORE a track has ever delivered data is a start
/// offset, not a deficit. Only once frames are flowing does missing frames mean something.
@Test func leadingSilenceBeforeTheCallStartsIsNotCorruption() {
var monitor = PadRatioMonitor()
var fired: PadRatioMonitor.Verdict?
// 60s of pure padding — the tap delivers nothing while the output device is idle.
for _ in 0..<60 {
let v = monitor.record(padFrames: 48_000, dataFrames: 0, rate: Self.rate())
if case .excessive = v, fired == nil { fired = v }
}
// Then the call connects and audio flows normally for 5 minutes.
for _ in 0..<300 {
let v = monitor.record(padFrames: 0, dataFrames: 48_000, rate: Self.rate())
if case .excessive = v, fired == nil { fired = v }
}
#expect(fired == nil, "a recording started before the call must not be called corrupted")
}

/// The corollary that must still hold: once a track HAS delivered data, a sustained deficit is
/// real and must fire. This is the 2026-08-04 shape — the tap was delivering (a call was
/// connected) at roughly half the declared rate.
@Test func deficitAfterDataStartsStillFires() {
var monitor = PadRatioMonitor()
var fired: PadRatioMonitor.Verdict?
// Leading silence first — must be ignored, not merely diluted.
for _ in 0..<60 {
_ = monitor.record(padFrames: 48_000, dataFrames: 0, rate: Self.rate())
}
// Now frames flow, but only half of what the declared rate implies.
for _ in 0..<60 {
let v = monitor.record(padFrames: 24_000, dataFrames: 24_000, rate: Self.rate())
if case .excessive = v, fired == nil { fired = v }
}
guard case .excessive(let ratio, _, _) = fired else {
Issue.record("a real deficit after data starts must still fire, got \(String(describing: fired))")
return
}
// The leading silence must not inflate the ratio either — this is ~0.5, not ~0.8.
#expect(ratio > 0.45 && ratio < 0.55, "leading silence must be excluded from the ratio, got \(ratio)")
}

// MARK: - Warm-up

/// Leading silence dominates the ratio before any real audio arrives, so a verdict must wait for
Expand Down
62 changes: 62 additions & 0 deletions SwiftTests/TranscriberTests/SummaryErrorMessageTests.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
import Testing
import Foundation
@testable import TranscriberCore

/// What the user is TOLD when a summary fails.
///
/// This has been wrong twice in production. A `-1001` request timeout fell into a catch-all that
/// assumed any non-`SummaryError` was a file failure and reported "check disk space and
/// permissions" — sending two separate debugging sessions after a permissions problem that did not
/// exist (gotcha #66). A message that misdescribes the cause is worse than a vague one, because it
/// is actively followed.
@Suite struct SummaryErrorMessageTests {

// MARK: - Auth failures must be actionable and must not leak the credential

@Test func authFailureNamesTheApiKeyAndTheStatus() {
let message = SummaryError.authenticationFailed(status: 401).localizedDescription
#expect(message.contains("API key"))
#expect(message.contains("401"))
}

/// The response body is deliberately NOT included: some servers echo the offending credential
/// back, and this text reaches a notification that can be on screen during a shared meeting.
/// The same concern already sanitizes `invalidEndpoint`.
@Test func authFailureNeverEchoesAServerBody() {
let leaky = "Bearer sk-lm-SECRETVALUE is not authorized"
let message = SummaryError.authenticationFailed(status: 403).localizedDescription
#expect(!message.contains("SECRETVALUE"))
#expect(!message.contains(leaky))
#expect(!message.lowercased().contains("bearer"))
}

@Test func forbiddenIsTreatedAsAnAuthFailureToo() {
#expect(SummaryError.authenticationFailed(status: 403).localizedDescription.contains("API key"))
}

// MARK: - The distinctions that were missing

/// Three different causes, three different instructions. Previously a timeout and an unreachable
/// server both produced the file-permissions message.
@Test func theThreeFailureKindsReadDifferently() {
let auth = SummaryError.authenticationFailed(status: 401).localizedDescription
let endpoint = SummaryError.invalidEndpoint("http://x").localizedDescription
let server = SummaryError.serverError(message: "model failed to load", code: nil).localizedDescription

#expect(auth != endpoint)
#expect(auth != server)
#expect(endpoint != server)
// And none of them blames the filesystem.
for m in [auth, endpoint, server] {
#expect(!m.lowercased().contains("disk space"))
}
}

/// The provider's own message survives for server-side faults — "model failed to load" is
/// exactly the kind of detail that makes a failure diagnosable (#134).
@Test func serverErrorForwardsTheProvidersOwnText() {
let message = SummaryError.serverError(message: "model failed to load", code: "500")
.localizedDescription
#expect(message.contains("model failed to load"))
}
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The suite title says "What the user is TOLD when a summary fails" and the PR narrative is about the wrong message reaching the user — but the URLError paths added to MeetingSummarizer.runSummary (.timedOut, .cannotConnectToHost / .cannotFindHost, and the default fallback) have no tests here.

Those three messages are the literal fix for bug #2. If runSummary's catch order is later reshuffled, or a new SummaryError case accidentally matches before URLError, the old misdirecting message can come back with nothing failing.

Suggested additions — these can be async tests that call runSummary with a stub SummaryProvider that throws the target URLError:

@Test func timeoutYieldsActionableMessage() async {
    let outcome = await MeetingSummarizer.runSummary(
        transcriptPath: URL(fileURLWithPath: "/dev/null"),
        provider: ThrowingProvider(URLError(.timedOut)),
        endpoint: "http://localhost"
    )
    guard case .failed(let msg) = outcome else { Issue.record("expected failed"); return }
    #expect(msg.contains("too long"))
    #expect(!msg.lowercased().contains("disk space"))
    #expect(!msg.lowercased().contains("permissions"))
}

@Test func unreachableHostYieldsActionableMessage() async {
    let outcome = await MeetingSummarizer.runSummary(
        transcriptPath: URL(fileURLWithPath: "/dev/null"),
        provider: ThrowingProvider(URLError(.cannotConnectToHost)),
        endpoint: "http://localhost"
    )
    guard case .failed(let msg) = outcome else { Issue.record("expected failed"); return }
    #expect(msg.lowercased().contains("server") || msg.lowercased().contains("endpoint"))
    #expect(!msg.lowercased().contains("disk space"))
}

(ThrowingProvider is a one-liner conforming to SummaryProvider that just re-throws its stored error — no real network needed.)

11 changes: 10 additions & 1 deletion TranscriberCore/Config.swift
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,12 @@ public struct SummaryConfig: Codable, Equatable, Sendable {
public var contextOverheadPercent: Int?
/// Tokens reserved for the summary response (default 2048).
public var maxOutputTokens: Int?
/// Seconds to wait for the model to produce a summary (default 600).
///
/// URLSession's stock 60s is the wrong order of magnitude for a LOCAL model summarising a long
/// meeting: it accepts the request immediately, then generates for minutes. Two real recordings
/// failed at exactly 60s (#173) — the transcript was fine both times, only the summary was lost.
public var requestTimeoutSeconds: Int?

public init(
enabled: Bool,
Expand All @@ -34,7 +40,8 @@ public struct SummaryConfig: Codable, Equatable, Sendable {
model: String,
contextLength: Int? = nil,
contextOverheadPercent: Int? = nil,
maxOutputTokens: Int? = nil
maxOutputTokens: Int? = nil,
requestTimeoutSeconds: Int? = nil
) {
self.enabled = enabled
self.provider = provider
Expand All @@ -44,6 +51,7 @@ public struct SummaryConfig: Codable, Equatable, Sendable {
self.contextLength = contextLength
self.contextOverheadPercent = contextOverheadPercent
self.maxOutputTokens = maxOutputTokens
self.requestTimeoutSeconds = requestTimeoutSeconds
}

enum CodingKeys: String, CodingKey {
Expand All @@ -55,6 +63,7 @@ public struct SummaryConfig: Codable, Equatable, Sendable {
case contextLength = "context_length"
case contextOverheadPercent = "context_overhead_percent"
case maxOutputTokens = "max_output_tokens"
case requestTimeoutSeconds = "request_timeout_seconds"
}

public init(from decoder: Decoder) throws {
Expand Down
20 changes: 19 additions & 1 deletion TranscriberCore/LMStudioSummaryProvider.swift
Original file line number Diff line number Diff line change
Expand Up @@ -21,16 +21,24 @@ public struct LMStudioSummaryProvider: SummaryProvider, Sendable {
model: String,
contextLength: Int? = nil,
contextOverheadPercent: Int? = nil,
maxOutputTokens: Int? = nil
maxOutputTokens: Int? = nil,
requestTimeoutSeconds: Int? = nil
) {
self.endpoint = endpoint
self.apiKey = apiKey
self.model = model
self.contextLength = contextLength
self.overheadPercent = contextOverheadPercent ?? Self.defaultOverheadPercent
self.outputBuffer = maxOutputTokens ?? Self.defaultOutputBuffer
self.requestTimeoutSeconds = requestTimeoutSeconds ?? Self.defaultRequestTimeoutSeconds
}

/// Generous by design: a local model accepts the request instantly and then generates for
/// minutes on a long transcript. Ten minutes is far past any real generation while still
/// bounded, so a genuinely dead server still fails rather than hanging forever.
public static let defaultRequestTimeoutSeconds = 600
private let requestTimeoutSeconds: Int

public func summarize(segments: [SummarySegment], metadata: SummaryMetadata) async throws -> String {
// Calibrate on first encounter with this model
await calibrateIfNeeded()
Expand Down Expand Up @@ -58,6 +66,10 @@ public struct LMStudioSummaryProvider: SummaryProvider, Sendable {
return try await retryRequest(segments: segments, metadata: metadata)
}

// 401/403 has a precise fix and a body that may echo the credential — classify it.
if httpResponse.statusCode == 401 || httpResponse.statusCode == 403 {
throw SummaryError.authenticationFailed(status: httpResponse.statusCode)
}
throw SummaryError.requestFailed("HTTP \(httpResponse.statusCode): \(body.prefix(200))")
}

Expand Down Expand Up @@ -90,6 +102,9 @@ public struct LMStudioSummaryProvider: SummaryProvider, Sendable {

if let httpResponse = response as? HTTPURLResponse, !(200...299).contains(httpResponse.statusCode) {
let body = String(data: data, encoding: .utf8) ?? ""
if httpResponse.statusCode == 401 || httpResponse.statusCode == 403 {
throw SummaryError.authenticationFailed(status: httpResponse.statusCode)
}
throw SummaryError.requestFailed("HTTP \(httpResponse.statusCode) (after retry): \(body.prefix(200))")
}

Expand Down Expand Up @@ -156,6 +171,9 @@ public struct LMStudioSummaryProvider: SummaryProvider, Sendable {

var request = URLRequest(url: url)
request.httpMethod = "POST"
// A local model accepts instantly and then generates for minutes; the stock 60s timeout is
// the wrong order of magnitude and cost two real summaries (#173).
request.timeoutInterval = TimeInterval(requestTimeoutSeconds)
request.setValue("application/json", forHTTPHeaderField: "Content-Type")
if !apiKey.isEmpty {
request.setValue("Bearer \(apiKey)", forHTTPHeaderField: "Authorization")
Expand Down
33 changes: 27 additions & 6 deletions TranscriberCore/MeetingSummarizer.swift
Original file line number Diff line number Diff line change
Expand Up @@ -94,11 +94,30 @@ public enum MeetingSummarizer {
// sanitized in the branch above.
Logger.transcription.error("Summary generation failed: \(error.localizedDescription, privacy: .public)")
return .failed(error.localizedDescription)
} catch let error as URLError {
// NOT a file failure. This branch exists because a URLError fell through to the
// catch-all below and was reported as "check disk space and permissions" — which read as
// a permissions problem and sent two separate debugging sessions down the wrong path
// (2026-08-04 and 2026-08-11, both actually `-1001` request timeouts against a local
// LM Studio that accepted the request and then generated for longer than the timeout).
// An error message that misdescribes the fault is worse than a vague one.
Logger.transcription.error(
"Summary generation failed: \(error.code.rawValue, privacy: .public) \(error.localizedDescription, privacy: .public)"
)
switch error.code {
case .timedOut:
return .failed("The model took too long to respond — the transcript is safe; try a smaller model or raise the summary timeout")
case .cannotConnectToHost, .cannotFindHost, .networkConnectionLost, .notConnectedToInternet:
return .failed("Couldn't reach the summary endpoint — is your model server running?")
default:
return .failed("Summary request failed: \(error.localizedDescription)")
}
} catch {
// A non-SummaryError here is a file read/write failure (transcript unreadable, summary
// write failed). CocoaError's description embeds the transcript/session filename, which
// would surface in the notification (visible during a screen-share) — so keep the detail
// in the local log and give the user a generic, actionable message (#134 review).
// A non-SummaryError, non-URLError here is a file read/write failure (transcript
// unreadable, summary write failed). CocoaError's description embeds the
// transcript/session filename, which would surface in the notification (visible during a
// screen-share) — so keep the detail in the local log and give the user a generic,
// actionable message (#134 review).
Logger.transcription.error("Summary generation failed: \(error.localizedDescription)")
return .failed("Couldn't read the transcript or write the summary — check disk space and permissions")
}
Expand All @@ -114,13 +133,15 @@ public enum MeetingSummarizer {
model: summary.model,
contextLength: summary.contextLength,
contextOverheadPercent: summary.contextOverheadPercent,
maxOutputTokens: summary.maxOutputTokens
maxOutputTokens: summary.maxOutputTokens,
requestTimeoutSeconds: summary.requestTimeoutSeconds
)
case .openai:
return OpenAISummaryProvider(
endpoint: summary.endpoint,
apiKey: summary.apiKey,
model: summary.model
model: summary.model,
requestTimeoutSeconds: summary.requestTimeoutSeconds
)
}
}
Expand Down
Loading
Loading