Skip to content

Add a MIME recovery mode to salvage headers when full envelope parsing fails #507

Description

@jesserobbins

What I was doing

Working through #505/#506 (Date header parsing bugs) and #504 (repair-dates), I checked what's left after those fixes: 144 archived messages with missing/implausible sent dates, of which 47 fail full envelope parsing in mime.Parse (enmime.ReadEnvelope) outright — no Message is returned at all.

What I found

Of those 47, 33 have a perfectly clean, standard Date: header sitting in the raw bytes — a plain regex scan found it immediately. The envelope parse is failing on something else entirely (Failed to ReadParts: malformed MIME header initial line, mime: unexpected content after media subtype, unable to locate boundary param in Content-Type header), not because the headers themselves are unreadable.

But today, one bad part anywhere in the message causes the importer to throw away everything. IngestRawMessage replaces the entire parsed message with a placeholder the moment mime.Parse errors:

parsed, parseErr := mime.Parse(raw)
if parseErr != nil {
    errMsg := textutil.FirstLine(parseErr.Error())
    parsed = &mime.Message{
        Subject:  "(MIME parse error)",
        BodyText: fmt.Sprintf("[MIME parsing failed: %s]\n\nRaw MIME data is preserved in message_raw table.", errMsg),
    }
}

internal/sync/sync.go builds the same placeholder independently. Subject, sender, recipients, and date are all recoverable for many of these messages — only the body/attachments genuinely depend on the multipart structure that broke.

Proposal: a MIME recovery mode

When full envelope parsing fails, fall back to a lenient, header-only pass instead of discarding everything:

  1. Scan the raw byte prefix (up to the first blank line, or a reasonable cap) with a simple per-line header tokenizer, independent of enmime's stricter multipart/RFC 5322 conformance checks.
  2. Extract whatever single-value headers are present and well-formed: Date, Subject, From, To, Cc, Message-ID, In-Reply-To, References.
  3. Feed Date through the existing parseDate/ResolveMessageDate chain — same plausibility rules that already apply to fully-parsed messages, so a recovered header still has to pass muster to be trusted.
  4. Leave BodyText/Attachments unavailable when the multipart structure is broken. Keep today's "raw MIME preserved in message_raw" note, but attach it as an annotation on an otherwise-real message instead of standing in for the entire message.
  5. internal/importer/ingest.go and internal/sync/sync.go construct this same placeholder independently today — they should share one recovery path.

Why it matters

This is broader than the date-repair work in #504/#505/#506: any message that fails full envelope parsing loses its subject, sender, and recipients too, not just its date — which affects search, sender/domain analytics, and the TUI's message list, not only date-range queries. A partial, best-effort message is more useful than a placeholder that says nothing but "something broke."

Data point

Against a real archive, this pattern (envelope parse failure with a recoverable Date header) accounted for 33 of the 144 messages repair-dates currently reports as unrepairable — about 23% of that set, and roughly 70% of the 47 that fail full envelope parsing outright.

Related

This issue covers the messages where the envelope parse fails outright — a different, broader failure mode than #505/#506.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions