What I was doing
Working through #505/#506 (Date header parsing bugs) and #504 (repair-dates), I checked what's left after those fixes: 144 archived messages with missing/implausible sent dates, of which 47 fail full envelope parsing in mime.Parse (enmime.ReadEnvelope) outright — no Message is returned at all.
What I found
Of those 47, 33 have a perfectly clean, standard Date: header sitting in the raw bytes — a plain regex scan found it immediately. The envelope parse is failing on something else entirely (Failed to ReadParts: malformed MIME header initial line, mime: unexpected content after media subtype, unable to locate boundary param in Content-Type header), not because the headers themselves are unreadable.
But today, one bad part anywhere in the message causes the importer to throw away everything. IngestRawMessage replaces the entire parsed message with a placeholder the moment mime.Parse errors:
parsed, parseErr := mime.Parse(raw)
if parseErr != nil {
errMsg := textutil.FirstLine(parseErr.Error())
parsed = &mime.Message{
Subject: "(MIME parse error)",
BodyText: fmt.Sprintf("[MIME parsing failed: %s]\n\nRaw MIME data is preserved in message_raw table.", errMsg),
}
}
internal/sync/sync.go builds the same placeholder independently. Subject, sender, recipients, and date are all recoverable for many of these messages — only the body/attachments genuinely depend on the multipart structure that broke.
Proposal: a MIME recovery mode
When full envelope parsing fails, fall back to a lenient, header-only pass instead of discarding everything:
- Scan the raw byte prefix (up to the first blank line, or a reasonable cap) with a simple per-line header tokenizer, independent of enmime's stricter multipart/RFC 5322 conformance checks.
- Extract whatever single-value headers are present and well-formed:
Date, Subject, From, To, Cc, Message-ID, In-Reply-To, References.
- Feed
Date through the existing parseDate/ResolveMessageDate chain — same plausibility rules that already apply to fully-parsed messages, so a recovered header still has to pass muster to be trusted.
- Leave
BodyText/Attachments unavailable when the multipart structure is broken. Keep today's "raw MIME preserved in message_raw" note, but attach it as an annotation on an otherwise-real message instead of standing in for the entire message.
internal/importer/ingest.go and internal/sync/sync.go construct this same placeholder independently today — they should share one recovery path.
Why it matters
This is broader than the date-repair work in #504/#505/#506: any message that fails full envelope parsing loses its subject, sender, and recipients too, not just its date — which affects search, sender/domain analytics, and the TUI's message list, not only date-range queries. A partial, best-effort message is more useful than a placeholder that says nothing but "something broke."
Data point
Against a real archive, this pattern (envelope parse failure with a recoverable Date header) accounted for 33 of the 144 messages repair-dates currently reports as unrepairable — about 23% of that set, and roughly 70% of the 47 that fail full envelope parsing outright.
Related
This issue covers the messages where the envelope parse fails outright — a different, broader failure mode than #505/#506.
What I was doing
Working through #505/#506 (Date header parsing bugs) and #504 (
repair-dates), I checked what's left after those fixes: 144 archived messages with missing/implausible sent dates, of which 47 fail full envelope parsing inmime.Parse(enmime.ReadEnvelope) outright — noMessageis returned at all.What I found
Of those 47, 33 have a perfectly clean, standard
Date:header sitting in the raw bytes — a plain regex scan found it immediately. The envelope parse is failing on something else entirely (Failed to ReadParts: malformed MIME header initial line,mime: unexpected content after media subtype,unable to locate boundary param in Content-Type header), not because the headers themselves are unreadable.But today, one bad part anywhere in the message causes the importer to throw away everything.
IngestRawMessagereplaces the entire parsed message with a placeholder the momentmime.Parseerrors:internal/sync/sync.gobuilds the same placeholder independently. Subject, sender, recipients, and date are all recoverable for many of these messages — only the body/attachments genuinely depend on the multipart structure that broke.Proposal: a MIME recovery mode
When full envelope parsing fails, fall back to a lenient, header-only pass instead of discarding everything:
Date,Subject,From,To,Cc,Message-ID,In-Reply-To,References.Datethrough the existingparseDate/ResolveMessageDatechain — same plausibility rules that already apply to fully-parsed messages, so a recovered header still has to pass muster to be trusted.BodyText/Attachmentsunavailable when the multipart structure is broken. Keep today's "raw MIME preserved in message_raw" note, but attach it as an annotation on an otherwise-real message instead of standing in for the entire message.internal/importer/ingest.goandinternal/sync/sync.goconstruct this same placeholder independently today — they should share one recovery path.Why it matters
This is broader than the date-repair work in #504/#505/#506: any message that fails full envelope parsing loses its subject, sender, and recipients too, not just its date — which affects search, sender/domain analytics, and the TUI's message list, not only date-range queries. A partial, best-effort message is more useful than a placeholder that says nothing but "something broke."
Data point
Against a real archive, this pattern (envelope parse failure with a recoverable
Dateheader) accounted for 33 of the 144 messagesrepair-datescurrently reports as unrepairable — about 23% of that set, and roughly 70% of the 47 that fail full envelope parsing outright.Related
repair-datescommandmime.ParseDate header fixes (narrower scope: messages where the envelope does parse)This issue covers the messages where the envelope parse fails outright — a different, broader failure mode than #505/#506.