Version: 0.19.3, macOS arm64
Import: msgvault import-pst, 46 GB PST exported from Microsoft 365, 75,188 messages, 110 folders, 1 error
Summary
After a PST import, the messages table has rfc822_message_id,
reply_to_message_id and thread_position empty on every row:
rfc822_message_id populated: 0 / 75,188
reply_to_message_id populated: 0 / 75,188
The schema provides for these columns, so this looks like the importer not
filling them rather than a design gap. Threading then falls back to a derived
key, which splits conversations.
The identifiers exist in the source mailbox
The same messages return a populated internetMessageId through the Microsoft
Graph API, in the normal form:
<XXXXXXXXXXXXXXXX@XXXXXXXX.prod.outlook.com>
So the values are present upstream and are lost at the import step.
source_message_id holds a synthetic value, e.g. pst-6bffbea2e8b4-3706084.
source_conversation_id is also empty on every row, so the PST conversation
index does not appear to be carried across either.
Concrete example
One exchange, six messages, same subject, same four participants, all within four
hours on the same day:
conversation A: five messages, 11:13 to 14:58
conversation B: one message, 15:12:49
The last reply, fourteen minutes after the message it answers, lands in its own
conversation.
Scale
Measured across the whole archive with a script comparing conversations that
share a normalised subject, using participant overlap and time proximity, and
excluding machine senders, calendar responses and messages filed twice:
conversations: 20,509
conversations that would merge away: 1,443 to 2,434
rate: 7.0% to 11.9%
7.0% is the strictest reading: machine traffic excluded, conversations within one
day, at least 80% participant overlap, and at least one member of the group
holding more than a single message. It is a floor, since the method only compares
conversations whose subjects already match.
The largest affected groups are ordinary human correspondence rather than noise,
with some threads split as many as ten ways.
Suggested fix
Populate rfc822_message_id from PR_INTERNET_MESSAGE_ID (0x1035) and
reply_to_message_id from PR_IN_REPLY_TO_ID (0x1042) during PST import, and
prefer those over the derived key when threading. PR_INTERNET_REFERENCES (0x1039)
would give the full chain where present.
Happy to run diagnostics against this archive if useful.
Version: 0.19.3, macOS arm64
Import:
msgvault import-pst, 46 GB PST exported from Microsoft 365, 75,188 messages, 110 folders, 1 errorSummary
After a PST import, the
messagestable hasrfc822_message_id,reply_to_message_idandthread_positionempty on every row:The schema provides for these columns, so this looks like the importer not
filling them rather than a design gap. Threading then falls back to a derived
key, which splits conversations.
The identifiers exist in the source mailbox
The same messages return a populated
internetMessageIdthrough the MicrosoftGraph API, in the normal form:
So the values are present upstream and are lost at the import step.
source_message_idholds a synthetic value, e.g.pst-6bffbea2e8b4-3706084.source_conversation_idis also empty on every row, so the PST conversationindex does not appear to be carried across either.
Concrete example
One exchange, six messages, same subject, same four participants, all within four
hours on the same day:
The last reply, fourteen minutes after the message it answers, lands in its own
conversation.
Scale
Measured across the whole archive with a script comparing conversations that
share a normalised subject, using participant overlap and time proximity, and
excluding machine senders, calendar responses and messages filed twice:
7.0% is the strictest reading: machine traffic excluded, conversations within one
day, at least 80% participant overlap, and at least one member of the group
holding more than a single message. It is a floor, since the method only compares
conversations whose subjects already match.
The largest affected groups are ordinary human correspondence rather than noise,
with some threads split as many as ten ways.
Suggested fix
Populate
rfc822_message_idfrom PR_INTERNET_MESSAGE_ID (0x1035) andreply_to_message_idfrom PR_IN_REPLY_TO_ID (0x1042) during PST import, andprefer those over the derived key when threading. PR_INTERNET_REFERENCES (0x1039)
would give the full chain where present.
Happy to run diagnostics against this archive if useful.