fix: Prevent TokenDecoder overflow causing substr exceptions in text summarization #9
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes rare token decoder overflow issues that cause
std::string::substrexceptions during text summarization tasks. The bug manifests when using thegliner-multitask-large-v0.5model with Q4 quantization on x86 Linux servers with ONNX Runtime 1.20.1, where token offsets can overflow signed 32-bit integers (reaching values around 1.6e9) and cause crashes when passed tostd::string::substr.Changes
adjustSpanToTextBounds()function to validate span indices against text bounds before processingsafeCopySpanText()function to guardsubstrcalls with proper validationSpanDecoder::decode()andTokenDecoder::decode()to use the new safety functionsRoot Cause
Both GPU and CPU inference can generate token offsets that exceed safe integer bounds during text chunking operations. When these invalid indices (startIdx/endIdx values around 1.6e9) are passed to
std::string::substron text of normal length (~820 characters), it throwsbasic_string::substr: __pos > this->size()exceptions.Testing
Impact
std::string::substrexceptions in text summarization workflows