Skip to content

Conversation

@ZbigniewTomanek
Copy link
Contributor

Fixes rare token decoder overflow issues that cause std::string::substr exceptions during text summarization tasks. The bug manifests when using the gliner-multitask-large-v0.5 model with Q4 quantization on x86 Linux servers with ONNX Runtime 1.20.1, where token offsets can overflow signed 32-bit integers (reaching values around 1.6e9) and cause crashes when passed to std::string::substr.

Changes

  • Added adjustSpanToTextBounds() function to validate span indices against text bounds before processing
  • Added safeCopySpanText() function to guard substr calls with proper validation
  • Updated both SpanDecoder::decode() and TokenDecoder::decode() to use the new safety functions
  • Invalid spans are now skipped instead of causing application crashes
  • Maintains backward compatibility with existing functionality

Root Cause

Both GPU and CPU inference can generate token offsets that exceed safe integer bounds during text chunking operations. When these invalid indices (startIdx/endIdx values around 1.6e9) are passed to std::string::substr on text of normal length (~820 characters), it throws basic_string::substr: __pos > this->size() exceptions.

Testing

  • Resolves crashes in text summarization tasks with affected model configurations
  • Existing regression tests continue to pass
  • Invalid spans are gracefully handled without affecting valid results

Impact

  • Eliminates std::string::substr exceptions in text summarization workflows
  • No configuration changes required - bounds checking operates transparently

…summarization

Both GPU and CPU inference can generate token offsets that overflow signed 32-bit integers,
particularly when using gliner-multitask-large-v0.5 with Q4 quantization on x86 Linux
servers with ONNX Runtime 1.20.1. This leads to spans with startIdx/endIdx values around
1.6e9 during text summarization tasks. When these invalid indices are passed to
std::string::substr, it throws "basic_string::substr: __pos > this->size()" exceptions.

Added bounds checking and safe text extraction:
- adjustSpanToTextBounds() validates span indices against text bounds
- safeCopySpanText() guards substr calls with validation
- Invalid spans are now skipped instead of causing crashes
- Both SpanDecoder and TokenDecoder use the safety functions

Resolves crashes in text summarization tasks and maintains compatibility with existing
regression tests.
@Ingvarstep
Copy link
Contributor

@ZbigniewTomanek , thanks for the contribution, @oleksandrlukashov , please review it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants