Skip to content

[Bug] Repeated field heading in title parse mode corrupts the next field and breaks parsing #271

Description

@graysoncooper

Describe the Bug

In LLMOutputParser title parse mode (parse_mode="title"), a recognized field heading that appears twice in the model output is not treated as a new section on its second occurrence. The repeated heading and its value get appended to the previous field's text, so the parser feeds a combined string like 1\n## count\n2 to the typed field. For int, float, list, and dict fields this raises a ValueError and aborts the whole structured parse; for bool and str fields it silently returns a wrong value.

The cause is in _parse_title_content in evoagentx/models/base_model.py. output_titles is built from the declared attributes, and is_output_title matches a line against that list. When a heading is first seen the code calls output_titles.remove(title), using removal as the duplicate detector. On a second occurrence of the same heading the title is no longer in the list, so is_output_title returns (False, None) and the else branch appends the heading line plus its body to current_attr_lines. When the section closes, process_lines joins those lines and passes the combined string to parse_data_from_text, which then fails (or, for bool/str, coerces the wrong text).

HEAD permalink (default branch main): https://github.com/EvoAgentX/EvoAgentX/blob/fd6b9a6352afc933b170e595bfb3dc5a28d9571a/evoagentx/models/base_model.py#L496-L534

Operating System

Not OS specific. Confirmed by reading and locally reproducing the parsing logic on the current default branch (main, commit fd6b9a6).

Python Version

Not version specific.

Steps to Reproduce

  1. Define a title-mode parser subclass with an int field count, a bool field enabled, and a str field label.
  2. Parse this content in title mode:
    ## count
    1
    ## count
    2
    ## enabled
    true
    ## label
    alpha
    
  3. Observe that parsing raises ValueError: Cannot parse text: '1\n## count\n2' into int data!, because the repeated ## count heading was folded into the first count value instead of starting a new section.
  4. Repeat with a duplicated ## enabled heading and observe the boolean silently resolves to the wrong value (no error), and with a duplicated ## label heading the string field silently returns the corrupted concatenation (for example alpha\n## label\nbeta).

Logs or Screenshots

ValueError: Cannot parse text: '1\n## count\n2' into int data!

Additional Context

The trigger is a model response that repeats a recognized field heading, which is uncommon but genuinely producible by an LLM. A targeted fix would stop using output_titles.remove(title) as the duplicate detector, keep declared headings recognizable across the whole scan, and explicitly define the behavior for a repeated heading (reject it, or overwrite/append deliberately) before its body text is assigned to the previous field.

Automated report: this issue was produced and filed automatically, with no human review before posting. Two independent checks agreed it is a real bug, but if it misreads the code please say so and we will close it.

Found while running Ito (AI code review that runs your application, free for open source) against recently merged PRs. Full analysis.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions