Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions docs/changelog/951.bugfix.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Reject a :class:`RELAX NG <turbohtml.validate.RelaxNG>` ``<ref>`` with no name, a reference cycle that never crosses an
``element``, and an ``interleave`` whose branches share an element name or both match text when the schema compiles, and
validate an ambiguous ``choice`` in bounded memory, instead of crashing, hanging, or exhausting memory.
11 changes: 7 additions & 4 deletions docs/explanation/validation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,11 @@ restriction chain. Namespaces resolve from the in-scope ``xmlns`` declarations,
compiles to that algebra, and validation takes the *derivative* of the pattern with respect to each start tag,
attribute, text run, and end tag: the pattern that remains after consuming one piece of the document. This is what makes
``interleave`` fall out for free -- the derivative of ``interleave(p1, p2)`` over an element is the choice of advancing
either side -- with no backtracking and no combinatorial blow-up, because smart constructors absorb ``notAllowed`` and
``empty`` to keep the residual pattern small.
either side -- with no backtracking. Smart constructors absorb ``notAllowed`` and ``empty``, the residual patterns are
hash-consed per validation, and ``choice`` drops a branch already present, so an ambiguous grammar stays bounded instead
of doubling the residual on each child. The restrictions the specification places on a schema are enforced when it
compiles: a ``<ref>`` with no name, a reference cycle that never crosses an ``element``, and an ``interleave`` whose
branches compete for an element name or text are rejected with a :class:`ValueError` rather than reached at validation.

****************
Why the C core
Expand All @@ -42,8 +45,8 @@ The datatype and facet layer is where validation spends its time: every leaf val
(is ``2020-13-40`` a date?) and then against its constraining facets (``minInclusive``, ``pattern``, ``length``, ...).
Doing that in the extension -- over the code-point buffers the parser already produced, with a compact Thompson-NFA
matcher for the ``pattern`` facet -- keeps a schema check close to the cost of the parse it follows, rather than a
second pass in Python. The recursion guards that protect the RELAX NG derivative from schemas whose refs recurse without
an element in between live there too, so an adversarial schema fails cleanly instead of overflowing the stack.
second pass in Python. A RELAX NG schema whose refs recurse without an element in between is rejected when it compiles,
so the derivative never reaches such a grammar and an adversarial schema fails cleanly instead of overflowing the stack.
Compilation and validation start with an iterative tree-depth scan. A schema or instance nested 400 levels or deeper
raises :class:`RecursionError` before a recursive grammar walk starts, including on small worker-thread stacks.

Expand Down
6 changes: 6 additions & 0 deletions docs/reference/validate.rst
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,12 @@ above ``m``. The message names the limit and the offset where the pattern reache
the repeat counts, or split the pattern into several ``pattern`` facets to stay within the limits. Matching a value
takes time linear in its length.

Compiling a RELAX NG schema raises :class:`ValueError` for a grammar the RELAX NG specification forbids: a ``<ref>``
with no ``name`` attribute (section 4.10), a reference cycle whose expansion never passes through an ``element``
(section 4.19), and an ``interleave`` whose branches can match an element with the same name or can both match text
(section 7.4). The message names the offending ``define`` or construct. A legal but ambiguous ``choice`` or
``interleave`` validates in memory bounded by the schema size rather than growing per child element.

.. autoclass:: XMLSchema
:members:
:inherited-members:
Expand Down
Loading
Loading