Skip to content

PQ could benefit by avoiding deserialization cost #17821

@yaauie

Description

@yaauie

While work has recently been done to pull PQ deserialization out of the lock by deferring the reification of events in a batch until after the lock has been released, we could further benefit by avoiding that deserialization cost entirely in the absence of memory pressure.

TODO: flesh out the requirements, using the proof-of-concept as a starting point.

  • held events should be soft references to allow GC to collect them if doing so can prevent an OOM
  • the soft-references should be removed after a hard-reference has been obtained and before said hard-reference becomes part of a batch
  • when a page is deactivated (and its memory-mapped file released), its cache of events should also be deleted
  • the local event cache should be feature-gated for safety (even if we are confident launching it as opt-out)

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions