Skip to content

Loading a serialized Instr object from file is slow #143

@g5t

Description

@g5t

Details

  • It's possible to serialize the entire state of an Instr object to file, including to an HDF5 file.
  • Retrieving the full Instr object from this state is slow for as-yet-undetermined reasons.
  • Sometimes only part of the state is needed, and a modification to load_hdf5 would be possible to enable specifying what part of the file to load more-quickly.

Ideas

  • Profile de-serialization to work out what part(s) are possibly worth improving
  • Add an in-file path keyword argument to load_hdf5 to select out just, e.g., the instrument parameters.

Temporary workaround

import h5py
from mccode_antlr.io.hdf5 import HDF5IO

with h5py.File(filename, 'r', driver='core', backing_store=False) as file:
    return HDF5IO.load(file['parameters'])

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions