Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python: Only pass metadata that we need #1600

Merged
merged 8 commits into from
Feb 5, 2024
Merged

Conversation

hoodmane
Copy link
Contributor

Rather than serializing the entire workerd config into a JSON string and then deserializing it at runtime this makes another readonly file system so we can copy only the data that Python wants to look at. This will copy all Data, Text, and Python files once at startup into the MetadataReader since I am not sure about the lifespan of the capnp config and it seems easier to understand if we make the module own its data.

Rather than serializing the entire workerd config into a JSON string and then
deserializing it at runtime this makes another readonly file system so we can
copy only the data that Python wants to look at. This will copy all Data, Text,
and Python files once at startup into the MetadataReader since I am not sure
about the lifespan of the capnp config and it seems easier to understand if
we make the module own its data.
@hoodmane hoodmane requested review from a team as code owners January 31, 2024 22:59
@hoodmane hoodmane requested review from dom96 and jasnell January 31, 2024 22:59
src/pyodide/internal/tarfs.js Show resolved Hide resolved
src/pyodide/python-entrypoint-helper.js Show resolved Hide resolved
Comment on lines +100 to +101
"pyodide.c++",
"pyodide/pyodide.c++",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These should be merged together, or the second one given a more descriptive name.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll leave that to a followup. But it would also be good to merge pyodide.h and pyodide/pyodide.h

@hoodmane hoodmane changed the title Python: Improve performance of metadata handling Python: Only pass metadata that we need Feb 5, 2024
@hoodmane hoodmane merged commit f007aa8 into main Feb 5, 2024
11 checks passed
@hoodmane hoodmane deleted the hoodmane/python-metadatafs branch February 5, 2024 21:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants