perf: reuse extract's file list when writing the traces README#24
Merged
Merged
Conversation
write_traces_readme re-walked the whole traces directory with rglob even though the extract flow just wrote the files and already holds the exact list. Accept an optional trace_files argument (exclusion filters still apply) and pass extract's copied_files from both the CLI and studio extraction paths. Callers without a known list, like the upload flow, keep the rescan. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
write_traces_readmere-walks the entire traces directory withrglobto rebuild a file list that the extract flow already holds — it just wrote those exact files (result.copied_files).This adds an optional
trace_filesparameter; when provided, the directory rescan is skipped. The partials/failures/excluded-dir filters and sorting still apply to the passed list, so the README content is identical. Both the CLI extract command and the studio extraction runner now pass their known list; callers without one (the Hugging Face upload flow, studio server, generation) keep the rescan behavior unchanged.Testing
pytest tests/test_extract_anonymize_cli.py tests/test_trace_readme.py tests/test_studio.py— 87 passed🤖 Generated with Claude Code