chore: strip [default0] prefix from trainer logs#2180
Open
mikasenghaas wants to merge 1 commit intomainfrom
Open
chore: strip [default0] prefix from trainer logs#2180mikasenghaas wants to merge 1 commit intomainfrom
mikasenghaas wants to merge 1 commit intomainfrom
Conversation
torchrun's --tee flag prepends every log line with [<role><rank>]: (e.g. [default0]:). This adds noise without value since we already filter to specific ranks. Strip the prefix with sed -u in the tail/tee pipelines and set --role=trainer for clearer per-rank log files. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
samsja
requested changes
Apr 2, 2026
Member
samsja
left a comment
There was a problem hiding this comment.
nah we need this for when we have multi rank trainer writting to the same file. Also I am against doing too many magic beyond tail. The file is the file
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
[default0]:prefix that torchrun's--teeflag prepends to every log line (note, that these can still be seen for in the per-rank logs)sed -uin the tail/tee pipelines to remove the prefix in real-time--role=trainerfor clearer per-rank log files in the torchrun log directoryBefore:
After:
🤖 Generated with Claude Code
Note
Low Risk
Logging-only changes that adjust
torchrunargs and shell pipelines; main risk is portability/quoting issues when spawningshell=Trueor running on systems without compatiblesed.Overview
Trainer launch commands now pass
torchrun --role=trainerso per-rank log files are labeled more clearly.For both local (
rl.py,sft.py) and multi-node SLURM templates, the displayed/tee’d trainer log stream is piped throughsed -uto strip the leading[...]rank prefix (e.g.,[default0]:) while keeping the underlying per-rank torchrun logs intact.Written by Cursor Bugbot for commit 0332018. This will update automatically on new commits. Configure here.