Skip to content

Yeyu/remove embedding from eagle #585

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 49 commits into
base: main
Choose a base branch
from
Draft

Yeyu/remove embedding from eagle #585

wants to merge 49 commits into from

Conversation

@yeyu-nvidia
Copy link
Contributor

@yeyu-nvidia yeyu-nvidia commented Nov 20, 2025

What does this PR do?

Type of change: ?

Overview: ?

Usage

# Add a code snippet demonstrating how to use this

Testing

Before your PR is "Ready for review"

  • Make sure you read and follow Contributor guidelines and your commits are signed.
  • Is this change backward compatible?: Yes/No
  • Did you write any new necessary tests?: Yes/No
  • Did you add or update any necessary documentation?: Yes/No
  • Did you update Changelog?: Yes/No

Additional Information

@yeyu-nvidia
refactor_multi_step_attn_mask_for_arbitrary_step
c00fb01 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
minor
1de7d0a 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
make new mask the same dtype and device as attn_mask
e6ec481 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
minor
89b136d 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
revert
fd8b455 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
integrate parallel draft to eagle auto regression
d3bd039 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
debug
15840d6 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
fix pseudo spec generate for parallel draft
af34721 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
implement kv cache (inference_context) for eagle training
0e57805 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
allow groundtruth mismatch in AcceptanceRateValidation
7eea4e2 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
set fail_when_mismatch to False
e934d62 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
debug
9b98412 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
debug
8ee7fe4 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
debug
2e09a73 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
debug
9162d00 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
remove redundant index as now eagle_logits_ has a length of seq-len
9ef5fe9 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
minor edit based on coderabbit's suggestions
5e11b81 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
make ttt_step configurable in forward
f9e37fe 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
fix the bug in pseudo_speculative_generate
b582c94 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
change variable name to make it clear
66ce87a 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
debug: reduce kv cache size from ttt*parallel to ttt+parallel-1; in e…
691d22e 
…ach ttt step, only the non_parallel tokens from previous ttt are used as context

Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
gitnore type for precommit
8f4ec81 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
consolidate acc printout
a169f37 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
fix the bug in pseudo_speculative_generate
8dcb48d 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
use embedding for mask tokens as hidden_states
31c2f45 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
debug
ee28311 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
remove mask tokens and use learnable embeddings and hidden_states ins…
ab94f5b 
…tead

Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
debug
f03935d 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
debug
34dd050 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
debug
cbf0285 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
debug: move learnable parallel draft embedding and hidden_states afte…
3a95869 
…r base model param freeze

Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
debug
75cd1aa 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
only compute acc on tokens whose label is not IGNORE_TOKEN_ID
095a302 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
revert acc compute
f0de5d1 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
implement eagle + medusa for parallel draft
64e2e75 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
debug
0be2c23 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
debug
4b52efd 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
debug: medusa labels should be shifted
ede621e 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
loss_decay only on ttt
10b8881 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
revert loss_decay; fix: medusa logit shift in the back
c42f87a 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
detach eagle hidden_states from medusa heads; this will make parallel…
0f9483f 
… draft training only trains medusa heads but not eagle_module

Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
no loss decay on parallel index
aa62ef5 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
debug: parallel logits should be cat in dim=0 in pseudo gen
770f28a 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
debug: scatter padded hidden_states in pseudo generate
823962b 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
rename layer names and hnorm for megatron eagle3 export to be consist…
8c8439a 
…ent with trtllm

Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
no detach for medusa feature
942a0d6 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
remove embedding from eagle
ec747b4 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
remove embedding
3bb3934 
Signed-off-by: Ye Yu <[email protected]>
@yeyu-nvidia
debug
4ec9975 
Signed-off-by: Ye Yu <[email protected]>
@copy-pr-bot
Copy link

copy-pr-bot bot commented Nov 20, 2025

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@yeyu-nvidia