Training data generation for eagle3

[dongyibo]

Thank you for the great work you've done in replicating eagle3.
I am using sglang as our inference engine.
Do you have plans to release the script for generating training data for eagle3? Thank you.

[NickL77]

I just added training data generation here: https://github.com/NickL77/BaldEagle/blob/master/generate_data/generate_data_fused_features.py

The layer selection is based on these lines from the EAGLE repo: https://github.com/SafeAILab/EAGLE/blob/main/eagle/model/modeling_llama_kv.py#L1137-L1139

Note: We don't have reference implementation from EAGLE repo, so the formatting may need to change in the future. Also this is experimental as I haven't trained any models using feature fusion.

[dongyibo]

I just added training data generation here: https://github.com/NickL77/BaldEagle/blob/master/generate_data/generate_data_fused_features.py

The layer selection is based on these lines from the EAGLE repo: https://github.com/SafeAILab/EAGLE/blob/main/eagle/model/modeling_llama_kv.py#L1137-L1139

Note: We don't have reference implementation from EAGLE repo, so the formatting may need to change in the future. Also this is experimental as I haven't trained any models using feature fusion.

Thank you~