Mini LLM train, finetune ... framework
Current Open Source Frameworks such as Transformers, Deepspeed, etc, are not be able to customize our own procedure in train, finetune, benchmark ... with max freedom.
This work provides mini and light framework for you. You can use them which have been implimented on your work or modify to satisfy your work. Besides, this work is also a tutorial to understand LLM train ... procedure
Integratee One GPU, DDP, FSDP, Deepspeed currently. Provide LoRA peft method for finetune. Support llama model, such as llama 3.2 vision ...
This work just begin..