Skip to content

Add fine-tuning guide documentation - #39

Merged
r33drichards merged 2 commits into
mainfrom
claude/vibrant-rubin-4uo2t3
Oct 1, 2026
Merged

r33drichards merged 2 commits into
mainfrom
claude/vibrant-rubin-4uo2t3

Conversation

@r33drichards

Copy link
Copy Markdown
Owner

Summary

This PR adds comprehensive documentation for fine-tuning the Laya vision model on custom datasets, including both local training and Modal-based GPU training approaches.

Key Changes

  • New guide document (site-docs/how-to/fine-tune.md): Complete fine-tuning workflow covering:

    • Data format specification (JSONL with images directory structure)
    • Detailed field descriptions and validation rules
    • Best practices for dataset preparation
    • Local training with Python using freeze options ("head", "last_n", "full")
    • Modal-based training on A100 GPUs with per-epoch evaluation
    • Temperature fitting for calibration
    • Model saving and sharing to Hugging Face Hub
    • Usage examples for the fine-tuned model
  • Updated navigation (mkdocs.yml): Added "Fine-tune on your own dataset" link to the How-to section

  • Updated overview (site-docs/how-to/overview.md): Added reference to the new fine-tuning guide with brief description

Notable Details

  • Provides clear comparison table of freeze options with use cases and performance characteristics
  • Includes practical code examples for both local and Modal training workflows
  • Documents dataset preparation best practices (question phrasing, split management, validation set sizing)
  • Explains temperature fitting for maintaining meaningful confidence scores after training
  • References related concepts and implementation details in laya/vlm_train.py

https://claude.ai/code/session_011MHDFrSKQo5r47X9SuTrpT

claude added 2 commits October 1, 2026 05:38
Covers the JSONL dataset format, a local fine-tune from the pinned published
checkpoint with laya.vlm_train.train, finetune_long on Modal (volume upload,
--init-from the Hub checkpoint, mixing with The Cauldron), and loading the result.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011MHDFrSKQo5r47X9SuTrpT
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011MHDFrSKQo5r47X9SuTrpT
@r33drichards
r33drichards merged commit 9e1e241 into main Oct 1, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants