Skip to content

Expose remaining training hyperparameters as CLI flags #72

Description

@will-lamerton

Problem

nanotune train exposes only --iterations and --lr. Four settings that exist in ConfigSchema have no flag at all:

  • batchSize
  • numLayers
  • stepsPerEval
  • saveEvery

So experimenting with any of them means editing .nanotune/config.json between runs.

Separately, mlx_lm lora supports a number of options Nanotune never passes (src/lib/mlx.ts):

  • --fine-tune-type (lora / dora / full)
  • LoRA rank, alpha, dropout
  • --max-seq-length
  • --grad-checkpoint
  • --val-batches
  • a training seed

LoRA rank in particular is one of the most useful knobs for fine-tuning quality, and it's currently unreachable.

Proposal

Two separable pieces:

  1. Add flags for the four existing config fields. Mechanical, matches the existing --iterations / --lr pattern in src/cli.tsx and src/commands/train.tsx.
  2. Extend the schema and the MLX invocation with LoRA rank/alpha/dropout, --fine-tune-type, --max-seq-length, and a training seed. Larger, needs schema changes and defaults that don't break existing projects.

Worth splitting if someone picks this up.

Acceptance criteria

  • Every training.* field in ConfigSchema has a corresponding CLI flag
  • CLI flags override config, matching current --iterations behaviour
  • New MLX options are schema-backed with sensible defaults
  • docs/configuration/index.md and docs/commands/train.md updated

Notes

Part 1 is a good first issue: follow --iterations from src/cli.tsx through to MLXTrainingOptions.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions