Skip to content

add --mproj to support multimodal model packaging #524

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ilopezluna merged 2 commits into from
Dec 18, 2025
Merged

add --mproj to support multimodal model packaging #524

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
f80a342
add --mproj to support multimodal model packaging
ilopezluna Dec 18, 2025
855eb86
extract common validation into validateAbsolutePath
ilopezluna Dec 18, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 25 additions & 3 deletions cmd/cli/commands/package.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,12 +28,13 @@ func newPackagedCmd() *cobra.Command {
var opts packageOptions

c := &cobra.Command{
Use: "package (--gguf <path> | --safetensors-dir <path> | --from <model>) [--license <path>...] [--context-size <tokens>] [--push] MODEL",
Use: "package (--gguf <path> | --safetensors-dir <path> | --from <model>) [--license <path>...] [--mmproj <path>] [--context-size <tokens>] [--push] MODEL",
Short: "Package a GGUF file, Safetensors directory, or existing model into a Docker model OCI artifact.",
Long: "Package a GGUF file, Safetensors directory, or existing model into a Docker model OCI artifact, with optional licenses. The package is sent to the model-runner, unless --push is specified.\n" +
Long: "Package a GGUF file, Safetensors directory, or existing model into a Docker model OCI artifact, with optional licenses and multimodal projector. The package is sent to the model-runner, unless --push is specified.\n" +
"When packaging a sharded GGUF model, --gguf should point to the first shard. All shard files should be siblings and should include the index in the file name (e.g. model-00001-of-00015.gguf).\n" +
"When packaging a Safetensors model, --safetensors-dir should point to a directory containing .safetensors files and config files (*.json, merges.txt). All files will be auto-discovered and config files will be packaged into a tar archive.\n" +
"When packaging from an existing model using --from, you can modify properties like context size to create a variant of the original model.",
"When packaging from an existing model using --from, you can modify properties like context size to create a variant of the original model.\n" +
"For multimodal models, use --mmproj to include a multimodal projector file.",
Args: func(cmd *cobra.Command, args []string) error {
if err := requireExactArgs(1, "package", "MODEL")(cmd, args); err != nil {
return err
Expand Down Expand Up @@ -116,6 +117,17 @@ func newPackagedCmd() *cobra.Command {
opts.licensePaths[i] = filepath.Clean(l)
}

// Validate mmproj path if provided
if opts.mmprojPath != "" {
if !filepath.IsAbs(opts.mmprojPath) {
return fmt.Errorf(
"mmproj path must be absolute.\n\n" +
"See 'docker model package --help' for more information",
)
}
opts.mmprojPath = filepath.Clean(opts.mmprojPath)
}
Comment on lines 138 to 144
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot Dec 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This validation logic for mmprojPath is very similar to the validation for ggufPath (lines 69-77) and licensePaths (lines 110-118). To improve maintainability and reduce code duplication, consider extracting this logic into a helper function.

For example, you could create a function like this:

func validateAbsPath(path, name string) (string, error) {
	if !filepath.IsAbs(path) {
		return "", fmt.Errorf(
			"%s path must be absolute.\n\n"+
				"See 'docker model package --help' for more information",
			name,
		)
	}
	return filepath.Clean(path), nil
}

This function could then be used for mmprojPath, ggufPath, and licensePaths, making the validation logic more concise and easier to maintain.


// Validate dir-tar paths are relative (not absolute)
for _, dirPath := range opts.dirTarPaths {
if filepath.IsAbs(dirPath) {
Expand Down Expand Up @@ -146,6 +158,7 @@ func newPackagedCmd() *cobra.Command {
c.Flags().StringVar(&opts.chatTemplatePath, "chat-template", "", "absolute path to chat template file (must be Jinja format)")
c.Flags().StringArrayVarP(&opts.licensePaths, "license", "l", nil, "absolute path to a license file")
c.Flags().StringArrayVar(&opts.dirTarPaths, "dir-tar", nil, "relative path to directory to package as tar (can be specified multiple times)")
c.Flags().StringVar(&opts.mmprojPath, "mmproj", "", "absolute path to multimodal projector file")
c.Flags().BoolVar(&opts.push, "push", false, "push to registry (if not set, the model is loaded into the Model Runner content store)")
c.Flags().Uint64Var(&opts.contextSize, "context-size", 0, "context size in tokens")
return c
Expand All @@ -159,6 +172,7 @@ type packageOptions struct {
fromModel string
licensePaths []string
dirTarPaths []string
mmprojPath string
push bool
tag string
}
Expand Down Expand Up @@ -305,6 +319,14 @@ func packageModel(ctx context.Context, cmd *cobra.Command, client *desktop.Clien
}
}

if opts.mmprojPath != "" {
cmd.PrintErrf("Adding multimodal projector file from %q\n", opts.mmprojPath)
pkg, err = pkg.WithMultimodalProjector(opts.mmprojPath)
if err != nil {
return fmt.Errorf("add multimodal projector file: %w", err)
}
}

// Check if we can use lightweight repackaging (config-only changes from existing model)
useLightweight := opts.fromModel != "" && pkg.HasOnlyConfigChanges()

Expand Down
14 changes: 12 additions & 2 deletions cmd/cli/docs/reference/docker_model_package.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,12 @@ command: docker model package
short: |
Package a GGUF file, Safetensors directory, or existing model into a Docker model OCI artifact.
long: |-
Package a GGUF file, Safetensors directory, or existing model into a Docker model OCI artifact, with optional licenses. The package is sent to the model-runner, unless --push is specified.
Package a GGUF file, Safetensors directory, or existing model into a Docker model OCI artifact, with optional licenses and multimodal projector. The package is sent to the model-runner, unless --push is specified.
When packaging a sharded GGUF model, --gguf should point to the first shard. All shard files should be siblings and should include the index in the file name (e.g. model-00001-of-00015.gguf).
When packaging a Safetensors model, --safetensors-dir should point to a directory containing .safetensors files and config files (*.json, merges.txt). All files will be auto-discovered and config files will be packaged into a tar archive.
When packaging from an existing model using --from, you can modify properties like context size to create a variant of the original model.
usage: docker model package (--gguf <path> | --safetensors-dir <path> | --from <model>) [--license <path>...] [--context-size <tokens>] [--push] MODEL
For multimodal models, use --mmproj to include a multimodal projector file.
usage: docker model package (--gguf <path> | --safetensors-dir <path> | --from <model>) [--license <path>...] [--mmproj <path>] [--context-size <tokens>] [--push] MODEL
pname: docker model
plink: docker_model.yaml
options:
Expand Down Expand Up @@ -69,6 +70,15 @@ options:
experimentalcli: false
kubernetes: false
swarm: false
- option: mmproj
value_type: string
description: absolute path to multimodal projector file
deprecated: false
hidden: false
experimental: false
experimentalcli: false
kubernetes: false
swarm: false
- option: push
value_type: bool
default_value: "false"
Expand Down
4 changes: 3 additions & 1 deletion cmd/cli/docs/reference/model_package.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
# docker model package

<!---MARKER_GEN_START-->
Package a GGUF file, Safetensors directory, or existing model into a Docker model OCI artifact, with optional licenses. The package is sent to the model-runner, unless --push is specified.
Package a GGUF file, Safetensors directory, or existing model into a Docker model OCI artifact, with optional licenses and multimodal projector. The package is sent to the model-runner, unless --push is specified.
When packaging a sharded GGUF model, --gguf should point to the first shard. All shard files should be siblings and should include the index in the file name (e.g. model-00001-of-00015.gguf).
When packaging a Safetensors model, --safetensors-dir should point to a directory containing .safetensors files and config files (*.json, merges.txt). All files will be auto-discovered and config files will be packaged into a tar archive.
When packaging from an existing model using --from, you can modify properties like context size to create a variant of the original model.
For multimodal models, use --mmproj to include a multimodal projector file.

### Options

Expand All @@ -16,6 +17,7 @@ When packaging from an existing model using --from, you can modify properties li
| `--from` | `string` | | reference to an existing model to repackage |
| `--gguf` | `string` | | absolute path to gguf file |
| `-l`, `--license` | `stringArray` | | absolute path to a license file |
| `--mmproj` | `string` | | absolute path to multimodal projector file |
| `--push` | `bool` | | push to registry (if not set, the model is loaded into the Model Runner content store) |
| `--safetensors-dir` | `string` | | absolute path to directory containing safetensors files and config |

Expand Down
Loading