Skip to content
This repository was archived by the owner on Oct 6, 2025. It is now read-only.

Support packaging sharded models #142

Merged
ekcasey merged 2 commits into from
Aug 22, 2025
Merged

Support packaging sharded models #142

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
703740f
Support packaging sharded models
ekcasey Aug 22, 2025
fd5d60a
Provide guidance for packaging sharded models in help text
ekcasey Aug 22, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion commands/package.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,9 @@ func newPackagedCmd() *cobra.Command {

c := &cobra.Command{
Use: "package --gguf <path> [--license <path>...] [--context-size <tokens>] [--push] MODEL",
Short: "Package a GGUF file into a Docker model OCI artifact, with optional licenses. The package is sent to the model-runner, unless --push is specified",
Short: "Package a GGUF file into a Docker model OCI artifact, with optional licenses.",
Long: "Package a GGUF file into a Docker model OCI artifact, with optional licenses. The package is sent to the model-runner, unless --push is specified.\n" +
"When packaging a sharded model --gguf should point to the first shard. All shard files should be siblings and should include the index in the file name (e.g. model-00001-of-00015.gguf).",
Args: func(cmd *cobra.Command, args []string) error {
if len(args) != 1 {
return fmt.Errorf(
Expand Down
7 changes: 4 additions & 3 deletions docs/reference/docker_model_package.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
command: docker model package
short: |
Package a GGUF file into a Docker model OCI artifact, with optional licenses. The package is sent to the model-runner, unless --push is specified
long: |
Package a GGUF file into a Docker model OCI artifact, with optional licenses. The package is sent to the model-runner, unless --push is specified
Package a GGUF file into a Docker model OCI artifact, with optional licenses.
long: |-
Package a GGUF file into a Docker model OCI artifact, with optional licenses. The package is sent to the model-runner, unless --push is specified.
When packaging a sharded model --gguf should point to the first shard. All shard files should be siblings and should include the index in the file name (e.g. model-00001-of-00015.gguf).
usage: docker model package --gguf <path> [--license <path>...] [--context-size <tokens>] [--push] MODEL
pname: docker model
plink: docker_model.yaml
Expand Down
36 changes: 18 additions & 18 deletions docs/reference/model.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,24 +5,24 @@ Docker Model Runner (EXPERIMENTAL)

### Subcommands

| Name | Description |
|:------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------|
| [`df`](model_df.md) | Show Docker Model Runner disk usage |
| [`inspect`](model_inspect.md) | Display detailed information on one model |
| [`install-runner`](model_install-runner.md) | Install Docker Model Runner (Docker Engine only) |
| [`list`](model_list.md) | List the models pulled to your local environment |
| [`logs`](model_logs.md) | Fetch the Docker Model Runner logs |
| [`package`](model_package.md) | Package a GGUF file into a Docker model OCI artifact, with optional licenses. The package is sent to the model-runner, unless --push is specified |
| [`ps`](model_ps.md) | List running models |
| [`pull`](model_pull.md) | Pull a model from Docker Hub or HuggingFace to your local environment |
| [`push`](model_push.md) | Push a model to Docker Hub |
| [`rm`](model_rm.md) | Remove local models downloaded from Docker Hub |
| [`run`](model_run.md) | Run a model and interact with it using a submitted prompt or chat mode |
| [`status`](model_status.md) | Check if the Docker Model Runner is running |
| [`tag`](model_tag.md) | Tag a model |
| [`uninstall-runner`](model_uninstall-runner.md) | Uninstall Docker Model Runner |
| [`unload`](model_unload.md) | Unload running models |
| [`version`](model_version.md) | Show the Docker Model Runner version |
| Name | Description |
|:------------------------------------------------|:------------------------------------------------------------------------------|
| [`df`](model_df.md) | Show Docker Model Runner disk usage |
| [`inspect`](model_inspect.md) | Display detailed information on one model |
| [`install-runner`](model_install-runner.md) | Install Docker Model Runner (Docker Engine only) |
| [`list`](model_list.md) | List the models pulled to your local environment |
| [`logs`](model_logs.md) | Fetch the Docker Model Runner logs |
| [`package`](model_package.md) | Package a GGUF file into a Docker model OCI artifact, with optional licenses. |
| [`ps`](model_ps.md) | List running models |
| [`pull`](model_pull.md) | Pull a model from Docker Hub or HuggingFace to your local environment |
| [`push`](model_push.md) | Push a model to Docker Hub |
| [`rm`](model_rm.md) | Remove local models downloaded from Docker Hub |
| [`run`](model_run.md) | Run a model and interact with it using a submitted prompt or chat mode |
| [`status`](model_status.md) | Check if the Docker Model Runner is running |
| [`tag`](model_tag.md) | Tag a model |
| [`uninstall-runner`](model_uninstall-runner.md) | Uninstall Docker Model Runner |
| [`unload`](model_unload.md) | Unload running models |
| [`version`](model_version.md) | Show the Docker Model Runner version |



Expand Down
3 changes: 2 additions & 1 deletion docs/reference/model_package.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
# docker model package

<!---MARKER_GEN_START-->
Package a GGUF file into a Docker model OCI artifact, with optional licenses. The package is sent to the model-runner, unless --push is specified
Package a GGUF file into a Docker model OCI artifact, with optional licenses. The package is sent to the model-runner, unless --push is specified.
When packaging a sharded model --gguf should point to the first shard. All shard files should be siblings and should include the index in the file name (e.g. model-00001-of-00015.gguf).

### Options

Expand Down
4 changes: 2 additions & 2 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@ require (
github.com/docker/docker v28.2.2+incompatible
github.com/docker/go-connections v0.5.0
github.com/docker/go-units v0.5.0
github.com/docker/model-distribution v0.0.0-20250813080006-2a983516ebb8
github.com/docker/model-runner v0.0.0-20250822151118-d8ed37445584
github.com/docker/model-distribution v0.0.0-20250822172258-8fe9daa4a4da
github.com/docker/model-runner v0.0.0-20250822173738-5341c9fc2974
github.com/fatih/color v1.15.0
github.com/google/go-containerregistry v0.20.6
github.com/mattn/go-isatty v0.0.20
Expand Down
8 changes: 4 additions & 4 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,10 +78,10 @@ github.com/docker/go-metrics v0.0.1/go.mod h1:cG1hvH2utMXtqgqqYE9plW6lDxS3/5ayHz
github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4=
github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
github.com/docker/libtrust v0.0.0-20160708172513-aabc10ec26b7/go.mod h1:cyGadeNEkKy96OOhEzfZl+yxihPEzKnqJwvfuSUqbZE=
github.com/docker/model-distribution v0.0.0-20250813080006-2a983516ebb8 h1:agH5zeO6tf8lHgMcBZxqCFKPuXHM/cA53gdsn895eMI=
github.com/docker/model-distribution v0.0.0-20250813080006-2a983516ebb8/go.mod h1:dThpO9JoG5Px3i+rTluAeZcqLGw8C0qepuEL4gL2o/c=
github.com/docker/model-runner v0.0.0-20250822151118-d8ed37445584 h1:8YAzh9lihwcFGyHTK9pTFqdM7IwYwb0R/YkrNxmQ2do=
github.com/docker/model-runner v0.0.0-20250822151118-d8ed37445584/go.mod h1:0IAh5ekLg8ipcPAF+Rdav1wbt9xF+zQPoRC1bblk/ik=
github.com/docker/model-distribution v0.0.0-20250822172258-8fe9daa4a4da h1:ml99WBfcLnsy1frXQR4X+5WAC0DoGtwZyGoU/xBsDQM=
github.com/docker/model-distribution v0.0.0-20250822172258-8fe9daa4a4da/go.mod h1:dThpO9JoG5Px3i+rTluAeZcqLGw8C0qepuEL4gL2o/c=
github.com/docker/model-runner v0.0.0-20250822173738-5341c9fc2974 h1:/uF17tBEtsE6T2Xgg4cgrrqNcQ02gY5Lp98je+2K0nQ=
github.com/docker/model-runner v0.0.0-20250822173738-5341c9fc2974/go.mod h1:1Q2QRB5vob542x6P5pQXlGTYs5bYPxNG6ePcjTndA0A=
github.com/dvsekhvalnov/jose2go v0.0.0-20170216131308-f21a8cedbbae/go.mod h1:7BvyPhdbLxMXIYTFPLsyJRFMsKmOZnQmzh6Gb+uquuM=
github.com/elastic/go-sysinfo v1.15.3 h1:W+RnmhKFkqPTCRoFq2VCTmsT4p/fwpo+3gKNQsn1XU0=
github.com/elastic/go-sysinfo v1.15.3/go.mod h1:K/cNrqYTDrSoMh2oDkYEMS2+a72GRxMvNP+GC+vRIlo=
Expand Down
20 changes: 8 additions & 12 deletions vendor/github.com/docker/model-distribution/distribution/client.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

42 changes: 42 additions & 0 deletions vendor/github.com/docker/model-distribution/internal/bundle/bundle.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

73 changes: 73 additions & 0 deletions vendor/github.com/docker/model-distribution/internal/bundle/parse.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

85 changes: 85 additions & 0 deletions vendor/github.com/docker/model-distribution/internal/bundle/unpack.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

30 changes: 19 additions & 11 deletions vendor/github.com/docker/model-distribution/internal/gguf/create.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading