From a3996087d1d7370b34b541afae6b49630f2590a8 Mon Sep 17 00:00:00 2001 From: Garrett Hensley Date: Wed, 16 Jul 2025 13:30:09 -0400 Subject: [PATCH 1/2] added a readme --- README.md | 84 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 84 insertions(+) create mode 100644 README.md diff --git a/README.md b/README.md new file mode 100644 index 00000000..06519b07 --- /dev/null +++ b/README.md @@ -0,0 +1,84 @@ +# Docker Model CLI + +A powerful command-line interface for managing, running, packaging, and deploying AI/ML models using Docker. This CLI lets you install and control the Docker Model Runner, interact with models, manage model artifacts, and integrate with OpenAI and other backends—all from your terminal. + +## Features +- **Install Model Runner**: Easily set up the Docker Model Runner for local or cloud environments with GPU support. +- **Run Models**: Execute models with prompts or in interactive chat mode, supporting multiline input and OpenAI-style backends. +- **List Models**: View all models available locally or via OpenAI, with options for JSON and quiet output. +- **Package Models**: Convert GGUF files into Docker model OCI artifacts and push them to registries, including license and context size options. +- **Configure Models**: Set runtime flags and context sizes for models. +- **Logs & Status**: Stream logs and check the status of the Model Runner and individual models. +- **Tag, Pull, Push, Remove, Unload**: Full lifecycle management for model artifacts. +- **Compose & Desktop Integration**: Advanced orchestration and desktop support for model backends. + +## Installation +1. **Clone the repo:** + ```bash + git clone https://github.com/docker/model-cli.git + cd model-cli + ``` +2. **Build the CLI:** + ```bash + make build + ``` +3. **Install Model Runner:** + ```bash + ./model install-runner + ``` + Use `--gpu cuda` for GPU support, or `--gpu auto` for automatic detection. + +## Usage +Run `./model --help` to see all commands and options. + +### Common Commands +- `model install-runner` — Install the Docker Model Runner +- `model run MODEL [PROMPT]` — Run a model with a prompt or enter chat mode +- `model list` — List available models +- `model package --gguf --push ` — Package and push a model +- `model logs` — View logs +- `model status` — Check runner status +- `model configure MODEL [flags]` — Configure model runtime +- `model unload MODEL` — Unload a model +- `model tag SOURCE TARGET` — Tag a model +- `model pull MODEL` — Pull a model +- `model push MODEL` — Push a model +- `model rm MODEL` — Remove a model + +## Example: Interactive Chat +```bash +./model run llama.cpp "What is the capital of France?" +``` +Or enter chat mode: +```bash +./model run llama.cpp +Interactive chat mode started. Type '/bye' to exit. +> """ +Tell me a joke. +""" +``` + +## Advanced +- **Compose Integration:** + Use `model compose up` to orchestrate model backends with Docker Compose. +- **OpenAI Backend:** + Use `--backend openai` and provide your API key for OpenAI-compatible models. +- **Packaging:** + Add licenses and set context size when packaging models for distribution. + +## Development +- **Run unit tests:** + ```bash + make unit-tests + ``` +- **Generate docs:** + ```bash + make docs + ``` + +## License +[Apache 2.0](LICENSE) + +--- + +Docker Model CLI — The fastest way to run, manage, and ship AI models with Docker. From 44e847d0dea33f6f5e1ab189ecc916a944b9e38b Mon Sep 17 00:00:00 2001 From: Garrett Hensley <62925734+garrettHensley@users.noreply.github.com> Date: Wed, 20 Aug 2025 15:48:44 -0400 Subject: [PATCH 2/2] Adjusted readme from PR feedback --- README.md | 9 +-------- 1 file changed, 1 insertion(+), 8 deletions(-) diff --git a/README.md b/README.md index 06519b07..a3f8d628 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,7 @@ A powerful command-line interface for managing, running, packaging, and deployin - **Tag, Pull, Push, Remove, Unload**: Full lifecycle management for model artifacts. - **Compose & Desktop Integration**: Advanced orchestration and desktop support for model backends. -## Installation +## Building 1. **Clone the repo:** ```bash git clone https://github.com/docker/model-cli.git @@ -59,10 +59,6 @@ Tell me a joke. ``` ## Advanced -- **Compose Integration:** - Use `model compose up` to orchestrate model backends with Docker Compose. -- **OpenAI Backend:** - Use `--backend openai` and provide your API key for OpenAI-compatible models. - **Packaging:** Add licenses and set context size when packaging models for distribution. @@ -79,6 +75,3 @@ Tell me a joke. ## License [Apache 2.0](LICENSE) ---- - -Docker Model CLI — The fastest way to run, manage, and ship AI models with Docker.