VideoAnnotator 1.6.0

VideoAnnotator 1.6.0 is the consolidation release: one repository, current dependencies, a clean first contact, results out in formats researchers already use, a guide for every pipeline, and new default models.

Released quietly. The viewer is being redesigned for v1.7.0, the public release, so expect its pages to change. Pilot testing and announcements wait for v1.7.0.

Highlights

• Results out: one tidy table of every pipeline's results (CSV or Parquet, one row per event and measure, the same 19 columns for every pipeline), ELAN .eaf files, and a methods paragraph with BibTeX that names each model and its exact weights. Available from the viewer's Export menu, videoannotator export table|elan|methods <folder> (no server needed) and the API. Includes worked R and Python notebooks.
• New default models, each chosen on its upstream's published evaluation and checked on the demo clip:
  • Speech: Whisper large-v3-turbo on faster-whisper (was base on openai-whisper). A 1.6 GB first download; on a CPU, whisper_model: small is much faster. turbo can't translate: a translate request now names a model that can.
  • Speaker turns: pyannote speaker-diarization-community-1. One model licence to accept instead of three.
  • People and pose: yolo26n-pose.
  • Faces: age and gender are off by default (trained on adults, uninformative for infants). The first download is 125 MB instead of 1.2 GB.
• A guide for every pipeline: what it finds, the few settings that matter, published benchmarks quoted from their sources, what to cite, and why each default is what it is (videoannotator pipelines describe <name>).
• Hugging Face token from the viewer, Settings or the CLI, with no file to edit and no restart.
• A container that feels local: videoannotator-start for Docker and Podman shares your video folder read-only at its real path. Results go to ~/VideoAnnotator/<run>/, and videos are read in place.
• Run it again: rerun a job, reuse its settings on new videos, saved datasets, a prompt library and workbench for the VLM pipeline, and side-by-side comparison of two VLM runs.

Upgrading from 1.5

• Results differ from v1.5's, because the default models changed. Each pipeline's previous model is still one setting away (see the CHANGELOG), except speech's runtime: v1.6.0-rc1 was the last version with openai-whisper, so v1.5 transcripts can't be reproduced bit for bit.
• Security: before this release, request logging wrote API keys into logs/api_server.log. Delete old logs and create a new key.
• Python 3.12 and 3.13. The audio extra no longer builds anything from source.

Full details: CHANGELOG.md.

VideoAnnotator 1.6.0 RC1

Release candidate 1 for VideoAnnotator 1.6.0. It's here so the install can be tested exactly as a new user would follow it from GitHub. Please report anything that doesn't work as described.

The headline is Start VideoAnnotator. Researchers with Docker Desktop or Podman Desktop, and no Python, run one installer. VideoAnnotator then starts on their own computer, reads videos where they already are, and opens the viewer already connected.

Install

Install Docker Desktop or Podman Desktop first. Then:

Windows (PowerShell):

powershell -ExecutionPolicy Bypass -c "irm https://github.com/InfantLab/VideoAnnotator/releases/latest/download/install.ps1 | iex"

macOS and Linux:

curl -LsSf https://github.com/InfantLab/VideoAnnotator/releases/latest/download/install.sh | sh

Then use the "Start VideoAnnotator" shortcut, or run videoannotator-start.

This release's launcher runs the image ghcr.io/infantlab/videoannotator:1.6.0-rc1.

Known issues

• Windows: Docker Desktop and Podman can't run at the same time. Podman's machine takes the pipe Docker Desktop needs, and Docker Desktop then fails to start. Stop one before starting the other.
• Windows with Podman on WSL 3.0: containers run without cgroups. WSL 3.0 gives Podman's machine none (podman#29749). The launcher handles this itself.
• A release candidate won't update itself to 1.6.0. videoannotator-start update treats 1.6.0-rc1 and 1.6.0 as the same version. Run the installer again once 1.6.0 is out.
• Not on PyPI. Release candidates are published here and on GHCR only.

Full list of changes: CHANGELOG.md.

VideoAnnotator v1.5.0

Extras-based modular install, admin-triggered extras install from the API, server-side folder ingest, batches as a unit, the viewer bundled at /viewer, the Ollama VLM backend, and the fixes from the v1.5.0 end-to-end runs.

Install the core with uv sync, then add the pipelines you need as extras (uv sync --extra face, audio, scene, person, or all). An admin can also install them from the running server via POST /api/v1/pipelines/extras/{extra}/install.

Known viewer issues deferred to v1.6.0 (face boxes and OpenFace overlays show only briefly around each ~1 s sample) are listed in docs/development/roadmap_v1.6.0.md.

Added

• Create jobs from videos already on the server — POST /api/v1/ingest: one request turns a
  server-side folder into a whole batch of jobs, with no upload. POST /api/v1/jobs takes one
  video per multipart request, which for a forty-video corpus means forty uploads streaming
  gigabytes through a browser to land somewhere else on the same disk. Ingested jobs reference
  the videos where they already are — nothing is copied — and are ordinary jobs in every other
  respect: they queue, report progress, cancel, retry, and share a batch_id. Deleting one
  removes its results and never the original video. Unusable files (empty, unreadable) are
  reported per file rather than failing the batch. GET /api/v1/ingest/browse lists the folders
  the server will read, so a client can offer a folder picker — a browser cannot discover a real
  filesystem path on its own. Gated three independent ways: admin-only (matching the
  extras-install precedent), loopback callers only, and confined to VIDEOANNOTATOR_INGEST_ROOTS
  (default: the server user's home directory), with paths fully resolved before the check so
  neither .. nor a symlink can escape.

• Batches are addressable as a unit: GET /api/v1/batches lists every batch the server knows
  about (newest first, paginated) with the same aggregate GET /api/v1/batches/{id} returns for
  one, GET /api/v1/jobs?batch_id=... drills into a batch's member jobs, and
  POST /api/v1/batches/{id}/cancel cancels a whole batch in one call (reporting already-finished
  jobs as skipped rather than failing, mirroring batch-retry). Job submission also accepts a
  batch_name, surfaced on the job and on every batch summary. Together these mean a client no
  longer has to remember client-side which jobs it submitted together to show them as one thing:
  submitting 12 videos is one batch that survives a reload, a different browser, and a different
  machine — where before it was 12 unrelated job rows and no way to ask the server which of them
  belonged together. GET /api/v1/jobs?unbatched_only=true completes the picture by returning
  only jobs that belong to no batch (CLI submissions, and anything predating batch tagging), so a
  client showing "batches, plus anything ungrouped" doesn't have to filter a paginated list
  client-side — which silently hid ungrouped jobs whenever batched ones filled the page.
  Extends specs/008-batch-group-workflow/.

• GET /api/v1/auth/me: any authenticated caller can now check its own identity, including
  is_admin — the direct explanation for a 403 Administrator privileges required from an
  admin-only endpoint (e.g. the extras-install API below), which previously had no way to be
  self-diagnosed from the frontend.

• Self-service extras install API: admins can now install a named pipeline extras group
  (face, audio, scene, person, all, ...) directly through the API —
  POST /api/v1/pipelines/extras/{extra}/install triggers it as a trackable background job
  (GET /api/v1/pipelines/extras/install-jobs/{job_id}), no terminal access required. A completed
  install needs a server restart to activate its pipeline(s); GET /api/v1/pipelines gained a
  top-level restart_required flag so a client can tell "installed, pending restart" apart from
  "still unavailable." Admin-only (require_admin, new — surfaces the User.is_admin column
  through the API auth layer for the first time), and the extras-group name is always validated
  against the running install's actual declared groups before anything runs. This is the write-side
  counterpart to 004-extras-based-install's existing read-only available/install_hint fields.
  See specs/005-pipeline-extras-install/. Backend-only — a video-annotation-viewer UI for this
  (locked pipeline cards, an Install action, a restart banner) is a separate, future spec in that
  project.

• vlm_annotation pipeline: per-frame classification/captioning via a locally-hosted
  vision-language model served by Ollama, driven by a user-supplied prompt. Every sample point is one
  independent, stateless model call. Supports single_frame and frame_burst (multi-image window)
  sampling modes. Gated behind a new llm extras group (pip install videoannotator[llm]). See
  docs/development/vlm_annotation_pipeline.md for the full writeup, config reference, and a
  step-by-step testing guide. Implements the roadmap_v1.6.0.md Phase 2 "Local LLM/VLM Backend" item.

Changed

• src/videoannotator/viewer_static/ (the vendored Video Annotation Viewer build served at
  /viewer) refreshed from the July 2026 v0.6.3 build to v0.7.0, picking up vlm_annotation
  pipeline display support, ELAN ground-truth comparison, and job-config form fixes. Also fixed a
  previously-unverified deep-linking bug this surfaced: direct navigation to any /viewer/* SPA
  route (not just the root) now correctly returns 200 and renders the app, via a proper
  SPAStaticFiles fallback (api/main.py) rather than a 404-with-app-shell-content workaround.

Fixed

• Face analysis saved frames with no face as full-frame faces: DeepFace, run with
  enforce_detection=False, reports a frame with no face as one face covering the whole image,
  with confidence 0, and still assigns it an emotion, age and gender. The pipeline saved these as
  real detections with score: 1.0. They made up 143 of 255 face boxes in the v1.5.0
  end-to-end run. They're now dropped. The fallback path (used when analysis fails) also records
  each face's real detected box instead of a fixed [0, 0, 100, 100] placeholder.

• First-run API key banner: it now prints a one-click viewer-connect link and
  videoannotator generate-token for new keys. Before, it pointed at localhost:8000 and at
  python -m scripts.manage_tokens, which pip installs don't include.

• api/job_processor.py (the job-execution path used by the API server's automatic background
  processor) instantiated pipeline classes with no config at all, silently ignoring any per-pipeline
  config submitted with a job — unlike batch/batch_orchestrator.py, which already passed it
  correctly. Any pipeline relying on non-default config submitted through the API (not just
  vlm_annotation) was affected.

• scripts/start_server.sh asked for an admin email/username on every restart, even when a
  database (and its admin account) already existed from a previous run — it now only prompts on a
  genuine first-time setup (no videoannotator.db yet), matching setup-db's own idempotency.

• scripts/start_server.sh's dependency sync used a bare uv sync, which defaults to an exact
  sync — every restart silently uninstalled any extras group (e.g. face, audio) that hadn't
  been named in that exact invocation's --extra/SYNC_EXTRAS, including one installed via the
  new self-service extras-install API above. Now runs with --inexact (additive-only), matching
  what a "restart to activate a newly-installed pipeline" workflow actually needs.

• videoannotator generate-token had no concept of admin at all — a brand-new user it created
  (e.g. re-issuing a viewer key with an email that didn't exactly match the original setup-db
  admin) was always non-admin, with no flag to change that and no way for the resulting client to
  find out why admin-only actions then failed. It now grants admin by default to a new user while
  the deployment is still single-user (0 or 1 existing users before this one — the common case of
  one person re-connecting the viewer to their own server), accepts an explicit --admin/
  --no-admin override in either direction (including promoting/demoting an existing user), and
  always prints the resulting admin status.

• Every server log line was printed twice (once formatted, once as INFO:name:message). A bare
  logging.info() at import time in api/main.py (and module-level logging.warning() calls in
  four pipeline modules) made Python install a default root handler, so videoannotator.*
  records were emitted by both it and our own handler. They now log through named loggers.

• HUGGINGFACE_TOKEN set in .env was ignored in the dev container: the container forwards
  host variables it doesn't have as empty strings, and .env never overrides an existing
  variable. .env values now fill variables that are set but blank (config_env.load_env_file).

• OLLAMA_BASE_URL in the dev container was the literal http: the devcontainer
  ${localEnv:NAME:default} syntax cuts the default at its first colon. It is now set directly.

• face_openface3_embedding always failed inside the server with "OpenFace 3.0 not installed",
  although it was: OpenFace's modules run argparse on import, which rejected the server's own
  command-line arguments and exited. The import now runs with those arguments hidden, and the
  error states the real reason.

Extras-Based Modular Install & Registry Refactor

Implements specs/004-extras-based-install/ — moves heavy ML pipeline dependencies out of the
core install and into opt-in pip extras, and replaces the registry's hardcoded pipeline↔module
mapping with metadata-driven resolution.

Added

• Per-pipeline-family extras: pip install videoannotator[scene] (or face, person,
  audio, `face-la...

v1.4.4

Modularity, viewer integration, and JOSS resubmission release, responding to pre-review feedback on openjournals/joss-reviews#10182 and #10183.

Added

• Bundled viewer: The FastAPI service now serves Video Annotation Viewer's static build at /viewer, same-origin and zero-config (VIDEOANNOTATOR_ENABLE_VIEWER to disable). VAV remains fully usable standalone.
• Contract test: Added tests/contract/test_viewer_contract.py, which validates VideoAnnotator's exporter output against VAV's actual Zod schemas.
• Roadmap docs: Added docs/development/roadmap_v1.6.0.md (plugin ecosystem, local LLM/Ollama backend) and rewrote roadmap_v1.5.0.md around the modularity spec (specs/003-modular-pipeline-architecture).

Fixed

• Person-tracking contract gap: COCO output was missing person_id/person_label/label_confidence/labeling_method when identity labeling was disabled, which VAV requires as non-optional.
• setup-db DetachedInstanceError on the first command in the README quickstart.
• Hidden CLI commands: validate-emotion, generate-token, and setup-db were missing from --help due to a misplaced __main__ guard.
• First-run friction: torch/CUDA warmup moved off the first health-check request into a startup task; added a one-click /viewer-connect login bridge.
• Silent face-pipeline fallback: the face pipeline alias could silently pick an unconfigured/experimental variant instead of erroring; OpenFace3 now fails fast with one clear error instead of one per frame.
• Windows SQLite teardown (openjournals/joss-reviews#10182): the engine-disposal fix from this release didn't cover two test fixtures that construct SQLiteStorageBackend directly; both now close the backend in teardown.

Documentation

• Full stale-link/version sanity pass ahead of JOSS resubmission: fixed ~14 stale src/* paths in reviewer-facing docs left over from the src/videoannotator/ package restructure, backfilled missing v1.4.3/v1.4.4 changelog entries, corrected pyproject.toml mypy config pointing at removed paths, and fixed several stale version pins (INSTALLATION.md, Docker.md, output_naming_conventions.md, CITATION.cff).
• Added the interface figure to the combined JOSS paper and documented the /viewer integration and contract test as evidence of coordinated maintenance between VideoAnnotator and VAV.

See CHANGELOG.md for the full itemized history, and PR #6 for the bulk of this release's changes.

v1.4.3

Patch release fixing installability and CI raised during JOSS review (openjournals/joss-reviews#10182).

Fixed

• Restored missing storage modules (storage/manager.py, storage/providers/*) that an unanchored storage/ rule in .gitignore had silently excluded from the repository. Fresh clones previously failed with No module named videoannotator.storage.manager, and the API/worker could not start.
• Cross-platform install: pinned the torch/torchvision/torchaudio trio to the matched 2.6.0 release and restricted the CUDA (cu124) index to Linux, so macOS/Windows resolve the CPU build from PyPI. Previously uv sync failed on any non-Linux/non-CUDA platform.
• Bumped openai-whisper from ==20240930 to >=20250625.
• Repointed reviewer documentation from the removed scripts/verify_installation.py to the built-in videoannotator diagnose command.
• Cleared pre-existing lint, type-check, and test failures. CI is now green on Linux and macOS; the Windows leg is informational pending #5.

See PR #4 for the full set of changes.

v1.4.2 - JOSS Review release

This release accompanies the submission of VideoAnnotator to the Journal of Open Source Software. A parallel submission is being made for the companion project Video Annotation Viewer.

Below is a summary of all changes across the v1.4.x release series (v1.4.0 -- v1.4.2).

---

Added (v1.4.0)

• Flexible storage: New artifact download capabilities, including source video retrieval.
• Database-backed authentication: Migrated from file-based to database-backed auth for improved security and scalability.
• Artifacts API: New endpoint GET /api/v1/jobs/{id}/artifacts to download job results as a ZIP archive.
• Container tooling: Baked hadolint into Docker images and devcontainer; added git-lfs to CPU/GPU Dockerfiles.

Changed

• CLIP migration (v1.4.2): Migrated scene-classification pipeline from clip to open_clip, using the LAION-2B pretrained ViT-B-32 model for improved availability and reproducibility.
• HuggingFace auth (v1.4.2): Updated diarization and Whisper pipelines to use the current token parameter instead of the deprecated use_auth_token.
• JOSS manuscript (v1.4.1): Consolidated the paper into paper/paper.md and replaced docs/joss.md with a pointer to avoid divergence.
• Repository hygiene (v1.4.1): Moved top-level helper scripts into organized subfolders under scripts/; updated imports to the videoannotator.* namespace.
• Entrypoints (v1.4.1): api_server.py now acts as a compatibility wrapper; documentation recommends the videoannotator CLI.
• README (v1.4.1): Rationalized setup/install instructions, fixed broken links, replaced hard-coded test/coverage claims with CI status badges.
• Devcontainer (v1.4.2): Simplified forwarded-port list to the single default API port (18011).

Fixed

• Artifact downloads (v1.4.0): Ensured source video files are included in the downloaded artifact ZIP.
• Database GUID handling (v1.4.2): Added defensive error handling in the GUID type decorator for malformed UUID values.
• Diarization init (v1.4.2): Wrapped model loading in explicit error handling with a clear log message on failure.
• Docs (v1.4.1): Standardized examples on canonical API port 18011 and corrected Docker run port mappings.
• Docs (v1.4.1): Replaced placeholder docs/usage/accessing_results.md with a real results retrieval guide.

Removed (v1.4.2)

• Voice emotion baseline: Removed voice_emotion_baseline pipeline metadata and associated tests (superseded by LAION EmoNet voice pipeline).

Documentation (v1.4.2)

• Added JOSS cover letter (paper/cover_letter.md).
• Updated paper bibliography and CITATION.cff to v1.4.2.

---

Full Changelog: v1.3.0...v1.4.2

VideoAnnotator v1.3.0 - Production Reliability & Critical Fixes

Released: October 31, 2025 🎃

This release addresses critical production blockers identified during client integration testing and establishes a production-ready foundation for JOSS publication.

Highlights

Core Improvements

• Job Cancellation: Cancel running jobs via API with proper GPU cleanup and worker management
• Persistent Storage: Configurable STORAGE_DIR with retention policies - no more data loss on restart
• Config Validation: Schema-based validation catches errors before job submission
• Secure by Default: AUTH_REQUIRED=true with automatic API key generation and restricted CORS

Infrastructure

• Package Namespace: Migrated to modern videoannotator.* namespace (PEP 517/518)
• Enhanced Diagnostics: New CLI commands (diagnose system/gpu/storage/database/all)
• Environment Config: 19 configurable environment variables with full documentation

Quality & Testing

• 720/763 tests passing (94.4%) - improved from 607 (79.6%)
• Fixed 113 tests with real audio/video fixtures
• ffmpeg installed across all Docker images
• Exceeds 95% target by 23 tests

Bug Fixes

• Fixed pipeline import and name resolution failures
• Added missing pipeline metadata
• Resolved ephemeral storage data loss risks
• Standardized error formats across all API endpoints

Documentation

• 10+ new documentation files
• Complete API documentation with curl examples
• JOSS reviewer quick start guide
• Security configuration guide
• Migration guide with automated script

Success Criteria - ALL MET ✅

• Zero job failures due to pipeline naming
• Zero data loss on server restart
• All running jobs cancellable via API
• Invalid configs rejected at submission
• Authentication required by default
• 84% task completion (56/67 tasks)

See CHANGELOG.md for complete details.

What's Next

v1.4.0 (Q2 2026): First Public Release + JOSS Paper submission

v1.2.2 - import fixes

Minor maintenance release.

Normalized internal imports (removed obsolete src. prefixes) resolving a server startup error.
Updated CLI server target to api.main:app and bumped version metadata.
No API or CLI breaking changes.

VideoAnnotator 1.2.1 – Registry Expansion & Stabilization

Highlights

• YAML-driven pipeline registry with richer taxonomy (family, variant, tasks, modalities, capabilities, backends, stability).
• Auto‑generated pipeline spec + emotion output format and validator.
• Standard error envelope + enriched system health endpoint.
• New face & voice emotion pipelines (OpenFace3 embedding, LAION face/voice, baseline emotion).
• Light post-freeze stabilization: optional legacy API key validation and Windows-safe ASCII logging.

Added

• Dynamic pipeline metadata exposure (/api/v1/pipelines, CLI videoannotator pipelines --detailed --json).
• Emotion schema + validator + naming conventions spec.
• Health endpoint now reports pipeline count, uptime, and queue snapshot.
• New pipelines registered (face_openface3_embedding, face_laion_clip, voice_emotion_baseline).

Changed

• Replaced legacy single category field with multi-dimensional taxonomy arrays.
• All pipeline discovery now sourced from registry (no hard-coded lists).
• Logging: removed emojis in runtime output in favor of [OK], [WARNING], [ERROR].

Compatibility

• Non-breaking for existing clients except: invalid API keys now correctly rejected (401).
• Anonymous endpoints remain accessible where previously allowed.

Upgrade Notes

If you referenced category, map logic to tasks, pipeline_family, or modalities.
Use: videoannotator pipelines --json for machine-readable metadata.
Regenerate pipeline spec after adding pipelines via the provided script.

VideoAnnotator v1.2.0 - REST API, Docker support, uv packaging

Major release featuring:

• Complete REST API with FastAPI and integrated background processing
• Modern Docker deployment system with CPU, GPU, and dev containers
• uv package manager migration for 10x faster dependency management
• Ruff integration for modern linting and formatting
• Comprehensive deployment documentation and troubleshooting

Production-ready features:

• Real-time job processing and status tracking
• Interactive API documentation at /docs
• Cross-platform compatibility and container orchestration ready
• Secure authentication framework with JWT tokens
• Complete backward compatibility with existing pipelines

Technical improvements:

• Fixed Docker builds with proper package management
• Optimized container sizes and build times
• Enhanced error handling and recovery systems
• Modern development workflow with integrated tooling