You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Summary of LUCE_* environment variables recognized across the
codebase, grouped by subsystem. Most are runtime toggles read via getenv /
os.environ; a few are build/compile-time or harness knobs (noted where relevant).
Policy (see server/docs/ENVIRONMENT.md): new features should ship as CLI flags
or defaults. Env vars are reserved for burn-in kill switches and debug
instrumentation. Treat undocumented variables as internal.
Status legend
Tags in the tables below flag variables that are not part of the intended
long-term serving surface:
🐛 debug — profiling/telemetry/ablation instrumentation. Zero-cost when
unset and never required for correct serving; safe to ignore in production.
🔀 kill-switch — burn-in toggle for a landed default, documented with the
intent to be deleted once the feature has soaked.
🧪 test/bench — only read by tests, benchmarks, or harness scripts.
⚠️removal candidate — legacy/one-off/likely-obsolete; prefer the CLI flag
or default where one exists.
Untagged variables are operational tuning knobs.
Server / runtime configuration
Variable
Purpose
LUCE_HOST
Server bind host.
LUCE_PORT
Server bind port.
LUCE_BIN
🧪 test/bench Path to test_dflash for the bench scripts.
LUCE_SERVER_BIN
Path to luce_server (container entrypoint, harness).
LUCE_BIN_AR
Alternate/AR binary path for benchmarks.
LUCE_DIR
Base working directory.
LUCE_SHARE_DIR
Static/share asset directory served by the HTTP server.
LUCE_MODEL_CARDS_DIR
Directory of model-card definitions.
LUCE_MODEL_NAME
Model name/identifier.
LUCE_TOKENIZER
Tokenizer path/identifier.
LUCE_TARGET
Target model path (container entrypoint, run.py).
LUCE_DRAFT
Draft file or directory (container entrypoint, run.py); none disables the container draft.
LUCE_TARGET_DEVICE
Default --target-device (backend:gpu or auto) when no flag or profile names one; the container sets auto.
LUCE_PROFILE / LUCE_ARGS
Container entrypoint: --profile name and extra luce_server flags.
LUCE_IMAGE_INFO_PATH
Path to image/build info metadata.
LUCE_MAX_CTX
Container entrypoint --max-ctx (default: sized from the GPU's memory).
LUCE_MAX_CONTEXT
KV sizing override for laguna and qwen35moe expert placement.
LUCE_DEFAULT_MAX_TOKENS
Default generation token cap.
LUCE_QUEUE_LIMIT
Container entrypoint --queue-limit: requests that may wait while every slot is busy; past them a request gets 503 (default: no limit).
LUCE_IGNORE_EOS
Ignore EOS token during generation.
LUCE_NO_PREAD
🔀 kill-switch Every model loader copies its weights out of the file mapping instead of using tensor_file_reader's threaded reads (which bypass the page cache for a file larger than half the available memory).
LUCE_LAZY
Container entrypoint: 1 adds --lazy-draft (needs a draft and LUCE_PREFILL_DRAFTER).
Disable automatic HIP UMA (unified memory) selection.
LUCE_HIP_UMA_MIN_FRAC
Minimum VRAM fraction before HIP UMA kicks in.
LUCE_HIP_UMA_NO_RAM_CHECK
🔀 kill-switch Skip the check that fails a managed buffer on an integrated GPU when it does not fit in available RAM, and the bound of an integrated GPU's free memory by its own VRAM left plus the available RAM (ggml_backend_cuda_get_device_memory, ggml_backend_cuda_get_device_room, ggml_backend_cuda_get_uma_room).
LUCE_WAVE_SIZE
HIP wave size (compile flag, e.g. gfx1151 needs 32).
Speculative decoding — drafter
Variable
Purpose
LUCE_DRAFT_KV
🔀 kill-switch=0 restores per-step drafter window recompute instead of the ring cache.
Speculative decoding for sampled requests (sampled verification). On by default for Qwen 3.5/3.6/3.8 targets, 0 disables it; 1 enables it for Laguna.
LUCE_VERIFY_WIDTH
Verify batch width.
LUCE_GPU_SAMPLE
GPU sampling path.
LUCE_GPU_ARGMAX / LUCE_GPU_VERIFY_ARGMAX
GPU argmax for sampling/verification.
LUCE_GPU_DRAFT_TOPK
GPU draft top-k.
LUCE_TQ3_VERIFY
TQ3-quantized verify path.
LUCE_N_SAMPLE / LUCE_SAMP
🧪 test/bench Sample count/mode.
LUCE_SV_DEBUG
🐛 debug Sampled-verify debug output.
Adaptive experts / adaptive verify width
Variable
Purpose
LUCE_ADAPTIVE_K_TAU
Cumulative combine-weight threshold for per-token expert gating (prefer --adaptive-experts).
LUCE_ADAPTIVE_K_DENSE
CSV of MoE layers kept dense under adaptive-K.
LUCE_ADAPTIVE_WIDTH_MIN
Minimum adaptive verify width.
LUCE_ADAPTIVE_WIDTH_THETA
Threshold controlling adaptive verify width.
LUCE_HYBRID_HOT_PCT
Hot-expert percentage for hybrid MoE.
KVFlash (KV cache pager)
Variable
Purpose
LUCE_KVFLASH
Enable KVFlash (prefer CLI --kvflash; token count or auto).
LUCE_KVFLASH_DRAFTER
KVFlash for the drafter cache.
LUCE_KVFLASH_MAX_POOL
Max KVFlash pool size.
LUCE_KVFLASH_POLICY
KVFlash eviction/placement policy.
LUCE_KVFLASH_TAU
KVFlash tau threshold.
LUCE_LAGUNA_SWA_RING
🔀 kill-switch=0 keeps SWA layers on pool-sized caches under KVFlash.
KV cache quantization / dtype
Variable
Purpose
LUCE_CACHE_TYPE_K / LUCE_CACHE_TYPE_V
KV cache K/V dtype.
LUCE_KV_TYPE
KV cache type selector.
LUCE_FEATURE_DTYPE
Draft feature-ring dtype.
LUCE_KV_F16
F16 KV cache.
LUCE_KV_K / LUCE_KV_V
Per-side (K/V) KV quantization.
LUCE_KV_Q4 / LUCE_KV_TQ3 / LUCE_KV_TBQ
Quantized KV formats (Q4 / TQ3 / TBQ).
FlashPrefill / prefill
Variable
Purpose
LUCE_FP_USE_BSA
Gemma 4 sliding-window prefill: dispatch the FlashPrefill sparse forward through the BSA launcher (CUTLASS on CUDA, rocWMMA on HIP). Enabled by presence; on by default with PFlash. The PFlash scorer does not read it.
LUCE_FP_HIP_ROW
HIP row-kernel path for flash prefill.
LUCE_FP_PROFILE / LUCE_FP_DUMP_COUNTS
🐛 debug FlashPrefill profiling/debug.
LUCE_PFLASH
Docker entrypoint: auto (default), on or off, passed as --pflash. With auto, PFlash is on for supported targets when LUCE_PREFILL_DRAFTER (an existing file) and PFLASH_SCORING_HEAD_GGUF are set.
LUCE_PREFILL_THRESHOLD
Prefill length threshold.
LUCE_PREFILL_DRAFTER
Docker entrypoint: the PFlash drafter GGUF (Qwen3.5-0.8B), passed as --prefill-drafter when LUCE_PFLASH is not off.
LUCE_PREFILL_KEEP
Docker entrypoint: fraction of source tokens PFlash keeps, passed as --prefill-keep-ratio.
Optional prefill/prefix cache slot override. When unset, the container preserves the native server defaults (prefix: 32; exact prefill: 0). Set either value to 0 for an explicit opt-out.
LUCE_PREFIX_CACHE_MAX_MIB
Container-entrypoint equivalent of --prefix-cache-max-mib; the native luce_server binary does not read it. When unset, the server default auto applies.
LUCE_PREFILL_POOL_TRIM_TOKENS
🔀 kill-switch Interval (default 8192 tokens, 0 = off) for trimming cached legacy CUDA/HIP pool allocations between completed Qwen3.5 prefill chunks, so a single long prefill does not exhaust VRAM before request cleanup.