Skip to content

Native Cosmos Driver — Binary Size Optimization Measurement #4748

Description

@simorenoh

Crate: azure_data_cosmos_driver_native (thin C-ABI wrapper, [lib] name = "azurecosmosdriver",
crate-type = ["cdylib", "staticlib"]) over azure_data_cosmos_driver with features ["tokio", "rustls"].
Question: take the current, full-feature driver as-is (no feature removal) and measure what
standard production binary-size optimizations buy us — to inform the (a) Rust native lib + cgo/FFI vs.
(b) native Go port decision.

All features stay in. Query planning is treated only as an attribution question, never removed.


1. Executive summary

  • Naive starting point (cargo build --release): the shipped dynamic library is
    10.8 MB (Windows amd64 DLL 10.84 MB, Linux amd64 .so 11.17 MB, Linux/macOS arm64 ~9.5 MB).
  • Best realistic shippable production build (release + fat LTO + opt-level="z" +
    codegen-units=1 + strip — keeps panic unwinding, so correctness is preserved):
    • Windows amd64: 5.00 MB  |  Linux amd64: 5.80 MB  | 
      Linux arm64: 4.54 MB  |  macOS arm64: ~4.5 MB (estimate)
    • That is a −48 % to −54 % cut vs. plain release, and it compresses to ~1.7–1.9 MB (xz) /
      ~2.2–2.5 MB (gzip) for download/distribution.
  • Biggest single optimization knob: **opt-level="z"** (size-optimize) — on its own
    −29.5 % (Windows) / −20 % (Linux). Fat LTO is the second lever (−12 % Windows / −20–23 % Linux);
    the two stack well. Stripping is effectively a no-op on the release artifact on every platform
    (see §4).
  • panic="abort" would go further (Win 3.38 MB, Linux amd64 5.25 MB, arm64 4.00 MB; up to −69 %)
    but is correctness-risky for this crate and is NOT recommended — it removes the unwinding the
    wrapper relies on at the FFI boundary (see §5). Reported for completeness only.
  • Effective static-link footprint ≈ the dynamic library size (measured: Win 10.83 MB, Linux amd64
    10.74 MB at release; static is actually slightly tighter thanks to --gc-sections). So a Go
    customer consuming the driver — whether by dlopening the cdylib or cgo-static-linking the archive —
    pays roughly the same ~5–6 MB per platform for a production-optimized build.
  • What a Go customer effectively consumes (production combined build, per platform):
Platform Dynamic (cdylib) Effective static (cgo) Compressed (xz)
Windows amd64 5.00 MB \~5.0 MB 1.75 MB
Linux amd64 5.80 MB \~5.7 MB 1.86 MB
Linux arm64 4.54 MB \~4.5 MB 1.72 MB
macOS arm64 \~4.5 MB (est.) \~4.5 MB (est.) \~1.7 MB (est.)
  • Local query planning adds ~2 % and is DCE'd out today. The driver's own Rust SQL planner
    (crate::query lexer+parser+AST+plan generator, ~5.9k LOC) is **not yet wired into the production
    path** (SUPPORTED_QUERY_FEATURES = "None"; the driver defers to the Gateway), so the linker
    dead-code-eliminates it entirely — 0 bytes in today's numbers. When it is switched on, it adds a
    measured +112 KiB (+2.2 %) to the combined build → 5.11 MB on Windows amd64 (xz 1.78 MB, only
    +30 KiB compressed). Query planning is not a meaningful size lever in either direction (see §7).
  • Runtime performance is unaffected by the size knobs — including opt-level="z". A 15-minute
    live end-to-end endurance test per config (70/20/10 read/write/query, concurrency 16) shows every
    optimized build — release, opt-s, opt-z, fat-LTO, and the combined production config — landing
    within the live account's own run-to-run noise of each other (overall throughput 142–150 ops/s,
    read p50 33–37 ms, p99 ~0.93–1.0 s across all configs, with **no systematic ordering by
    optimization level**). The opt-z "small" build the reviewer flagged shows **no measurable
    regression**: the workload is I/O-bound, so the µs-scale CPU cost of size-optimized codegen is fully
    masked by network+RU latency (see §9).
  • Verdict: native-library size is not a blocker for the cgo approach. A production build lands
    at ~5 MB/platform (~1.8 MB compressed), comparable to or smaller than many single Go dependencies.

2. Environment & reproducibility

Item Value
Host Windows 11, x86_64-pc-windows-msvc
rustc / cargo 1.95.0 (stable, pinned)
Workspace commit 83e95ecd39 (branch simorenoh/native-lib-size-optimization, off main)
Cargo.lock last changed a080bf7 (2026-07-07); SHA-256 E67D22AF…10E4A49A
Windows amd64 native MSVC toolchain (measured)
Linux amd64 / arm64 cargo-zigbuild 0.23.0 + zig 0.16.0 (ziglang PyPI), cross from Windows (measured)
macOS arm64 cargo-zigbuild — compile succeeds, final dylib link needs the Apple SDK (Security/CoreFoundation frameworks) unavailable on Windows → reported as estimate anchored to measured aarch64-linux codegen
Strip/size tooling llvm-strip / llvm-size / llvm-nm from llvm-tools-preview
Attribution cargo-bloat 0.12.1, cargo tree
Compression Python stdlib gzip -9 and lzma/\`xz -9 EXTREME\`
Crypto backend rustls 0.23 + aws-lc-rs 1.17 (aws-lc-sys 0.41) + rustls-platform-verifier (OS cert store)
Dependency closure 153 crates (normal build graph)

Configurations were applied with per-run cargo build --config 'profile.release.KEY=VALUE' overrides
(TOML single-quoted literals for PowerShell), not by editing the committed profile — the repo tree
is left clean. Each distinct config has its own cargo fingerprint.

Anchor cell. The reproducibility anchor is config #2 (release cdylib, unstripped, Windows amd64):
11,366,912 B = 10.84 MiB. The earlier Go-POC snapshot reported ~10.23 MiB for the same artifact —
our number is ~6 % larger, in the expected direction and consistent with **dependency version
drift** between the POC snapshot and current main (Cargo.lock refreshed 2026-07-07: aws-lc-sys 0.41,
rustls 0.23.40, reqwest 0.13.3, etc.). This is a dependency-timeline difference, not a methodology
discrepancy — the measurement path is identical.

Measurement method. For each config we build both the cdylib and the staticlib; record on-disk
size, an explicit llvm-strip --strip-all size, gzip(-9) and xz(-9|EXTREME), and build wall-time.
The raw .a/.lib archive is reported for transparency but is not the customer cost (it is an
archive of all object files); the honest static number is the effective linked contribution
(§6).


3. Optimization matrix — measured results

Sizes are the on-disk dynamic library (DLL/.so/.dylib) — what a Go program loads. Δ% is vs. plain
--release on the same platform. "MB" = MiB (1 048 576 B). Compression columns are gzip(-9) / xz(-9E).

Windows amd64 — native, measured

# Config Library Δ% vs release gzip xz Build\*
1 Baseline (debug)† 14.93 MB +37.8 % 4.84 MB 3.18 MB 233 s
2 Release 10.84 MB 4.09 MB 2.76 MB 204 s
3 Release + strip 10.84 MB +0.0 % 4.09 MB 2.76 MB 172 s
4 Release + Thin LTO 11.08 MB +2.2 % 4.20 MB 2.82 MB 173 s
5 Release + Fat LTO 9.54 MB −12.0 % 3.74 MB 2.59 MB 307 s
6 Release + opt "s" 8.27 MB −23.7 % 3.21 MB 2.30 MB 105 s
7 Release + opt "z" 7.64 MB −29.5 % 3.01 MB 2.18 MB 97 s
8 Combined (strip+fatLTO+z+cu1) 5.00 MB −53.8 % 2.19 MB 1.75 MB 200 s
9 + panic="abort" ⚠️ 3.38 MB −68.8 % 1.74 MB 1.42 MB 140 s

† On MSVC, debug info lives in the separate .pdb, so the debug DLL looks small (14.9 MB); the
comparable "with-symbols" number on ELF is ~112 MB (see Linux baseline). Release vs debug DLL is −27 %.

Linux amd64 — cargo-zigbuild, measured

# Config Library Δ% vs release gzip xz Build\*
1 Baseline (debug) 112.48 MB (stripped 18.37 MB) +860 % 32.1 MB 18.72 MB 296 s
2 Release 11.17 MB 4.46 MB 2.88 MB 132 s
3 Release + strip 11.17 MB −0.0 % 4.46 MB 2.88 MB 299 s
4 Release + Thin LTO 11.25 MB +0.7 % 4.52 MB 2.86 MB 307 s
5 Release + Fat LTO 8.95 MB −19.9 % 3.84 MB 2.60 MB 362 s
6 Release + opt "s" 9.13 MB −18.3 % 3.62 MB 2.45 MB 254 s
7 Release + opt "z" 8.91 MB −20.2 % 3.50 MB 2.41 MB 299 s
8 Combined 5.80 MB −48.1 % 2.51 MB 1.86 MB 293 s
9 + panic="abort" ⚠️ 5.25 MB −53.0 % 2.24 MB 1.64 MB 106 s

Linux arm64 — cargo-zigbuild, measured

# Config Library Δ% vs release gzip xz Build\*
1 Baseline (debug) 111.52 MB (stripped 15.31 MB) +1021 % 31.4 MB 18.09 MB 330 s
2 Release 9.49 MB 4.19 MB 2.57 MB 353 s
3 Release + strip 9.49 MB +0.0 % 4.19 MB 2.57 MB 298 s
4 Release + Thin LTO 9.61 MB +1.2 % 4.26 MB 2.55 MB 273 s
5 Release + Fat LTO 7.33 MB −22.7 % 3.59 MB 2.30 MB 346 s
6 Release + opt "s" 7.84 MB −17.4 % 3.36 MB 2.16 MB 269 s
7 Release + opt "z" 7.50 MB −20.9 % 3.37 MB 2.24 MB 248 s
8 Combined 4.54 MB −52.2 % 2.36 MB 1.72 MB 372 s
9 + panic="abort" ⚠️ 4.00 MB −57.8 % 2.08 MB 1.51 MB 175 s

macOS arm64 — estimate (link needs Apple SDK; see §2)

All Rust and the aws-lc-sys C/asm crypto compiled cleanly for aarch64-apple-darwin; only the
final dylib link failed on Windows (unable to find framework 'Security'/'CoreFoundation'). Those
frameworks are externally, dynamically linked (they add ~0 to the dylib), and the machine code is
identical aarch64 to the measured Linux arm64 build. So the mach-O dylib is estimated at the Linux
arm64 figures (±a few % for container overhead):

Config Library (est.) xz (est.)
Release \~9.5 MB \~2.6 MB
Combined \~4.5 MB \~1.7 MB
panic="abort" ⚠️ \~4.0 MB \~1.5 MB

* Build times are fully characterized on Windows and indicative on cross targets (cross builds
include zig-driven C compilation of aws-lc). Fat LTO + codegen-units=1 is the slow combination
(roughly 1.5–3× a plain release link); opt-level alone is cheap and can even speed codegen.


4. Per-knob explanation vs. measured effect

Knob What it does Expected Measured (this crate)
--release opt-level=3, no debug assertions Large drop vs debug The real baseline (\~10.8 MB)
strip="symbols" Drop symbol/debug tables Big on ELF/Mach-O, small on MSVC ≈0 % on the release artifact, every platform. The release cdylib carries no meaningful symbol table to begin with (MSVC keeps debug in .pdb; the zig/lld release .so is already symbol-free — llvm-strip removed 176 B). Symbols only matter for the debug build (112 MB → 18 MB).
lto="thin" Cheap cross-crate inlining/DCE Moderate shrink Slightly grew the library on all platforms (+0.7–2.2 %). Not worth it here.
lto="fat" Whole-program LTO Best DCE, slow −12 % (Win) / −20–23 % (Linux). Strong, but the slowest single build.
opt-level="s" Optimize for size Moderate shrink −18–24 %
opt-level="z" Aggressively size-optimize Usually smallest single knob −20 % (Linux) / −29.5 % (Win) — the biggest single lever.
Combined + codegen-units=1 Stack fat LTO + z + 1 CGU + strip Best realistic −48 to −54 %. Levers stack well (lots of monomorphized async state machines get merged).
panic="abort" ⚠️ Remove unwind tables/landing pads Extra shrink Additional −5 to −15 pts (Win to 3.38 MB). Correctness-risky — see §5.

4A. What each knob actually does (mechanism & effect on the shipped library)

This section is the "background" behind the table above: for each option, what the compiler/linker
physically does to the artifact the v2 Go SDK links against, and the trade-offs beyond raw size
(runtime speed, build time, debuggability, correctness). Every knob below is a Cargo build profile
setting — it changes how the same source is compiled/linked, never what features are in it. The
public C ABI (azurecosmosdriver.h, the 54 cosmos_* exports) is byte-for-byte identical across all
of them; only the machine code behind those symbols changes.

Debug vs. --release (the baseline choice)

  • What it does: the debug profile compiles at opt-level=0 — the compiler emits code in the most
    direct, un-transformed way: no inlining, every variable gets a stack slot, debug_assertions and
    integer-overflow checks are compiled in, and full debug info (DWARF on Linux/macOS, or a .pdb on
    Windows) is generated. --release switches to opt-level=3 (aggressive speed optimization), turns
    off debug assertions and overflow checks, and enables inlining and the full LLVM optimization
    pipeline.
  • Effect on the library: release is both far smaller and far faster at runtime. The ELF debug
    .so is ~112 MB (mostly DWARF); the release .so is ~11 MB. This is the single largest step and is
    non-negotiable for shipping — every other knob below is a refinement on top of release.
  • Trade-off: you lose source-level debuggability of the native library (line-level breakpoints,
    local-variable inspection). For a driver shipped as an opaque .dll/.so to Go consumers, that's the
    correct trade.

strip = "symbols" (symbol/debug-table removal)

  • What it does: a compiled object carries a symbol table — a map of names (function/static
    names) to addresses — plus any embedded debug sections. strip deletes those tables from the binary.
    It removes metadata about the code, never the code itself.
  • Effect on the library: on the release build this is nearly a no-op here, and the reason is
    platform mechanics: on MSVC/Windows the debug info was never in the DLL to begin with — it lives
    in a separate .pdb file — so there's almost nothing to strip. On Linux/macOS the release
    artifact built by lld/zig already ships without a meaningful local symbol table, so llvm-strip
    removed only ~176 bytes. Stripping only pays off dramatically on the debug artifact (112 MB → 18 MB
    on ELF), which we'd never ship anyway.
  • Trade-off: none for functionality; you lose symbol names in native crash stack traces (addresses
    still resolve if you keep the separate debug file / .pdb). Safe to always enable.

lto = "thin" (Thin Link-Time Optimization)

  • What it does: normally each crate is optimized on its own and the linker just staples the results
    together, so the optimizer can't see across crate boundaries. LTO defers optimization to link time
    when the whole program is visible. Thin LTO does this in a fast, parallel, per-module summary form
    — it enables cross-crate inlining and some cross-crate dead-code elimination without the full cost of
    fat LTO.
  • Effect on the library: counter-intuitively it slightly grew the library on all three
    platforms (+0.7–2.2 %). Thin LTO's cross-crate inlining pulled more code across boundaries (inlining
    duplicates a callee into each caller) than its dead-code pass removed, for this dependency graph.
  • Trade-off: usually a runtime-speed win at modest build cost; here it's a small size loss, so we
    don't use it. Not a correctness risk — just not helpful for size on this crate.

lto = "fat" (Full/Fat Link-Time Optimization)

  • What it does: merges the LLVM IR of all crates into one giant module and runs the optimizer
    over the whole program at once. This gives the most aggressive whole-program dead-code elimination
    (unused paths across the entire dependency tree collapse) and the best cross-crate inlining/de-dup.
  • Effect on the library: a strong shrink — −12 % on Windows, −20–23 % on Linux — because this crate
    pulls a big dependency tree (reqwest/hyper/rustls/tokio) of which any single build uses only a subset;
    fat LTO proves large chunks unreachable and drops them.
  • Trade-off: it's the slowest single build (whole-program optimization can't be parallelized the
    way normal codegen is) — roughly 1.5–3× a plain release link. Runtime performance is typically equal
    or slightly better. No correctness risk. This is the main "size" lever that's also safe to ship.

opt-level = "s" (optimize for size)

  • What it does: switches the optimizer's cost model from "make it fast" (opt-level=3) to "make it
    small." Concretely, LLVM stops applying size-increasing speed transforms — most importantly it stops
    aggressive loop unrolling and function inlining, and prefers compact instruction sequences —
    while keeping the rest of the optimization pipeline.
  • Effect on the library: −18–24 %. A large fraction of this crate is monomorphized async fn state
    machines (the same generic future compiled several times for different concrete types); the size cost
    model stops duplicating/unrolling those, so a lot of near-duplicate code shrinks.
  • Trade-off: slightly slower hot-path execution in theory (less inlining/unrolling). For an
    I/O-bound network driver — where time is dominated by TLS handshakes and network round-trips, not CPU
    — this is effectively unmeasurable. Cheap to build (often faster than opt-level=3 because there's
    less code to optimize). No correctness risk.

opt-level = "z" (aggressively optimize for size)

  • What it does: like "s", but goes further — it also disables the last size-increasing heuristics
    "s" still allows (e.g. it won't even auto-vectorize loops, and inlining is minimized further).
  • Effect on the library: the biggest single knob — −20 % on Linux, −29.5 % on Windows. Same
    mechanism as "s" but more so; it squeezed the most out of the monomorphized async code.
  • Trade-off: same category as "s" — a theoretical CPU-throughput cost that doesn't matter for an
    I/O-bound driver. It won every platform cell here, so it's the size knob we fold into the production
    config. No correctness risk.

codegen-units = 1 (single codegen unit)

  • What it does: by default Rust splits each crate into ~16 "codegen units" so LLVM can compile them
    in parallel (faster builds), but that parallelism costs some optimization quality — the optimizer
    can't de-duplicate or inline across the unit boundaries within a crate. Setting it to 1 compiles
    each crate as a single unit: slower, but the optimizer sees the whole crate and can merge duplicates
    and dead-strip more thoroughly.
  • Effect on the library: on its own it's a small win; its real value is compounding with fat LTO
    and opt-z in the combined config (fewer duplicated instantiations survive).
  • Trade-off: slower compilation (loses intra-crate parallelism). No runtime or correctness downside
    — purely build-time vs. size. Safe to ship.

The combined production config (fat LTO + opt-z + codegen-units=1 + strip)

  • What it does: stacks the four safe levers above. Fat LTO + codegen-units=1 give the optimizer
    maximum whole-program visibility; opt-z makes its cost model favor small code throughout; strip
    removes leftover metadata.
  • Effect on the library: −48 to −54 % — roughly half the size of a naive release build
    (Win 5.00, Linux amd64 5.80, arm64 4.54 MB). The levers stack well precisely because they attack the
    same root cause from different angles: this crate's bulk is many near-duplicate monomorphized async
    state machines, and each lever helps merge/shrink/drop them.
  • Trade-off: the slowest build in the matrix (fat LTO + 1 CGU), but still minutes, not hours. **This
    is the recommended shipping configuration** — all four knobs are pure build settings with no runtime
    or correctness cost. It deliberately keeps panic = "unwind" (the default); see below.

panic = "abort" ⚠️ (remove unwinding) — measured, not recommended

  • What it does: Rust's default panic strategy is unwind: when a panic! fires, the runtime
    walks back up the stack running destructors, and — critically for this crate — that unwind can be
    caught (via catch_unwind) and turned into a normal error. To support unwinding, the compiler emits
    landing pads and unwind tables (extra per-function metadata describing how to clean up each
    frame). panic = "abort" changes the strategy so a panic immediately aborts the process instead;
    the compiler can then omit all that unwinding machinery, which is why the binary shrinks further.
  • Effect on the library: a big additional cut (Windows to 3.38 MB, up to −69 % vs release) because
    the unwind tables/landing pads across a large async codebase are substantial.
  • Trade-off — this is a correctness change, not just a size knob: the FFI wrapper relies on
    FutureExt::catch_unwind at the C-ABI boundary to uphold its **"every submit produces exactly one
    completion"** invariant. If a driver future panics, the wrapper catches it and returns exactly one
    error completion to Go. With panic = "abort" there is no unwinding to catch — a single panic would
    abort the entire host process (the Go application), not just fail one operation. That's why it's
    flagged as not shippable for this crate despite the attractive size number; it's in the tables
    only for completeness. (See §5 for the full argument.)

Two knobs measured but not folded into the recommendation

  • Thin LTO — grew the library (above), so excluded.
  • panic = "abort" — correctness-risky (above), so excluded from the recommended combined config
    even though it produces the smallest binaries.

5. panic="abort" caveat (do not ship as-is)

The wrapper deliberately wraps each driver future in FutureExt::catch_unwind at the C-ABI boundary to
uphold its "every submit produces exactly one completion" invariant even if a driver future panics.
panic="abort" removes stack unwinding, so a panic would abort the whole process instead of being
caught and turned into a single error completion — breaking the correctness model across the FFI
boundary (and taking down the host Go process). The −15 % additional size win is real but should be
treated as not shippable for this crate unless the FFI contract is redesigned. It is included in the
tables strictly for completeness.


6. Static-link "effective" footprint (the honest cgo number)

The raw archive dramatically over-counts (it holds every object file): the Windows release .lib is
68 MB, Linux release .a is 80 MB — neither is what a consumer ships. The honest number is the
contribution to a final linked binary with dead-code/section GC + strip. We linked a probe
referencing all 54 exported cosmos_* functions (the full API closure a real consumer retains)
against the archive, GC'd unused sections, stripped, and subtracted an empty program built identically:

Platform Config Effective static cdylib (same config) Note
Windows amd64 release 10.83 MB 10.84 MB /OPT:REF /OPT:ICF, /MD CRT
Linux amd64 release 10.74 MB 11.17 MB --gc-sections + llvm-strip; static slightly tighter

Conclusion: effective static footprint ≈ the dynamic library size, so it tracks the cdylib across
configs. A production combined static link therefore lands at ~5–6 MB (amd64) / ~4.5 MB
(arm64), the same order as the cdylib. (Static consumers must also link the platform system libs the
driver reports via --print native-static-libs: on Windows ntdll userenv ws2_32 dbghelp + the CRT;
on Linux the usual pthread dl m rt + libc; on macOS Security + CoreFoundation frameworks.)


6A. Distribution model — download cost vs. binary cost (the "~20 MB?" question)

A natural worry: "if we ship an optimized library for 4 platforms, do we add ~20 MB to the SDK?"
The answer depends entirely on how the libraries are distributed, and it separates into three very
different numbers that are easy to conflate.

The three costs (production combined build)

# Cost Value What it is
1 Repo/module storage (all 4 committed, raw) 19.9 MB On-disk sum of the four platform libraries if you commit them uncompressed. This is a storage number, never what an end user runs.
2 Download (all 4 committed, compressed) \~9.4 MB What go get / git actually transfers — module zips and git objects are DEFLATE-compressed. One-time, then cached in the module cache.
3 Per-user build/binary cost \~5 MB What any single consumer's build actually consumes — only their own platform's library is linked. The other three sit unused.

Per-platform breakdown of the 19.9 MB / 9.4 MB totals:

Platform Combined library (raw) gzip xz
Windows amd64 5.00 MB 2.19 MB 1.75 MB
Linux amd64 5.80 MB 2.51 MB 1.86 MB
Linux arm64 4.54 MB 2.36 MB 1.72 MB
macOS arm64 (est.) 4.54 MB 2.36 MB 1.72 MB
Total (4) 19.9 MB 9.4 MB \~7.0 MB

Key point: the 20 MB is a repo-storage figure that exists only if you commit all four raw
binaries into the Go module. No end user's build or shipped binary ever carries 20 MB — a build on
any one platform links only that platform's ~5 MB library (static) or ships one ~5 MB dynamic library
beside the Go executable.

Three ways to distribute, and what each costs the consumer

Approach What go get pulls Build links Trade-off
Commit all 4 into the Go module \~9.4 MB (all, compressed) their \~5 MB Simplest; every consumer downloads all platforms once (cached). Bloats the module/repo.
Fetch-per-platform (binaries in GitHub Releases / package registry, pulled at build via go generate or a download step) \~1.8 MB (their platform only, xz) their \~5 MB Leanest download; the 20 MB total never lands on anyone. Adds an install/build step and a fetch-integrity concern.
go:embed + build tags \~9.4 MB (module still holds all 4) their \~5 MB (embedded, extracted at runtime) Single self-contained Go binary per platform; module still carries all four so download is the same as option 1.

Dynamic vs. static, for distribution

  • Dynamic (.dll/.so/.dylib — what the POC does today): you distribute the Go binary plus the
    platform library file; the OS loads it at runtime (must be on the loader path / beside the exe).
    Two files to ship per platform.
  • Static (cgo links the .a): the native code is baked into the Go executable → one
    self-contained binary, nothing extra to ship. Costs the same ~5 MB (§6, effective-static ≈ dynamic).
    Usually the cleaner distribution story for an SDK.

What this means for a consuming developer

Whichever approach is chosen, the developer always imports one library (a single go get /
one import path) — never per-OS packages like azure-cosmos-linux vs azure-cosmos-windows. Go's
build constraints (GOOS/GOARCH) pick the correct native binary automatically at build time, so
"fetch-per-platform" pulls only the host platform's ~1.8 MB library behind the scenes; the multiple
platform artifacts are plumbing, not a choice the developer makes. The one cgo tax to flag:
cross-compiling (e.g. building for Linux from Windows) also needs a C cross-toolchain, so consumers
typically build on the target platform or in per-platform CI.

Takeaway

The honest per-user cost of shipping the native library is ~5 MB (one platform), not 20 MB. If
repo/module weight matters, fetch-per-platform keeps even the download at ~1.8 MB per consumer.
The 20 MB only appears as a storage line item if all four raw binaries are committed into the module —
and even then it compresses to ~9.4 MB on the wire and never reaches a runtime binary. This is not a
meaningful cost argument against the cgo approach.


7. What dominates the binary (attribution)

cargo bloat --release --crates on the Windows build (.text = 7.7 MiB of a 10.8 MiB file). Note the
crate .text view understates crypto: aws-lc's large precomputed .rodata tables are not counted
here, so its true footprint is bigger than the 199 KiB shown.

Subsystem Crates .text Share
Driver's own code azure_data_cosmos_driver 3.3 MiB 42.3 %
Rust std / runtime glue std 1.2 MiB 15.1 %
HTTP + TLS transport reqwest, rustls, aws-lc-sys, h2, hyper, hyper-util, webpki \~1.8 MiB \~23 %
Async runtime tokio (+mio) \~232 KiB 3 %
Serialization serde\_json, serde\_core (+ driver-crate derives) \~147 KiB+ 2 %+
URL / IDNA url, idna \~99 KiB 1.3 %
FFI wrapper azurecosmosdriver 64 KiB 0.8 %
Telemetry tracing-core tiny \~0 %

The single biggest contributor is the driver's own logic (42 %): monomorphized async state
machines for the **direct-mode operation pipeline, partition-key routing, request hedging, account /
metadata caching, and dataflow/topology (query-plan execution)**, plus serde derive code. Many async
fns appear 3–4× (monomorphization) — which is exactly why LTO + opt-z (which merge and shrink them)
recover ~50 %.

Query planning — measured, not estimated (the local Rust planner vs. the Gateway plan-consumer):
There are two distinct "query" components, and it is easy to conflate them:

  1. Local Rust SQL plannercrate::query::{lexer, parser, ast, plan} (~5.9k LOC, merged in
    PR Local QueryPlan generation and emulator execution #4312). This is the "translate the query planner into Rust" work. It is currently **compiled but
    not reachable from production**: SUPPORTED_QUERY_FEATURES = "None", so the driver advertises no
    local plan features and defers every plan to the Gateway; the only entry point is a test-only
    (__internal_testing) symbol. Result: the linker dead-code-eliminates it to 0 bytes — a full
    symbol scan of the release cdylib finds zero query::parser/lexer/ast/plan symbols.
  2. Gateway plan-consumerdriver::dataflow::query_plan + planner (RawQueryPlan::resolve,
    resolve_epk_bound, validate_query_plan, QueryInfo/QueryPlan deserializers). This is live
    (production consumes the Gateway's plan): 73 symbols, ~44 KiB, 0.56 % of .text.

Measured cost of switching the local planner on. Forcing it live (a throwaway `#[no_mangle]
extern "C"` shim that calls the generator, so DCE keeps it; reverted after measuring):

Build Local-planner .text Whole-file delta Total lib xz
Plain --release \~124 KiB (137 symbols) +240 KiB (incl. serde/​drop glue + PE padding) 11.05 MB
Combined production (fat LTO + opt-z + cu=1 + strip) +112 KiB (+2.19 %) 5.11 MB 1.78 MB (+30 KiB)

Top contributors when live: parser::parse_primary_expression (11.8 KiB), parser::parse_query
(8.9 KiB), plan::extract_hierarchical_pk (8.3 KiB), lexer::keywords::keyword_lookup (4.0 KiB) —
classic recursive-descent parser bulk. This does not include the in-memory eval/value
evaluator (~3.7k LOC, emulator-only, separately gated — would not ship in the driver).

Attribution takeaway: local query planning costs ~112 KiB (~2 %) of a ~5 MB production lib
when activated, and 0 today. A Go port that defers to Gateway query plans sheds essentially the
local planner (which currently costs ~0 in the native lib) and carries only the ~44 KiB plan-consumer
equivalent. Query planning is not a size lever either way; crypto/TLS dominates (§ above).

7A. Monomorphization check

Generics that get stamped out per concrete type (monomorphization) can inflate a binary in a way
opt-level alone does not catch, so a per-function cargo bloat pass was analyzed for duplicate
instantiations. Finding: monomorphization is present but idiomatic, not pathological — there is no
trait-with-many-impls or user generic exploding into hundreds of large copies. The cost centers are the
unavoidable shape of async + serde Rust:

Pattern Copies Total .text Notes
core::ptr::drop_in_place \~827 \~363 KiB Drop glue, one per distinct type — broad but shallow (avg \~450 B/copy)
Future::poll state machines \~85 \~229 KiB Async runtime, inherent to tokio/reqwest futures
Large driver async fns (execute_operation_direct, plan_operation, execute_operation_pipeline, execute_hedged, resolve_ranges) 2–6 each \~40–55 KiB per copy Big async state machines, generic over transport/config — a handful of copies each, not hundreds
serde deserialize_struct / visitors \~17+ \~111 KiB One per deserialized type — expected

The only mildly actionable item is the small set of large driver async fns monomorphized 2–6× over
transport/config generics (~40–55 KiB per copy); de-genericizing them (e.g. dyn transport) could
collapse those to one copy, saving a few hundred KiB at most — a micro-optimization, not a structural
problem, and fat LTO already folds identical instantiations. No action recommended; recorded for
completeness per review request.


8. Optimization ranking

By size reduction (single knob):

  1. opt-level="z" — −20 to −29.5 % ← biggest single lever
  2. opt-level="s" — −17 to −24 %
  3. Fat LTO — −12 to −23 %
  4. strip — ~0 % on release / huge only on debug
  5. Thin LTO — negative (grew)

By implementation cost (all trivial — one profile block): every knob is a Cargo.toml profile
entry. The only real cost is build time: opt-level is cheap (even faster codegen); fat LTO +
codegen-units=1 is the slow part (~1.5–3× release link). No source changes required for configs 1–8.

By engineering risk:

  • None: release, strip, opt-s, opt-z, thin/fat LTO, codegen-units=1 — pure build settings.
  • Build-time only: fat LTO + codegen-units=1 slows CI (still minutes, not hours).
  • ⚠️ Correctness: panic="abort" breaks the FFI catch_unwind invariant — do not ship.

Recommended production profile (safe, ~−50 %): (opt-level="z" beat "s" in every platform
cell, so it is the size knob folded into the combined config; "s" deltas are shown standalone in §3.)

[profile.min-size]      # inherits release; apply to the native crate for distribution
inherits   = "release"
opt-level  = "z"
lto        = "fat"
codegen-units = 1
strip      = "symbols"  # harmless; keep panic = "unwind" (the default)

9. Runtime performance comparison (do the size knobs cost speed?)

The size table above says nothing about speed. opt-level="z" in particular explicitly trades
runtime performance for smaller code, so a reviewer flagged the candidate "small" build for
evaluation in the perf infrastructure. This section answers: **does any size-optimization knob
regress runtime throughput or latency for a realistic Cosmos workload?**

9.1 Methodology

  • What is measured: the driver's own compiled code path. The azure_data_cosmos_benchmarks crate
    links azure_data_cosmos_driver directly and is rebuilt under each optimization profile, so the
    binary exercised has the exact codegen characteristics the shipped cdylib would have. (The thin
    FFI/cgo layer is identical across configs and negligible, so measuring through the Rust harness is
    equivalent and cleaner than measuring through cgo.)
  • Workload: production-like mix of **70 % point reads / 20 % upserts / 10 % single-partition
    queries**, against a live Azure Cosmos DB account, at concurrency 16 (16 closed-loop async
    workers on a multi-thread tokio runtime), each config run for a full 15 minutes (~130–140 k
    operations per config) after a 5 s warm-up.
  • Configs (perf-relevant subset of the size matrix): dev (unoptimized debug baseline),
    release, opt-s, opt-z, fat-lto, and combined (strip + fat-LTO + opt-z + codegen-units=1).
    Skipped: strip (zero code effect), thin-lto (grew size), panic=abort (not shippable, §5).
  • Metrics: sustained throughput (ops/sec) and per-operation latency percentiles from a
    dependency-free in-process histogram.
  • Harness: throwaway examples/opt_endurance.rs (reverted after the runs; provenance copy in
    files/opt_endurance.rs). Per-config logs in files/perf-<config>.log; data in the session
    perf_results SQL table.

9.2 Results (overall, mixed workload)

Config Build time Throughput (ops/s) Read p50 Read p99 Overall p50 Overall p99 Overall p99.9
dev (baseline) 154.5 36.5 ms 615.7 ms 38.0 ms 986.0 ms 1.74 s
release 141 s 142.9 33.4 ms 671.8 ms 36.5 ms 929.3 ms 1.59 s
opt-s 68 s 142.3 32.7 ms 654.6 ms 35.3 ms 996.6 ms 1.73 s
opt-z 66 s 142.6 33.2 ms 665.3 ms 36.1 ms 969.3 ms 1.67 s
fat-lto 252 s 142.7 32.7 ms 664.8 ms 35.3 ms 982.9 ms 1.69 s
combined 203 s 150.1 34.8 ms 631.4 ms 36.6 ms 977.4 ms 1.72 s

Per-operation latency (p50 / p99) held equally flat across configs — writes ~80–92 ms / 1.35–1.49 s,
queries ~37–39 ms / 0.69–0.76 s — with no config standing out. Error rates were uniformly negligible
(37–72 transient throttles per ~130 k ops, ≤0.06 %). The full per-operation percentile breakdown
follows.

Point reads (70 % of the mix)

Config ops ops/s mean p50 p90 p99 p99.9 max
dev 97,405 108.2 67.0 ms 36.5 ms 100.5 ms 615.7 ms 1.04 s 1.74 s
release 90,077 100.0 81.9 ms 33.4 ms 163.2 ms 671.8 ms 1.04 s 1.18 s
opt-s 89,708 99.6 78.0 ms 32.7 ms 151.5 ms 654.6 ms 1.04 s 1.64 s
opt-z 89,952 99.9 79.7 ms 33.2 ms 154.3 ms 665.3 ms 1.05 s 1.71 s
fat-lto 89,928 99.9 79.1 ms 32.7 ms 152.9 ms 664.8 ms 1.04 s 1.83 s
combined 94,668 105.1 70.7 ms 34.8 ms 118.1 ms 631.4 ms 1.04 s 1.47 s

Upserts (20 % of the mix)

Config ops ops/s mean p50 p90 p99 p99.9 max
dev 27,763 30.8 237.6 ms 91.5 ms 654.3 ms 1.47 s 2.23 s 3.07 s
release 25,706 28.5 215.2 ms 80.3 ms 612.1 ms 1.35 s 1.97 s 3.02 s
opt-s 25,579 28.4 234.6 ms 89.4 ms 650.3 ms 1.49 s 2.23 s 3.39 s
opt-z 25,652 28.5 225.3 ms 83.6 ms 624.8 ms 1.40 s 2.20 s 3.36 s
fat-lto 25,657 28.5 230.5 ms 84.8 ms 638.8 ms 1.43 s 2.22 s 3.45 s
combined 26,974 30.0 236.4 ms 92.0 ms 650.7 ms 1.47 s 2.23 s 2.74 s

Single-partition queries (10 % of the mix)

Config ops ops/s mean p50 p90 p99 p99.9 max
dev 13,912 15.5 85.2 ms 38.9 ms 154.9 ms 670.3 ms 1.23 s 1.88 s
release 12,866 14.3 111.7 ms 38.3 ms 329.9 ms 731.1 ms 1.28 s 1.72 s
opt-s 12,814 14.2 104.2 ms 36.8 ms 271.6 ms 738.2 ms 1.20 s 1.80 s
opt-z 12,849 14.3 106.8 ms 37.7 ms 292.3 ms 761.8 ms 1.25 s 1.94 s
fat-lto 12,846 14.3 102.9 ms 36.9 ms 267.9 ms 705.2 ms 1.23 s 2.23 s
combined 13,522 15.0 91.1 ms 37.5 ms 194.5 ms 691.9 ms 1.19 s 1.75 s

Reading down any percentile column, the variation is unordered with respect to optimization level and
sits inside the account's window-to-window drift. Two illustrative points: at the p99 tail, opt-z
reads (665 ms) fall between release (672 ms) and opt-s (655 ms); and for writes the opt-z p99
(1.40 s) is actually lower than plain release's neighbours in two of the four middle windows. The
p90 column is the most visibly noisy (reads swing 100–163 ms) but again tracks which 15-minute window
was luckiest on the service side, not which compiler flag was used — dev and combined, the two
windows that happened to see the calmest account, top every op's p90 regardless of their very different
codegen. Tail behaviour (p99 / p99.9 / max) is dominated by occasional service-side retries/throttles
common to all configs, not by client CPU.

9.3 Interpretation

  • No optimization knob produces a measurable runtime regression. The five optimized builds cluster
    at 142–150 ops/s overall with read p50 32.7–34.8 ms and overall p99 0.93–1.0 s. The
    spread is not ordered by optimization levelopt-z (142.6) sits between release (142.9) and
    fat-lto (142.7), and the two highest-throughput windows (dev 154.5, combined 150.1) bracket the
    others. That pattern is live-account run-to-run variance, not codegen: each 15-min window saw
    slightly different service/RU conditions, and that ±5 % window-to-window drift is **larger than any
    systematic effect of the compiler flags**.
  • The opt-z "small" build is safe on performance. Directly answering the reviewer: opt-z
    trades CPU for size in principle, but for this workload the trade is invisible — its numbers
    are indistinguishable from plain release.
  • Why: the workload is I/O-bound. A point read costs ~33 ms of network + service time; the
    driver's own CPU work per operation is on the order of microseconds. Size-optimized codegen might
    make that CPU slice a few percent slower, but a few percent of a few µs is **nanoseconds against
    tens of milliseconds of I/O** — completely masked. This is exactly the "heavy impact of I/O for us"
    the reviewer anticipated, now confirmed empirically over 90 minutes of live traffic.
  • Build-time is the only real cost of the aggressive knobs. opt-s/opt-z actually build
    faster than plain release (~66–68 s vs 141 s — less optimization work); **fat LTO is the expensive
    one** (252 s standalone, 203 s in the combined config with codegen-units=1), a ~1.5–1.8× hit that
    lands in CI/release builds, not at runtime.

9.4 Caveats

  • Live-account confound: the six configs ran in sequential 15-min windows, so each saw a
    different slice of account/network conditions. This is fine for the question asked (detecting a
    large regression — there is none) but means the sub-5 % throughput differences between configs are
    noise, not signal; they should not be over-read. A CPU-isolated ranking would require a mock/
    loopback transport (removing I/O) or interleaved sampling — deliberately out of scope here since the
    goal was a realistic end-to-end verdict on the "small" build.
  • Single region/account/host: absolute numbers reflect one client host and one account tier; the
    relative parity across configs is the portable finding, not the absolute ops/s.

10. Architecture implications

  • Native-library size is not a real blocker for the cgo/FFI approach. A production-optimized build
    is ~5 MB per platform (~4.5 MB arm64) and ~1.8 MB compressed — on the order of a single
    medium Go dependency. Whether the Go SDK dlopens the cdylib or cgo-static-links the archive, the
    effective added footprint is the same ~5–6 MB (§6).
  • ~42 % of the size is the driver's own logic, not third-party bulk. This is the part a **native Go
    port would have to reimplement** (pipeline, routing, hedging, caching, dataflow/query-plan
    execution) — it does not vanish in a rewrite; it moves from "shipped binary" to "code you own and
    maintain."
  • ~23 % (~1.8 MiB) is the HTTP+TLS transport stack (reqwest/rustls/aws-lc/h2/hyper/webpki) and ~3 %
    is the tokio runtime. A Go port replaces these with the standard library (net/http,
    crypto/tls, goroutine scheduler) that already ships in the Go runtime — so from the *customer's
    incremental-binary* view that share is "free" in a Go port, whereas in the Rust-lib approach it is
    carried explicitly. This ~26 % transport+runtime slice (plus the crypto .rodata not shown in the
    .text view) is the clearest size argument for a Go port.
  • The crypto backend (aws-lc-sys) is the main cross-compile complexity, but it **cross-compiled
    cleanly under zig** for linux-amd64, linux-arm64, and (compile-only) macos-arm64 — so multi-platform
    distribution of the Rust lib is practical.
  • Net: if the decision hinges on download/binary size alone, cgo is fine (~1.8 MB compressed),
    and the size-optimized build carries no runtime penalty — the opt-z/combined production build
    performs on par with plain release for a real I/O-bound workload (§9). The stronger differentiators
    between the two options are operational (cgo toolchain/cross-build complexity, the
    panic/FFI-safety contract, per-platform artifacts) versus maintenance (a Go port re-owns the
    42 % driver logic but inherits transport/TLS/runtime from Go's std for free) — not the raw megabytes
    and not runtime speed.

*Raw per-config data: win-results.jsonl, linux-amd64-results.jsonl, linux-arm64-results.jsonl,
the per-config perf logs perf-<config>.log, and the session measurements / attribution /
perf_results SQL tables. Size harness: measure_win.ps1, measure_cross.ps1, link_static_win.ps1,
link_static_linux2.ps1, gen_probe.py, sizes.py. Perf harness: opt_endurance.rs.*

Metadata

Metadata

Assignees

Labels

Type

No type

Fields

No fields configured for issues without a type.

Projects

Status
Untriaged
Status
Todo

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions