Skip to content
Open
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Architecture Decision Records — Go v2 FFI direction

ADRs (Architecture Decision Records) capture **what we decided** and a brief
**why**, in a minimal, easy-to-reference form. They are **numbered and
immutable**: once accepted, an ADR is not edited; a later ADR may supersede it.
Detailed discussion and exploration live in
[`go-v2-ffi-exploration.md`](https://github.com/Azure/azure-sdk-for-rust/blob/main/sdk/cosmos/azure_data_cosmos_driver/docs/native-driver-distribution/go-native-distribution/go-v2-ffi-exploration.md),
not here.

**Format (template):** Status (immediately under the title) · Context (2-4
sentences) · Decision (1-3 bullets) · Consequences (2-4 bullets) · Alternatives
considered (1 line each).

| # | Title | Status |
|---|-------|--------|
| [0001](https://github.com/Azure/azure-sdk-for-rust/blob/main/sdk/cosmos/azure_data_cosmos_driver/docs/native-driver-distribution/go-native-distribution/adr/0001-go-v2-uses-ffi.md) | Go v2 uses the Rust driver through FFI | Proposed |

> This ADR is proposed for review. Packaging mechanics are intentionally left to
> the native distribution ADRs; this record captures the Go v2 implementation
> direction.
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# ADR 0001 — Go v2 uses the Rust driver through FFI

**Status:** Proposed

## Context

Go v2 needs a path that can deliver in the current timeframe while staying close
to the Rust driver's behavior. A time-boxed pure-Go spike proved that a
gateway-mode core path can be ported and validated, but it also clarified the
central trade-off: avoiding cgo and native packaging means re-owning driver
logic in Go and carrying long-term Rust-to-Go drift risk.

## Decision

- Go v2 proceeds with the **Rust-driver FFI path** for the current delivery
window.
- The Go binding consumes the **prebuilt Rust native driver** through cgo and the
C ABI surface.
- Packaging mechanics, platform matrix, signing, ABI versioning, and fan-out are
handled by the native distribution design and ADRs.
Comment thread
ananth7592 marked this conversation as resolved.

## Consequences

- Go v2 gets the same driver implementation as Rust, reducing short-term feature
drift and implementation risk.
- Go v2 takes a dependency on `CGO_ENABLED=1` for the native driver path.
- The Go packaging model becomes part of the product decision: customers should
not manually install a Rust toolchain or copy native libraries for the common
path, but cgo still requires an available C build toolchain.
- Differential validation remains important so Go-visible behavior can be tested
against the Rust driver.

## Alternatives considered

- **Pure-Go driver port** — technically feasible for the core path tested, but
not selected for Go v2 because it shifts cost from packaging to long-term Go
ownership and Rust-to-Go drift.
- **Manual native-library installation** — rejected for the common path because
it breaks the normal `go get` / `go build` expectation for a first-party Go
SDK.
- **Pure-Go downloader shim** — rejected by the native distribution ADRs; the
native driver must be packaged and linked through the FFI path, not fetched by
a downloader stub.

## Discussion

See
[`go-v2-ffi-exploration.md`](https://github.com/Azure/azure-sdk-for-rust/blob/main/sdk/cosmos/azure_data_cosmos_driver/docs/native-driver-distribution/go-native-distribution/go-v2-ffi-exploration.md)
for the meeting context, pure-Go spike evidence, packaging concerns, tentative
platform matrix, and market reference points.
Original file line number Diff line number Diff line change
@@ -0,0 +1,162 @@
<!-- cspell:ignore amd64 azcosmos GSSAPI librdkafka -->
# Go v2 FFI exploration notes
Comment thread
Copilot marked this conversation as resolved.

> **Status:** Discussion notes for ADR review. The decision is recorded in
> [`ADR 0001`](https://github.com/Azure/azure-sdk-for-rust/blob/main/sdk/cosmos/azure_data_cosmos_driver/docs/native-driver-distribution/go-native-distribution/adr/0001-go-v2-uses-ffi.md).

## 1. Purpose

This document captures the exploration behind the Go v2 decision to proceed with
the Rust-driver FFI path in the current delivery window. It is intentionally more
descriptive than the ADR: it records the trade-offs, pure-Go spike evidence,
packaging implications, and market reference points that informed the decision.

## 2. Implementation paths evaluated

The team evaluated two paths:

| Path | Shape |
|---|---|
| **FFI path** | Go calls the prebuilt Rust driver through cgo and a C ABI. |
| **Pure-Go port** | Go reimplements the driver behavior directly in Go. |

Both were technically feasible. The decision is not "can we port it?" The
decision is:

> Is avoiding cgo and native packaging worth re-owning driver logic that Rust
> already implements?

For the Go v2 timeframe, the team aligned on **FFI** because it gives faster
delivery, stronger parity with the Rust driver, and lower short-term
implementation risk.

## 3. What the pure-Go spike proved

The pure-Go spike showed that a gateway-mode core path can be ported in
idiomatic Go:

- `CGO_ENABLED=0`
- zero third-party dependencies
- point create/read
- retry and routing behavior
- basic failover behavior
- cross-partition unordered query fan-out
- unsupported-query rejection parity

The spike was validated with a scenario-based differential harness. The harness
put an HTTP endpoint in front of the Rust driver's in-memory emulator so the Go
port and the Rust driver could run against the same emulator-backed state and
compare caller-visible outcomes.

This is valuable evidence: the Go port is credible. It is also a useful
validation reference for Go-visible behavior. It is not the selected Go v2
delivery path because every Rust-driver change would need to be tracked, ported,
and revalidated in Go.

## 4. Why FFI for Go v2

FFI has the better short-term delivery shape:

- **Parity:** Go consumes the same driver implementation as Rust.
- **Velocity:** new Rust-driver behavior does not need to be re-ported before Go
can benefit.
- **Risk containment:** the hard driver logic stays in one implementation.
- **Testing leverage:** the Go binding can validate language-level behavior
against the Rust driver's known behavior instead of re-proving the full driver.

The main cost is not binary size. The main cost is **Go packaging and customer
deployment experience**.

## 5. Packaging implications

Cosmos Go v1 is consumed like a normal Go SDK:

```bash
go get github.com/Azure/azure-sdk-for-go/sdk/data/azcosmos
go build ./...
```

There is no Rust compiler, native shared library, or post-install step in the
customer path. If Go v2 uses FFI, prebuilt Rust artifacts can preserve that part
of the experience, but cgo still requires a C build toolchain. The packaging bar
Comment thread
ananth7592 marked this conversation as resolved.
is therefore:

> Can the Go SDK still feel like one normal Go module across the supported
> Windows, macOS, and Linux targets?

That means native Rust driver artifacts must be produced, versioned, selected,
and linked for the supported Go OS/architecture matrix.

This discussion scopes the initial packaging estimate to the mainstream
OS/architecture targets:

| OS family | Target combinations discussed |
|---|---|
| Windows | x64/amd64, ARM64, possibly x86 as a consideration |
| macOS | ARM64/Apple Silicon, x64/Intel |
| Linux glibc | x64/amd64, ARM64 |
Comment thread
ananth7592 marked this conversation as resolved.

That is **six firm targets**, or **seven** if Windows x86 is included. Linux
musl/Alpine is intentionally not counted here until the Go packaging plan
decides whether it is in scope for the initial release.

With an estimated **~5 MB optimized native binary per platform**, this scoped
matrix is roughly **~30-35 MB before compression**. Any additional targets, such
as musl/Alpine, add to that footprint. This is manageable for an initial release,
but it is a real packaging design point and should not be treated as an
implementation detail.

## 6. Packaging models to evaluate

Two practical models were discussed:

| Model | Customer experience | Trade-off |
|---|---|---|
| **Bundle the full platform matrix in the Go package** | Closest to `go get` / `go build` just working everywhere | Larger module/package footprint; all platform binaries travel together |
| **Per-platform native dependency** | Customer obtains only the platform-specific binary they need | Smaller per-customer footprint, but more setup burden and more room for install mistakes |

A bundle-first hybrid has industry precedent. For example,
[`confluent-kafka-go`](https://github.com/confluentinc/confluent-kafka-go/blob/master/README.md#librdkafka)
includes prebuilt `librdkafka` binaries for common platforms, requires
`CGO_ENABLED` to remain enabled, and falls back to a manual dynamic install only
when customers need unsupported platforms or special features such as
GSSAPI/Kerberos.

That reference shows a viable market pattern for a Go library backed by native
code: **bundle the common path, document unsupported/special paths clearly, and
make native requirements explicit**. It aligns most closely with the first model,
with a manual native dependency as an escape hatch. It also confirms that this
is a materially different customer experience from a pure-Go Azure SDK.

## 7. Pure-Go port trade-off

The pure-Go path has a strong developer-experience story:

- `CGO_ENABLED=0`
- normal Go cross-compilation
- no native runtime dependency
- Go standard library transport/TLS/runtime

But it moves the cost into ownership:

- driver behavior must be reimplemented in Go
- every Rust-driver change has to be tracked, ported, and revalidated
- long-term drift becomes the main risk
- the hardest remaining areas still need evidence: richer query execution,
continuation/split-resume behavior, broader multi-region behavior, change
feed, and bulk/batch

The pure-Go spike remains valuable because it proved the fallback is not
fictional. It also produced a useful differential-validation approach. But for
Go v2, it is better treated as evidence and risk retirement, not the shipping
plan.

## 8. Non-goals

- This discussion does not define the final Go packaging shape.
- This discussion does not require customers to manually install native
libraries.
- This discussion does not introduce direct mode; both the Rust and Go v2 paths
are gateway-mode SDKs.
- This discussion does not remove the pure-Go spike as a reference or validation
asset.
Loading