-
Notifications
You must be signed in to change notification settings - Fork 13
feat(wasm): add WebAssembly build and @columnar-tech/dbc-wasm npm package #395
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
37 commits
Select commit
Hold shift + click to select a range
c23b00d
feat(wasm): add WebAssembly build with JS/TS interface (Phase 0)
zeroshade 551ebee
feat(wasm): browser/node build seam, resolve API, credential-resolver…
zeroshade dc223b0
test(wasm): exercise resolve in the node spike harness
zeroshade 7534fe7
fix(wasm): address roborev 6502 — partial search results and list env…
zeroshade c5abfe9
fix(wasm): preserve registry error in resolve not-found path (roborev…
zeroshade 54d2cb3
feat(wasm): add @columnar-tech/dbc-wasm npm package (Phase 3)
zeroshade e96f1e0
fix(wasm): per-call client config so loadDbc instances are isolated (…
zeroshade b40e4bc
feat(wasm): stream HTTP response bodies instead of buffering (Phase 4)
zeroshade 82aab72
fix(wasm): per-instance client handle to retain refreshed tokens (rob…
zeroshade 28893ec
feat(wasm): Windows-host path groundwork via splitConfigList seam (Ph…
zeroshade a1c8a00
ci(wasm): Windows-host smoke gate + normalizeLocation unit test (Phas…
zeroshade 03ae190
fix(wasm): only normalize locations on Windows (roborev 6527)
zeroshade c23b35e
ci(wasm): trigger smoke on testdata fixture changes (roborev 6528)
zeroshade 2f8b4b7
feat(wasm): opt-in Node Worker Thread runtime via loadDbc({worker:tru…
zeroshade 1ede706
fix(wasm): reject worker RPCs after close or exit instead of hanging …
zeroshade 2aef7ee
fix(wasm): terminate worker when loadDbc({worker:true}) init fails (r…
zeroshade 5a3eaf5
refactor(wasm): unify dbc-wasm backends and fix API inconsistencies (…
zeroshade 75658b1
refactor(wasm): extract shared Go/wasm boot sequence into boot.cjs
zeroshade f32cb2b
refactor(wasm): fold worker close/reject lifecycle into markClosed he…
zeroshade ace0f46
refactor(wasm): fold worker init-gate rejection into markClosed
zeroshade 86f0636
fix(wasm): prefix in-process dbcNewClient init errors with dbc-wasm: …
zeroshade 35b127b
test(wasm): cover in-process loadDbc init-error dbc-wasm: prefix (rob…
zeroshade d6379ea
docs(wasm): resolve design-review findings (roborev 6565)
zeroshade da96a32
fix(wasm): curate go.env to avoid wasm_exec argv/env size limit on Wi…
zeroshade 772858a
fix(wasm): map TMPDIR from TEMP/TMP for Go js/wasm os.TempDir on Wind…
zeroshade 1fb85ac
test(wasm): dump listInstalled result + dir contents on smoke mismatch
zeroshade 49820e5
fix(config): discover driver manifests via os.ReadDir for GOOS=js Win…
zeroshade 6fc700e
ci(wasm): drop experimental windows-latest from matrix (#396)
zeroshade e627427
docs(wasm): note baseURL defaults to the Columnar driver CDN
zeroshade 83b4832
chore(wasm): remove Phase 0 de-risk spike (superseded by shipped pack…
zeroshade 4920054
docs(wasm): document runtime support (Node + Deno; Bun blocked by ups…
zeroshade 45e3f9b
ci(wasm): run the npm-wasm test suite under Deno alongside Node
zeroshade 16ec53a
docs(wasm): link upstream Bun issue (oven-sh/bun#32505) in runtime-su…
zeroshade c033e68
Update packages/npm-wasm/scripts/build.js
zeroshade 3d27969
fix(wasm): terminate dbc-wasm package description string
zeroshade 044ae69
ci(wasm): address review feedback on the workflow
zeroshade 6c2ca74
ci(wasm): build with Go 1.26 to match go.mod
zeroshade File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,93 @@ | ||
| # Copyright 2026 Columnar Technologies Inc. | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
|
|
||
| name: Wasm | ||
| on: | ||
| push: | ||
| branches: | ||
| - 'main' | ||
| paths: | ||
| - '**/*.go' | ||
| - 'auth/**' | ||
| - 'config/**' | ||
| - 'internal/**' | ||
| - 'wasm/**' | ||
| - 'packages/npm-wasm/**' | ||
| - 'cmd/dbc/testdata/**' | ||
| - 'go.mod' | ||
| - 'go.sum' | ||
| - '.github/workflows/wasm.yml' | ||
| pull_request: | ||
| paths: | ||
| - '**/*.go' | ||
| - 'auth/**' | ||
| - 'config/**' | ||
| - 'internal/**' | ||
| - 'wasm/**' | ||
| - 'packages/npm-wasm/**' | ||
| - 'cmd/dbc/testdata/**' | ||
| - 'go.mod' | ||
| - 'go.sum' | ||
| - '.github/workflows/wasm.yml' | ||
|
|
||
| concurrency: | ||
| group: ${{ github.repository }}-${{ github.head_ref || github.sha }}-${{ github.workflow }} | ||
| cancel-in-progress: ${{ github.event_name == 'pull_request' }} | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| jobs: | ||
| build-and-smoke: | ||
| name: ${{ matrix.os }} | ||
| runs-on: ${{ matrix.os }} | ||
| strategy: | ||
| fail-fast: false | ||
| matrix: | ||
| # windows-latest is intentionally excluded: the WASM Windows-host path is | ||
| # experimental and not yet passing (driver discovery under GOOS=js — see | ||
| # https://github.com/columnar-tech/dbc/issues/396). Re-add when fixed. | ||
| os: [ 'ubuntu-latest', 'macos-latest' ] | ||
| steps: | ||
| - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 | ||
| with: | ||
| persist-credentials: false | ||
| - name: Install Go | ||
| uses: actions/setup-go@924ae3a1cded613372ab5595356fb5720e22ba16 # v6.5.0 | ||
| with: | ||
| go-version: 1.26 | ||
| cache: true | ||
| cache-dependency-path: go.sum | ||
| - name: Node version | ||
| run: node --version | ||
| - name: Build @columnar-tech/dbc-wasm | ||
| run: node packages/npm-wasm/scripts/build.js | ||
| - name: Node loader unit tests | ||
| run: node packages/npm-wasm/test/normalize.test.cjs | ||
| - name: Node smoke test | ||
| run: node packages/npm-wasm/test/smoke.cjs | ||
| - name: Node worker smoke test | ||
| run: node packages/npm-wasm/test/worker.test.cjs | ||
| - name: Setup Deno | ||
| uses: denoland/setup-deno@667a34cdef165d8d2b2e98dde39547c9daac7282 # v2.0.4 | ||
| with: | ||
| deno-version: v2.x | ||
| - name: Deno version | ||
| run: deno --version | ||
| - name: Deno loader unit tests | ||
| run: deno run -A packages/npm-wasm/test/normalize.test.cjs | ||
| - name: Deno smoke test | ||
| run: deno run -A packages/npm-wasm/test/smoke.cjs | ||
| - name: Deno worker smoke test | ||
| run: deno run -A packages/npm-wasm/test/worker.test.cjs |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,66 @@ | ||
| // Copyright 2026 Columnar Technologies Inc. | ||
| // | ||
| // Licensed under the Apache License, Version 2.0 (the "License"); | ||
| // you may not use this file except in compliance with the License. | ||
| // You may obtain a copy of the License at | ||
| // | ||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, software | ||
| // distributed under the License is distributed on an "AS IS" BASIS, | ||
| // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
|
|
||
| package config_test | ||
|
|
||
| import ( | ||
| "os" | ||
| "path/filepath" | ||
| "testing" | ||
|
|
||
| "github.com/columnar-tech/dbc/config" | ||
| "github.com/stretchr/testify/assert" | ||
| "github.com/stretchr/testify/require" | ||
| ) | ||
|
|
||
| const findInManifestTOML = ` | ||
| name = 'Test Driver' | ||
| publisher = 'Test Publisher' | ||
| license = 'MIT' | ||
| version = '1.2.3' | ||
| source = 'dbc' | ||
|
|
||
| [ADBC] | ||
| version = '1.1.0' | ||
|
|
||
| [Driver] | ||
| entrypoint = 'AdbcDriverInit' | ||
|
|
||
| [Driver.shared] | ||
| linux_amd64 = '/path/to/driver.so' | ||
| ` | ||
|
|
||
| func TestFindDriverConfigsIn(t *testing.T) { | ||
| t.Run("lists drivers from an explicit location without env", func(t *testing.T) { | ||
| t.Setenv("ADBC_DRIVER_PATH", "/should/not/be/read") | ||
| dir := t.TempDir() | ||
| require.NoError(t, os.WriteFile(filepath.Join(dir, "drv1.toml"), []byte(findInManifestTOML), 0o644)) | ||
| require.NoError(t, os.WriteFile(filepath.Join(dir, "drv2.toml"), []byte(findInManifestTOML), 0o644)) | ||
|
|
||
| got := config.FindDriverConfigsIn(dir) | ||
| ids := make([]string, len(got)) | ||
| for i, d := range got { | ||
| ids[i] = d.ID | ||
| } | ||
| assert.ElementsMatch(t, []string{"drv1", "drv2"}, ids) | ||
| }) | ||
|
|
||
| t.Run("empty location returns nil", func(t *testing.T) { | ||
| assert.Empty(t, config.FindDriverConfigsIn("")) | ||
| }) | ||
|
|
||
| t.Run("nonexistent location returns empty", func(t *testing.T) { | ||
| assert.Empty(t, config.FindDriverConfigsIn(filepath.Join(t.TempDir(), "nope"))) | ||
| }) | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| // Copyright 2026 Columnar Technologies Inc. | ||
| // | ||
| // Licensed under the Apache License, Version 2.0 (the "License"); | ||
| // you may not use this file except in compliance with the License. | ||
| // You may obtain a copy of the License at | ||
| // | ||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, software | ||
| // distributed under the License is distributed on an "AS IS" BASIS, | ||
| // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
|
|
||
| //go:build js | ||
|
|
||
| package config | ||
|
|
||
| // SetPlatformTupleOverride sets the host platform tuple (e.g. "linux_amd64"). | ||
| // Required under GOARCH=wasm, where the detected tuple is "unknown_wasm64". It | ||
| // sets the package var directly so every reader observes it. Call once at | ||
| // startup before any install/uninstall. | ||
| func SetPlatformTupleOverride(tuple string) { | ||
| platformTuple = tuple | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,28 @@ | ||
| // Copyright 2026 Columnar Technologies Inc. | ||
| // | ||
| // Licensed under the Apache License, Version 2.0 (the "License"); | ||
| // you may not use this file except in compliance with the License. | ||
| // You may obtain a copy of the License at | ||
| // | ||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, software | ||
| // distributed under the License is distributed on an "AS IS" BASIS, | ||
| // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
|
|
||
| //go:build js | ||
|
|
||
| package config | ||
|
|
||
| // splitConfigList does not split under GOOS=js: the wasm API passes a single | ||
| // explicit directory as the location, and ':' (the Unix list separator that | ||
| // filepath.SplitList uses under js) would corrupt a Windows drive-lettered | ||
| // path such as "C:/drivers". | ||
| func splitConfigList(s string) []string { | ||
| if s == "" { | ||
| return nil | ||
| } | ||
| return []string{s} | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| // Copyright 2026 Columnar Technologies Inc. | ||
| // | ||
| // Licensed under the Apache License, Version 2.0 (the "License"); | ||
| // you may not use this file except in compliance with the License. | ||
| // You may obtain a copy of the License at | ||
| // | ||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, software | ||
| // distributed under the License is distributed on an "AS IS" BASIS, | ||
| // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
|
|
||
| //go:build !js | ||
|
|
||
| package config | ||
|
|
||
| import "path/filepath" | ||
|
|
||
| func splitConfigList(s string) []string { return filepath.SplitList(s) } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,55 @@ | ||
| // Copyright 2026 Columnar Technologies Inc. | ||
| // | ||
| // Licensed under the Apache License, Version 2.0 (the "License"); | ||
| // you may not use this file except in compliance with the License. | ||
| // You may obtain a copy of the License at | ||
| // | ||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, software | ||
| // distributed under the License is distributed on an "AS IS" BASIS, | ||
| // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
|
|
||
| package dbc | ||
|
|
||
| import ( | ||
| "net/url" | ||
| "testing" | ||
|
|
||
| "github.com/columnar-tech/dbc/auth" | ||
| ) | ||
|
|
||
| func TestWithCredentialResolver(t *testing.T) { | ||
| want := &auth.Credential{} | ||
| var gotHost string | ||
| c, err := NewClient(WithCredentialResolver(func(u *url.URL) (*auth.Credential, error) { | ||
| gotHost = u.Host | ||
| return want, nil | ||
| })) | ||
| if err != nil { | ||
| t.Fatalf("NewClient: %v", err) | ||
| } | ||
|
|
||
| got, err := c.credentialResolver(&url.URL{Host: "registry.example.com"}) | ||
| if err != nil { | ||
| t.Fatalf("resolver returned error: %v", err) | ||
| } | ||
| if gotHost != "registry.example.com" { | ||
| t.Fatalf("resolver host = %q, want registry.example.com", gotHost) | ||
| } | ||
| if got != want { | ||
| t.Fatal("resolver returned a different credential than provided") | ||
| } | ||
| } | ||
|
|
||
| func TestWithCredentialResolverDefault(t *testing.T) { | ||
| c, err := NewClient() | ||
| if err != nil { | ||
| t.Fatalf("NewClient: %v", err) | ||
| } | ||
| if c.credentialResolver == nil { | ||
| t.Fatal("default credentialResolver should be non-nil") | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| node_modules/ | ||
|
|
||
| # Build artifacts generated by scripts/build.js — not committed | ||
| dbc.wasm | ||
| wasm_exec.js | ||
| package.json | ||
| LICENSE | ||
| *.tgz |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this really needed? Since SplitList only ever runs on a single platform, mixing isn't an issue? Not sure if a change is needed here but just commenting.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is, but not because of platform mixing — it's what
GOOS=jsdoes tofilepath. Underjs,filepath.ListSeparatoris:(js is treated as a Unix target regardless of the actual host OS), so on a Windows host a drive-lettered location likeC:/driverswould split into["C", "/drivers"]. The wasm API always passes a single explicit directory, so the correct behavior underjsis "don't split," which is what this override does; native builds keep the normalfilepath.SplitListinsplitlist_other.go.