javascript: add packaging of 1st party drivers (postgresql, sqlite, flightsql)

[H-Plus-Time]

What feature or improvement would you like to see?

My thinking here would be to piggyback on the build process for the python wheels (which vendor in things like libpq - the node ecosystem has similar conventions) and package them up mechanically as:

@apache-arrow/adbc-driver-postgresql/
├── package.json
├── index.js // containing `getDriverPath` and `getDriverManifestPath` (if that's desirable)

// platform-dependent packages
@apache-arrow/adbc-driver-postgresql-linux-x64/
├── package.json
├── libadbc_driver_postgresql.so
└── manifest.toml // dependent on whether distributing driver manifests is a good idea

Is there any desire for a direct dbapi-like interface for the per-driver packages (wherein you can import the driver package directly and connect via that), or should the behaviour be left the same as for the C, R and Go libraries?

[ianmcook]

Hi Nicholas, thanks for asking this.

For several reasons, we are moving away from packaging ADBC drivers as language-specific libraries and distributing them on language-specific package registries. For details, see the discussion at https://github.com/orgs/adbc-drivers/discussions/7. (This is similar to what you're asking here, but about PyPI instead of npm.) Going forward, the ADBC Driver Registry will be the best place to get drivers.

But we recognize the need to create a seamless experience for the TS/JS community, so we're doing two things to enable this:

1. As of today, it's now possible to install dbc from npm:

   npm install -g @columnar-tech/dbc

   Then you can install drivers with dbc:

   dbc install postgresql

   Or you can collapse these twp steps into one with npx:

   npx --yes @columnar-tech/dbc install postgresql

   For full details, see https://www.npmjs.com/package/@columnar-tech/dbc.

2. We are in the process of creating a WASM build of dbc with JS/TS bindings. This will make it easier for TS/JS apps to install ADBC drivers programmatically. This isn't released yet. If you're interested, please take a look at feat(wasm): add WebAssembly build and @columnar-tech/dbc-wasm npm package columnar-tech/dbc#395. Comments welcome.

Please let us know if these address your need or if you have any questions.

[ianmcook]

@H-Plus-Time also ICYMI, there's an ADBC driver manager for TS/JS, which is how you load and use the dbc-installed drivers: https://www.npmjs.com/package/@apache-arrow/adbc-driver-manager

See examples showing how to use this with 30+ databases here: https://github.com/columnar-tech/adbc-quickstarts/tree/main/javascript#javascript-adbc-quickstarts

[H-Plus-Time]

Yes (and I was aware of the driver manager, that's what prompted this), that addresses my questions definitely; agreed that dbc in lieu of language-specific bindings (which, when they weren't present for a given language, led to ugly hybrid environments) is the correct solution for this.