Migrate database if prefered option is available

[Cunibon]

Created this PR to simplify migrating an existing Flutter app using Drift from one storage implementation to another.

The new implementation allows developers to specify a preferredImplementation. If this implementation is available, the current database will be exported and used to initialize the new database via the initializeDatabase function.

This change should improve developer experience and make it easier to take advantage of the benefits of OPFS.

[simolus3]

I love this idea 🚀 I think we need an integration test for this as well (extras/integration_tests/web_wasm), but these are tricky to write. We could perhaps open a database with IndexedDB first and then pass this option to assert that data has been migrated to OPFS.

[Cunibon]

I agree, but I am having a lot of trouble running the tests localy, is there any setup you can point me to? The CONTRIBUTING.md does not have any information regarding this.

[simolus3]

If you run dart test in that directory, what's the error message you get? You need chromedriver and geckodriver installed for the tests to work, but if you only have chrome you could use dart test -N chrome and skip the firefox setup.

The basic layout of those tests is that we start a website making some functionality we want to call available as top-level JavaScript functions. The first argument is an optional parameter, the last argument is a completion callback. So you can call them with e.g. detectImplementations(null, console.log). If you run dart run tool/serve_manually.dart, you can open localhost:8080 to call these functions from the console.

[Cunibon]

This is my log:

PS F:\FlutterProjekt\drift\extras\integration_tests\web_wasm> dart test
Resolving dependencies in F:\FlutterProjekt\drift...
Downloading packages...
Got dependencies in F:\FlutterProjekt\drift.
Resolving dependencies in F:\FlutterProjekt\drift...
Downloading packages...
Got dependencies in F:\FlutterProjekt\drift.
00:02 +0: test\drift_wasm_test.dart: (setUpAll)
Generating the build script.
Compiling the build script.
Starting daemon...
Reading the asset graph.
Creating the asset graph.
Doing initial build cleanup.
00:13 +0 -1: test\drift_wasm_test.dart: (setUpAll) [E]
Instance of 'MissingPortFile'
package:build_daemon/client.dart 24:37 _existingPort
===== asynchronous gap ===========================
package:build_daemon/client.dart 180:9 BuildDaemonClient.connect
===== asynchronous gap ===========================
package:web_wasm/driver.dart 43:25 TestAssetServer.start
===== asynchronous gap ===========================
test\drift_wasm_test.dart 59:14 main.

It seems like the daemon is not starting correctly, since MissingPortFile comes from build_daemon -> client.dart.

[Cunibon]

Figured it out, there is this line of code:

    if (packageDir.endsWith('/')) {
      packageDir = packageDir.substring(0, packageDir.length - 1);
    }

but my path ends with "\"

Changing it to:

    if (packageDir.endsWith('/') || packageDir.endsWith('\\')) {
      packageDir = packageDir.substring(0, packageDir.length - 1);
    }

Fixed that part at least ^^

[Cunibon]

I ran into another problem, the sqlite3.wasm file in the web folder just says ../../../../extras/assets/sqlite3.wasm which for some reason does not work for me.

I replaced it with the file at the path and that seems to have fixed it.

[Cunibon]

The only thing currently bothering me is that I dont clean up the "old" db after the move.

I would need to add a completer to web.EventStreamProviders.messageEvent in wasm_setup.dart to make sure the old db only gets deleted if the new one is acutally there. Ideally there would also be a fail safe in case the move did not work out for whatever reason.

[simolus3]

to make sure the old db only gets deleted if the new one is acutally there

We could potentially move this into the worker? We'd have to expose the fact that we're initializing by moving another database over in _ProbeResult.open, but what if instead of sending the ArrayBuffer we could send an encoded ExistingDatabase over as well. The worker would then have to do a typecheck and:

• Keep the existing initialization logic if we receive an array buffer.
• If we receive an object encoding the previous location, open that VFS, copy contents, then delete the old storage.

Doing that in the worker ensures we'll only delete old databases when the new one is ready.