Resumable host calls

[jedisct1]

Some functions, such as password stretching functions and key pair creation functions, can take a long time to return.

This is an issue in contexts that don't expect calls to be blocking. JavaScript is an obvious example of this.

The usual way to workaround it is to make the API asynchronous (as in scrypt-async). A computation step is performed, then the function returns but signals that more steps are needed in order to complete.

We can do something very similar. Reserve an error code for this, and bindings would try again until the operation returns a different error code.

Is there a better way to address this? I remember discussions about having the equivalent of promises in WASI. Is it still something being considered, and that we could use instead?

[tniessen]

This is a fundamental problem with WebAssembly. Most JavaScript applications only use WebAssembly in a background thread for this very reason. For standalone WebAssembly applications, threading will eventually be necessary to be able to compete with the performance of native applications.

[jedisct1]

I couldn't agree more, threading will be absolutely necessary.

But what should we do now?

Should we assume that all runtimes are going to have support for threads in a very near future?

That would makes things a little be easier, but is it a reasonable assertion?

What do you think?

[tniessen]

Sorry @jedisct1. I really appreciate being involved in this, but I have barely had time to work through hundreds and hundreds of GitHub notifications from other projects for the last few months.

I think asynchronous behavior adds an incredible amount of complexity, and security aspects. That would require a background thread pool, which would somehow have to work together with explicit WebAssembly threads. (For example, WebAssembly in a browser should probably not be allowed to use 8 threads + 8 parallel asynchronous operations, which could easily use all available CPU resources.)

Asynchronous is what WebCrypto does, and it has its benefits for such a high-level environment (JavaScript in a browser). Node.js already has a thread pool and therefore also allows off-loading some crypto work to the thread pool. However, as long as WebAssembly does not have a unified concept of asynchronous background work (which will probably never be the case due to its low-level nature), I don't think we should add that complexity here.