-
Notifications
You must be signed in to change notification settings - Fork 45
Description
when working on https://github.com/haskell-wasm/bsdtar-wasm i noticed that it takes about 2 seconds to extract an archive to native file system with wasmtime or nodejs/uvwasi, but more than a minute to extract to in-memory vfs when using this library. so i took a look at the profile and noticed that the hotspot is fd_write, which concatenates buffers when appending a new buffer to a file. so when we do a lot of small sequential writes to a file, this quickly blows up due to the O(n^2) time complexity in repeated concatenation.
we can fix it by implementing exponential growth of the file data buffer, so the amortized time complexity of appending data would drop down to O(n) where n is the length of data chunk to be appended. though that'll have the side effect of increased memory overhead, so i'd like to hear your opinion: would you be okay to accept a patch to address this issue at the cost of increased memory overhead? or would you rather gate it behind a feature flag to be opted in? thanks.