Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft: Threadpool refactor fsds #55

Open
wants to merge 76 commits into
base: threadpool-refactor
Choose a base branch
from

Conversation

elcritch
Copy link
Contributor

@elcritch elcritch commented Sep 27, 2023

Refactor sqliteds into a synchronous backend. The ThreadDatastore was then refactored to use the synchronous sqlite backend. The ThreadDatastore is based on #48 but with simplifications due to not needing to handle Futures on the backend.

The FsDs should now be pretty easy to refactor by re-using the design for SqliteDs.

Other changes:

  • SqliteDs should be fully thread-safe now barring errors
  • SqliteDs is now fully generic and can use keys of string | KeyId and data as seq[byte] | DataBuffer
  • ThreadDatastore is about ~20% shorter than previously since there's no futures on the backend
  • ThreadDatastore seems to be about ~10% faster, but just based off timing tests
  • Queries run on a single thread simplifying key conflicts and preventing queries from swamping the system

Some todos:

  • fix potential FsBackend race conditions

@elcritch elcritch marked this pull request as draft September 27, 2023 04:16
@elcritch elcritch marked this pull request as ready for review September 28, 2023 22:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant