Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Evaluate the impact of sharding in MongoDB indexes #91

Open
andre-senna opened this issue Jun 27, 2024 · 7 comments · May be fixed by singnet/das-pre-infra-vultr#16
Open

Evaluate the impact of sharding in MongoDB indexes #91

andre-senna opened this issue Jun 27, 2024 · 7 comments · May be fixed by singnet/das-pre-infra-vultr#16
Assignees
Labels
Public SHould be cc'ed to the public board

Comments

@andre-senna
Copy link
Contributor

No description provided.

@levisingularity
Copy link
Collaborator

@andre-senna My proposal is to create two distinct environments: one with a standalone database and another configured with sharding. Both environments should have data loaded and indexes set up according to the requirements of the queries to be executed. A script will be developed to run a series of queries on both databases using the defined indexes. At the end of the process, a comparative chart will be generated showing the average query execution times for each configuration, allowing us to evaluate the impact of sharding on performance. Additionally, a document will be created to explain the test methodology, the infrastructure components used, and a conclusion describing the results obtained.

@andre-senna
Copy link
Contributor Author

andre-senna commented Jul 25, 2024

Depends on singnet/das-atom-db#150

@levisingularity
Copy link
Collaborator

Image

Finishing the script to prepare the environment

@andre-senna andre-senna transferred this issue from singnet/das-atom-db Aug 23, 2024
@andre-senna andre-senna added the Public SHould be cc'ed to the public board label Aug 23, 2024
@andre-senna andre-senna added this to the Enhanced version of AtomDB milestone Aug 23, 2024
@levisingularity
Copy link
Collaborator

Image

Shard diagram

@levisingularity
Copy link
Collaborator

I'm debugging the benchmark code trying to understand it better and getting some logs to output what the code is doing

@levisingularity
Copy link
Collaborator

output.txt
This is the output of the "smoke test"

@levisingularity
Copy link
Collaborator

After the edgar's fix seems it's working as expected:

Image

@andre-senna andre-senna removed this from the Enhanced version of AtomDB milestone Dec 10, 2024
@levisingularity levisingularity moved this from In Progress to Blocked in DAS - Public Board Dec 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Public SHould be cc'ed to the public board
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants