Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

High memory usage #86

Open
baszalmstra opened this issue Nov 28, 2024 · 0 comments
Open

High memory usage #86

baszalmstra opened this issue Nov 28, 2024 · 0 comments

Comments

@baszalmstra
Copy link
Contributor

Description

We encountered a few cases where the solver uses more than 1.5GB of RAM. Almost all memory usage seems to be related to clauses, specifically ForbidMultipleInstances.

Total number of solvables:  65335
Total number of watches:    68317 (1021 KiB)
Total number of clauses:    46291650 (1.4 GiB)
- Requires:                 423637
- Constrains:               4882449
- Lock:                     0
- Learned:                  1
- ForbidMultiple:           40985343
  - pyarrow:                22101276
  - arrow-cpp:              9885681
  - botocore:               1798356
  - boto3:                  1335795
  - hypothesis:             1191196
  ...

This happens when there are a relatively large number of candidates to consider for a single version (a few thousand). The number of ForbidMultiple clauses that we allocate is O(n^2).

Possible solution

We could look at implementing commander encoding to encode these clauses differently which should be O(n). We already implemented binary encoding in #37 but with a number of other improvements this PR has fallen to the wayside.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant