Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change the want list to work in on demand mode #15

Merged
merged 6 commits into from
Jul 5, 2022

Conversation

boreq
Copy link
Contributor

@boreq boreq commented Jul 4, 2022

After this change the Swift code can request blobs to be downloaded and
the blobs remain in the want list for a specific amount of time. This
replace the previous approach of downloading all blobs right away and
copies the current behaviour exhibited by go-ssb. The new code behaves
slightly better as the want list isn't cleared when application exists
and is persisted.

After this change the Swift code can request blobs to be downloaded and
the blobs remain in the want list for a specific amount of time. This
replace the previous approach of downloading all blobs right away and
copies the current behaviour exhibited by go-ssb. The new code behaves
slightly better as the want list isn't cleared when application exists
and is persisted.
@boreq boreq force-pushed the on-demand-want-list branch from 5b682bc to 977a59e Compare July 4, 2022 15:31
@boreq boreq changed the title Change the want list to work in the on demand mode Change the want list to work in on demand mode Jul 4, 2022
@boreq boreq marked this pull request as ready for review July 5, 2022 11:10
Comment on lines +124 to +127
if now.After(until) {
toDelete = append(toDelete, id)
return nil
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering if we could somehow make the cleanup part independent to make reads faster. Maybe we should not care about cleaning up at this point and add it later as an independent process, along with removing blobs from the disk?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or we can leave it as it is for now, and add a point to #1 to take care of it later.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a point to #1 already as I think this is good enough for now as it "kinda works" but I definitely want to fix this soon.

service/adapters/bolt/want_list_repository_test.go Outdated Show resolved Hide resolved
service/domain/blobs/replication/has_handler.go Outdated Show resolved Hide resolved
@boreq boreq merged commit b525460 into feature/blobs Jul 5, 2022
boreq added a commit that referenced this pull request Jul 5, 2022
After this change the Swift code can request blobs to be downloaded and
the blobs remain in the want list for a specific amount of time. This
replace the previous approach of downloading all blobs right away and
copies the current behaviour exhibited by go-ssb. The new code behaves
slightly better as the want list isn't cleared when application exists
and is persisted.
@boreq boreq deleted the on-demand-want-list branch July 5, 2022 13:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

2 participants