libkiwix shoudl handle remote library HTTP requests

[kelson42]

As suggested somewhere (I cannot find where), I think we should introduce a "RemoteLibrary" in libkiwix.
This remote library should share the same (subset) api of Library :

• getBookById
• getBookCount
• getBooksLanguages
• getBooksLanguagesWithCounts
• getBooksCategories
• getBooksCreators
• getBooksPublishers
• getBooksIds
• filter (?)
• sort (?)
• filter_sorted_paginated (to create)

However, the RemoteLibrary would do request to the server for each methods (we almost already have a specific endpoint for each method)

Request and caching strategy:

We will use partial_entries endpoint to get list of books (getBooksIds and filter_sorted_paginated).
Theses methods already return bookId only (as partial_entries endpoint, so we are good).

When user call getBookById with a bookId, we do a request to entry endpoint and get the book information and create a Book. The Book itself is cached and so, no duplicate request to entry is made.

About Illustration:
OPDS already give a url to the illustration instead of the data (already to reduce the size of the opds stream).
Illustration class itself is downloading the data when we call Illustration::getData(). This is somehow duplicated with ThumbnailDownloader implemented in kiwix-desktop by @juuz0 (sorry, I missed that)

This lead to download management:

The problem here is that don't want the download to freeze the ui.

• Easy solution:
  Don't care. If we do multiple small requests (about few kB), the io latency may be negligeable and if it is, fix it later.
• Middle solution:
  Libkiwix don't care. It is up to kiwix-desktop (and other client ?) to call the remote library method in a different thread and not block its ui
• Difficult solution:
  Build a two step system: methods don't return the results directly but return a tuple (url, callback). url being the url to request (by threaded requests). callback being a function to call with the response content and which actually return the "useful results".
  A blocking application would use :

auto tuple = library.getBooksIds();
auto bookIds = tuple.1(download(tuple.0));
do_stuff_with_bookids(bookIds);

A event driven application would use:

auto tuple = library.getBooksIds();
auto request = Request(tuple.0);
request.on_completion([](response) {
  auto bookIds = tuple.1(response.content);
  do_stuff_with_bookids(bookIds);
});
request.do();
do_other_things_without_waiting_request_to_finish();

• Hard solution:
  Design a full async/callback api on libkiwix.

I would go with the easy solution and if not enough, move to the difficult solution.
(The "Easy remote library" becoming a wrapper around the "difficult remote library" with the "blocking application" code)

---

I have played a bit with the api:

• full catalog opds full entry content is about 5MB
• full catalog opds with partial entries only (only the list of books) is 1.4MB
• 50 entries opds full entry content is 60K
• 50 entries opds partial entries only is 18K
• One full entry 1.2K

(All done without filter)

From a comment by @mgautierfr at kiwix/kiwix-desktop#957 (comment)