Configure requests and collectors #1946
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…string in their heresphere api
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This change allows users to specify request headers, cookies, and the request body. It does not modify headers, cookies, or the body for existing requests made by XBVR—rather, it simply provides users with the ability to set them if they choose to.
This can be useful in scenarios such as bypassing age verification prompts or skipping redirects to advertising pages. The effectiveness of these settings depends entirely on how the site has been implemented.
The configured headers, cookies, and body are stored in a record within the kvs table. When a scraper collector is created or a trailer is requested from the Heresphere API, the system checks whether any modifications need to be applied to the request. For trailers, this can be used to authenticate with the site and access the full scene instead of just the trailer.
The key for the kvs table entry is, by default, the domain of the request followed by either -scraper or -trailers. A UI for configuring these settings is available under Options > Advanced > "Cookies/Headers". Since these details are stored in the kvs table, they can be imported and exported along with bundle imports/exports by selecting "Include config settings". Users can also import individual settings.
Additionally, I have updated the Naughty America trailer functionality to use their Heresphere API instead of a direct URL. This change provides users with more codec and resolution options for trailers.
I will also provide a Wiki page with more complete instructions on how to set this up.