Update scrapingbee-cli package to support Header based Authorization - #32
Open
sahilsunny wants to merge 3 commits into
Conversation
The google --help variant matched on the raw PTY stream, but the REPL's differential renderer can repaint straight to the final frame without ever emitting the scrolled-off warning line into the stream, so the assertion raced on slow CI runners (failed on ubuntu/3.11 in PR CI and macos/3.10 on main — different cells, same mechanism). Restore the 1.5.1-era approach: usage --help keeps the output short so the warning stays on screen, and the transient pump checks every intermediate screen state.
…SCR-585)
- All Client requests now authenticate via 'Authorization: Bearer' instead
of the deprecated api_key query parameter, so the key no longer appears
in request URLs. Applies to every command including auth validation and
usage; crawl's scrapy-scrapingbee middleware still builds api_key URLs
and migrates separately.
- google --nb-results: requested results per page. Absent from llms.txt
but verified live: the API strictly rejects unknown params (400 'Unknown
field' control) yet accepts nb_results with 200.
- amazon-product --autoselect-variant: matches the existing amazon-search
flag; same live verification (200 vs 400 bogus-param control).
- Skill docs: --custom-google price corrected 15 -> 20 credits per the
API's own error message ('Each request will cost 20 credits!').
Live-verified via pipx build: usage/GET/PUT all 200 under Bearer auth,
nb-results and autoselect-variant accepted. 863 unit tests green (new:
Bearer session header, no api_key in GET/POST/usage params, param
forwarding for both new options); ruff + ty clean.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Switches API auth from the deprecated
api_keyquery param to theAuthorization: Bearerheader, so the key no longer shows up in URLs. The only exception iscrawl, whose scrapy-scrapingbee middleware will be migrated separately.Also adds two params that are missing from the docs but confirmed working against the live API (it rejects unknown params with 400, these return 200):
google --nb-resultsamazon-product --autoselect-variantStays on 1.6.0 since it isn't published yet; release date set to 2026-08-24.
Tested live on a pipx install (usage, GET, PUT, both new flags) plus unit tests for the new behavior. Based on #31 — merge that first, with a merge commit (not squash).