[robbrad]

Summary

Consolidates the vetted open PRs into the next release, tested together on one branch.

• chore: bump docker/login-action 4.5.2 → 4.6.0, github/codeql-action 4.37.4 → 4.38.1 (from chore: bump docker/login-action from 4.5.2 to 4.6.0 #2227, chore: bump github/codeql-action from 4.37.4 to 4.38.1 #2237, dependabot)
• fix(vale-of-white-horse): don't infer a collection date into a bank holiday or Christmas week — the page publishes the current fortnight's pair of collections rather than the next occurrence of each bin, so a past date is rolled forward by 14 days; that's wrong when it lands in a week the council collects on a different day (bank holidays, and the 24 Dec–17 Jan slippage period), which only the page itself knows. Also fixes a separate bug where an early-January page still showing December's date was read as eleven months in the future instead of last year's. (from fix: ValeofWhiteHorseCouncil: don't infer a collection date into a bank holiday week #2229, credit @dpage, follows up on ValeofWhiteHorseCouncil: returns the previous collection date for the bin collected earlier in the fortnight #2213) — live-verified passing.
• fix(sunderland): use curl_cffi to pass Cloudflare's TLS fingerprint challenge, plus a full realistic header set — since mid-September the council's site 403s any client whose TLS handshake doesn't look like a real browser. Swaps the session for curl_cffi impersonating Chrome and, matching the existing pattern for Gateshead/Powys (same GOSS iCM platform, also Cloudflare-fronted), sends a complete browser-like header set rather than just a User-Agent. Not conclusively verifiable from this branch's own CI: the live integration run still gets Cloudflare's challenge page (cf-mitigated: challenge), which matches a known, already-documented limitation for these councils — Cloudflare's IP reputation scoring flags GitHub-hosted runner IPs regardless of client sophistication (see the comment already in PowysCouncil.py/GatesheadCouncil.py). Real-world verification will need to come from an actual deployment. (from fix(councils): use curl_cffi for SunderlandCityCouncil to pass Cloudflare TLS fingerprint challenge #2233, credit @jonny190, targets SunderlandCityCouncil: 403 from Cloudflare TLS fingerprint challenge since 15 Sep 2026 (curl_cffi fix tested) #2232)
• fix(bolsover): use a continuous fortnight index for bin parity, not a relative week number — the old parity calc was positioned relative to "today", so the same calendar date could get a different (wrong) bin type depending on when the scraper ran, and didn't reliably survive ISO week-53 year boundaries. Anchored to a fixed epoch validated against the council's published calendar instead. (from fix(BolsoverCouncil): bin type flips + days jump by +6 due to relative parity #2236, credit @JoeGandy) — live-verified passing.
• fix(ha): redact household and endpoint values from config-flow diagnostics — HA config-flow/setup logs previously included full config dicts and CLI args (postcodes, UPRNs, house numbers, private URLs) at debug/info level, which HA diagnostics can collect and share outside the local install. (from fix: redact sensitive configuration values from logs #2239, credit @dajiaohuang) — CI-verified passing on Python 3.14 across all supported HA versions (2026.2 through 2026.9, plus dev).
• fix(solihull): handle bin cards with no collection history — a bin card with no collection history yet (e.g. a newly added food caddy) renders only the "Next collection" row instead of both "Last collected" and "Next collection", so the previous fixed-index lookup went out of range. Reads the last of whichever date rows are present instead, and skips a card with none rather than crashing. (fixes Solihull Council: Unexpected error: list index out of range #2235) — live-verified passing.
• fix(exeter): send realistic browser headers on the address-finder request — the request previously sent no headers at all (not even a User-Agent), which points at bot/fingerprint detection given the Exeter City Council 403 error #2240 reporter could load the site fine in an ordinary browser. Adds the same kind of realistic header set already used for Gateshead/Powys. Also not conclusively verifiable from this branch's own CI — same known limitation as Sunderland: the site's Azure Application Gateway still 403s the runner's IP with a generic static error page, not a payload-validation error, which is a source-IP issue rather than proof the fix is wrong. (targets Exeter City Council 403 error #2240)

Investigated but not included

• Canterbury (Canterbury Council - 403 Error #2215, 403 error): tried switching the retired "Beta" API stage to "prod" (the stage another open-source project confirmed the council's site now uses), but this branch's own live CI still got a clean ForbiddenException from the prod stage too. Comparing against that other project's working implementation shows the real gap: it derives a genuine uprn+usrn pair from a postcode+house-number address lookup, while this scraper only ever accepts a raw UPRN with a hardcoded placeholder usrn. Matching that flow means changing this scraper's expected inputs (postcode+house number instead of UPRN alone) — a breaking change for existing users, not a same-shape fix — so it's reverted out of this release and left as a larger follow-up, with findings posted on Canterbury Council - 403 Error #2215.
• Buckinghamshire (Buckinghamshire Council - Empty Response error from council #2238 empty table, Unexpected error in async_setup_entry: Timeout while updating data #2228 timeout): this branch's own live CI ran the existing scraper unmodified against the real site and it passed cleanly - the collection table was populated, no timeout. Looks like both reports were transient/council-side rather than a code defect; commented on both issues asking reporters to confirm if still occurring.

Not included

#2180 ("harden South Kesteven scraper and Home Assistant runtime") remains excluded. It's still the same 199-file, +13.5k/-3.6k change bundling the actual SKDC fix with an unrelated cross-council print-redaction sweep, a new test harness, and a rewrite of the release/CI pipeline. I asked the author in July whether they'd split it into reviewable pieces; there's been no response in two months, so it's left as-is for a maintainer to decide on separately.

Test plan

• New regression tests all pass: 7 for Vale of White Horse (bank holiday/Christmas/January-rollover cases), 9 for Bolsover (parity alternation, ISO week-53/52 boundaries, green-bin suspension), 4 for Solihull (with/without collection history, mixed cards, no-date-rows skip)
• Full non-live uk_bin_collection unit suite green (no regressions)
• black --check clean on all changed files
• Live CI integration run: Bolsover, Buckinghamshire, Solihull, Vale of White Horse all passed live
• custom_components (HA redact fix) tests: green on Python 3.14 across every supported HA version in the compatibility matrix
• Sunderland and Exeter fail live from CI specifically (their respective edge/gateway layers challenge or block the runner's IP itself, not the client); neither can be conclusively verified from this environment — see notes above

---

🤖 Generated with Claude Code

https://claude.ai/code/session_015MaCS29izthZp2dwtvLwf4

Summary by CodeRabbit

• Bug Fixes
  • Improved bin schedules for Bolsover, Solihull, and Vale of White Horse, including alternating collection rounds, handling missing collection history, and accounting for rescheduled weeks and year boundaries.
  • Improved Sunderland and Exeter council lookups for more reliable address requests.
• Security & Privacy
  • Sensitive configuration details are now redacted from diagnostic logs.
• Chores
  • Updated security-analysis and release workflow actions.

[coderabbitai]

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: dcba4cc7-a4c5-4cbb-9a7b-f80de4fa9025

📥 Commits

Reviewing files that changed from the base of the PR and between 9fb64ea and 0a9ab8b.

⛔ Files ignored due to path filters (1)

• poetry.lock is excluded by !**/*.lock

📒 Files selected for processing (15)

• .github/workflows/codeql-analysis.yml
• .github/workflows/release.yml
• custom_components/uk_bin_collection/__init__.py
• custom_components/uk_bin_collection/config_flow.py
• custom_components/uk_bin_collection/const.py
• custom_components/uk_bin_collection/tests/test_config_flow.py
• pyproject.toml
• uk_bin_collection/tests/test_bolsover_council.py
• uk_bin_collection/tests/test_solihull_council.py
• uk_bin_collection/tests/test_vale_of_white_horse_council.py
• uk_bin_collection/uk_bin_collection/councils/BolsoverCouncil.py
• uk_bin_collection/uk_bin_collection/councils/ExeterCityCouncil.py
• uk_bin_collection/uk_bin_collection/councils/SolihullCouncil.py
• uk_bin_collection/uk_bin_collection/councils/SunderlandCityCouncil.py
• uk_bin_collection/uk_bin_collection/councils/ValeofWhiteHorseCouncil.py

 __________________________________________________
< Making NVIDIA proud with all this bug detection. >
 --------------------------------------------------
  \
   \   (\__/)
       (•ㅅ•)
       / 　 づ

✨ Finishing Touches

📝 Generate docstrings

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[codecov]

❌ 2 Tests Failed:

Tests completed	Failed	Passed	Skipped
5	2	3	0

View the full list of 2 ❄️ flaky test(s)

uk_bin_collection.tests.step_defs.test_validate_council::test_scenario_outline[ExeterCityCouncil]

Flake rate in main: 100.00% (Passed 0 times, Failed 623 times)

Stack Traces | 0.497s run time

fixturefunc = <function scrape_step at 0x7fae8cdd4670>
request = <FixtureRequest for <Function test_scenario_outline[ExeterCityCouncil]>>
kwargs = {'context': <test_validate_council.Context object at 0x7fae8cc263c0>, 'headless_mode': 'True', 'local_browser': 'False', 'selenium_url': 'http://localhost:4444'}

    def call_fixture_func(
        fixturefunc: _FixtureFunc[FixtureValue], request: FixtureRequest, kwargs
    ) -> FixtureValue:
        if inspect.isgeneratorfunction(fixturefunc):
            fixturefunc = cast(Callable[..., Generator[FixtureValue]], fixturefunc)
            generator = fixturefunc(**kwargs)
            try:
                fixture_result = next(generator)
            except StopIteration:
                raise ValueError(f"{request.fixturename} did not yield a value") from None
            finalizer = functools.partial(_teardown_yield_fixture, fixturefunc, generator)
            request.addfinalizer(finalizer)
        else:
            fixturefunc = cast(Callable[..., FixtureValue], fixturefunc)
>           fixture_result = fixturefunc(**kwargs)
                             ^^^^^^^^^^^^^^^^^^^^^

../../../..../pypoetry/virtualenvs/uk-bin-collection-EwS6Gn8s-py3.14/lib/python3.14.../site-packages/_pytest/fixtures.py:1005: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.../tests/step_defs/test_validate_council.py:101: in scrape_step
    context.parse_result = CollectData.run()
                           ^^^^^^^^^^^^^^^^^
uk_bin_collection/uk_bin_collection/collect_data.py:109: in run
    return self.client_code(
uk_bin_collection/uk_bin_collection/collect_data.py:130: in client_code
    return get_bin_data_class.template_method(address_url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
uk_bin_collection/uk_bin_collection/get_bin_data.py:64: in template_method
    bin_data_dict = self.get_and_parse_data(this_url, **kwargs)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
uk_bin_collection/uk_bin_collection/get_bin_data.py:87: in get_and_parse_data
    bin_data_dict = self.parse_data("", url=address_url, **kwargs)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.../uk_bin_collection/councils/ExeterCityCouncil.py:78: in parse_data
    response.raise_for_status()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <Response [403]>

    def raise_for_status(self) -> None:
        """Raises :class:`HTTPError`, if one occurred."""
    
        http_error_msg = ""
        if isinstance(self.reason, bytes):
            # We attempt to decode utf-8 first because some servers
            # choose to localize their reason strings. If the string
            # isn't utf-8, we fall back to iso-8859-1 for all other
            # encodings. (See PR #3538)
            try:
                reason = self.reason.decode("utf-8")
            except UnicodeDecodeError:
                reason = self.reason.decode("iso-8859-1")
        else:
            reason = self.reason
    
        if 400 <= self.status_code < 500:
            http_error_msg = (
                f"{self.status_code} Client Error: {reason} for url: {self.url}"
            )
    
        elif 500 <= self.status_code < 600:
            http_error_msg = (
                f"{self.status_code} Server Error: {reason} for url: {self.url}"
            )
    
        if http_error_msg:
>           raise HTTPError(http_error_msg, response=self)
E           requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://exeter.gov..../repositories/hidden-pages/address-finder/?qsource=UPRN&qtype=bins&term=100040212270

../../../..../pypoetry/virtualenvs/uk-bin-collection-EwS6Gn8s-py3.14/lib/python3.14.../site-packages/requests/models.py:1167: HTTPError

uk_bin_collection.tests.step_defs.test_validate_council::test_scenario_outline[SunderlandCityCouncil]

Flake rate in main: 59.42% (Passed 254 times, Failed 372 times)

Stack Traces | 0.401s run time

fixturefunc = <function scrape_step at 0x7fbf62642980>
request = <FixtureRequest for <Function test_scenario_outline[SunderlandCityCouncil]>>
kwargs = {'context': <test_validate_council.Context object at 0x7fbf62619be0>, 'headless_mode': 'True', 'local_browser': 'False', 'selenium_url': 'http://localhost:4444'}

    def call_fixture_func(
        fixturefunc: _FixtureFunc[FixtureValue], request: FixtureRequest, kwargs
    ) -> FixtureValue:
        if inspect.isgeneratorfunction(fixturefunc):
            fixturefunc = cast(Callable[..., Generator[FixtureValue]], fixturefunc)
            generator = fixturefunc(**kwargs)
            try:
                fixture_result = next(generator)
            except StopIteration:
                raise ValueError(f"{request.fixturename} did not yield a value") from None
            finalizer = functools.partial(_teardown_yield_fixture, fixturefunc, generator)
            request.addfinalizer(finalizer)
        else:
            fixturefunc = cast(Callable[..., FixtureValue], fixturefunc)
>           fixture_result = fixturefunc(**kwargs)
                             ^^^^^^^^^^^^^^^^^^^^^

../../../..../pypoetry/virtualenvs/uk-bin-collection-EwS6Gn8s-py3.14/lib/python3.14.../site-packages/_pytest/fixtures.py:1005: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.../tests/step_defs/test_validate_council.py:101: in scrape_step
    context.parse_result = CollectData.run()
                           ^^^^^^^^^^^^^^^^^
uk_bin_collection/uk_bin_collection/collect_data.py:109: in run
    return self.client_code(
uk_bin_collection/uk_bin_collection/collect_data.py:130: in client_code
    return get_bin_data_class.template_method(address_url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
uk_bin_collection/uk_bin_collection/get_bin_data.py:64: in template_method
    bin_data_dict = self.get_and_parse_data(this_url, **kwargs)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
uk_bin_collection/uk_bin_collection/get_bin_data.py:87: in get_and_parse_data
    bin_data_dict = self.parse_data("", url=address_url, **kwargs)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.../uk_bin_collection/councils/SunderlandCityCouncil.py:64: in parse_data
    r.raise_for_status()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <Response [403]>

    def raise_for_status(self):
        """Raise an error if status code is not in [200, 400)"""
        if not self.ok:
>           raise HTTPError(f"HTTP Error {self.status_code}: {self.reason}", 0, self)
E           curl_cffi.requests.exceptions.HTTPError: HTTP Error 403:

../../../..../pypoetry/virtualenvs/uk-bin-collection-EwS6Gn8s-py3.14/lib/python3.14.../curl_cffi/requests/models.py:241: HTTPError

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}