[robbrad]

Summary

Priority fix for #2158, and one of the 3 new failures found in a fresh full-suite run.

Root cause: this is a self-inflicted regression from the #2073 retry fix (PR #2155, already merged). AberdeenshireCouncil.parse_data() completely ignores the page it's given and makes its own request to a specific, working endpoint (/Apps/Waste-Collections/Routes/Route/{uprn}) - but the generic framework fetch to the bare root URL (online.aberdeenshire.gov.uk/, from input.json's url field) ran first regardless, since this council was never marked skip_get_url. That root URL is currently returning a persistent HTTP 500 on the council's own end - previously harmless (a bare requests.get() doesn't raise on non-2xx status), but the #2073 fix's retry session correctly (and now, for this council, unfortunately) turns 5 straight 500s into a hard RetryError, before parse_data() (which never needed that page anyway) even runs.

Verified live: online.aberdeenshire.gov.uk/ → 500, but online.aberdeenshire.gov.uk/Apps/Waste-Collections/Routes/Route/{uprn} → 200 with valid data, at the same moment.

Fix: override get_data() in AberdeenshireCouncil.py to skip the fetch entirely (return "") rather than relying on skip_get_url - that flag comes from each user's already-saved HA config data, which existing installs don't have, so setting it in input.json alone would only fix the test suite, not real users. This fix works immediately on upgrade with zero config changes needed. Verified via the actual CLI entrypoint HA uses, with no skip_get_url flag passed (matching existing users' saved config).

Broader finding (not fixed here, flagging for awareness)

Found 59 councils with this same shape - parse_data() ignores page and makes its own request, but skip_get_url isn't set. Most are harmless (their url field happens to point at a healthy resource that's just redundantly fetched), so I haven't touched them. Cross-referenced against a fresh full-suite run: only Aberdeenshire and (initially suspected, then ruled out - see below) Fylde are currently affected. Worth a slower systematic audit at some point, but not urgent since only one is actually live.

Test plan

• Verified live: root URL 500s, actual data endpoint 200s with valid data
• poetry run pytest uk_bin_collection/tests/step_defs/ -k AberdeenshireCouncil - passed
• Verified via CLI (collect_data.py) with no skip_get_url flag, matching existing users' saved config - returns correct bin data
• black --check clean

Fixes #2158

Summary by CodeRabbit

• Bug Fixes
  • Improved waste collection information handling for Aberdeenshire Council when no root-page content is available.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: b5c775d2-2f8d-4c27-8479-5d9a06438bc3

📥 Commits

Reviewing files that changed from the base of the PR and between 46d94c1 and 28627dc.

📒 Files selected for processing (1)

• uk_bin_collection/uk_bin_collection/councils/AberdeenshireCouncil.py

---

📝 Walkthrough

Walkthrough

AberdeenshireCouncil adds a class method that skips fetching the supplied root URL and returns an empty string.

Changes

Aberdeenshire data handling

Layer / File(s)	Summary
Skip root URL fetch uk_bin_collection/uk_bin_collection/councils/AberdeenshireCouncil.py	get_data returns an empty string without fetching the provided URL, with comments describing the root-page behavior.

Estimated code review effort: 1 (Trivial) | ~2 minutes

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly matches the main change: AberdeenshireCouncil now skips fetching the unused root URL.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/aberdeenshire-get-data-regression

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.89%. Comparing base (4d6f787) to head (28627dc).
⚠️ Report is 2 commits behind head on master.
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #2159   +/-   ##
=======================================
  Coverage   86.89%   86.89%           
=======================================
  Files           9        9           
  Lines        1160     1160           
=======================================
  Hits         1008     1008           
  Misses        152      152           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

[robbrad]

Folded into #2163 to keep this to a single PR. Closing this one - the commit is now part of #2163's branch.