Repository navigation
fix: #2153 Lincoln stale anchor date + Flintshire schedule-note cleanup - #2166
Merged
Merged
Conversation
The council's API returns a "next date" field for each bin type that can lag behind reality by weeks (confirmed live: refusenextdate was 2026-06-03, over 6 weeks in the past, identical across every address in the postcode - a staleness issue on their end, not address-specific). The scraper only ever projected a small fixed window of occurrences forward from that anchor, so when the anchor itself was stale enough, every generated date - including the last one in the window - was still in the past. HA's coordinator then filtered out all of them, producing "No bin data found" with nothing to fall back to (the reported symptom). Now rolls the anchor date forward by the collection's own frequency (weekly/fortnightly) until it's genuinely upcoming before generating the occurrence window, so a stale anchor no longer produces an entirely-past result. Verified live: previously all-past dates (03/06-01/07) now correctly resolve to 15/07 onward.
Contributor
📝 WalkthroughWalkthroughLincoln council parsing now rolls collection dates forward using recognized weekly or fortnightly frequencies, skips invalid past dates, and limits generated occurrences. Flintshire parsing ignores non-bin schedule notices, while Lincoln property matching formatting remains behaviorally unchanged. ChangesCouncil parsing
Estimated code review effort: 2 (Simple) | ~10 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
4 tasks done
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@uk_bin_collection/uk_bin_collection/councils/LincolnCouncil.py`:
- Around line 90-107: Update the frequency handling in the BIN_TYPES loop to
distinguish missing/empty frequency values from unrecognized non-empty values.
Preserve the existing one-off behavior only when the frequency is absent, but
raise an explicit error for any non-empty value not present in FREQ_DAYS instead
of entering the else branch; retain the existing roll-forward logic for
recognized frequencies.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: ee034516-fc1e-49b2-88fb-6b8174b33cbb
📒 Files selected for processing (1)
uk_bin_collection/uk_bin_collection/councils/LincolnCouncil.py
Comment on lines
90
to
+107
| for key, bin_type, freq in BIN_TYPES: | ||
| if not data[key]: | ||
| continue | ||
| offsets = [0] | ||
| if data[freq] == "fortnightly": | ||
| offsets.extend(list(range(14, 30, 14))) | ||
| elif data[freq] == "weekly": | ||
| offsets.extend(list(range(7, 30, 7))) | ||
| date = datetime.strptime(data[key], "%Y-%m-%d").date() | ||
| step = FREQ_DAYS.get(data[freq]) | ||
| if step: | ||
| # The council's "next date" field can lag behind reality | ||
| # by weeks - roll it forward by the collection frequency | ||
| # until it's genuinely upcoming, rather than trusting it | ||
| # as always being the next one. | ||
| while date < today: | ||
| date += timedelta(days=step) | ||
| occurrences = 3 if step == 14 else 5 | ||
| offsets = [step * n for n in range(occurrences)] | ||
| else: | ||
| if date < today: | ||
| continue | ||
| offsets = [0] |
Contributor
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Unrecognized/empty freq values silently skip the roll-forward fix.
step = FREQ_DAYS.get(data[freq]) returns None for any frequency string other than exactly "weekly"/"fortnightly" (typo, casing mismatch, empty string, or a new frequency category added by the council). In that case the code falls into the else branch and treats the anchor date as a one-off, silently reproducing this PR's original bug (stale past dates causing unavailable sensors) for that bin type instead of surfacing the unexpected value.
Based on learnings, bin collection parsing should prefer explicit failures over silent defaults when encountering unexpected formats, so format regressions are caught early.
🐛 Suggested fix: raise on unrecognized non-empty frequency values
date = datetime.strptime(data[key], "%Y-%m-%d").date()
- step = FREQ_DAYS.get(data[freq])
+ freq_value = data[freq]
+ if freq_value and freq_value not in FREQ_DAYS:
+ raise ValueError(
+ f"Unrecognized collection frequency '{freq_value}' for {bin_type}"
+ )
+ step = FREQ_DAYS.get(freq_value)
if step:📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| for key, bin_type, freq in BIN_TYPES: | |
| if not data[key]: | |
| continue | |
| offsets = [0] | |
| if data[freq] == "fortnightly": | |
| offsets.extend(list(range(14, 30, 14))) | |
| elif data[freq] == "weekly": | |
| offsets.extend(list(range(7, 30, 7))) | |
| date = datetime.strptime(data[key], "%Y-%m-%d").date() | |
| step = FREQ_DAYS.get(data[freq]) | |
| if step: | |
| # The council's "next date" field can lag behind reality | |
| # by weeks - roll it forward by the collection frequency | |
| # until it's genuinely upcoming, rather than trusting it | |
| # as always being the next one. | |
| while date < today: | |
| date += timedelta(days=step) | |
| occurrences = 3 if step == 14 else 5 | |
| offsets = [step * n for n in range(occurrences)] | |
| else: | |
| if date < today: | |
| continue | |
| offsets = [0] | |
| for key, bin_type, freq in BIN_TYPES: | |
| if not data[key]: | |
| continue | |
| date = datetime.strptime(data[key], "%Y-%m-%d").date() | |
| freq_value = data[freq] | |
| if freq_value and freq_value not in FREQ_DAYS: | |
| raise ValueError( | |
| f"Unrecognized collection frequency '{freq_value}' for {bin_type}" | |
| ) | |
| step = FREQ_DAYS.get(freq_value) | |
| if step: | |
| # The council's "next date" field can lag behind reality | |
| # by weeks - roll it forward by the collection frequency | |
| # until it's genuinely upcoming, rather than trusting it | |
| # as always being the next one. | |
| while date < today: | |
| date += timedelta(days=step) | |
| occurrences = 3 if step == 14 else 5 | |
| offsets = [step * n for n in range(occurrences)] | |
| else: | |
| if date < today: | |
| continue | |
| offsets = [0] |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@uk_bin_collection/uk_bin_collection/councils/LincolnCouncil.py` around lines
90 - 107, Update the frequency handling in the BIN_TYPES loop to distinguish
missing/empty frequency values from unrecognized non-empty values. Preserve the
existing one-off behavior only when the frequency is absent, but raise an
explicit error for any non-empty value not present in FREQ_DAYS instead of
entering the else branch; retain the existing roll-forward logic for recognized
frequencies.
Source: Learnings
Found while re-investigating #2118: some collection weeks include a row like "*New Round* - Tuesday AHP" in the same column as the actual bin type, which was getting appended to the bins list as if it were a collection. These notice rows are consistently prefixed with "*", so skip them rather than treating them as a bin type. Verified live - "Brown Bin" entries (the subject of #2118) are still correctly extracted; the junk entries are gone.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #2166 +/- ##
=======================================
Coverage 86.89% 86.89%
=======================================
Files 9 9
Lines 1160 1160
=======================================
Hits 1008 1008
Misses 152 152 ☔ View full report in Codecov by Harness. |
…into fix/lincoln-stale-anchor-date
robbrad
changed the title
4 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Root cause of #2153 ("Lincoln city no longer works", all sensors unavailable, log shows "No bin data found... No previous data to fall back to").
The council's API returns a "next date" per bin type that can lag behind reality by weeks. Verified live right now:
refusenextdateis2026-06-03- over 6 weeks in the past - and identical across every single address in the test postcode, confirming this is a staleness issue on the council's own backend, not anything address-specific (rules out a bad UPRN/postcode on the reporter's end).The scraper only ever projected a small fixed window of occurrences forward from that anchor (
[0, 7, 14, 21, 28]days for weekly,[0, 14, 28]for fortnightly). Once the anchor is stale by more than that window, every generated date - including the last one - is still in the past. HA's coordinator filters out all past dates, leaving nothing, which is exactly the reported "No bin data found, no previous data to fall back to" symptom.Fix: roll the anchor date forward by the bin type's own frequency (weekly/fortnightly) until it's genuinely upcoming, then generate the occurrence window from there. A stale anchor no longer produces an entirely-past result.
Also folds in #2167 (FlintshireCountyCouncil)
Found while re-investigating #2118 (Brown Bin / Garden Waste). The Brown Bin extraction itself was already working correctly (unrelated to this fix), but along the way found a genuine, separate bug: some collection weeks include a schedule-notice row (
*New Round* - Tuesday AHP) in the same table column as the actual bin type, which was getting appended to the bins list as if it were a real collection. These notice rows are consistently prefixed with*, so skip them.Test plan
*New Round*entries gone,Brown Binand everything else still correctly presentpoetry run pytest uk_bin_collection/tests/step_defs/ -k "LincolnCouncil or FlintshireCountyCouncil"- both passedpoetry run pytest uk_bin_collection/tests custom_components/uk_bin_collection/tests --ignore=uk_bin_collection/tests/step_defs/- 221 passedblack --checkcleanFixes #2153
Summary by CodeRabbit