Skip to content

feat: expose coordinator polling intervals as user-configurable options - #234

Open
fullstackPrincess wants to merge 1 commit into
Jezza34000:mainfrom
fullstackPrincess:feat/configurable-polling-intervals
Open

feat: expose coordinator polling intervals as user-configurable options#234
fullstackPrincess wants to merge 1 commit into
Jezza34000:mainfrom
fullstackPrincess:feat/configurable-polling-intervals

Conversation

@fullstackPrincess

Copy link
Copy Markdown

📝 Proposed Change / Description

Adds a new Advanced polling section to the integration's options flow that surfaces three coordinator timings that previously lived as hard-coded constants in const.py:

Option Default Range Replaces
scan_interval_slow 190 s 5–600 s SCAN_INTERVAL_SLOW
scan_interval_default 60 s 5–600 s DEFAULT_SCAN_INTERVAL
smart_polling_boost_duration 15 s 5–300 s enable_smart_polling(3) (3 ticks × 5 s)

Why

With the current /<product_key>/<deviceName>/user/get-only MQTT subscription, the IoT MQTT listener typically does not receive any device-state push messages from the PetKit cloud (messages_received stays at 0 for many setups, including mine on the api-ru.petkit.cn regional cloud). State updates therefore come exclusively from the REST polling path inside PetkitDataUpdateCoordinator.

The hard-coded 190 s slow interval is fine for low cloud load, but inconvenient when an event-triggered binary sensor (e.g. binary_sensor.toilet_occupied) needs to fire an automation in something closer to real time. With this PR, users who are happy to accept the extra REST traffic can drop the slow interval to e.g. 5–30 s without forking the integration; users who want to be gentle on the cloud can raise it.

The defaults are unchanged so behaviour is preserved out of the box.

Fixes: N/A


🔖 Type of change

  • ✨ New feature (non-breaking change which adds functionality)

🐾 Affected devices

  • 🌐 All devices

The change touches the shared coordinator + options flow, so it benefits every device that relies on cloud polling — most relevant for litter boxes (per-visit events), feeders (manual feed feedback), and air purifiers, but applies across the board.


🧪 How has this been tested?

  • Tested locally with a real PetKit device
  • Tested with Home Assistant version: 2026.5.x (Docker ghcr.io/home-assistant/home-assistant:stable)

Device(s) tested on: PetKit PuraMax 2 + Air Smart Spray (K3) on api-ru.petkit.cn.

What I verified:

  1. Fresh install — config flow seeds the new section with defaults (190/60/15) and the integration loads without errors.
  2. Upgrade from version 8 — async_migrate_entry bumps the entry to version 9 and adds the new section with defaults; existing entries keep their previous behaviour.
  3. Options flow — the new "Advanced polling" section appears (collapsed by default), pre-populated with current values; saving new values reloads the entry and the coordinator picks them up immediately via the existing async_reload_entry update listener.
  4. Polling — set scan_interval_slow=5 and confirmed in debug logs (pypetkitapi.client) that REST refreshes happen every ~5 s with no 429 / auth errors over an extended observation window. Reverted to defaults — refreshes return to ~190 s as expected.
  5. Smart polling boost — binary_sensor.*_toilet_occupied flipping on still triggers the temporary 5 s ticking; with smart_polling_boost_duration=15 it lasts 3 fast ticks as before. Setting it to 30 correctly extends the burst to 6 ticks.

✅ Checklist

  • Code follows project style: pre-commit run over the touched files passes (black / ruff / codespell / prettier / detect-secrets / prettier-json all green)
  • No new linting/type errors introduced
  • Documentation updated (if needed) — see "Additional context" below regarding the wiki
  • Branch is up-to-date with main

ℹ️ Additional context

  • Backwards compatibility. All hard-coded intervals stay as the defaults for both new entries and the migration step, so users who never touch the new section continue to see the existing 190 s / 60 s / 15 s behaviour.
  • enable_smart_polling() signature. The argument was made optional (boost_duration: int | None = None). Existing callers were updated to call it without arguments so the coordinator uses the user-configured duration; explicit overrides still work for any future caller that needs a one-off duration.
  • Translations. English strings (name, data, data_description, description) are added for the new section. A Russian translation for the new section is included; while editing ru.json I also added the previously-missing notifications_options block (it already exists in en.json but had no Russian counterpart) — happy to split this into a separate PR if preferred.
  • Wiki. The user-facing Configuration wiki page does not live in this repo, so it is not updated here. After merge it would benefit from a short paragraph documenting the new section — I can prepare a wiki diff in a follow-up if useful.

📸 Logs

Polling interval taking effect after setting scan_interval_slow=5 via UI (debug logs from pypetkitapi.client):

14:21:07.503 DEBUG ... List devices data for account: ...
14:21:07.504 DEBUG ... Request: POST https://api-ru.petkit.cn/latest/t4/device_detail
14:21:09.446 DEBUG [custom_components.petkit] Finished fetching petkit.devices data in 1.943 seconds (success: True)
14:21:14.5xx DEBUG ... (next refresh, ~5 s later)

⚠️ Breaking Changes

  • No breaking changes.

The new section is additive. Default values reproduce the exact prior behaviour and the migration step seeds them automatically for upgraded entries, so no user action is required after this PR is merged.


Thanks for the helping paw! 🐾

Adds a new "Advanced polling" section to the integration's options flow
with three knobs that previously lived as hard-coded constants in
const.py:

- scan_interval_slow (sec) - refresh interval used while the IoT MQTT
  listener is connected. Default 190 s, matches previous behaviour.
- scan_interval_default (sec) - fallback interval used when the IoT
  MQTT listener is disconnected. Default 60 s.
- smart_polling_boost_duration (sec) - how long the temporary fast
  poll boost lasts after an event-triggered binary sensor flips on.
  Default 15 s, equivalent to the previous 3 ticks * 5 s.

Rationale: with the current /user/get-only MQTT subscription, the
coordinator essentially polls the PetKit REST API for state updates.
The hard-coded 190 s slow interval is conservative for cloud usage
but inconvenient for users who want near-real-time updates from event
sensors (e.g. binary_sensor.toilet_occupied) and are happy to accept
the extra REST traffic.

Implementation notes:

- Coordinator reads the new section in __init__ and exposes the
  values as instance attributes (scan_interval_slow,
  scan_interval_default, smart_polling_boost_duration).
- enable_smart_polling() now derives the tick count from the boost
  duration option; an explicit boost_duration argument is still
  accepted for backwards compatibility.
- Existing constants stay as defaults so missing options fall back to
  identical historical behaviour. A migration step
  (entry version 8 -> 9) seeds the new section on upgrade.
- Updated existing callers in binary_sensor.py and button.py to use
  the option-driven default.
- English and Russian translations added (existing options sections
  also gained the previously-untranslated notifications_options
  Russian strings).
@sonarqubecloud

Copy link
Copy Markdown

@Jezza34000

Copy link
Copy Markdown
Owner

Hello @fullstackPrincess
Thank you for the suggestion. However, this feature was designed to avoid configuration confusion and automate the process, as very few people actually modify this configuration manually in the vast majority of cases.

However, your PR raises a very interesting point: the Petkit integration connects to MQTT but never receives any notifications from the servers?
If this is the case, then the design of my time-based polling MQTT IoT architecture needs to be reviewed. Rather than offering the option to configure this manually.

@fullstackPrincess

Copy link
Copy Markdown
Author

Hi @Jezza34000 — thanks for the thoughtful response, and especially for highlighting that "do they actually get pushed at all?" is the more interesting question. I went and tried to answer it empirically, and the result was surprising enough that I think it changes the picture for this PR.

TL;DR

The Aliyun IoT broker does push device events — the integration just isn't receiving them. A standalone paho-mqtt client connected with the exact same credentials, client id, clean_session=False, protocol, and topic the integration uses captured 47 push messages in 6 minutes — including the litter-box state updates I care about. With the integration re-enabled in the same conditions, sensor.petkit_mqtt.messages_received stayed at 0 for the entire 3-minute follow-up window despite identical hardware activity.

Scenario Topic subscribed Messages received
Standalone paho-mqtt sniffer (HA petkit entry disabled) /<product_key>/<device_name>/user/get 47 in 6 min
PetkitIotMqttListener (HA petkit entry enabled) /<product_key>/<device_name>/user/get 0 in 3 min

So time-based polling isn't compensating for cloud silence — there's an integration-side blind spot or threading bug that drops these messages on the floor.

What I'm trying to react to

This PR's motivating use case for me is firing a Home Assistant automation the moment a cat enters the litter box — the binary_sensor.petkit_puramax_2_toilet_occupied off → on transition. That state is exactly what the cloud is pushing out:

TOPIC=/<product_key>/app_<USER_ID>/user/get
PAYLOAD={"msgType":0,"payload":{"contentAsString":"{\"type\":\"api.t4.immediate.stateupdate\",\"content\":{\"type\":3,\"deviceId\":<LITTER_DEVICE_ID>}}","from":{"username":"d.t4.<LITTER_DEVICE_ID>"},"to":{"username":"app_<USER_ID>"},"timestamp":1778493107},"type":"notifitation_app","deviceName":"app_<USER_ID>","timestamp":1778493107}

These api.t4.immediate.stateupdate envelopes show up within a couple of seconds of any litter activity — exactly the latency I'd want for an automation. With pure REST polling at the default SCAN_INTERVAL_SLOW = 190, that automation can run up to ~3 minutes late, which is a long time for a "the cat just walked in" trigger. Hence the PR.

Methodology

  1. Disabled the HA petkit entry to release the IoT identity.
  2. Called pypetkitapi.PetKitClient.get_iot_mqtt_config() with the same (username, password, region, timezone) to get the same mqtt_host, product_key, device_name, device_secret.
  3. Connected with paho-mqtt using the same Aliyun HMAC-SHA256 signing, clean_session=False, MQTTv311, and client_id derived from device_name.
  4. Subscribed to a wide set of patterns to also probe what ACL allows. Of the 17 patterns, only three were Granted QoS 1 by the Aliyun broker (the remaining 14 were silently Unspecified error denials):
    • /<product_key>/<device_name>/user/get — same as the integration
    • /sys/<product_key>/<device_name>/thing/event/+/post
    • /sys/<product_key>/<device_name>/thing/+/+/+
  5. Held a ~5 kg weight in the litter for ~10 s during the test so the device would actively publish state.
  6. Waited the rest of the 6 minutes, logging every received frame.
  7. Re-enabled the HA petkit entry, repeated the same activity, captured sensor.petkit_mqtt and the litter entities for 3 min.

PetKit mobile app was logged out of this account everywhere during both phases, so there was no second client competing for the Aliyun IoT identity.

Result detail

  • All 47 standalone-sniffer messages came in on the one user-level topic the integration already subscribes to. So # The official Android app only subscribes to /user/get is correct, and that single subscription is sufficient — Aliyun will deliver real-time device events to it.
  • 45 of 47 messages were api.t4.immediate.stateupdate events for the litter device (from.username=d.t4.<LITTER_DEVICE_ID>), with content.type values of 3, 5, and 10. Their timestamps line up perfectly with the periods where I was interacting with the litter box.
  • The remaining 2 were api.system.immediate.messagestatus housekeeping frames.
  • Subscriptions to anything outside our own (product_key, device_name) namespace (e.g. /<product_key>/<litter_device_id>/event/+, /+/+/event/+, #) were ACL-denied. The /user/get notify-fanout we already have really is the only delivery channel here.

So where could the integration be losing them?

I haven't tracked down the root cause yet — but a few things to look at, since the on-the-wire delivery is provably fine:

  • _on_message is never invoked in the failing case (messages_received is updated unconditionally at the top of async_handle_message, so 0 means paho's callback itself never fires).
  • The integration uses client.loop_start() (background thread) plus self.hass.add_job(self.async_handle_message, ...) to cross into the HA event loop. Worth checking whether the paho thread is actually reading from the socket after connect_async() returns; on some HA versions I've seen loop_start() race with later connect_async() reconnect attempts.
  • Aliyun-side, clean_session=False means each connection takes over the previous one with the same client id. Are there any stray reconnection paths in the integration that could be racing the listener and grabbing the broker session away from it?
  • A try/except Exception log line around the body of _on_message would tell us instantly if some Pydantic parse or _parse_iot_message is throwing inside the paho thread and the callback is being silently nuked.

Suggested next step

Given the above, I'd argue this PR is still useful — it gives users a knob to keep their setup usable today without forking the integration — but the more important fix is the underlying iot_mqtt.py listener bug. Happy to:

  • close this PR if you'd rather just track the listener bug separately, or
  • keep it open as a stop-gap until the listener is fixed (the new section is additive, defaults preserve current behaviour, and the migration is in place), or
  • pivot it into a fix for the listener if you can point me at where you'd like that to live (I can repro on demand from here on the RU regional cloud).

Whichever you prefer.

Logs

Sanitized full logs (user id, device ids, MQTT tenant subdomain, secrets all masked):
https://gist.github.com/fullstackPrincess/76d40aaff55214b6b665e860cf37abad

  • 01-sniffer-without-ha-360s.log — full 6-minute sniffer run (47 messages with topic + payload)
  • 02-ha-with-mqtt-3min-zero-messages.log — note the absence of any Received: ... lines while messages_received stayed at 0
  • 03-ha-mqtt-state-snapshot.logsensor.petkit_mqtt and litter entity dump after the 3-minute window

@fullstackPrincess

Copy link
Copy Markdown
Author

One more thought, in case the "expose three more knobs in the options form" part is what bothers you most about the PR (rather than the values themselves being configurable at all):

If the concern is keeping the options UI small and unintimidating, the same configurability could be done without surfacing anything in the form — the options would still survive HACS updates because they live in the config entry, not in the integration code:

  1. Hidden options in the entry, no UI exposure. The migration step still seeds advanced_polling_options with the historical defaults so coordinator._reload_polling_options() keeps working unchanged. Power users who actually need to change the values edit them through services.yaml action like a hypothetical petkit.set_polling_intervals (or just patch the entry once via the WebSocket config_entries/update). Everyone else never sees them.
  2. Environment variable overrides — read e.g. PETKIT_SCAN_INTERVAL_SLOW, PETKIT_SCAN_INTERVAL_DEFAULT, PETKIT_SMART_POLLING_BOOST_DURATION once at startup with int(os.getenv(...)) and fall back to the constants. They live in docker-compose.yml / the HAOS add-on env, so they survive every HACS update of this integration without any storage involvement.
  3. YAML configuration.yaml blockpetkit: schema with optional polling: keys validated through voluptuous on setup. HA still officially supports this even for cloud-polling integrations (it's how a number of other community integrations let users override hard-coded timings). Same survival-across-updates property.

Any of those would let me delete the OptionsFlow part of this PR and keep just the coordinator changes plus a thin override-reading layer. Happy to rework the diff in whichever direction you prefer — including "merge nothing, fix the listener bug, no knobs needed." Just want to flag that the persistence-across-updates concern can be solved without growing the UI if that's the sticking point.

@Jezza34000

Copy link
Copy Markdown
Owner

Hi @fullstackPrincess,

Thank you for this incredibly thorough and well-documented investigation

On my side, I ran my own tests and I'm unable to reproduce the bug: the listener does receive messages as expected in my environment. So there must be something else going on that we haven't identified yet unfortunately
I'm out of ideas at the moment...

If anything new comes to mind or if you manage to narrow it down further, I'm very interested to hear it.

For me, the main concern remains finding why this bug occurs in the first place. This PR only has value if the bug is actually present if the integration works correctly out of the box, there is simply no need to expose polling interval configuration. In my case, even the most ephemeral entities update correctly: for example, yumshare_feeding detects when the feeder starts running that only lasts a few seconds (8 seconds exactly) and was never caught before, but now it's detected every single time.

Regarding the configuration options you proposed: the option is actually already there, just hidden a user can already edit const.py directly and change PETKIT_SCAN_INTERVAL_SLOW manually with a text editor.
As for the configuration.yaml approach that's a no for me: when a config_flow is in place, configuration should not go through that file, it goes against HA design principles.

My last concern is about exposing polling intervals to end users: the risk is that users will inevitably set very low values to get more frequent updates, which leads to excessive load on Petkit's servers. There is actually a precedent of a Home Assistant integration that got blocked by a manufacturer because of too aggressive polling (I can't remember the name right now), but it happened.
Giving users that knob is a real risk of getting the entire integration banned by Petkit.

@github-actions

Copy link
Copy Markdown

This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants