HA high CPU usage/spikes since 2024.11.x update

[mihaicristianpetrescu]

Describe the issue you are experiencing

Home Assistant Yellow with RPI CM4 8GB RAM - so maxed out.

Since updating the supervisor to 2024.11.x and then the core to 2024.11.x, Home Assistant CPU usage started going up, it spikes a lot and the interface loads slow - in comparison before these update the CPU was pretty much sitting flush between 10-20-ish%, now it is constantly above 20ish% with spikes to 40%.

No other updates have been performed or other new add-ons, HACS installs, even new configuration or devices added.
Just the supervisor and core updates.

I've downgraded the core to 2024.10.4 and it somehow helps getting the spikes down a bit, but the system is still running slow. Unfortunately supervisor cannot be downgraded otherwise I would have tried that as well.

Not sure what got broken but there's definitely something seriously broken, either in the supervisor update or the core update.
There is a difference in CPU spikes between core 2024.10.x and 2024.11.x, but there still are spikes in comparison to the before core 2024.10.x and supervisor 2024.10.x.

I've installed Glances add-on and saw python-3 being the one process that's spiking and in return increasing the homeassistant docker CPU usage.

I've tried every single core 2024.11.x update so far and not 1 of them fixes the problem, have been constantly going back to 2024.10.4 to have a somewhat semi-responsive system.

Seems like this might be more of a supervisor problem than a core, because there is still high CPU even on older core (Ex 2024.10.4), but with the latest available supervisor (since it cannot be downgraded to test out) 2024.11.2.

What type of installation are you running?

Home Assistant OS

Which operating system are you running on?

Home Assistant Operating System

Steps to reproduce the issue

Simply start HAOS and notice how the CPU start spiking up constantly going to 30% or more.
Spikes are a whole lot more worse on newer core 2024.11.x than on 2024.10.x but they are still there.
Before updating the supervisor things were running well - as far as I read there's no way to downgrade the supervisor otherwise I would have tried that as well.
Core was updated after supervisor.

Anything in the Supervisor logs that might be useful for us?

Doesn't seems so.

System Health information

System Information

version	core-2024.10.4
installation_type	Home Assistant OS
dev	false
hassio	true
docker	true
user	root
virtualenv	false
python_version	3.12.4
os_name	Linux
os_version	6.6.31-haos-raspi
arch	aarch64
timezone	Europe/Bucharest
config_dir	/config

Home Assistant Community Store

GitHub API	ok
GitHub Content	ok
GitHub Web	ok
HACS Data	ok
GitHub API Calls Remaining	5000
Installed Version	2.0.1
Stage	running
Available Repositories	1539
Downloaded Repositories	27

AccuWeather

can_reach_server	ok
remaining_requests	44

Home Assistant Cloud

logged_in	false
can_reach_cert_server	ok
can_reach_cloud_auth	ok
can_reach_cloud	ok

Home Assistant Supervisor

host_os	Home Assistant OS 13.2
update_channel	stable
supervisor_version	supervisor-2024.11.2
agent_version	1.6.0
docker_version	27.2.0
disk_total	916.9 GB
disk_used	26.9 GB
healthy	true
supported	true
host_connectivity	true
supervisor_connectivity	true
ntp_synchronized	true
virtualization
board	yellow
supervisor_api	ok
version_api	ok
installed_addons	Home Assistant Google Drive Backup (0.112.1), File editor (5.8.0), Samba share (12.3.2), Mosquitto broker (6.4.1), Terminal & SSH (9.15.0), Paradox Alarm Interface (3.6.0), Cloudflared (5.2.1), Studio Code Server (5.17.3), Zigbee2MQTT (1.41.0-1), SQLite Web (4.2.2), Node-RED (18.1.1), Zigbee2MQTT (1.41.0-1), Zigbee2MQTT (1.41.0-1), TheengsGateway (1.11.4), ZigStar TI CC2652P7 FW Flasher (0.2.0), Glances (0.21.1)

Dashboards

dashboards	9
resources	16
views	22
mode	storage

Recorder

oldest_recorder_run	October 18, 2024 at 10:51 PM
current_recorder_run	November 16, 2024 at 10:30 PM
estimated_db_size	11163.42 MiB
database_engine	sqlite
database_version	3.45.3

Sonoff

version	3.8.1 (ffa7e22)
cloud_online	8 / 9
local_online	8 / 8

Xiaomi Miot Auto

component_version	0.7.22
can_reach_server	ok
can_reach_spec	ok
logged_accounts	1
total_devices	3

Supervisor diagnostics

No response

Additional information

No response

[HaraldGithub]

Same problem here. 😩

[fribse]

Same here, geeez, and it shows a 2024.11.4 in the release page, but there is nothing newer when I ask for supervisor update in the CLI?

[mihaicristianpetrescu]

@fribse If you go to System -> Updates and join the beta channel you'll be able to update supervisor to 2024.11.4, it's only available in the beta channel (I did do that). No improvements though.

[fribse]

AH, gotcha, but this is getting worse and worse. It's completely unusable now, everything is running in melasses. I did contemplate making a new VM, and restore in that, but as far as I can see in the window, it starts with updating the supervisor to 2024.11.2, even before I get the backup restored?

[fribse]

I have quite a few addons installed, I've disabled automatic start for them, rebooted and have started the essentials manually now.

• AdGuard Home
• Advanced SSH & Web Terminal
• AppDaemon (Started)
• Chrony (Started)
• ESPHome
• Govee to MQTT Bridge
• Grafana
• Home Assistant Google Drive Backup
• InfluxDB
• Let's Encrypt
• MariaDB (Started)
• Matter Server (Started)
• Mosquitto Broker (Started)
• Node-Red (Started)
• OpenThread Border Router (Started)
• Samba share
• Studio Code Server
• Zigbee2MQTT (Started)

After doing that last night the CPU load was very high (around 80%, not 100+%), around 5 am this morning, it normalized to 20%.

[fribse]

My problem is gone. I found out that it stemmed from something completely different, just a question of coincidence.
A failed psu for the fans blowing air into the rack, so the temp went up, so then the internal fan in the NUC failed, and the temp in the NUC went even higher, and finally the fan failed, and it started thermal throttling.
After temporarily opening the rack, replacing the fan in the NUC, everything is back to normal, a CPU load of 15% with everything running.

[mihaicristianpetrescu]

@fribse You are one of the lucky ones...my system has passive heat control via included large radiator with the HA Yellow so...nothing to fix there. Just the software that's not working right.

[dominikandreas]

I had a similar issue and looks like it was backup related. For one, supervisor comes nearly unresponsive when it's creating or restoring a backup. And another issue for me was that I had a slow NFS storage defined for the backup location, which caused significant constant performance degradation to the supervisor and therefore also home assistant whenever he supervisor API was used (e.g. accessing the add-ons page). After removing the NFS storage, everything runs smoothly.

Not sure if NFS was actually the issue or old backup files that were in there (or combination of both)