Skip to content

Rework of firmware upload #112

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
hmbacher wants to merge 67 commits into from
Closed

Rework of firmware upload #112

hmbacher wants to merge 67 commits into from

Conversation

@hmbacher
Copy link
Contributor

@hmbacher hmbacher commented Dec 14, 2025

Hello! :)

The firmware upload user experience was quite basic, so I've enhanced it with better validation, progress tracking, and error handling.

Unified Firmware Update Dialog

  • Shared dialog for both firmware downloads (from GitHub) and firmware uploads (via browser)
  • Real-time progress indicator now available for file uploads
  • Server-side error display with descriptive messages
  • Success confirmation with automatic browser reload countdown
updating-progress updating_error updating-success

WebSocket Event Enhancements

  • Unified WebSocket event (EVENT_OTA_UPDATE) for both upload and download flows
  • Extended event message to include bytes_written and total_bytes for accurate progress tracking

Client-Side Validation

  • File type validation: Only .bin and .md5 files accepted
  • File size validation: Prevents uploading firmware files exceeding 2.3 MB limit
  • MD5 upload confirmation: Visual feedback when valid MD5 hash is uploaded
upload-filetype-not-supported upload-md5-invalid upload-md5-ok upload-firmware-too-large

Server-Side Validation

  • File type verification via file extension
  • MD5 hash length validation (must be exactly 32 bytes)
  • Firmware magic byte detection to verify firmware format
  • Platform compatibility check via chip type validation

Enhanced Error Handling

  • Comprehensive error handling in both backend and frontend
  • Clear error messages displayed to users

Some Refactoring

  • Using instance member variables instead of static variables in UploadFirmwareService (thread-safe)
  • Fixed memory leak around request->_tempObject
  • Fixed some typos
  • Added documentation

hmbacher added 30 commits May 19, 2025 21:11
@hmbacher
first commit
079e101
@hmbacher
Implement fans configuration feature with UI and backend support
6f4dbf3
@hmbacher
feat: Add sensor management and control functionality
55ccebd 
- Implemented a new page for managing sensors in the frontend.
- Created an EditSensor component for editing sensor details with validation.
- Developed ControllerSettingsService to manage controller settings via HTTP endpoints.
- Introduced FanController to handle fan control logic based on temperature readings.
- Added OneWireESP32 library for interfacing with DS18B20 temperature sensors.
- Implemented TempSensorsService to manage temperature sensors and their data acquisition.
- Established persistence for sensor settings and states using file storage.
- Enhanced error handling and logging throughout the services.
@hmbacher
feat: Enhance temperature sensor handling with error tracking and eve…
ac2c695 
…nt updates
@hmbacher
migrated from ESP32C3 to ESP32S3
9e024e2
@hmbacher
feat: Implement FFC MQTT Configuration and Alarm Logging
4dc040f 
- Added FFCMqttConfig component for configuring MQTT settings.
- Removed GatewayMQTTConfig component to streamline MQTT configuration.
- Updated MQTT routing to use FFCMqttConfig.
- Introduced new alarms page for managing alarm events.
- Created backend services for alarm logging and MQTT settings.
- Integrated alarm notifications via MQTT.
- Enhanced temperature sensor service to publish alarms on sensor status changes.
- Added utility functions for ISO 8601 date conversion.
- Updated UI to include alarm log and relevant error handling.
@hmbacher
feat: Integrate alarm state management and update alarm handling in UI
6f11828
@hmbacher
feat: Add RPM sensor functionality and integrate with fan controller
151e871
@hmbacher
feat: Refactor RPM sensor to include alarm service and enhance error …
ac5620a 
…handling
@hmbacher
feat: Implement sensor discovery functionality and update controller …
3f179a6 
…state management
@hmbacher
feat: Remove unused icons and configuration files; update logo in UI
0fe3bd1
@hmbacher
feat: Update navigation to settings page and enhance alarm display wi…
b6be847 
…th ID
@hmbacher
feat: Enhance alarm notifications and add fan/temperature monitoring …
c50ef9a 
…settings
@hmbacher
chore: Update TODO file for project organization and future tasks
ce4df56
@hmbacher
feat: Add configuration file and update PWM settings for fan control;…
5309bb6 
… enhance temperature sensor discovery logging
@hmbacher
- Removed obsolete (default) docs
9cf6e5a 
- Changed to PCB v2.0 settings
- Updated version string
@hmbacher
chore: Add initial TODO file with features and known bugs
0c2f433
@hmbacher
Reworked network settings
94d4439
@hmbacher
Fixed complier/IDE warnings
af42908
@hmbacher
Updated pioarduino platfom and switched to explicit versioning
5d80712
@hmbacher
Merge remote-tracking branch 'esp32sveltekit/main'
2d9440a 
# Conflicts:
#	.gitignore
#	README.md
#	factory_settings.ini
#	features.ini
#	interface/package-lock.json
#	interface/package.json
#	interface/src/app.css
#	interface/src/lib/DaisyUiHelper.ts
#	interface/src/lib/assets/logo.png
#	interface/src/lib/components/Collapsible.svelte
#	interface/src/lib/components/ConfirmDialog.svelte
#	interface/src/lib/components/InputPassword.svelte
#	interface/src/lib/components/SettingsCard.svelte
#	interface/src/lib/components/Spinner.svelte
#	interface/src/lib/components/UpdateIndicator.svelte
#	interface/src/lib/components/toasts/Toast.svelte
#	interface/src/lib/stores/analytics.ts
#	interface/src/lib/stores/socket.ts
#	interface/src/lib/types/models.ts
#	interface/src/routes/+layout.svelte
#	interface/src/routes/+layout.ts
#	interface/src/routes/+page.svelte
#	interface/src/routes/connections/mqtt/+page.svelte
#	interface/src/routes/connections/mqtt/MQTT.svelte
#	interface/src/routes/connections/ntp/NTP.svelte
#	interface/src/routes/login.svelte
#	interface/src/routes/menu.svelte
#	interface/src/routes/statusbar.svelte
#	interface/src/routes/system/metrics/BatteryMetrics.svelte
#	interface/src/routes/system/metrics/SystemMetrics.svelte
#	interface/src/routes/system/status/SystemStatus.svelte
#	interface/src/routes/system/update/GithubFirmwareManager.svelte
#	interface/src/routes/system/update/UploadFirmware.svelte
#	interface/src/routes/user/+page.svelte
#	interface/src/routes/user/EditUser.svelte
#	interface/src/routes/wifi/ap/Accesspoint.svelte
#	interface/src/routes/wifi/sta/Scan.svelte
#	interface/src/routes/wifi/sta/Wifi.svelte
#	interface/static/favicon.png
#	interface/static/manifest.json
#	interface/vite.config.ts
#	lib/PsychicHttp/src/PsychicCore.h
#	lib/PsychicHttp/src/PsychicEndpoint.cpp
#	lib/PsychicHttp/src/PsychicEndpoint.h
#	lib/PsychicHttp/src/PsychicHttp.h
#	lib/PsychicHttp/src/PsychicHttpServer.cpp
#	lib/PsychicHttp/src/PsychicStaticFileHander.cpp
#	lib/PsychicHttp/src/PsychicUploadHandler.cpp
#	lib/framework/APSettingsService.cpp
#	lib/framework/APSettingsService.h
#	lib/framework/APStatus.cpp
#	lib/framework/APStatus.h
#	lib/framework/AnalyticsService.h
#	lib/framework/ArduinoJsonJWT.cpp
#	lib/framework/ArduinoJsonJWT.h
#	lib/framework/AuthenticationService.cpp
#	lib/framework/AuthenticationService.h
#	lib/framework/BatteryService.cpp
#	lib/framework/BatteryService.h
#	lib/framework/DownloadFirmwareService.cpp
#	lib/framework/DownloadFirmwareService.h
#	lib/framework/ESP32SvelteKit.cpp
#	lib/framework/ESP32SvelteKit.h
#	lib/framework/ESPFS.h
#	lib/framework/EventEndpoint.h
#	lib/framework/EventSocket.cpp
#	lib/framework/EventSocket.h
#	lib/framework/FSPersistence.h
#	lib/framework/FactoryResetService.cpp
#	lib/framework/FactoryResetService.h
#	lib/framework/Features.h
#	lib/framework/FeaturesService.cpp
#	lib/framework/FeaturesService.h
#	lib/framework/HttpEndpoint.h
#	lib/framework/IPUtils.h
#	lib/framework/JsonUtils.h
#	lib/framework/MqttEndpoint.h
#	lib/framework/MqttSettingsService.cpp
#	lib/framework/MqttSettingsService.h
#	lib/framework/MqttStatus.cpp
#	lib/framework/MqttStatus.h
#	lib/framework/NTPSettingsService.cpp
#	lib/framework/NTPSettingsService.h
#	lib/framework/NTPStatus.cpp
#	lib/framework/NTPStatus.h
#	lib/framework/NotificationService.h
#	lib/framework/RestartService.cpp
#	lib/framework/RestartService.h
#	lib/framework/SecurityManager.h
#	lib/framework/SecuritySettingsService.cpp
#	lib/framework/SecuritySettingsService.h
#	lib/framework/SettingValue.cpp
#	lib/framework/SettingValue.h
#	lib/framework/SleepService.cpp
#	lib/framework/SleepService.h
#	lib/framework/StatefulService.cpp
#	lib/framework/StatefulService.h
#	lib/framework/SystemStatus.cpp
#	lib/framework/SystemStatus.h
#	lib/framework/UploadFirmwareService.cpp
#	lib/framework/UploadFirmwareService.h
#	lib/framework/WebSocketClient.bak
#	lib/framework/WebSocketServer.h
#	lib/framework/WiFiScanner.cpp
#	lib/framework/WiFiScanner.h
#	lib/framework/WiFiSettingsService.cpp
#	lib/framework/WiFiSettingsService.h
#	lib/framework/WiFiStatus.cpp
#	lib/framework/WiFiStatus.h
#	platformio.ini
#	scripts/build_interface.py
#	scripts/rename_fw.py
#	src/main.cpp
@hmbacher
Cleanup up async worker
db32c2f
@hmbacher
Update development server configuration and dependencies
17e0c0a
@hmbacher
Updated SSL cert creation config, updated pipeline scripts
9150064
@hmbacher
Fixed vite dev server issue
c6d826b
@hmbacher
Fixed bug if now handlers are passed
c6d2462
@hmbacher
Added coredump
aa9437c
@hmbacher
Finalized Wifi Sta settings
a5ac26d
@hmbacher
Merge branch 'main' of github.com:hmbacher/fridge-fan-control
8d21f11 
# Conflicts:
#	platformio.ini
@hmbacher
Optimized dirty flag for SettingsCard and Collapsible
973e280
hmbacher and others added 26 commits September 26, 2025 17:59
@hmbacher
Implemented draggable list with svelte-dnd-action
4d06842 
# Conflicts:
#	interface/src/routes/gateway/smoke-detectors/+page.svelte
#	platformio.ini
@hmbacher
Fixed overflow properties
99ce894 
# Conflicts:
#	interface/src/routes/gateway/smoke-detectors/+page.svelte
@hmbacher
Fixed Websockets implementation in frontend
0d3eb9f
@hmbacher
Changed drag & drop styling
1926029 
# Conflicts:
#	interface/src/lib/components/DraggableList.svelte
@hmbacher
Fixed re-ordering of Smoke detectors
8263db0 
# Conflicts:
#	interface/src/routes/gateway/smoke-detectors/+page.svelte
#	src/GatewayDevicesService.cpp
@hmbacher
Cleanup up obsolete comments
8fcd683
@hmbacher
Show HTML in Info and Confirm dialog
dceea17
@hmbacher
Removed deprecated commands in build script
0a89485
@hmbacher
Enhanced execution of pre-build scripts
985b872 
# Conflicts:
#	scripts/generate_enums.py
@hmbacher
Add print statements to indicate skipped execution for non-build task…
989c378 
…s in scripts

# Conflicts:
#	scripts/generate_enums.py
@theelims
Merge branch 'main' into pr/hmbacher/100
033ab93
@theelims
Latest Arduino
8531a3c
@hmbacher
Merge pull request #1 from theelims/main
3691c5f 
Sync fork
@hmbacher
Merge branch 'update'
5ec8e39
@hmbacher
Added an extensible health check to ESP32SvelteKit
be9e831
@hmbacher
Fixed overflow of WiFi networks list
14c2b71
@hmbacher
Merge branch 'main' of github.com:hmbacher/ESP32-sveltekit
a81b54f
@hmbacher
Hiding DHCP/Static badge for small screens
d0b9491
@hmbacher
Merge pull request #2 from theelims/main
35f0d9a 
Sync with 0.6.0
@hmbacher
Merge branch 'update'
54eb625 
# Conflicts:
#	interface/src/routes/wifi/sta/Wifi.svelte
@hmbacher
Merge remote-tracking branch 'origin-theelims/main' into update
f2645b6
@hmbacher
Merge branch 'update'
a29db33
@hmbacher
Refactor firmware update handling and telemetry integration
a331440 
- Removed GithubUpdateDialog component and replaced it with FirmwareUpdateDialog for consistency.
- Updated telemetry store to manage OTA status with additional fields: bytes_written and total_bytes.
- Modified UpdateIndicator and UploadFirmware components to utilize the new FirmwareUpdateDialog and updated telemetry structure.
- Enhanced UploadFirmwareService to emit detailed progress updates via WebSocket, including bytes written and total bytes.
- Introduced validation for firmware chip type in UploadFirmwareService to ensure compatibility.
- Improved error handling and user feedback during firmware upload processes.
- Added new FirmwareUpdateEvents header for standardized WebSocket event names and structures.
@hmbacher
Refactor firmware update handling and telemetry integration
a9b306d 
- Removed GithubUpdateDialog component and replaced it with FirmwareUpdateDialog for consistency.
- Updated telemetry store to manage OTA status with additional fields: bytes_written and total_bytes.
- Modified UpdateIndicator and UploadFirmware components to utilize the new FirmwareUpdateDialog and updated telemetry structure.
- Enhanced UploadFirmwareService to emit detailed progress updates via WebSocket, including bytes written and total bytes.
- Introduced validation for firmware chip type in UploadFirmwareService to ensure compatibility.
- Improved error handling and user feedback during firmware upload processes.
- Added new FirmwareUpdateEvents header for standardized WebSocket event names and structures.
@hmbacher
Merge: Keep file size validation in UploadFirmware.svelte
0b9ac40
@hmbacher
Remove files not to be part of PR
08ba005
@hmbacher
Copy link
Contributor Author

hmbacher commented Dec 14, 2025

Closing pull request due to missed inconsistency.

@hmbacher hmbacher closed this Dec 14, 2025
@theelims
Copy link
Owner

theelims commented Dec 14, 2025

@hmbacher I hope you'll resubmit this once you solved your inconsistency. Just one remark, the binary size is not strictly limited to 2.3 MB. Could you make this more generic and check it against the available space on the OTA partition instead? I use >4MB controllers and most of my projects exceed the 2.3 MB binary size.

@hmbacher
Copy link
Contributor Author

hmbacher commented Dec 15, 2025

@theelims, good idea. Unfortunately, the most elegant approach to leveraging PsychicHttp’s built‑in validation did not work, since internally httpd_resp_send_err() is used. This results in an ERR_CONNECTION_RESET in the browser before the HTTP 400 response can be evaluated. I also did not want to perform the file size check in the frontend anymore, because that would have required transferring the partition size from the backend to the frontend. The current solution completely bypasses file size validation in PsychicHttp and instead performs it within the UploadFirmwareService. This achieves the most consistent integration with the existing error handling, without bloating the system with new endpoints or additional frontend‑backend synchronization. The determination of the maximum firmware size now takes place at runtime in the backend using the corresponding ESP‑IDF function.

I create PR #113.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@hmbacher @theelims