Skip to content

Use smaller files on the SD card #1650 #2809

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 7 commits into from
Closed

Use smaller files on the SD card #1650 #2809

wants to merge 7 commits into from

Conversation

@harshithsunku
Copy link
Contributor

@harshithsunku harshithsunku commented Aug 16, 2025 •
edited
Loading

Audio Chunking System Implementation (Under Development)

Overview

This PR implements a robust audio chunking system for the OMI firmware that automatically rotates audio recordings into time-based chunks for improved file management and storage efficiency.

Key Features

Automatic 5-Minute Audio Chunks

  • Audio recordings are automatically segmented into 5-minute chunks
  • Seamless rotation between chunks without audio data loss
  • Configurable chunk duration (currently set to 5 minutes via CHUNK_DURATION_CYCLES)

File Naming

  • Unique timestamp-based filenames: audio/chunk_HHMMSS_NNNNN.bin
  • Persistent counter system that survives device reboots
  • Counter stored in chunk_counter.txt for filename uniqueness across sessions

Thread-Safe Implementation

  • Uses Zephyr atomic operations for thread-safe chunk rotation
  • Prevents race conditions between audio recording and chunk management

Existing System

  • Global chunking_enabled flag allows toggling between new chunking system and legacy storage
  • Existing audio recording functionality remains unchanged when chunking is disabled

Technical Implementation

Main Loop Integration

  • Added check_chunk_rotation_timing() call to main loop every 500ms
  • set_system_boot_complete() called after initialization to prevent chunking during boot

Storage System Updates

  • Enhanced write_to_storage() in transport.c to check for chunk rotation
  • Automatic buffer flushing before chunk rotation to prevent data loss

Files Modified

  • src/sdcard.c - Core chunking implementation
  • src/sdcard.h - API definitions
  • src/main.c - Main loop integration and boot completion
  • src/transport.c - Storage system integration

@harshithsunku
Initial commit for audio chunk
fded56d
@harshithsunku
Add persistent chunk counter for unique audio filenames
772c36c 
- Introduced a chunk counter to generate unique filenames across reboots.
- Implemented functions to save and load the chunk counter from persistent storage.
- Updated filename format to include the chunk counter for better file management.
@beastoin
Copy link
Collaborator

love to see this. keep going man.

@harshithsunku
Add persistent chunk counter for unique audio filenames and optimizat…
5e77caf 
…ions by reducing system calls.

- Introduced a chunk counter to generate unique filenames across reboots.
- Implemented functions to save and load the chunk counter from persistent storage.
- Updated filename format to include the chunk counter for better file management.
@harshithsunku
Add SD card configuration header and refactor sdcard.c for improved m…
b0d9233 
…aintainability

- Introduced sdcard_config.h to centralize SD card related constants and paths.
- Updated sdcard.c to utilize defined constants for mount points and file paths, enhancing code readability and configurability.
- Refactored chunking logic to improve error handling and maintainability, including updates to filename generation and chunk counter management.
@harshithsunku
Add storage file management features and sync enhancements
13e2a7b 
- Updated AndroidManifest.xml to include permissions for external storage access.
- Enhanced asset management by adding new image assets and updating the asset list.
- Implemented a new DeviceAudioFilesWidget to display audio files on the device, including connection status and file management options.
- Added functionality in CaptureProvider to manage downloaded files, including listing, deleting, and downloading files from the device.
- Improved error handling and user notifications for file operations.
- Refactored device connection logic to support fetching storage file names from connected devices.
@harshithsunku
Implement automatic file processing in CaptureProvider
075011a 
- Added a new method to automatically process downloaded files after successful download.
- Introduced checks for backend connectivity and user authentication before processing.
- Implemented legacy-compatible filename generation for processed files.
- Enhanced error handling with specific notifications for various failure scenarios.
- Updated the CaptureProvider to include new imports and methods for improved functionality.
@harshithsunku
Copy link
Contributor Author

love to see this. keep going man.

Hi @beastoin ,

Thanks! The draft version of the firmware is ready and tested. I’m now exploring the Android app part.

@harshithsunku
Add audio chunking feature and update SD card management
bda93b1 
- Introduced CONFIG_OMI_ENABLE_AUDIO_CHUNKING to enable a time-based audio chunking system for better file management.
- Updated sdcard.c to control chunking behavior based on the new configuration option.
- Enhanced sdcard_config.h documentation to reflect the new audio chunking feature.
- Modified project configuration file to enable audio chunking by default.
@beastoin
Copy link
Collaborator

beastoin commented Sep 8, 2025 •
edited
Loading

wait, sry, did you test the firmware ? harshithsunku

---

prev:

sry man. closing this for now // the update speed is too low haha.

keep working on your branch, then feel free to create a new PR when you're ready.

thank you // @harshithsunku

@beastoin beastoin closed this Sep 8, 2025
@beastoin beastoin reopened this Sep 8, 2025
@harshithsunku
Copy link
Contributor Author

wait, sry, did you test the firmware ? harshithsunku

prev:

sry man. closing this for now // the update speed is too low haha.

keep working on your branch, then feel free to create a new PR when you're ready.

thank you // @harshithsunku

Hi @beastoin ,

Yes, I’ve tested the firmware and it’s working fine from the initial commit. I still need to check things on the Flutter side, which is already in progress. I was tied up with other work, so I couldn’t spend much time on this, but I’ll try to get back to it within the next two weeks.

FYI:
syncing the files to Flutter is a bit tricky. I initially had it working, but that approach consumed too many resources. To optimize, I’m updating the firmware to remove timestamps from the files and instead use only sequence numbers. This should reduce I/O and fs operations on the SD card and help with battery usage.

Since I mainly work on the firmware side and I’m still quite new to the Android/Flutter part, I wasn’t fully aware of how the app sync was implemented at first. Once I started digging into that part, I realized the current design would use a lot of resources, which caused the delay. I’m now working on these optimizations so the sync will be more efficient with minimal resource usage.

@beastoin
Copy link
Collaborator

beastoin commented Sep 16, 2025 •
edited
Loading

just checked your code quickly (firmware only). qq — what happens to the audio files after a device reset? i’m worried the new file will override the old one haha.

anw, to move faster you could split this PR into 2 parts: firmware + app. i’d like to help 1 by 1. let’s finish the firmware first.

tip: set up pre-commit to resolve the formatting issue → https://docs.omi.me/doc/developer/AppSetup#9-code-formatting%E2%80%8B

@harshithsunku

@harshithsunku
Copy link
Contributor Author

just checked your code quickly (firmware only). qq — what happens to the audio files after a device reset? i’m worried the new file will override the old one haha.

anw, to move faster you could split this PR into 2 parts: firmware + app. i’d like to help 1 by 1. let’s finish the firmware first.

tip: set up pre-commit to resolve the formatting issue → https://docs.omi.me/doc/developer/AppSetup#9-code-formatting%E2%80%8B

@harshithsunku

Hi @beastoin,

Thanks for reviewing the firmware code

The reload scenario is already handled to avoid overwrites.

How it works

  • Each audio file name includes both a timestamp and a persistent counter → chunk_HHMMSS_NNNNN.b
  • The counter is stored on the SD card (/chunk_counter.txt) and incremented with every new file
  • On reset, the firmware loads the last counter value so numbering continues safely
  • First boot is handled gracefully: if the counter file doesn’t exist, it starts at 0

Implementation

  • Counter persistence: save_chunk_counter() / get_chunk_counter()
  • Filename generation: generate_timestamp_audio_filename()
  • Init: counter is loaded during mount_sd_card()

So, no worries files won’t ever be overwritten, even across multiple reloads.

I will split the PR and let you know in a day.

@harshithsunku
Copy link
Contributor Author

There is other PR for this : #3019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@harshithsunku @beastoin