Add ML Model component OTA update on keyword-detection application #106
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The `SignTfmImage.cmake` module is refactored to to accept an input binary name and an input signing layout file. These modifications are going to be used to sign the non_secure and the ML Model images separately which is essential to add the ML Model component OTA update feature. Signed-off-by: Ahmed Ismail <[email protected]>
`--fast` FVP NPU option is removed as it's not completely supported by all platforms. Signed-off-by: Ahmed Ismail <[email protected]>
Eventually, we want the MCUBoot (and the rest of TF-M) to handle the ML model in the same way as other components. To achieve that, the ML model component will be kept in flash during boot, for the MCUBoot to validate the image, and perform the swap operation if needed. Since the Ethos NPU doesn't have access to flash, the model will be copied back to DDR at runtime. This is why the model is still kept in the DDR memory region in the linker script. Patches for the trusted_firmware-m component: - Add support for the third image for cs-300 platform. - Configure the signing layout for the ML model, and reuse the NS key. - Add a modified flash map for the cs-300, for the MCUBoot to handle the third image. This is based on the default flash layout used before. Signed-off-by: Filip Jagodzinski <[email protected]> Signed-off-by: Ahmed Ismail <[email protected]>
Eventually, we want the MCUBoot (and the rest of TF-M) to handle the ML model in the same way as other components. To achieve that, the ML model component will be kept in flash during boot, for the MCUBoot to validate the image, and perform the swap operation if needed. Since the Ethos NPU doesn't have access to flash, the model will be copied back to DDR at runtime. This is why the model is still kept in the DDR memory region in the linker script. Patches for the trusted_firmware-m component: - Add support for the third image for cs-310 platform. - Configure the signing layout for the ML model, and reuse the NS key. - Add a modified flash map for the cs-310, for the MCUBoot to handle the third image. This is based on the default flash layout used before. Signed-off-by: Filip Jagodzinski <[email protected]> Signed-off-by: default avatarAhmed Ismail <[email protected]>
Eventually, we want the MCUBoot (and the rest of TF-M) to handle the ML model in the same way as other components. To achieve that, the ML model component will be kept in flash during boot, for the MCUBoot to validate the image, and perform the swap operation if needed. Since the Ethos NPU doesn't have access to flash, the model will be copied back to DDR at runtime. This is why the model is still kept in the DDR memory region in the linker script. Patches for the trusted_firmware-m component: - Add support for the third image for mps4 platforms. - Configure the signing layout for the ML model, and reuse the NS key. - Add a modified flash map for the mps4 platforms, for the MCUBoot to handle the third image. This is based on the default flash layout used before. These changes applies to both: * Corstone-315 Platform. * Corstone-320 Platform. Signed-off-by: Filip Jagodzinski <[email protected]> Signed-off-by: Ahmed Ismail <[email protected]>
keyword-detection example: - Configure the MCUBoot to work with 3 components instead of 2. - Configure the MCUBoot to use a non-default flash map. - Set the ML model image version. - Sign the ML model image using the same key as the NS image. - Generate an update signature for the ML model image. - At the ML task init, add a new step, where the ML model is copied from flash to DDR. - Update the merge images CMake function in TF-M integration layer to handle the third component (the ML model image). - Update the extract_sections_from_axf CMake function. Patches for the freertos_ota_pal_psa component: - Add the ML model file path. - Stop using a global variable to store the NS image version. - Fix the GetImageVersionPSA to return the version to an output param for any given component (rather than update the global var). Patches for the ml_embedded_evaluation_kit component: - Override EthosU55 NPU default IRQ handler to avoid modifying the vector table in run-time which alter the non-secure image result in MCUBoot validation failure. Additions for the OTA Orchestrator: - Stop using a global variable to store the NS image version. - Replace all the uses of the appFirmwareVersion global var with the appropriate call to the new image version getter. Signed-off-by: Filip Jagodzinski <[email protected]> Signed-off-by: Ahmed Ismail <[email protected]>
Add a modified tflite file that produces no inference results at runtime. This file can be used to build an application that is fully functional, but the ML inference is unsuccessful in detecting any keyword. Applying an OTA ML model update in this state is very visible since the updated model does work as expected (detects keywords from audio samples). Signed-off-by: Filip Jagodzinski <[email protected]> Signed-off-by: Ahmed Ismail <[email protected]>
Since we don't run OTA tests as part of the intgeration tests anymore, we don't need to sign the update binary and there is no need to upload the update binary to AWS. Signed-off-by: Ahmed Ismail <[email protected]>
Add a new nightly test to verify the ML model OTA update, this is done for Keyword-Detection application compiled with GNU toolchain only as this is the currently supported combination. This nightly test runs on all the Corstone platforms. Signed-off-by: Ahmed Ismail <[email protected]>
ig15
approved these changes
Jan 29, 2025
urutva
approved these changes
Jan 29, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR adds the support to perform ML Model component OTA update on Keyword-Detection application on all the FRI's supported platforms (CS-300, CS-310, CS-315, and CS-320).
For more information on how to use the new feature, please follow the steps mentioned in ML Model Update document.
Test Steps
Checklist:
Related Issue
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.