QEMU X86-64 running Aktualizr not able to receive Meta Data Updates from Repo Server running on OTA CE

[DukesOnSkyline]

I have been trying to get the OTA Community Edition (CE) to work with Aktualizr running on QEMU. The GITHub OTA CE Micro Services (Repo Server, Tree-Hub, Director etc) are running on Kubernetes (1.14 Version) without any issues (No modifications). (Reference: https://github.com/advancedtelematic/ota-community-edition)

After copying the Credentials.zip from OTA CE Build into QEMU X86-64 Aktualizr Build, see that the PUT request on REPO Server logs (HTTPS Status Code: 204) after a successful “bitbake core-image-minimal” Build. (Reference: https://docs.ota.here.com/ota-client/latest/build-qemu.html)

Expectation:
Aktualizr-QEMU Build (Bitbake core-image-minimal) uploads the first version of Meta-Data (Without VIM Binary) on to OTA CE’s Repo Server.
Add changes to build/conf/local.conf: IMAGE_INSTALL_APPEND = “ vim “ and rebuild the QEMU Aktualizr image and boot it up on QEMU. As part of the Bitbake build, it should upload only the changes (“ vim ” Binary) in meta data to the Repo Server of OTA CE.
Run Aktualizr to pull in the change in Meta-Data from the Repo Server on to the QEMU Virtual Machine. After syncing the meta data from repo server, should be able to see the “ vim “ binary on QEMU VM running Aktualizr.

Issue:
Not able to pull in VIM binary from the Repo Server on to the QEMU VM running Aktulizr. Keep seeing the message “ no meta data updates found”.
I have attached the Aktualizr Journalctl logs (Running on QEMU VM), config files of QEMU X86-64 Build (build/conf/local.conf) and config/images.yaml (OTA CE pulling in Docker Images of Repo Server, Director, Tree Hub etc.) for reference. Please check out the logs for more information.

Please let me know If you need more information. Appreciate all the help!

Thanks
Desh

aktualizr-journalctl.log
images.yaml.docx
localconf.docx

[DukesOnSkyline]

Adding additional Aktualizr logs with loglevel=0 and loglevel=1.
aktualizr-journalctl-v2.log

[pattivacek]

AFAIK this project isn't actively maintained anymore. You may note there haven't been any human-made changes for well over a year. That said, it should still work.

From the sound of things, you are booting a new qemu instance using your second image, hence there is nothing to update to. Is vim already available when you boot it? It sounds like what you want is to boot the first image and update to the second. Alternatively, if my suspicion is correct, you could issue an update back to the first image, and after it updates, verify that vim is no longer there.

Or perhaps the problem is that you may not have actually issued the update... I see no reference to that step. You have to do that via the API or web UI, unless it's already been configured to always update devices to the latest version.

[DukesOnSkyline]

Thanks, Patti, for looking into it, appreciate your time. Yes, I understand that this Project is not actively maintained but I would like to get it to work to understand OTA Software updates to an embedded device.

Infrastructure: Building the QEMU Image and running OTA CE(Community Edition) Server on the same Linux Server (Ubuntu 20.04 LTS)

Step 1:
To answer your question, the very first QEMU Image that I built using “bitbake core-image-minimal” doesn’t include VIM (Built it without VIM). I see a successful bitbake build with “do_ostree_push” messages on the Build machine. Also, see the HTTP Status Code 204 in response to the PUT message on Repo Server (OTA CE Micro Service) running on Kubernetes.
After this step, I boot up the QEMU image and run Aktualizr client agent on QEMU using the following command: “aktualizr –loglevel=0”. I do not see any meta-data updates (its understandable) since it’s the same QEMU Image with the same set of Binaries and files.

Step 2:
Next Step. I rebuild the QEMU Image with changes to build/conf/local.conf by adding VIM binary using IMAGE_INSTALL_APPEND directive. Again, the upload to the REPO Server is successful. Assuming status code 204 on REPO Server indicates the upload to the Server is successful.

Step 3:
As part of the Final Step, go back to running QEMU image(no reboots) and try to run aktualizr agent again in order to pull the meta-data updates (VIM Binary) from the Server (OTA CE). Again, using the same command: “aktualizr –loglevel=0” to pull in the VIM binary from the Repo server.

I was thinking running the Aktualizr agent that's built as part of meta-updater yocto layer (QEMU image) is used to pull the OTA Updates from the Repo server. As per the Meta-updater Yocto Build configuration, If aktualizr agent is running on QEMU, it is supposed to poll the Server for updates evert (SOTA_POLLING_SEC=”10”) 10 seconds(build/conf/local.conf). Since you mention that I have to issue an OTA Update explicitly through an API/WEB UI , I wonder If I am missing something. Could you please give me some pointers on using these API/Web UI interfaces to issue an OTA update? Also, Do you mean issuing the OTA Update from the OTA CE Server Side?

For your reference: When I run “aktualizr” to pull the OTA update, aktualizr agent is using the default configuration that was built as part of meta-updater layer. Do you see any issues with it? Also, I did not make changes to the following default Configuration.

root@qemux86-64:/usr/lib/sota/conf.d# aktualizr
Aktualizr version 2020.10-26-g11784490c starting
Reading config: "/usr/lib/sota/conf.d/05-log-debug.toml"
Reading config: "/usr/lib/sota/conf.d/20-sota-device-cred.toml"
Reading config: "/usr/lib/sota/conf.d/40-hardware-id.toml"
Final configuration that will be used:
[logger]
loglevel = 0

[p11]
module = ""
pass = ""
uptane_key_id = ""
tls_ca_id = ""
tls_pkey_id = ""
tls_clientcert_id = ""

[tls]
server = "https://ota.ce:30443"
server_url_path = "/var/sota/import/gateway.url"
ca_source = "file"
pkey_source = "file"
cert_source = "file"

[provision]
server = "https://ota.ce:30443"
p12_password = ""
expiry_days = "36000"
provision_path = ""
device_id = ""
primary_ecu_serial = ""
primary_ecu_hardware_id = "qemux86-64"
ecu_registration_endpoint = "https://ota.ce:30443/director/ecus"
mode = "DeviceCred"

[uptane]
polling_sec = 10
director_server = "https://ota.ce:30443/director"
repo_server = "https://ota.ce:30443/repo"
key_source = "file"
key_type = "RSA2048"
force_install_completion = false
secondary_config_file = ""
secondary_preinstall_wait_sec = 600

[pacman]
type = "ostree"
os = ""
sysroot = ""
ostree_server = "https://ota.ce:30443/treehub"
images_path = "/var/sota/images"
packages_file = "/usr/package.manifest"
fake_need_reboot = false

[storage]
type = "sqlite"
path = "/var/sota"
sqldb_path = "sql.db"
uptane_metadata_path = "metadata"
uptane_private_key_path = "ecukey.der"
uptane_public_key_path = "ecukey.pub"
tls_cacert_path = "root.crt"
tls_pkey_path = "pkey.pem"
tls_clientcert_path = "client.pem"

[import]
base_path = "/var/sota/import"
uptane_private_key_path = ""
uptane_public_key_path = ""
tls_cacert_path = "root.crt"
tls_pkey_path = "pkey.pem"
tls_clientcert_path = "client.pem"

[telemetry]
report_network = true
report_config = true

[bootloader]
rollback_mode = "none"
reboot_sentinel_dir = "/var/run/aktualizr-session"
reboot_sentinel_name = "need_reboot"
reboot_command = "/sbin/reboot"

[network]
curl_proxy = ""
curl_bandwidth = 0
use_oscp = false

Current directory: /usr/lib/sota/conf.d
Use existing SQL storage: "/var/sota/sql.db"

[pattivacek]

Could you please give me some pointers on using these API/Web UI interfaces to issue an OTA update? Also, Do you mean issuing the OTA Update from the OTA CE Server Side?

Yes, you are missing something. See for example https://docs.ota.here.com/ota-web/dev/manage-devices.html and https://api.ota.here.com/docs. I know very little about running the backend, though, so if that doesn't make sense, you'll have better luck contacting Here or another company that uses this software (to my knowledge, Foundries and Toradex).

When I run “aktualizr” to pull the OTA update, aktualizr agent is using the default configuration that was built as part of meta-updater layer. Do you see any issues with it?

I really can't say. They are defaults for a reason, but whether they are what you need is another matter.

[DukesOnSkyline]

I did reach out to HERE technologies on using OTA Connect rather than using OTA Community edition but it looks like OTA Connect support is only available for contracting customers. About Foundries.IO and Toradex, they have developed some sort of wrappers on top of OTA Community edition which is available for paid customers again.

It looks like OTA Community Edition (CE) does not support the same level of features as OTA Connect. When I bring up OTA CE through Kubernetes, all the Services come up without issues but there is no utility available to mange devices/software version or even to deploy software packages to devices. I wonder If anyone has ever gotten OTA CE to work with meta-updater yocto layer to push/pull OTA updates(Using Aktaulizr Agent).

Again, appreciate all the pointers and help. Thanks.