aws-for-fluent-bit image migration to AL2023

[mmkeverestek]

Hi Team,

Currently, this image is on Amazon Linux 2. The Amazon Linux is set for EOL in June 2025.

By when this image will be migrated to Amazon Linux 2023.

Regards,
Mushkoor

[EAlf91]

Also relevant for discussion #830 as the current fluent-bit version reached EOL in 2022

[cwoebker]

Would be great to get an estimate on this. I assume, that as of now, this will not work on any node with an AL2023 basis?

[jenademoodley]

Tested on my end using the latest 2.32.2.20240724 version with Tail and Systemd plugins, and can confirm that the Tail plugin work on AL2023 instances, however the Systemd plugin does not work. Systemd is especially necessary in AL2023 instances due to the absence of rsyslog on default installations:
https://repost.aws/knowledge-center/ec2-linux-al2023-find-log-files

Researching into it reveals it may be due to the compression of the journal files in systemd >=246 as detailed here fluent/fluent-bit#2998

The official fluent-bit image has this change as of version 1.8.11 fluent/fluent-bit#4567 but aws-for-fluent-bit is supposed to be using version 1.9.10 as of version 2.32.1 as seen in the release notes
https://github.com/aws/aws-for-fluent-bit/releases/tag/v2.32.1

As such the changes may not have been added to the aws-for-fluent-bit. Hoping that the image is updated with this change so that aws-for fluent-bit can be used on AL2023

[mmkeverestek]

Hi Team,

Any ETA for the resolution of this issue? Now ~8 months are left for EOL of AmazonLinux 2.

The consumers of aws-for-fluent-bit should get some time for migration to new version with AL2023.

Fluent bit officially supports AL2023: https://docs.fluentbit.io/manual/installation/linux/amazon-linux

Regards,
Mushkoor

[sparrc]

AL2 EOL is currently scheduled for June 30, 2026. We don't currently have any plans to migrate to AL2023 this year. We may migrate off of OS-based containers altogether and make aws-for-fluent-bit based on a scratch or distroless container, as the official fluent/fluent-bit containers are: https://docs.fluentbit.io/manual/installation/docker#multi-architecture-images

[waderobson]

Trying to get logging working in our shinny new AL2023 EKS cluster and ended up here. Besides the fact that there is no paint by numbers docs is there any reason one should not just use the fluent-bit provided images?

I'm guessing the main benefit to using aws-for-fluent-bit is AWS will support it?

[jenademoodley]

Created a pull request for this #913

Tested and can confirm systemd logs are now successfully obtained from systemd on AL2023 with the above changes