TC358743 produces BGR instead of RGB

[mdevaev]

Describe the bug

I'm trying to use RGB24 format with TC358743 instead of UYVY, and noticed that it produces BGR sequence in the raw data instead of RGB. I used the blue fullscreen image as an HDMI source and made a stream dump that confirms this, getting something like [255, 0, 0] for first 3 bytes.

If you have a top secret datasheet under Toshiba NDA, you can check pages 36/261 and 37/261. This confirms that UYVY is implemented correctly, but RGB has the reverse sequence. Apparently, this bug went unnoticed because everyone was using UYVY. I need RGB to output the image in DRM and encode in M2M at the same time.

Steps to reproduce the behaviour

Use any capture software with TC358743 in RGB24 format and dump the raw image.

Device (s)

Raspberry Pi CM4, Raspberry Pi CM4 Lite

System

• Arch Linux ARM, but it doesn't matter
• version c14213886ae1e31951022b74ce776ce7ee7023ee (clean) (release) (start)
• Linux pikvm 6.6.21-4-rpi #1 SMP Wed Mar 13 20:36:42 UTC 2024 armv7l GNU/Linux
• Apparently, the problem is relevant for any version of kernel.

Logs

No response

Additional context

I found this bug when encoding JPEG in M2M and using libjpeg. It's the same with M2M H.264. Interestingly, because of the endianness, DRM_FORMAT_RGB888 is actually BGR, so the rendering of the TC* chip output in DRM was correct colors, while the encoding was not.

I don't know if it is possible to fix this with little blood without loss of performance, given that it looks like you need to use the reverse order to put bytes in memory after receiving from CSI. This is not a big problem and I can make a userspace workaround for this, but it would probably be better to fix it upstream if the fix is not complex or fatal to performance.

[6by9]

Life with component ordering is always complicated.

tc358743 is an upstream driver with relatively minimal changes. It was written by Cisco for some of their video conferencing kit. IIRC it was against an ST SoC, but it may have been TI.

They've defined that the media bus code is MEDIA_BUS_FMT_RGB888_1X24

Checking the definition for MEDIA_BUS_FMT_RGB888_1X24, it is putting B in the LSBs of the bus. CSI-2 does send the data LSB and blue first, so would make sense as a mapping,

As a cause for further confusion, the CSI-2 spec (I'm looking at v1.01.00 r0.04 2-Apr-2009) does include 12.2 RGB888 Data Reception

The RGB888 data format byte to 32-bit memory word mapping follows the generic CSI-2 rule.

Figure 110 that follows writes B0 into the LSB of a 32bit word.

Section 12.7 YUV422 8-bit Data Reception and figure 115 that follows says that format does NOT follow the generic CSI-2 rule, and whilst U1 is sent first on the wire, it is written into the MSB of a 32bit word.

However the whole of section 12 is marked as informative.

I can't see anything in the Unicam docs that imply it does implement the generic CSI-2 rule. I don't see why it would ever be implemented on a big endian system such as ARM, as it means that the byte order is totally messed if wanting to access individual components as bytes (memory order would be B2 R1 G1 B1 G3 B3 R2 G2!)

Looking at how MEDIA_BUS_FMT_RGB888_1X24 should map to a V4L2_PIX_FMT_xx define, comparing against V4L2_PIX_FMT_SRGGB16 and MEDIA_BUS_FMT_SRGGB16_1X16, I can see an argument that MEDIA_BUS_FMT_RGB888_1X24 should be V4L2_PIX_FMT_BGR24.
TC358743 is the only vaguely supported device on the Pi using the format (we don't support OV5640, ADV7511, ADV7605, or ADV7842), so swapping them would be of limited risk, but needs a chunk more studying. I don't see an equivalent in any other driver which would have given me more confidence in the change.

[6by9]

And it wasn't unnoticed - there are various forum threads where GStreamer's capssetter is being used to swap bgr to rgb when using tc358743.

[mdevaev]

Thanks for researching this. It looks like total hell. I believe CISCO can use byte swapping in its userspace (or a format overriding, whatever), and it looks like this is the path I should follow right now to avoid headache.

[MarkusLange]

Silly question I try to pipe a signal (HDMI. RGB) through the pi, TC358743 input and PI HDMI output the tricks with capssetter and bgr and rgb swap works as long I didn't scale, for example 1080p or 1080i to 1080p or 1080i works flawless but if I scale from 720 or to 720 and use videoconvert and videoscale the colorshift with capssetter did not work anymore can someone help me with that?

[6by9]

FWIW there are a couple of threads upstream, but nothing resolved yet.

https://lore.kernel.org/linux-media/20250606-rpi-unicam-rgb-bgr-fix-v1-1-9930b963f3eb@kernel.org/ - expected that MEDIA_BUS_FMT_BGR888_1X24 is used.

https://lore.kernel.org/linux-media/20250612-csi-bgr-rgb-v1-3-dc8a309118f8@kernel.org/ - changing to use that MBUS code, however it will break any existing userspace app that currently select MEDIA_BUS_FMT_RGB888_1X24 in their Media Controller setup (eg TC358743 on Pi5 I expect, so likely PiKVM). Userspace regressions are a big no-no.

Seeing as there are several drivers that use RGB888_1X24, it would be easier to fix it in the Unicam driver (just not in the way that the first patch proposed). We can be a little easier going there as the Unicam driver has only just been merged to mainline, so in their view there are no/very few users of it.
If mainline did adopt that, then for downstream I'd be tempted to put in the same mapping twice, so both BGR888_1X24 and RGB888_1X24 map to V4L2_PIX_FMT_BGR24. The handling is generally dodgy, and the CSI2 spec specifies the layout that is V4L2_PIX_FMT_BGR24. If your random source device produces RGB24 then it's non-CSI2 compliant and shouldn't be using that CSI2 data ID.

[6by9]

Silly question I try to pipe a signal (HDMI. RGB) through the pi, TC358743 input and PI HDMI output the tricks with capssetter and bgr and rgb swap works as long I didn't scale, for example 1080p or 1080i to 1080p or 1080i works flawless but if I scale from 720 or to 720 and use videoconvert and videoscale the colorshift with capssetter did not work anymore can someone help me with that?

What's your full pipeline that you're trying to run? And which version of GStreamer?

Actually I see you've posted on the forums too - https://forums.raspberrypi.com/viewtopic.php?t=389417

Please don't do that, or at least provide a link between the two. Otherwise you just waste the time of those who might help - I wouldn't have needed to ask the above question as you've given the details in that thread.

[MarkusLange]

Sorry for that. On the forum I hat to wait as it is moderated so I can't get the link to it instant and I wasn't near any device when it comes online so I hadn't adopt it in this issue in proper time.

[MarkusLange]

If mainline did adopt that, then for downstream I'd be tempted to put in the same mapping twice, so both BGR888_1X24 and RGB888_1X24 map to V4L2_PIX_FMT_BGR24. The handling is generally dodgy, and the CSI2 spec specifies the layout that is V4L2_PIX_FMT_BGR24. If your random source device produces RGB24 then it's non-CSI2 compliant and shouldn't be using that CSI2 data ID.

Since I have no clue, so asking cautious.

from the link to the first patch, BGR and RGB both use MIPI_CSI2_DT_RGB888 is this correct?

[diff](https://lore.kernel.org/linux-media/20250606-rpi-unicam-rgb-bgr-fix-v1-1-9930b963f3eb@kernel.org/#iZ31drivers:media:platform:broadcom:bcm2835-unicam.c) --git a/drivers/media/platform/broadcom/bcm2835-unicam.c b/drivers/media/platform/broadcom/bcm2835-unicam.c
index f10064107d543caf867249d0566a0f42d6d8c4c6..1f549019efd53c9aae83193e74f1a3601ebf274d 100644
--- a/drivers/media/platform/broadcom/bcm2835-unicam.c
+++ b/drivers/media/platform/broadcom/bcm2835-unicam.c
@@ -338,15 +338,10 @@ static const struct unicam_format_info unicam_image_formats[] = {
 	/* RGB Formats */
 		.fourcc		= V4L2_PIX_FMT_RGB565, /* gggbbbbb rrrrrggg */
 		.code		= MEDIA_BUS_FMT_RGB565_1X16,
 		.depth		= 16,
 		.csi_dt		= MIPI_CSI2_DT_RGB565,
-	}, {
-		.fourcc		= V4L2_PIX_FMT_RGB24, /* rgb */
-		.code		= MEDIA_BUS_FMT_RGB888_1X24,
-		.depth		= 24,
-		.csi_dt		= MIPI_CSI2_DT_RGB888,
 	}, {
 		.fourcc		= V4L2_PIX_FMT_BGR24, /* bgr */
 		.code		= MEDIA_BUS_FMT_BGR888_1X24,
 		.depth		= 24,
 		.csi_dt		= MIPI_CSI2_DT_RGB888,

[superware]

FWIW there are a couple of threads upstream, but nothing resolved yet.

Hello, also experiencing this color-ordering issue with TC358743, anything resolved yet? In the meantime, is it possible to switch to UYVY on Pi5 without patching the driver?

[mdevaev]

@superware I just use a userspace workaround: pikvm/ustreamer@7015a26