NXP RT1064 CSI implementation of OV5640

[mariopaja]

Hi everyone,

We have a custom board & a custom module of ov5640 from an old project and I am trying to test the tcpserversink sample.

Additionally I have tried also to implement it with mimxrt1064_evk for easier testing from others too. For this case I am using the Adafruit OV5640 Camera Breakout - 120 Degree Lens with Autofocus with the mimxrt1064_evk

I noticed that the code gets stuck at kernel/kheap.c: (void) z_pend_curr(&heap->lock, key, &heap->wait_q, timeout);

Sample: /video/tcpserversink

mimxrt1064_evk.overlay

#include <zephyr/dt-bindings/video/video-interfaces.h>

/ {
	chosen {
		zephyr,camera = &csi;
	};
};

&csi {
	pinctrl-0 = <&pinmux_csi>;
	pinctrl-names = "default";
	status = "okay";

	port {
		ep_in: endpoint {
			remote-endpoint-label = "ep_out";
            bus-width = <8>;
			hsync-active = <0>;
			vsync-active = <0>;
			pclk-sample = <1>;
		};
	};
};

csi_i2c: &lpi2c1 {
	status = "okay";
	pinctrl-0 = <&pinmux_lpi2c1>;
	pinctrl-names = "default";

	ov5640: ov5640@3c {
		compatible = "ovti,ov5640";
		reg = <0x3c>;
		status = "okay";

		port {
			ep_out: endpoint {
				remote-endpoint-label = "ep_in";
				bus-type = <VIDEO_BUS_TYPE_PARALLEL>;
			};
		};
	};
};

prj.conf

# Generic networking options
CONFIG_NETWORKING=y
CONFIG_NET_TCP=y
CONFIG_NET_IPV4=y
CONFIG_NET_SOCKETS=y
CONFIG_ZVFS_OPEN_MAX=6
CONFIG_POSIX_API=y

# Kernel options
CONFIG_MAIN_STACK_SIZE=2048
CONFIG_ENTROPY_GENERATOR=y
CONFIG_TEST_RANDOM_GENERATOR=y
CONFIG_INIT_STACKS=y

# Logging
CONFIG_NET_LOG=y
CONFIG_LOG=y
CONFIG_NET_STATISTICS=y
CONFIG_PRINTK=y

# Network buffers
CONFIG_NET_PKT_RX_COUNT=16
CONFIG_NET_PKT_TX_COUNT=64
CONFIG_NET_BUF_RX_COUNT=16
CONFIG_NET_BUF_TX_COUNT=512
CONFIG_NET_BUF_DATA_SIZE=1024

CONFIG_NET_CONTEXT_NET_PKT_POOL=y

# IP address options
CONFIG_NET_IF_UNICAST_IPV6_ADDR_COUNT=3
CONFIG_NET_IF_MCAST_IPV6_ADDR_COUNT=4
CONFIG_NET_MAX_CONTEXTS=10

# Network shell
CONFIG_NET_SHELL=y

# Network application options and configuration
CONFIG_NET_CONFIG_SETTINGS=y
CONFIG_NET_CONFIG_MY_IPV4_ADDR="10.0.0.134"

CONFIG_VIDEO=y
CONFIG_VIDEO_SW_GENERATOR=y

CONFIG_HEAP_MEM_POOL_SIZE=16384
CONFIG_THREAD_RUNTIME_STATS=y
CONFIG_SCHED_THREAD_USAGE=y
CONFIG_MAIN_STACK_SIZE=16000
CONFIG_IDLE_STACK_SIZE=2048
CONFIG_ISR_STACK_SIZE=8192

Output:

[00:00:02.972,000] <inf> phy_mc_ksz8081: PHY 0 is up
[00:00:02.972,000] <inf> phy_mc_ksz8081: PHY (0) Link speed 100 Mb, full duplex

[00:00:02.973,000] <inf> eth_nxp_enet_mac: Link is up
*** Booting Zephyr OS build v4.1.0-4810-gf432d5d510ae ***
[00:00:02.974,000] <inf> net_config: Initializing network
[00:00:02.974,000] <inf> net_config: IPv4 address: 10.0.0.134
Video device detected, format: RGBP 160x120

INFO:
drivers/video/ov5640.c
The following is removed from ov5640_init():

	if (!gpio_is_ready_dt(&cfg->reset_gpio)) {
		LOG_ERR("%s: device %s is not ready", dev->name, cfg->reset_gpio.port->name);
		return -ENODEV;
	}

	if (!gpio_is_ready_dt(&cfg->powerdown_gpio)) {
		LOG_ERR("%s: device %s is not ready", dev->name, cfg->powerdown_gpio.port->name);
		return -ENODEV;
	}

	/* Power up sequence */
	if (cfg->powerdown_gpio.port != NULL) {
		ret = gpio_pin_configure_dt(&cfg->powerdown_gpio, GPIO_OUTPUT_ACTIVE);
		if (ret) {
			return ret;
		}
	}

	if (cfg->reset_gpio.port != NULL) {
		ret = gpio_pin_configure_dt(&cfg->reset_gpio, GPIO_OUTPUT_ACTIVE);
		if (ret) {
			return ret;
		}
	}

	k_sleep(K_MSEC(5));

	if (cfg->powerdown_gpio.port != NULL) {
		gpio_pin_set_dt(&cfg->powerdown_gpio, 0);
	}

	k_sleep(K_MSEC(1));

	if (cfg->reset_gpio.port != NULL) {
		gpio_pin_set_dt(&cfg->reset_gpio, 0);
	}

[DerekSnell]

Hi @mariopaja ,

There is recent support of DVP with the OV5640, see #76124. We believe this should work using the parallel interface on the RT10xx. However, I am not aware of an example for this today.

You mentioned your board does not boot. I suspect it does boot, but the execution gets stuck in the startup code initializing the drivers, and never gets to main(). Can you debug and diagnose this startup on your board and provide more details?

Best regards

[mariopaja]

@DerekSnell I found the problem out, reset_gpio & powerdown_gpio were missing. I removed them from the init on the driver and it seems like the application boots to a certain point

[mariopaja]

@ngphibang i noticed on video related PR that you have been working on this sensor, do you have an idea about this issue?

[mariopaja]

@ngphibang i noticed on video related PR that you have been working on this sensor, do you have an idea about this issue?

[ngphibang]

Hi @mariopaja ,

We only have ov5640 working on RT1170-EVK in CSI mode and don't have RT1064 with ov5640 in DVP mode. So, with your custom boards and your very little given input, it is very hard to know where is the issue. Some of my ideas for debugging are below:

• Maybe better to start with the video/capture sample (without display) than the tcpserversink sample to simplify the problem ? The capture sample focuses on the camera only.
• Maybe adding logs into the sample's main.c, the csi and the ov5640 drivers to see where is the stucked point ? Is the CSI successfully initialized ? then the ov5640 is initialized ? then the application ? at which point it stucks ...
• The init orfer priority : the CSI driver is initialized before the sensor driver to give master clock to the sensor on RT1064. With your custom boards, is the ov5640 needs clock from the CSI ? If not, maybe you need to change the init order.
• Maybe you could post the schematic of your custom boards for the camera module here to give some more information ?

BTW, @CharlesDias has made the ov5640 working on ST boards in DVP mode, maybe he can help at some point ...

[CharlesDias]

Hi @mariopaja and @ngphibang,

Please, take a look at this shield configuration:

https://docs.zephyrproject.org/latest/boards/shields/st_b_cams_omv_mb1683/doc/index.html