PinePhone Pro/IMX258 Camera Debugging - Revision history

Diederik: /* Initial status / Add note about DeviceTree binding* which tells the DT description is lacking those voltage supply entries

2024-11-10T15:01:32Z

Initial status: Add note about DeviceTree *binding* which tells the DT description is lacking those voltage supply entries

← Older revision		Revision as of 15:01, 10 November 2024
Line 18:		Line 18:
	Are these errors bad/fatal? Inspecting the [https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/drivers/media/i2c/imx258.c?h=linux-6.6.y source] shows us that these supplies are defined in the <code>imx258_supply_names</code> array, and <code>imx258_probe</code> calls <code>devm_regulator_bulk_get</code> to set them up.		Are these errors bad/fatal? Inspecting the [https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/drivers/media/i2c/imx258.c?h=linux-6.6.y source] shows us that these supplies are defined in the <code>imx258_supply_names</code> array, and <code>imx258_probe</code> calls <code>devm_regulator_bulk_get</code> to set them up.
	<code>imx258_probe</code> checks the return code from <code>devm_regulator_bulk_get</code>, emitting a fatal "Failed to get supplies" message if the return code is negative (indicating failure). Since we don't see this message (or any others from the driver), it looks like it loads successfully!		<code>imx258_probe</code> checks the return code from <code>devm_regulator_bulk_get</code>, emitting a fatal "Failed to get supplies" message if the return code is negative (indicating failure). Since we don't see this message (or any others from the driver), it looks like it loads successfully!

			==== Note ====
			The DeviceTree description of the Sony IMX258 Image Sensor should conform to its [https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/Documentation/devicetree/bindings/media/i2c/sony,imx258.yaml DeviceTree binding].<br />
			What those error messages tell you is that the <code>vana</code> and <code>vdig</code> supplies have NOT been defined in the DeviceTree description. The best thing to do would be to add those.

	=== How can we test the camera? ===		=== How can we test the camera? ===

Diederik: Update various links to (current) latest upstream LTS kernel (6.6)

2024-11-10T14:42:45Z

Update various links to (current) latest upstream LTS kernel (6.6)

← Older revision		Revision as of 14:42, 10 November 2024
Line 16:		Line 16:
	This tells us the kernel loads the driver, which is something.		This tells us the kernel loads the driver, which is something.

	Are these errors bad/fatal? Inspecting the [https://~~github~~.~~com~~/~~megous~~/linux/~~blob~~/~~orange-pi-5~~.17/drivers/media/i2c/imx258.c source] shows us that these supplies are defined in the <code>imx258_supply_names</code> array, and <code>imx258_probe</code> calls <code>devm_regulator_bulk_get</code> to set them up.		Are these errors bad/fatal? Inspecting the [https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/drivers/media/i2c/imx258.c?h=linux-6.6.y source] shows us that these supplies are defined in the <code>imx258_supply_names</code> array, and <code>imx258_probe</code> calls <code>devm_regulator_bulk_get</code> to set them up.
	<code>imx258_probe</code> checks the return code from <code>devm_regulator_bulk_get</code>, emitting a fatal "Failed to get supplies" message if the return code is negative (indicating failure). Since we don't see this message (or any others from the driver), it looks like it loads successfully!		<code>imx258_probe</code> checks the return code from <code>devm_regulator_bulk_get</code>, emitting a fatal "Failed to get supplies" message if the return code is negative (indicating failure). Since we don't see this message (or any others from the driver), it looks like it loads successfully!

Line 89:		Line 89:
	</pre>		</pre>

	[https://www.kernel.org/doc/html/v4.13/media~~/uapi~~/v4l/vidioc-qbuf.html VIDIOC_DQBUF] corresponds to dequeuing a buffer once it has been filled with an image by the v4l2 device. Apparently, the device never gives us back a captured frame.		[https://www.kernel.org/doc/html/v6.6/userspace-api/media/v4l/vidioc-qbuf.html VIDIOC_DQBUF] corresponds to dequeuing a buffer once it has been filled with an image by the v4l2 device. Apparently, the device never gives us back a captured frame.

	=== Now what? ===		=== Now what? ===
Line 132:		Line 132:
	=== <code>rkisp1</code> debugfs info ===		=== <code>rkisp1</code> debugfs info ===

	The <code>rkisp1</code> driver keeps some [https://~~github~~.~~com~~/~~megous~~/linux/~~blob~~/~~c5af9f4f874a66b65c73c450b79f6a86b1b46332~~/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h#~~L335-L363~~ error/debug counters] in debugfs, which can be viewed at <code>/sys/kernel/debug/ff910000.isp0</code>.		The <code>rkisp1</code> driver keeps some [https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h?h=linux-6.6.y#n418 error/debug counters] in debugfs, which can be viewed at <code>/sys/kernel/debug/ff910000.isp0</code>.

	Sadly, the only one of these that comes up nonzero is <code>sp_stop_timeout</code> ("upon stream stop, the capture waits 1 second for the isr to stop the stream. This param is incremented in case of timeout"). I observe it to increment once per time I terminate the <code>"v4l2-ctl" ... "--stream-mmap"</code> command. This tells us what we already know: interrupts are not firing, but it also tells us that we aren't hitting any other errors that rkisp1 knows about.		Sadly, the only one of these that comes up nonzero is <code>sp_stop_timeout</code> ("upon stream stop, the capture waits 1 second for the isr to stop the stream. This param is incremented in case of timeout"). I observe it to increment once per time I terminate the <code>"v4l2-ctl" ... "--stream-mmap"</code> command. This tells us what we already know: interrupts are not firing, but it also tells us that we aren't hitting any other errors that rkisp1 knows about.

Gtw: Describe configuration required on 6.x kernels

2024-10-13T15:19:38Z

Describe configuration required on 6.x kernels

← Older revision		Revision as of 15:19, 13 October 2024
Line 191:		Line 191:

	In some cases, the isp exposes 2 devices nodes, and the nodes have separate topologies with a different sensor attached to each. In this situation, you may need to reference the isp using the correct device node for the commands to work.		In some cases, the isp exposes 2 devices nodes, and the nodes have separate topologies with a different sensor attached to each. In this situation, you may need to reference the isp using the correct device node for the commands to work.

			=== Newer (6.x) kernels ===

			The <code>rkisp1</code> driver has changed significantly in more recent kernel versions, and the V4L2 node topology varies, often breaking the example commands shown here.

			In particular, newly introduced CSI and resizer subdevices mean that a configuration similar to the following is required to make a 4208x3120 capture from the imx258 camera when running [https://codeberg.org/megi/linux/archive/orange-pi-6.9-20240512-2332.tar.gz Megi's 6.9 kernel]:

			<pre>
			media-ctl -d 1 -r
			media-ctl -d 1 -l "'imx258 1-001a':0 -> 'rkisp1_csi':0 [1]"
			media-ctl -d 1 -l "'rkisp1_csi':1 -> 'rkisp1_isp':0 [1]"
			media-ctl -d 1 -l "'rkisp1_isp':2 -> 'rkisp1_resizer_selfpath':0 [0]"
			media-ctl -d 1 -l "'rkisp1_isp':2 -> 'rkisp1_resizer_mainpath':0 [1]"
			media-ctl -d 1 --set-v4l2 '"imx258 1-001a":0 [fmt:SRGGB10_1X10/4208x3120]'
			media-ctl -d 1 --set-v4l2 '"rkisp1_isp":0 [fmt:SRGGB10_1X10/4208x3120 crop: (0,0)/4208x3120]'
			media-ctl -d 1 --set-v4l2 '"rkisp1_isp":2 [fmt:YUYV8_2X8/4208x3120 crop: (0,0)/4208x3120]'
			media-ctl -d 1 --set-v4l2 '"rkisp1_resizer_mainpath":0 [fmt:YUYV8_2X8/4208x3120 crop: (0,0)/4208x3120]'
			media-ctl -d 1 --set-v4l2 '"rkisp1_resizer_mainpath":1 [fmt:YUYV8_2X8/4208x3120]'
			v4l2-ctl -d 5 -v width=4208,height=3120,pixelformat=422P
			v4l2-ctl -d 5 --stream-mmap --stream-count 1 --stream-to=/tmp/frame.raw
			ffmpeg -y -s:v 4208x3120 -pix_fmt yuv422p -i /tmp/frame.raw /tmp/frame.jpeg
			</pre>

	=== What's left? ===		=== What's left? ===

Fxc at 16:41, 20 February 2023

2023-02-20T16:41:37Z

← Older revision		Revision as of 16:41, 20 February 2023
Line 202:		Line 202:
	* [https://blog.krybot.com/a?ID=01650-cea27a80-a1ab-4da1-9cb5-3945be91e3e1 Background on the GIC]		* [https://blog.krybot.com/a?ID=01650-cea27a80-a1ab-4da1-9cb5-3945be91e3e1 Background on the GIC]
	* [https://elinux.org/images/9/94/ISP-presentation.pdf Background on the rkisp1 ISP]		* [https://elinux.org/images/9/94/ISP-presentation.pdf Background on the rkisp1 ISP]


			[[Category:PinePhone Pro]]

Memleek: explain better how one figures out what function is responsible for frame completion notification

2022-07-02T23:07:58Z

explain better how one figures out what function is responsible for frame completion notification

← Older revision		Revision as of 23:07, 2 July 2022
Line 121:		Line 121:
	=== If i2c works, what doesn't? ===		=== If i2c works, what doesn't? ===

	Our program that asks for a video frame is still hung! We ask the camera to start streaming frames, it presumably does, but the v4l2 stack never tells us a frame has finished. Doing some digging in the v4l2 stack (and the rkisp1 driver), we find out that rkisp1 is notified about frame status via interrupts.		Our program that asks for a video frame is still hung! We ask the camera to start streaming frames, it presumably does, but the v4l2 stack never tells us a frame has finished. Doing some digging in the v4l2 stack (and the rkisp1 driver), we find out that rkisp1 is notified about frame status via interrupts. We figure this out by manually backtracking through the code to see when <code>vb2_buffer_done</code>, the function in the vb2 API to call when a frame is finished, would be called. In the rkisp1 code, <code>vb2_buffer_done</code> is only called from <code>rkisp1_handle_buffer</code> which is only called from <code>rkisp1_capture_isr</code>, which (for the PPP's rk3399 SoC) is only called from <code>rkisp1_isr</code>, which is an interrupt handler. We know that it's an interrupt handler from the name (<code>_isr</code>), but also because it gets passed (indirectly) to <code>devm_request_irq</code> by way of being the <code>.isr</code> field of <code>rk3399_isp_isrs</code>.

	Perhaps the hardware is never actually emitting the interrupt that signals a frame being finished. Indeed, grepping our FTrace log shows that <code>rkisp1_isr</code> is never called. A quick look at <code>/proc/interrupts</code> shows that '''the only interrupt associated with the isp has never triggered''' (0 count on every CPU):		Perhaps the hardware is never actually emitting the interrupt that signals a frame being finished. Indeed, grepping our FTrace log shows that <code>rkisp1_isr</code> is never called. A quick look at <code>/proc/interrupts</code> shows that '''the only interrupt associated with the isp has never triggered''' (0 count on every CPU):

Kgmt0: /* Megi saves the day */ Fix ffmpeg command for front camera example

2022-06-29T18:29:29Z

Megi saves the day: Fix ffmpeg command for front camera example

← Older revision		Revision as of 18:29, 29 June 2022
Line 187:		Line 187:
	"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "--stream-mmap" "--stream-count" "1" "--stream-to=frame.raw"		"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "--stream-mmap" "--stream-count" "1" "--stream-to=frame.raw"

	ffmpeg -y -s:v ~~1048x780~~ -pix_fmt yuv422p -i frame.raw frame.jpg		ffmpeg -y -s:v 1632x1224 -pix_fmt yuv422p -i frame.raw frame.jpg
	</pre>		</pre>

Kgmt0: /* Megi saves the day */ DTS patch is now included in 5.18

2022-06-14T00:56:11Z

Megi saves the day: DTS patch is now included in 5.18

← Older revision		Revision as of 00:56, 14 June 2022
Line 151:		Line 151:
	</pre>		</pre>

	Turns out, the .dts had the wrong ISP connected to the sensor!		Turns out, the .dts had the wrong ISP connected to the sensor! This is fixed in [https://github.com/megous/linux/commit/9afd884f8b36121fb6097e77b6d35fe46ab48ad9 this commit], which is included in kernel version 5.18 or newer.

	Apply [https://github.com/megous/linux/commit/9afd884f8b36121fb6097e77b6d35fe46ab48ad9 this commit] ([https://github.com/megous/linux/commit/9afd884f8b36121fb6097e77b6d35fe46ab48ad9.patch patch]) to your kernel ~~tree, rebuild, and reinstall. Now~~, the following should produce an image (<code>frame.jpg</code>):		With a sufficiently new kernel, the following should produce an image (<code>frame.jpg</code>):

	<pre>		<pre>

Kgmt0: /* Megi saves the day */ Remove excess empty line

2022-06-13T17:07:10Z

Megi saves the day: Remove excess empty line

← Older revision		Revision as of 17:07, 13 June 2022
Line 186:		Line 186:
	"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "width=1632,height=1224,pixelformat=422P"		"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "width=1632,height=1224,pixelformat=422P"
	"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "--stream-mmap" "--stream-count" "1" "--stream-to=frame.raw"		"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "--stream-mmap" "--stream-count" "1" "--stream-to=frame.raw"


	ffmpeg -y -s:v 1048x780 -pix_fmt yuv422p -i frame.raw frame.jpg		ffmpeg -y -s:v 1048x780 -pix_fmt yuv422p -i frame.raw frame.jpg

Kgmt0: /* Megi saves the day */ Fix typo in the commands for front camera

2022-06-13T17:04:48Z

Megi saves the day: Fix typo in the commands for front camera

Kgmt0: /* Megi saves the day */ Add info about the case where the isp has 2 device nodes

2022-06-13T17:01:59Z

Megi saves the day: Add info about the case where the isp has 2 device nodes

← Older revision		Revision as of 17:01, 13 June 2022
Line 189:		Line 189:
	ffmpeg -y -s:v 1048x780 -pix_fmt yuv422p -i frame.raw frame.jpg		ffmpeg -y -s:v 1048x780 -pix_fmt yuv422p -i frame.raw frame.jpg
	</pre>		</pre>

			In some cases, the isp exposes 2 devices nodes, and the nodes have separate topologies with a different sensor attached to each. In this situation, you may need to reference the isp using the correct device node for the commands to work.

	=== What's left? ===		=== What's left? ===

← Older revision		Revision as of 17:04, 13 June 2022
Line 174:		Line 174:

	<pre>		<pre>
	"media-ctl" "-d" "~~pltform~~:rkisp1" "-r"		"media-ctl" "-d" "platform:rkisp1" "-r"
	"media-ctl" "-d" "~~pltform~~:rkisp1" "--set-v4l2" "'m00_f_ov8858 1-0036':0 [fmt:SBGGR10_1X10/1632x1224]"		"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" "'m00_f_ov8858 1-0036':0 [fmt:SBGGR10_1X10/1632x1224]"
	"media-ctl" "-d" "~~pltform~~:rkisp1" "-l" "'m00_f_ov8858 1-0036':0 -> 'rkisp1_isp':0 [1]"		"media-ctl" "-d" "platform:rkisp1" "-l" "'m00_f_ov8858 1-0036':0 -> 'rkisp1_isp':0 [1]"
	"media-ctl" "-d" "~~pltform~~:rkisp1" "-l" "'rkisp1_isp':2 -> 'rkisp1_resizer_selfpath':0 [1]"		"media-ctl" "-d" "platform:rkisp1" "-l" "'rkisp1_isp':2 -> 'rkisp1_resizer_selfpath':0 [1]"
	"media-ctl" "-d" "~~pltform~~:rkisp1" "--set-v4l2" "'rkisp1_isp':0 [fmt:SBGGR10_1X10/1632x1224 crop:(0,0)/1632x1224]"		"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" "'rkisp1_isp':0 [fmt:SBGGR10_1X10/1632x1224 crop:(0,0)/1632x1224]"
	"media-ctl" "-d" "~~pltform~~:rkisp1" "--set-v4l2" "'rkisp1_isp':2 [fmt:YUYV8_2X8/1632x1224 crop:(0,0)/1632x1224]"		"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" "'rkisp1_isp':2 [fmt:YUYV8_2X8/1632x1224 crop:(0,0)/1632x1224]"
	"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "width=1632,height=1224,"		"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "width=1632,height=1224,"
	"media-ctl" "-d" "~~pltform~~:rkisp1" "--set-v4l2" "'rkisp1_resizer_selfpath':0 [fmt:YUYV8_2X8/1632x1224 crop:(0,0)/1632x1224]"		"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" "'rkisp1_resizer_selfpath':0 [fmt:YUYV8_2X8/1632x1224 crop:(0,0)/1632x1224]"
	"media-ctl" "-d" "~~pltform~~:rkisp1" "--set-v4l2" "'rkisp1_resizer_selfpath':1 [fmt:YUYV8_2X8/1632x1224]"		"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" "'rkisp1_resizer_selfpath':1 [fmt:YUYV8_2X8/1632x1224]"

	"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "width=1632,height=1224,pixelformat=422P"		"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "width=1632,height=1224,pixelformat=422P"
	"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "--stream-mmap" "--stream-count" "1" "--stream-to=frame.raw"		"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "--stream-mmap" "--stream-count" "1" "--stream-to=frame.raw"


	ffmpeg -y -s:v 1048x780 -pix_fmt yuv422p -i frame.raw frame.jpg		ffmpeg -y -s:v 1048x780 -pix_fmt yuv422p -i frame.raw frame.jpg