PINE64 - User contributions [en]

PinePhone Pro/IMX258 Camera Debugging

2022-06-29T18:29:29Z

Kgmt0: /* Megi saves the day */ Fix ffmpeg command for front camera example

{{Under construction}}

== Background ==

The Pinephone Pro's rear camera is a Sony IMX258. The camera driver is present upstream, and at least [https://gitlab.manjaro.org/manjaro-arm/packages/core/linux-pinephonepro Manjaro's kernel] (downstream of Megi's) enables the driver. The camera does not seem to work; this page catalogs work on figuring out why.

== Initial status ==

What do we see in dmesg?

<pre>
imx258 1-001a: supply vana not found, using dummy regulator
imx258 1-001a: supply vdig not found, using dummy regulator
</pre>

This tells us the kernel loads the driver, which is something.

Are these errors bad/fatal? Inspecting the [https://github.com/megous/linux/blob/orange-pi-5.17/drivers/media/i2c/imx258.c source] shows us that these supplies are defined in the <code>imx258_supply_names</code> array, and <code>imx258_probe</code> calls <code>devm_regulator_bulk_get</code> to set them up.
<code>imx258_probe</code> checks the return code from <code>devm_regulator_bulk_get</code>, emitting a fatal "Failed to get supplies" message if the return code is negative (indicating failure). Since we don't see this message (or any others from the driver), it looks like it loads successfully!

=== How can we test the camera? ===

==== Megapixels ====

[https://gitlab.com/postmarketOS/megapixels Megapixels] requires a per-device file to configure cameras. This one isn't complete (most of its numbers are complete nonsense), but it should let us see what happens when Megapixels tries to access the camera:
<code>pine64,pinephone-pro.ini</code>:
<pre>
[device]
make=PINE64
model=PinePhone Pro

[rear]
driver=imx258
media-driver=rkisp1
capture-width=4208
capture-height=3118
capture-rate=60
capture-fmt=GRBG10P
preview-width=1280
preview-height=960
preview-rate=60
#preview-fmt=BGGR8
capture-fmt=GRBG10P
rotate=90
mirrored=true
focallength=2.6
cropfactor=12.7
fnumber=2.8
flash-display=true
</pre>

This segfaulted Megapixels for me. After patching some trivial segfaults, I get fatal GTK4 issues and other tangential problems. Let's try something else.

==== Manually setting up a camera pipeline ====

We can roughly follow [https://www.kernel.org/doc/html/latest/admin-guide/media/rkisp1.html this guide]. Doing so gives us the following commands:
<pre>
"media-ctl" "-d" "platform:rkisp1" "-r"
"media-ctl" "-d" "platform:rkisp1" "-l" "'imx258 1-001a':0 -> 'rkisp1_isp':0 [1]"
"media-ctl" "-d" "platform:rkisp1" "-l" "'rkisp1_isp':2 -> 'rkisp1_resizer_selfpath':0 [1]"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_isp":0 [fmt:SGRBG10_1X10/1048x780]'
"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "width=1048,height=780,"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_resizer_selfpath":0 [fmt:SGRBG10_1X10/1048x780 crop:(0,0)/1048x780]'

"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "width=800,height=600,pixelformat=422P"
"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "--stream-mmap" "--stream-count" "1" "--verbose"
</pre>

Careful readers will note that this is setting the camera up with a low resolution compared to what the sensor can do. For some reason I can't get the rkisp1 node to configure its sink pad with the maximum resolution of the sensor (4208x3118). Instead, it sets itself to 4032x3024. This mismatch is forbidden and causes v4l2 to give us EPIPE when we try to enable the stream. Using the lower resolution gets us further--the last command (which attempts to save a single frame of video) gives output like this:
<pre>
VIDIOC_QUERYCAP: ok
VIDIOC_REQBUFS returned 0 (Success)
VIDIOC_QUERYBUF returned 0 (Success)
VIDIOC_QUERYBUF returned 0 (Success)
VIDIOC_QUERYBUF returned 0 (Success)
VIDIOC_QUERYBUF returned 0 (Success)
VIDIOC_QBUF returned 0 (Success)
VIDIOC_QBUF returned 0 (Success)
VIDIOC_QBUF returned 0 (Success)
VIDIOC_QBUF returned 0 (Success)
VIDIOC_STREAMON returned 0 (Success)
</pre>

But then it just hangs there indefinitely.

Running under strace, we see the program stuck in an ioctl:
<pre>
ioctl(3, VIDIOC_DQBUF, {type=V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE
</pre>

[https://www.kernel.org/doc/html/v4.13/media/uapi/v4l/vidioc-qbuf.html VIDIOC_DQBUF] corresponds to dequeuing a buffer once it has been filled with an image by the v4l2 device. Apparently, the device never gives us back a captured frame.

=== Now what? ===

==== Does i2c work at all? ====

The camera driver uses i2c; we can see the camera [https://github.com/megous/linux/blob/c5af9f4f874a66b65c73c450b79f6a86b1b46332/arch/arm64/boot/dts/rockchip/rk3399-pinephone-pro.dts#L890-L914 in the .dts] to determine that it's on i2c bus 1, address 0x1a.

The imx258 is controlled over i2c, and sends image data over MIPI. The first step is to see if the i2c control is working.

Looking back at the driver, it seems we should see errors in dmesg if i2c communication fails. The driver checks the chip id in its probe function, which uses i2c, so we can conclude that i2c gets set up properly initially. Does it keep working?

==== Does i2c keep working? (FTrace) ====

We can see if other functions that use i2c produce errors--for example, when we run our <code>"v4l2-ctl" ... "--stream-mmap"</code> command, we expect that <code>imx258_start_streaming</code> will get called, and this function also writes a load of registers over i2c.

Build and install a kernel with CONFIG_FTRACE (not hard, but outside the scope of this write-up) and enable function graph tracing:
<pre>
$ sudo su
# cd /sys/kernel/debug/tracing
# echo function_graph > current_tracer
# cat available_filter_functions | grep -E 'rkisp|imx258|v4l2' | awk '{print $1}' > set_graph_function
# echo 'vm_map_pages,__iommu_map,iommu_map_sg_atomic,rk_iommu_map,__alloc_pages,dma_alloc_attrs,dma_mmap_attrs,__vm_map_pages,vb2_mmap,clk_e,able,schedule_timeout,clk_disable,regmap_write,schedule,__i2c_tra,sfer,i2c_tra,sfer_buffer_flags,dma_free_attrs' | tr , '\n' > set_graph_notrace
# tee trace_pipe > ~/function-trace.log
</pre>

Now, in another terminal session, run the <code>"v4l2-ctl" ... "--stream-mmap"</code> command again. We end up with a bunch (but hopefully not *too* much) of output in our first terminal and the <code>~/function-trace.log</code> file.

A quick grep shows that <code>imx258_start_streaming</code> is indeed called, so '''i2c is working properly'''.

=== If i2c works, what doesn't? ===

Our program that asks for a video frame is still hung! We ask the camera to start streaming frames, it presumably does, but the v4l2 stack never tells us a frame has finished. Doing some digging in the v4l2 stack (and the rkisp1 driver), we find out that rkisp1 is notified about frame status via interrupts.

Perhaps the hardware is never actually emitting the interrupt that signals a frame being finished. Indeed, grepping our FTrace log shows that <code>rkisp1_isr</code> is never called. A quick look at <code>/proc/interrupts</code> shows that '''the only interrupt associated with the isp has never triggered''' (0 count on every CPU):
<pre>
# head -n1 /proc/interrupts; grep isp /proc/interrupts
CPU0 CPU1 CPU2 CPU3 CPU4 CPU5
57: 0 0 0 0 0 0 GICv3 75 Level ff914000.iommu, rkisp1
</pre>

=== <code>rkisp1</code> debugfs info ===

The <code>rkisp1</code> driver keeps some [https://github.com/megous/linux/blob/c5af9f4f874a66b65c73c450b79f6a86b1b46332/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h#L335-L363 error/debug counters] in debugfs, which can be viewed at <code>/sys/kernel/debug/ff910000.isp0</code>.

Sadly, the only one of these that comes up nonzero is <code>sp_stop_timeout</code> ("upon stream stop, the capture waits 1 second for the isr to stop the stream. This param is incremented in case of timeout"). I observe it to increment once per time I terminate the <code>"v4l2-ctl" ... "--stream-mmap"</code> command. This tells us what we already know: interrupts are not firing, but it also tells us that we aren't hitting any other errors that rkisp1 knows about.

== Next steps ==

Where do we go from here? I don't know--something may be wrong with the way the .dts specifies interrupts, some kind of firmware/GIC (the rk3399's interrupt controller) configuration issue, or something else. It would be good to try to determine:

* Whether the MIPI lines actually show traffic, if someone has a logic analyzer and a dissected PPP.
* Whether the DMA of frames to memory actually happens, regardless of (lack of) emission of the interrupt that tells us when said DMA finishes a frame.
* Whether the regulator messages we see during boot are actually significant. I don't know enough about the Linux regulator framework to say whether these being not-found means power is not correctly supplied, or if Linux just isn't being properly informed of the power supply requirements for precise tracking/power-saving when the camera is off.

== Megi saves the day ==

<pre>
[I] <megi> you might be interested in this https://megous.com/dl/tmp/0ae6ba03a17a3d53.png
[I] <megi> https://megous.com/git/linux/tag/?h=orange-pi-5.18-20220521-1759
</pre>

Turns out, the .dts had the wrong ISP connected to the sensor! This is fixed in [https://github.com/megous/linux/commit/9afd884f8b36121fb6097e77b6d35fe46ab48ad9 this commit], which is included in kernel version 5.18 or newer.

With a sufficiently new kernel, the following should produce an image (<code>frame.jpg</code>):

<pre>
"media-ctl" "-d" "platform:rkisp1" "-r"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"imx258 1-001a":0 [fmt:SGRBG10_1X10/1048x780]'
"media-ctl" "-d" "platform:rkisp1" "-l" "'imx258 1-001a':0 -> 'rkisp1_isp':0 [1]"
"media-ctl" "-d" "platform:rkisp1" "-l" "'rkisp1_isp':2 -> 'rkisp1_resizer_selfpath':0 [1]"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_isp":0 [fmt:SGRBG10_1X10/1048x780 crop:(0,0)/1048x780]'
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_isp":2 [fmt:YUYV8_2X8/1048x780 crop:(0,0)/1048x780]'
"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "width=1048,height=780,"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_resizer_selfpath":0 [fmt:YUYV8_2X8/1048x780 crop:(0,0)/1048x780]'

"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "width=1048,height=780,pixelformat=422P"
"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "--stream-mmap" "--stream-count" "1" "--stream-to=frame.raw"

ffmpeg -y -s:v 1048x780 -pix_fmt yuv422p -i frame.raw frame.jpg
</pre>

Similarly, you can take a photo from the front camera with the following:

<pre>
"media-ctl" "-d" "platform:rkisp1" "-r"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" "'m00_f_ov8858 1-0036':0 [fmt:SBGGR10_1X10/1632x1224]"
"media-ctl" "-d" "platform:rkisp1" "-l" "'m00_f_ov8858 1-0036':0 -> 'rkisp1_isp':0 [1]"
"media-ctl" "-d" "platform:rkisp1" "-l" "'rkisp1_isp':2 -> 'rkisp1_resizer_selfpath':0 [1]"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" "'rkisp1_isp':0 [fmt:SBGGR10_1X10/1632x1224 crop:(0,0)/1632x1224]"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" "'rkisp1_isp':2 [fmt:YUYV8_2X8/1632x1224 crop:(0,0)/1632x1224]"
"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "width=1632,height=1224,"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" "'rkisp1_resizer_selfpath':0 [fmt:YUYV8_2X8/1632x1224 crop:(0,0)/1632x1224]"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" "'rkisp1_resizer_selfpath':1 [fmt:YUYV8_2X8/1632x1224]"

"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "width=1632,height=1224,pixelformat=422P"
"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "--stream-mmap" "--stream-count" "1" "--stream-to=frame.raw"

ffmpeg -y -s:v 1632x1224 -pix_fmt yuv422p -i frame.raw frame.jpg
</pre>

In some cases, the isp exposes 2 devices nodes, and the nodes have separate topologies with a different sensor attached to each. In this situation, you may need to reference the isp using the correct device node for the commands to work.

=== What's left? ===

* libcamera and megapixels still don't work. Megapixels appears to need support for debayering YUYV, and both appear to be configuring the rkisp1 pipeline wrong.
* The images captured are pretty green and seem to have two pixels of garbage at their bottom edge. rkisp1 supports auto white-balance (AWB) using the params/stats pads, and libcamera has some smarts that may hook these up properly. In the meantime, raw images can be postprocessed to improve white balance and exposure.

== Useful links ==

* [https://files.pine64.org/doc/PinePhonePro/PinephonePro-Schematic-V1.0-20211127.pdf Pinephone Pro Schematic]
* [https://blog.krybot.com/a?ID=01650-cea27a80-a1ab-4da1-9cb5-3945be91e3e1 Background on the GIC]
* [https://elinux.org/images/9/94/ISP-presentation.pdf Background on the rkisp1 ISP]

PinePhone Pro/IMX258 Camera Debugging

2022-06-14T00:56:11Z

Kgmt0: /* Megi saves the day */ DTS patch is now included in 5.18

{{Under construction}}

== Background ==

The Pinephone Pro's rear camera is a Sony IMX258. The camera driver is present upstream, and at least [https://gitlab.manjaro.org/manjaro-arm/packages/core/linux-pinephonepro Manjaro's kernel] (downstream of Megi's) enables the driver. The camera does not seem to work; this page catalogs work on figuring out why.

== Initial status ==

What do we see in dmesg?

<pre>
imx258 1-001a: supply vana not found, using dummy regulator
imx258 1-001a: supply vdig not found, using dummy regulator
</pre>

This tells us the kernel loads the driver, which is something.

Are these errors bad/fatal? Inspecting the [https://github.com/megous/linux/blob/orange-pi-5.17/drivers/media/i2c/imx258.c source] shows us that these supplies are defined in the <code>imx258_supply_names</code> array, and <code>imx258_probe</code> calls <code>devm_regulator_bulk_get</code> to set them up.
<code>imx258_probe</code> checks the return code from <code>devm_regulator_bulk_get</code>, emitting a fatal "Failed to get supplies" message if the return code is negative (indicating failure). Since we don't see this message (or any others from the driver), it looks like it loads successfully!

=== How can we test the camera? ===

==== Megapixels ====

[https://gitlab.com/postmarketOS/megapixels Megapixels] requires a per-device file to configure cameras. This one isn't complete (most of its numbers are complete nonsense), but it should let us see what happens when Megapixels tries to access the camera:
<code>pine64,pinephone-pro.ini</code>:
<pre>
[device]
make=PINE64
model=PinePhone Pro

[rear]
driver=imx258
media-driver=rkisp1
capture-width=4208
capture-height=3118
capture-rate=60
capture-fmt=GRBG10P
preview-width=1280
preview-height=960
preview-rate=60
#preview-fmt=BGGR8
capture-fmt=GRBG10P
rotate=90
mirrored=true
focallength=2.6
cropfactor=12.7
fnumber=2.8
flash-display=true
</pre>

This segfaulted Megapixels for me. After patching some trivial segfaults, I get fatal GTK4 issues and other tangential problems. Let's try something else.

==== Manually setting up a camera pipeline ====

We can roughly follow [https://www.kernel.org/doc/html/latest/admin-guide/media/rkisp1.html this guide]. Doing so gives us the following commands:
<pre>
"media-ctl" "-d" "platform:rkisp1" "-r"
"media-ctl" "-d" "platform:rkisp1" "-l" "'imx258 1-001a':0 -> 'rkisp1_isp':0 [1]"
"media-ctl" "-d" "platform:rkisp1" "-l" "'rkisp1_isp':2 -> 'rkisp1_resizer_selfpath':0 [1]"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_isp":0 [fmt:SGRBG10_1X10/1048x780]'
"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "width=1048,height=780,"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_resizer_selfpath":0 [fmt:SGRBG10_1X10/1048x780 crop:(0,0)/1048x780]'

"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "width=800,height=600,pixelformat=422P"
"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "--stream-mmap" "--stream-count" "1" "--verbose"
</pre>

Careful readers will note that this is setting the camera up with a low resolution compared to what the sensor can do. For some reason I can't get the rkisp1 node to configure its sink pad with the maximum resolution of the sensor (4208x3118). Instead, it sets itself to 4032x3024. This mismatch is forbidden and causes v4l2 to give us EPIPE when we try to enable the stream. Using the lower resolution gets us further--the last command (which attempts to save a single frame of video) gives output like this:
<pre>
VIDIOC_QUERYCAP: ok
VIDIOC_REQBUFS returned 0 (Success)
VIDIOC_QUERYBUF returned 0 (Success)
VIDIOC_QUERYBUF returned 0 (Success)
VIDIOC_QUERYBUF returned 0 (Success)
VIDIOC_QUERYBUF returned 0 (Success)
VIDIOC_QBUF returned 0 (Success)
VIDIOC_QBUF returned 0 (Success)
VIDIOC_QBUF returned 0 (Success)
VIDIOC_QBUF returned 0 (Success)
VIDIOC_STREAMON returned 0 (Success)
</pre>

But then it just hangs there indefinitely.

Running under strace, we see the program stuck in an ioctl:
<pre>
ioctl(3, VIDIOC_DQBUF, {type=V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE
</pre>

[https://www.kernel.org/doc/html/v4.13/media/uapi/v4l/vidioc-qbuf.html VIDIOC_DQBUF] corresponds to dequeuing a buffer once it has been filled with an image by the v4l2 device. Apparently, the device never gives us back a captured frame.

=== Now what? ===

==== Does i2c work at all? ====

The camera driver uses i2c; we can see the camera [https://github.com/megous/linux/blob/c5af9f4f874a66b65c73c450b79f6a86b1b46332/arch/arm64/boot/dts/rockchip/rk3399-pinephone-pro.dts#L890-L914 in the .dts] to determine that it's on i2c bus 1, address 0x1a.

The imx258 is controlled over i2c, and sends image data over MIPI. The first step is to see if the i2c control is working.

Looking back at the driver, it seems we should see errors in dmesg if i2c communication fails. The driver checks the chip id in its probe function, which uses i2c, so we can conclude that i2c gets set up properly initially. Does it keep working?

==== Does i2c keep working? (FTrace) ====

We can see if other functions that use i2c produce errors--for example, when we run our <code>"v4l2-ctl" ... "--stream-mmap"</code> command, we expect that <code>imx258_start_streaming</code> will get called, and this function also writes a load of registers over i2c.

Build and install a kernel with CONFIG_FTRACE (not hard, but outside the scope of this write-up) and enable function graph tracing:
<pre>
$ sudo su
# cd /sys/kernel/debug/tracing
# echo function_graph > current_tracer
# cat available_filter_functions | grep -E 'rkisp|imx258|v4l2' | awk '{print $1}' > set_graph_function
# echo 'vm_map_pages,__iommu_map,iommu_map_sg_atomic,rk_iommu_map,__alloc_pages,dma_alloc_attrs,dma_mmap_attrs,__vm_map_pages,vb2_mmap,clk_e,able,schedule_timeout,clk_disable,regmap_write,schedule,__i2c_tra,sfer,i2c_tra,sfer_buffer_flags,dma_free_attrs' | tr , '\n' > set_graph_notrace
# tee trace_pipe > ~/function-trace.log
</pre>

Now, in another terminal session, run the <code>"v4l2-ctl" ... "--stream-mmap"</code> command again. We end up with a bunch (but hopefully not *too* much) of output in our first terminal and the <code>~/function-trace.log</code> file.

A quick grep shows that <code>imx258_start_streaming</code> is indeed called, so '''i2c is working properly'''.

=== If i2c works, what doesn't? ===

Our program that asks for a video frame is still hung! We ask the camera to start streaming frames, it presumably does, but the v4l2 stack never tells us a frame has finished. Doing some digging in the v4l2 stack (and the rkisp1 driver), we find out that rkisp1 is notified about frame status via interrupts.

Perhaps the hardware is never actually emitting the interrupt that signals a frame being finished. Indeed, grepping our FTrace log shows that <code>rkisp1_isr</code> is never called. A quick look at <code>/proc/interrupts</code> shows that '''the only interrupt associated with the isp has never triggered''' (0 count on every CPU):
<pre>
# head -n1 /proc/interrupts; grep isp /proc/interrupts
CPU0 CPU1 CPU2 CPU3 CPU4 CPU5
57: 0 0 0 0 0 0 GICv3 75 Level ff914000.iommu, rkisp1
</pre>

=== <code>rkisp1</code> debugfs info ===

The <code>rkisp1</code> driver keeps some [https://github.com/megous/linux/blob/c5af9f4f874a66b65c73c450b79f6a86b1b46332/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h#L335-L363 error/debug counters] in debugfs, which can be viewed at <code>/sys/kernel/debug/ff910000.isp0</code>.

Sadly, the only one of these that comes up nonzero is <code>sp_stop_timeout</code> ("upon stream stop, the capture waits 1 second for the isr to stop the stream. This param is incremented in case of timeout"). I observe it to increment once per time I terminate the <code>"v4l2-ctl" ... "--stream-mmap"</code> command. This tells us what we already know: interrupts are not firing, but it also tells us that we aren't hitting any other errors that rkisp1 knows about.

== Next steps ==

Where do we go from here? I don't know--something may be wrong with the way the .dts specifies interrupts, some kind of firmware/GIC (the rk3399's interrupt controller) configuration issue, or something else. It would be good to try to determine:

* Whether the MIPI lines actually show traffic, if someone has a logic analyzer and a dissected PPP.
* Whether the DMA of frames to memory actually happens, regardless of (lack of) emission of the interrupt that tells us when said DMA finishes a frame.
* Whether the regulator messages we see during boot are actually significant. I don't know enough about the Linux regulator framework to say whether these being not-found means power is not correctly supplied, or if Linux just isn't being properly informed of the power supply requirements for precise tracking/power-saving when the camera is off.

== Megi saves the day ==

<pre>
[I] <megi> you might be interested in this https://megous.com/dl/tmp/0ae6ba03a17a3d53.png
[I] <megi> https://megous.com/git/linux/tag/?h=orange-pi-5.18-20220521-1759
</pre>

Turns out, the .dts had the wrong ISP connected to the sensor! This is fixed in [https://github.com/megous/linux/commit/9afd884f8b36121fb6097e77b6d35fe46ab48ad9 this commit], which is included in kernel version 5.18 or newer.

With a sufficiently new kernel, the following should produce an image (<code>frame.jpg</code>):

<pre>
"media-ctl" "-d" "platform:rkisp1" "-r"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"imx258 1-001a":0 [fmt:SGRBG10_1X10/1048x780]'
"media-ctl" "-d" "platform:rkisp1" "-l" "'imx258 1-001a':0 -> 'rkisp1_isp':0 [1]"
"media-ctl" "-d" "platform:rkisp1" "-l" "'rkisp1_isp':2 -> 'rkisp1_resizer_selfpath':0 [1]"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_isp":0 [fmt:SGRBG10_1X10/1048x780 crop:(0,0)/1048x780]'
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_isp":2 [fmt:YUYV8_2X8/1048x780 crop:(0,0)/1048x780]'
"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "width=1048,height=780,"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_resizer_selfpath":0 [fmt:YUYV8_2X8/1048x780 crop:(0,0)/1048x780]'

"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "width=1048,height=780,pixelformat=422P"
"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "--stream-mmap" "--stream-count" "1" "--stream-to=frame.raw"

ffmpeg -y -s:v 1048x780 -pix_fmt yuv422p -i frame.raw frame.jpg
</pre>

Similarly, you can take a photo from the front camera with the following:

<pre>
"media-ctl" "-d" "platform:rkisp1" "-r"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" "'m00_f_ov8858 1-0036':0 [fmt:SBGGR10_1X10/1632x1224]"
"media-ctl" "-d" "platform:rkisp1" "-l" "'m00_f_ov8858 1-0036':0 -> 'rkisp1_isp':0 [1]"
"media-ctl" "-d" "platform:rkisp1" "-l" "'rkisp1_isp':2 -> 'rkisp1_resizer_selfpath':0 [1]"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" "'rkisp1_isp':0 [fmt:SBGGR10_1X10/1632x1224 crop:(0,0)/1632x1224]"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" "'rkisp1_isp':2 [fmt:YUYV8_2X8/1632x1224 crop:(0,0)/1632x1224]"
"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "width=1632,height=1224,"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" "'rkisp1_resizer_selfpath':0 [fmt:YUYV8_2X8/1632x1224 crop:(0,0)/1632x1224]"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" "'rkisp1_resizer_selfpath':1 [fmt:YUYV8_2X8/1632x1224]"

"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "width=1632,height=1224,pixelformat=422P"
"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "--stream-mmap" "--stream-count" "1" "--stream-to=frame.raw"

ffmpeg -y -s:v 1048x780 -pix_fmt yuv422p -i frame.raw frame.jpg
</pre>

In some cases, the isp exposes 2 devices nodes, and the nodes have separate topologies with a different sensor attached to each. In this situation, you may need to reference the isp using the correct device node for the commands to work.

=== What's left? ===

* libcamera and megapixels still don't work. Megapixels appears to need support for debayering YUYV, and both appear to be configuring the rkisp1 pipeline wrong.
* The images captured are pretty green and seem to have two pixels of garbage at their bottom edge. rkisp1 supports auto white-balance (AWB) using the params/stats pads, and libcamera has some smarts that may hook these up properly. In the meantime, raw images can be postprocessed to improve white balance and exposure.

== Useful links ==

* [https://files.pine64.org/doc/PinePhonePro/PinephonePro-Schematic-V1.0-20211127.pdf Pinephone Pro Schematic]
* [https://blog.krybot.com/a?ID=01650-cea27a80-a1ab-4da1-9cb5-3945be91e3e1 Background on the GIC]
* [https://elinux.org/images/9/94/ISP-presentation.pdf Background on the rkisp1 ISP]

PinePhone Pro/IMX258 Camera Debugging

2022-06-13T17:07:10Z

Kgmt0: /* Megi saves the day */ Remove excess empty line

{{Under construction}}

== Background ==

The Pinephone Pro's rear camera is a Sony IMX258. The camera driver is present upstream, and at least [https://gitlab.manjaro.org/manjaro-arm/packages/core/linux-pinephonepro Manjaro's kernel] (downstream of Megi's) enables the driver. The camera does not seem to work; this page catalogs work on figuring out why.

== Initial status ==

What do we see in dmesg?

<pre>
imx258 1-001a: supply vana not found, using dummy regulator
imx258 1-001a: supply vdig not found, using dummy regulator
</pre>

This tells us the kernel loads the driver, which is something.

Are these errors bad/fatal? Inspecting the [https://github.com/megous/linux/blob/orange-pi-5.17/drivers/media/i2c/imx258.c source] shows us that these supplies are defined in the <code>imx258_supply_names</code> array, and <code>imx258_probe</code> calls <code>devm_regulator_bulk_get</code> to set them up.
<code>imx258_probe</code> checks the return code from <code>devm_regulator_bulk_get</code>, emitting a fatal "Failed to get supplies" message if the return code is negative (indicating failure). Since we don't see this message (or any others from the driver), it looks like it loads successfully!

=== How can we test the camera? ===

==== Megapixels ====

[https://gitlab.com/postmarketOS/megapixels Megapixels] requires a per-device file to configure cameras. This one isn't complete (most of its numbers are complete nonsense), but it should let us see what happens when Megapixels tries to access the camera:
<code>pine64,pinephone-pro.ini</code>:
<pre>
[device]
make=PINE64
model=PinePhone Pro

[rear]
driver=imx258
media-driver=rkisp1
capture-width=4208
capture-height=3118
capture-rate=60
capture-fmt=GRBG10P
preview-width=1280
preview-height=960
preview-rate=60
#preview-fmt=BGGR8
capture-fmt=GRBG10P
rotate=90
mirrored=true
focallength=2.6
cropfactor=12.7
fnumber=2.8
flash-display=true
</pre>

This segfaulted Megapixels for me. After patching some trivial segfaults, I get fatal GTK4 issues and other tangential problems. Let's try something else.

==== Manually setting up a camera pipeline ====

We can roughly follow [https://www.kernel.org/doc/html/latest/admin-guide/media/rkisp1.html this guide]. Doing so gives us the following commands:
<pre>
"media-ctl" "-d" "platform:rkisp1" "-r"
"media-ctl" "-d" "platform:rkisp1" "-l" "'imx258 1-001a':0 -> 'rkisp1_isp':0 [1]"
"media-ctl" "-d" "platform:rkisp1" "-l" "'rkisp1_isp':2 -> 'rkisp1_resizer_selfpath':0 [1]"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_isp":0 [fmt:SGRBG10_1X10/1048x780]'
"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "width=1048,height=780,"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_resizer_selfpath":0 [fmt:SGRBG10_1X10/1048x780 crop:(0,0)/1048x780]'

"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "width=800,height=600,pixelformat=422P"
"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "--stream-mmap" "--stream-count" "1" "--verbose"
</pre>

Careful readers will note that this is setting the camera up with a low resolution compared to what the sensor can do. For some reason I can't get the rkisp1 node to configure its sink pad with the maximum resolution of the sensor (4208x3118). Instead, it sets itself to 4032x3024. This mismatch is forbidden and causes v4l2 to give us EPIPE when we try to enable the stream. Using the lower resolution gets us further--the last command (which attempts to save a single frame of video) gives output like this:
<pre>
VIDIOC_QUERYCAP: ok
VIDIOC_REQBUFS returned 0 (Success)
VIDIOC_QUERYBUF returned 0 (Success)
VIDIOC_QUERYBUF returned 0 (Success)
VIDIOC_QUERYBUF returned 0 (Success)
VIDIOC_QUERYBUF returned 0 (Success)
VIDIOC_QBUF returned 0 (Success)
VIDIOC_QBUF returned 0 (Success)
VIDIOC_QBUF returned 0 (Success)
VIDIOC_QBUF returned 0 (Success)
VIDIOC_STREAMON returned 0 (Success)
</pre>

But then it just hangs there indefinitely.

Running under strace, we see the program stuck in an ioctl:
<pre>
ioctl(3, VIDIOC_DQBUF, {type=V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE
</pre>

[https://www.kernel.org/doc/html/v4.13/media/uapi/v4l/vidioc-qbuf.html VIDIOC_DQBUF] corresponds to dequeuing a buffer once it has been filled with an image by the v4l2 device. Apparently, the device never gives us back a captured frame.

=== Now what? ===

==== Does i2c work at all? ====

The camera driver uses i2c; we can see the camera [https://github.com/megous/linux/blob/c5af9f4f874a66b65c73c450b79f6a86b1b46332/arch/arm64/boot/dts/rockchip/rk3399-pinephone-pro.dts#L890-L914 in the .dts] to determine that it's on i2c bus 1, address 0x1a.

The imx258 is controlled over i2c, and sends image data over MIPI. The first step is to see if the i2c control is working.

Looking back at the driver, it seems we should see errors in dmesg if i2c communication fails. The driver checks the chip id in its probe function, which uses i2c, so we can conclude that i2c gets set up properly initially. Does it keep working?

==== Does i2c keep working? (FTrace) ====

We can see if other functions that use i2c produce errors--for example, when we run our <code>"v4l2-ctl" ... "--stream-mmap"</code> command, we expect that <code>imx258_start_streaming</code> will get called, and this function also writes a load of registers over i2c.

Build and install a kernel with CONFIG_FTRACE (not hard, but outside the scope of this write-up) and enable function graph tracing:
<pre>
$ sudo su
# cd /sys/kernel/debug/tracing
# echo function_graph > current_tracer
# cat available_filter_functions | grep -E 'rkisp|imx258|v4l2' | awk '{print $1}' > set_graph_function
# echo 'vm_map_pages,__iommu_map,iommu_map_sg_atomic,rk_iommu_map,__alloc_pages,dma_alloc_attrs,dma_mmap_attrs,__vm_map_pages,vb2_mmap,clk_e,able,schedule_timeout,clk_disable,regmap_write,schedule,__i2c_tra,sfer,i2c_tra,sfer_buffer_flags,dma_free_attrs' | tr , '\n' > set_graph_notrace
# tee trace_pipe > ~/function-trace.log
</pre>

Now, in another terminal session, run the <code>"v4l2-ctl" ... "--stream-mmap"</code> command again. We end up with a bunch (but hopefully not *too* much) of output in our first terminal and the <code>~/function-trace.log</code> file.

A quick grep shows that <code>imx258_start_streaming</code> is indeed called, so '''i2c is working properly'''.

=== If i2c works, what doesn't? ===

Our program that asks for a video frame is still hung! We ask the camera to start streaming frames, it presumably does, but the v4l2 stack never tells us a frame has finished. Doing some digging in the v4l2 stack (and the rkisp1 driver), we find out that rkisp1 is notified about frame status via interrupts.

Perhaps the hardware is never actually emitting the interrupt that signals a frame being finished. Indeed, grepping our FTrace log shows that <code>rkisp1_isr</code> is never called. A quick look at <code>/proc/interrupts</code> shows that '''the only interrupt associated with the isp has never triggered''' (0 count on every CPU):
<pre>
# head -n1 /proc/interrupts; grep isp /proc/interrupts
CPU0 CPU1 CPU2 CPU3 CPU4 CPU5
57: 0 0 0 0 0 0 GICv3 75 Level ff914000.iommu, rkisp1
</pre>

=== <code>rkisp1</code> debugfs info ===

The <code>rkisp1</code> driver keeps some [https://github.com/megous/linux/blob/c5af9f4f874a66b65c73c450b79f6a86b1b46332/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h#L335-L363 error/debug counters] in debugfs, which can be viewed at <code>/sys/kernel/debug/ff910000.isp0</code>.

Sadly, the only one of these that comes up nonzero is <code>sp_stop_timeout</code> ("upon stream stop, the capture waits 1 second for the isr to stop the stream. This param is incremented in case of timeout"). I observe it to increment once per time I terminate the <code>"v4l2-ctl" ... "--stream-mmap"</code> command. This tells us what we already know: interrupts are not firing, but it also tells us that we aren't hitting any other errors that rkisp1 knows about.

== Next steps ==

Where do we go from here? I don't know--something may be wrong with the way the .dts specifies interrupts, some kind of firmware/GIC (the rk3399's interrupt controller) configuration issue, or something else. It would be good to try to determine:

* Whether the MIPI lines actually show traffic, if someone has a logic analyzer and a dissected PPP.
* Whether the DMA of frames to memory actually happens, regardless of (lack of) emission of the interrupt that tells us when said DMA finishes a frame.
* Whether the regulator messages we see during boot are actually significant. I don't know enough about the Linux regulator framework to say whether these being not-found means power is not correctly supplied, or if Linux just isn't being properly informed of the power supply requirements for precise tracking/power-saving when the camera is off.

== Megi saves the day ==

<pre>
[I] <megi> you might be interested in this https://megous.com/dl/tmp/0ae6ba03a17a3d53.png
[I] <megi> https://megous.com/git/linux/tag/?h=orange-pi-5.18-20220521-1759
</pre>

Turns out, the .dts had the wrong ISP connected to the sensor!

Apply [https://github.com/megous/linux/commit/9afd884f8b36121fb6097e77b6d35fe46ab48ad9 this commit] ([https://github.com/megous/linux/commit/9afd884f8b36121fb6097e77b6d35fe46ab48ad9.patch patch]) to your kernel tree, rebuild, and reinstall. Now, the following should produce an image (<code>frame.jpg</code>):

<pre>
"media-ctl" "-d" "platform:rkisp1" "-r"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"imx258 1-001a":0 [fmt:SGRBG10_1X10/1048x780]'
"media-ctl" "-d" "platform:rkisp1" "-l" "'imx258 1-001a':0 -> 'rkisp1_isp':0 [1]"
"media-ctl" "-d" "platform:rkisp1" "-l" "'rkisp1_isp':2 -> 'rkisp1_resizer_selfpath':0 [1]"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_isp":0 [fmt:SGRBG10_1X10/1048x780 crop:(0,0)/1048x780]'
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_isp":2 [fmt:YUYV8_2X8/1048x780 crop:(0,0)/1048x780]'
"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "width=1048,height=780,"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_resizer_selfpath":0 [fmt:YUYV8_2X8/1048x780 crop:(0,0)/1048x780]'

"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "width=1048,height=780,pixelformat=422P"
"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "--stream-mmap" "--stream-count" "1" "--stream-to=frame.raw"

ffmpeg -y -s:v 1048x780 -pix_fmt yuv422p -i frame.raw frame.jpg
</pre>

Similarly, you can take a photo from the front camera with the following:

<pre>
"media-ctl" "-d" "platform:rkisp1" "-r"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" "'m00_f_ov8858 1-0036':0 [fmt:SBGGR10_1X10/1632x1224]"
"media-ctl" "-d" "platform:rkisp1" "-l" "'m00_f_ov8858 1-0036':0 -> 'rkisp1_isp':0 [1]"
"media-ctl" "-d" "platform:rkisp1" "-l" "'rkisp1_isp':2 -> 'rkisp1_resizer_selfpath':0 [1]"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" "'rkisp1_isp':0 [fmt:SBGGR10_1X10/1632x1224 crop:(0,0)/1632x1224]"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" "'rkisp1_isp':2 [fmt:YUYV8_2X8/1632x1224 crop:(0,0)/1632x1224]"
"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "width=1632,height=1224,"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" "'rkisp1_resizer_selfpath':0 [fmt:YUYV8_2X8/1632x1224 crop:(0,0)/1632x1224]"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" "'rkisp1_resizer_selfpath':1 [fmt:YUYV8_2X8/1632x1224]"

"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "width=1632,height=1224,pixelformat=422P"
"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "--stream-mmap" "--stream-count" "1" "--stream-to=frame.raw"

ffmpeg -y -s:v 1048x780 -pix_fmt yuv422p -i frame.raw frame.jpg
</pre>

In some cases, the isp exposes 2 devices nodes, and the nodes have separate topologies with a different sensor attached to each. In this situation, you may need to reference the isp using the correct device node for the commands to work.

=== What's left? ===

* libcamera and megapixels still don't work. Megapixels appears to need support for debayering YUYV, and both appear to be configuring the rkisp1 pipeline wrong.
* The images captured are pretty green and seem to have two pixels of garbage at their bottom edge. rkisp1 supports auto white-balance (AWB) using the params/stats pads, and libcamera has some smarts that may hook these up properly. In the meantime, raw images can be postprocessed to improve white balance and exposure.

== Useful links ==

* [https://files.pine64.org/doc/PinePhonePro/PinephonePro-Schematic-V1.0-20211127.pdf Pinephone Pro Schematic]
* [https://blog.krybot.com/a?ID=01650-cea27a80-a1ab-4da1-9cb5-3945be91e3e1 Background on the GIC]
* [https://elinux.org/images/9/94/ISP-presentation.pdf Background on the rkisp1 ISP]

PinePhone Pro/IMX258 Camera Debugging

2022-06-13T17:04:48Z

Kgmt0: /* Megi saves the day */ Fix typo in the commands for front camera

{{Under construction}}

== Background ==

The Pinephone Pro's rear camera is a Sony IMX258. The camera driver is present upstream, and at least [https://gitlab.manjaro.org/manjaro-arm/packages/core/linux-pinephonepro Manjaro's kernel] (downstream of Megi's) enables the driver. The camera does not seem to work; this page catalogs work on figuring out why.

== Initial status ==

What do we see in dmesg?

<pre>
imx258 1-001a: supply vana not found, using dummy regulator
imx258 1-001a: supply vdig not found, using dummy regulator
</pre>

This tells us the kernel loads the driver, which is something.

Are these errors bad/fatal? Inspecting the [https://github.com/megous/linux/blob/orange-pi-5.17/drivers/media/i2c/imx258.c source] shows us that these supplies are defined in the <code>imx258_supply_names</code> array, and <code>imx258_probe</code> calls <code>devm_regulator_bulk_get</code> to set them up.
<code>imx258_probe</code> checks the return code from <code>devm_regulator_bulk_get</code>, emitting a fatal "Failed to get supplies" message if the return code is negative (indicating failure). Since we don't see this message (or any others from the driver), it looks like it loads successfully!

=== How can we test the camera? ===

==== Megapixels ====

[https://gitlab.com/postmarketOS/megapixels Megapixels] requires a per-device file to configure cameras. This one isn't complete (most of its numbers are complete nonsense), but it should let us see what happens when Megapixels tries to access the camera:
<code>pine64,pinephone-pro.ini</code>:
<pre>
[device]
make=PINE64
model=PinePhone Pro

[rear]
driver=imx258
media-driver=rkisp1
capture-width=4208
capture-height=3118
capture-rate=60
capture-fmt=GRBG10P
preview-width=1280
preview-height=960
preview-rate=60
#preview-fmt=BGGR8
capture-fmt=GRBG10P
rotate=90
mirrored=true
focallength=2.6
cropfactor=12.7
fnumber=2.8
flash-display=true
</pre>

This segfaulted Megapixels for me. After patching some trivial segfaults, I get fatal GTK4 issues and other tangential problems. Let's try something else.

==== Manually setting up a camera pipeline ====

We can roughly follow [https://www.kernel.org/doc/html/latest/admin-guide/media/rkisp1.html this guide]. Doing so gives us the following commands:
<pre>
"media-ctl" "-d" "platform:rkisp1" "-r"
"media-ctl" "-d" "platform:rkisp1" "-l" "'imx258 1-001a':0 -> 'rkisp1_isp':0 [1]"
"media-ctl" "-d" "platform:rkisp1" "-l" "'rkisp1_isp':2 -> 'rkisp1_resizer_selfpath':0 [1]"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_isp":0 [fmt:SGRBG10_1X10/1048x780]'
"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "width=1048,height=780,"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_resizer_selfpath":0 [fmt:SGRBG10_1X10/1048x780 crop:(0,0)/1048x780]'

"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "width=800,height=600,pixelformat=422P"
"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "--stream-mmap" "--stream-count" "1" "--verbose"
</pre>

Careful readers will note that this is setting the camera up with a low resolution compared to what the sensor can do. For some reason I can't get the rkisp1 node to configure its sink pad with the maximum resolution of the sensor (4208x3118). Instead, it sets itself to 4032x3024. This mismatch is forbidden and causes v4l2 to give us EPIPE when we try to enable the stream. Using the lower resolution gets us further--the last command (which attempts to save a single frame of video) gives output like this:
<pre>
VIDIOC_QUERYCAP: ok
VIDIOC_REQBUFS returned 0 (Success)
VIDIOC_QUERYBUF returned 0 (Success)
VIDIOC_QUERYBUF returned 0 (Success)
VIDIOC_QUERYBUF returned 0 (Success)
VIDIOC_QUERYBUF returned 0 (Success)
VIDIOC_QBUF returned 0 (Success)
VIDIOC_QBUF returned 0 (Success)
VIDIOC_QBUF returned 0 (Success)
VIDIOC_QBUF returned 0 (Success)
VIDIOC_STREAMON returned 0 (Success)
</pre>

But then it just hangs there indefinitely.

Running under strace, we see the program stuck in an ioctl:
<pre>
ioctl(3, VIDIOC_DQBUF, {type=V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE
</pre>

[https://www.kernel.org/doc/html/v4.13/media/uapi/v4l/vidioc-qbuf.html VIDIOC_DQBUF] corresponds to dequeuing a buffer once it has been filled with an image by the v4l2 device. Apparently, the device never gives us back a captured frame.

=== Now what? ===

==== Does i2c work at all? ====

The camera driver uses i2c; we can see the camera [https://github.com/megous/linux/blob/c5af9f4f874a66b65c73c450b79f6a86b1b46332/arch/arm64/boot/dts/rockchip/rk3399-pinephone-pro.dts#L890-L914 in the .dts] to determine that it's on i2c bus 1, address 0x1a.

The imx258 is controlled over i2c, and sends image data over MIPI. The first step is to see if the i2c control is working.

Looking back at the driver, it seems we should see errors in dmesg if i2c communication fails. The driver checks the chip id in its probe function, which uses i2c, so we can conclude that i2c gets set up properly initially. Does it keep working?

==== Does i2c keep working? (FTrace) ====

We can see if other functions that use i2c produce errors--for example, when we run our <code>"v4l2-ctl" ... "--stream-mmap"</code> command, we expect that <code>imx258_start_streaming</code> will get called, and this function also writes a load of registers over i2c.

Build and install a kernel with CONFIG_FTRACE (not hard, but outside the scope of this write-up) and enable function graph tracing:
<pre>
$ sudo su
# cd /sys/kernel/debug/tracing
# echo function_graph > current_tracer
# cat available_filter_functions | grep -E 'rkisp|imx258|v4l2' | awk '{print $1}' > set_graph_function
# echo 'vm_map_pages,__iommu_map,iommu_map_sg_atomic,rk_iommu_map,__alloc_pages,dma_alloc_attrs,dma_mmap_attrs,__vm_map_pages,vb2_mmap,clk_e,able,schedule_timeout,clk_disable,regmap_write,schedule,__i2c_tra,sfer,i2c_tra,sfer_buffer_flags,dma_free_attrs' | tr , '\n' > set_graph_notrace
# tee trace_pipe > ~/function-trace.log
</pre>

Now, in another terminal session, run the <code>"v4l2-ctl" ... "--stream-mmap"</code> command again. We end up with a bunch (but hopefully not *too* much) of output in our first terminal and the <code>~/function-trace.log</code> file.

A quick grep shows that <code>imx258_start_streaming</code> is indeed called, so '''i2c is working properly'''.

=== If i2c works, what doesn't? ===

Our program that asks for a video frame is still hung! We ask the camera to start streaming frames, it presumably does, but the v4l2 stack never tells us a frame has finished. Doing some digging in the v4l2 stack (and the rkisp1 driver), we find out that rkisp1 is notified about frame status via interrupts.

Perhaps the hardware is never actually emitting the interrupt that signals a frame being finished. Indeed, grepping our FTrace log shows that <code>rkisp1_isr</code> is never called. A quick look at <code>/proc/interrupts</code> shows that '''the only interrupt associated with the isp has never triggered''' (0 count on every CPU):
<pre>
# head -n1 /proc/interrupts; grep isp /proc/interrupts
CPU0 CPU1 CPU2 CPU3 CPU4 CPU5
57: 0 0 0 0 0 0 GICv3 75 Level ff914000.iommu, rkisp1
</pre>

=== <code>rkisp1</code> debugfs info ===

The <code>rkisp1</code> driver keeps some [https://github.com/megous/linux/blob/c5af9f4f874a66b65c73c450b79f6a86b1b46332/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h#L335-L363 error/debug counters] in debugfs, which can be viewed at <code>/sys/kernel/debug/ff910000.isp0</code>.

Sadly, the only one of these that comes up nonzero is <code>sp_stop_timeout</code> ("upon stream stop, the capture waits 1 second for the isr to stop the stream. This param is incremented in case of timeout"). I observe it to increment once per time I terminate the <code>"v4l2-ctl" ... "--stream-mmap"</code> command. This tells us what we already know: interrupts are not firing, but it also tells us that we aren't hitting any other errors that rkisp1 knows about.

== Next steps ==

Where do we go from here? I don't know--something may be wrong with the way the .dts specifies interrupts, some kind of firmware/GIC (the rk3399's interrupt controller) configuration issue, or something else. It would be good to try to determine:

* Whether the MIPI lines actually show traffic, if someone has a logic analyzer and a dissected PPP.
* Whether the DMA of frames to memory actually happens, regardless of (lack of) emission of the interrupt that tells us when said DMA finishes a frame.
* Whether the regulator messages we see during boot are actually significant. I don't know enough about the Linux regulator framework to say whether these being not-found means power is not correctly supplied, or if Linux just isn't being properly informed of the power supply requirements for precise tracking/power-saving when the camera is off.

== Megi saves the day ==

<pre>
[I] <megi> you might be interested in this https://megous.com/dl/tmp/0ae6ba03a17a3d53.png
[I] <megi> https://megous.com/git/linux/tag/?h=orange-pi-5.18-20220521-1759
</pre>

Turns out, the .dts had the wrong ISP connected to the sensor!

Apply [https://github.com/megous/linux/commit/9afd884f8b36121fb6097e77b6d35fe46ab48ad9 this commit] ([https://github.com/megous/linux/commit/9afd884f8b36121fb6097e77b6d35fe46ab48ad9.patch patch]) to your kernel tree, rebuild, and reinstall. Now, the following should produce an image (<code>frame.jpg</code>):

<pre>
"media-ctl" "-d" "platform:rkisp1" "-r"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"imx258 1-001a":0 [fmt:SGRBG10_1X10/1048x780]'
"media-ctl" "-d" "platform:rkisp1" "-l" "'imx258 1-001a':0 -> 'rkisp1_isp':0 [1]"
"media-ctl" "-d" "platform:rkisp1" "-l" "'rkisp1_isp':2 -> 'rkisp1_resizer_selfpath':0 [1]"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_isp":0 [fmt:SGRBG10_1X10/1048x780 crop:(0,0)/1048x780]'
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_isp":2 [fmt:YUYV8_2X8/1048x780 crop:(0,0)/1048x780]'
"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "width=1048,height=780,"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_resizer_selfpath":0 [fmt:YUYV8_2X8/1048x780 crop:(0,0)/1048x780]'

"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "width=1048,height=780,pixelformat=422P"
"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "--stream-mmap" "--stream-count" "1" "--stream-to=frame.raw"

ffmpeg -y -s:v 1048x780 -pix_fmt yuv422p -i frame.raw frame.jpg
</pre>

Similarly, you can take a photo from the front camera with the following:

<pre>
"media-ctl" "-d" "platform:rkisp1" "-r"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" "'m00_f_ov8858 1-0036':0 [fmt:SBGGR10_1X10/1632x1224]"
"media-ctl" "-d" "platform:rkisp1" "-l" "'m00_f_ov8858 1-0036':0 -> 'rkisp1_isp':0 [1]"
"media-ctl" "-d" "platform:rkisp1" "-l" "'rkisp1_isp':2 -> 'rkisp1_resizer_selfpath':0 [1]"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" "'rkisp1_isp':0 [fmt:SBGGR10_1X10/1632x1224 crop:(0,0)/1632x1224]"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" "'rkisp1_isp':2 [fmt:YUYV8_2X8/1632x1224 crop:(0,0)/1632x1224]"
"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "width=1632,height=1224,"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" "'rkisp1_resizer_selfpath':0 [fmt:YUYV8_2X8/1632x1224 crop:(0,0)/1632x1224]"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" "'rkisp1_resizer_selfpath':1 [fmt:YUYV8_2X8/1632x1224]"

"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "width=1632,height=1224,pixelformat=422P"
"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "--stream-mmap" "--stream-count" "1" "--stream-to=frame.raw"

ffmpeg -y -s:v 1048x780 -pix_fmt yuv422p -i frame.raw frame.jpg
</pre>

In some cases, the isp exposes 2 devices nodes, and the nodes have separate topologies with a different sensor attached to each. In this situation, you may need to reference the isp using the correct device node for the commands to work.

=== What's left? ===

* libcamera and megapixels still don't work. Megapixels appears to need support for debayering YUYV, and both appear to be configuring the rkisp1 pipeline wrong.
* The images captured are pretty green and seem to have two pixels of garbage at their bottom edge. rkisp1 supports auto white-balance (AWB) using the params/stats pads, and libcamera has some smarts that may hook these up properly. In the meantime, raw images can be postprocessed to improve white balance and exposure.

== Useful links ==

* [https://files.pine64.org/doc/PinePhonePro/PinephonePro-Schematic-V1.0-20211127.pdf Pinephone Pro Schematic]
* [https://blog.krybot.com/a?ID=01650-cea27a80-a1ab-4da1-9cb5-3945be91e3e1 Background on the GIC]
* [https://elinux.org/images/9/94/ISP-presentation.pdf Background on the rkisp1 ISP]

PinePhone Pro/IMX258 Camera Debugging

2022-06-13T17:01:59Z

Kgmt0: /* Megi saves the day */ Add info about the case where the isp has 2 device nodes

{{Under construction}}

== Background ==

The Pinephone Pro's rear camera is a Sony IMX258. The camera driver is present upstream, and at least [https://gitlab.manjaro.org/manjaro-arm/packages/core/linux-pinephonepro Manjaro's kernel] (downstream of Megi's) enables the driver. The camera does not seem to work; this page catalogs work on figuring out why.

== Initial status ==

What do we see in dmesg?

<pre>
imx258 1-001a: supply vana not found, using dummy regulator
imx258 1-001a: supply vdig not found, using dummy regulator
</pre>

This tells us the kernel loads the driver, which is something.

Are these errors bad/fatal? Inspecting the [https://github.com/megous/linux/blob/orange-pi-5.17/drivers/media/i2c/imx258.c source] shows us that these supplies are defined in the <code>imx258_supply_names</code> array, and <code>imx258_probe</code> calls <code>devm_regulator_bulk_get</code> to set them up.
<code>imx258_probe</code> checks the return code from <code>devm_regulator_bulk_get</code>, emitting a fatal "Failed to get supplies" message if the return code is negative (indicating failure). Since we don't see this message (or any others from the driver), it looks like it loads successfully!

=== How can we test the camera? ===

==== Megapixels ====

[https://gitlab.com/postmarketOS/megapixels Megapixels] requires a per-device file to configure cameras. This one isn't complete (most of its numbers are complete nonsense), but it should let us see what happens when Megapixels tries to access the camera:
<code>pine64,pinephone-pro.ini</code>:
<pre>
[device]
make=PINE64
model=PinePhone Pro

[rear]
driver=imx258
media-driver=rkisp1
capture-width=4208
capture-height=3118
capture-rate=60
capture-fmt=GRBG10P
preview-width=1280
preview-height=960
preview-rate=60
#preview-fmt=BGGR8
capture-fmt=GRBG10P
rotate=90
mirrored=true
focallength=2.6
cropfactor=12.7
fnumber=2.8
flash-display=true
</pre>

This segfaulted Megapixels for me. After patching some trivial segfaults, I get fatal GTK4 issues and other tangential problems. Let's try something else.

==== Manually setting up a camera pipeline ====

We can roughly follow [https://www.kernel.org/doc/html/latest/admin-guide/media/rkisp1.html this guide]. Doing so gives us the following commands:
<pre>
"media-ctl" "-d" "platform:rkisp1" "-r"
"media-ctl" "-d" "platform:rkisp1" "-l" "'imx258 1-001a':0 -> 'rkisp1_isp':0 [1]"
"media-ctl" "-d" "platform:rkisp1" "-l" "'rkisp1_isp':2 -> 'rkisp1_resizer_selfpath':0 [1]"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_isp":0 [fmt:SGRBG10_1X10/1048x780]'
"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "width=1048,height=780,"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_resizer_selfpath":0 [fmt:SGRBG10_1X10/1048x780 crop:(0,0)/1048x780]'

"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "width=800,height=600,pixelformat=422P"
"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "--stream-mmap" "--stream-count" "1" "--verbose"
</pre>

Careful readers will note that this is setting the camera up with a low resolution compared to what the sensor can do. For some reason I can't get the rkisp1 node to configure its sink pad with the maximum resolution of the sensor (4208x3118). Instead, it sets itself to 4032x3024. This mismatch is forbidden and causes v4l2 to give us EPIPE when we try to enable the stream. Using the lower resolution gets us further--the last command (which attempts to save a single frame of video) gives output like this:
<pre>
VIDIOC_QUERYCAP: ok
VIDIOC_REQBUFS returned 0 (Success)
VIDIOC_QUERYBUF returned 0 (Success)
VIDIOC_QUERYBUF returned 0 (Success)
VIDIOC_QUERYBUF returned 0 (Success)
VIDIOC_QUERYBUF returned 0 (Success)
VIDIOC_QBUF returned 0 (Success)
VIDIOC_QBUF returned 0 (Success)
VIDIOC_QBUF returned 0 (Success)
VIDIOC_QBUF returned 0 (Success)
VIDIOC_STREAMON returned 0 (Success)
</pre>

But then it just hangs there indefinitely.

Running under strace, we see the program stuck in an ioctl:
<pre>
ioctl(3, VIDIOC_DQBUF, {type=V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE
</pre>

[https://www.kernel.org/doc/html/v4.13/media/uapi/v4l/vidioc-qbuf.html VIDIOC_DQBUF] corresponds to dequeuing a buffer once it has been filled with an image by the v4l2 device. Apparently, the device never gives us back a captured frame.

=== Now what? ===

==== Does i2c work at all? ====

The camera driver uses i2c; we can see the camera [https://github.com/megous/linux/blob/c5af9f4f874a66b65c73c450b79f6a86b1b46332/arch/arm64/boot/dts/rockchip/rk3399-pinephone-pro.dts#L890-L914 in the .dts] to determine that it's on i2c bus 1, address 0x1a.

The imx258 is controlled over i2c, and sends image data over MIPI. The first step is to see if the i2c control is working.

Looking back at the driver, it seems we should see errors in dmesg if i2c communication fails. The driver checks the chip id in its probe function, which uses i2c, so we can conclude that i2c gets set up properly initially. Does it keep working?

==== Does i2c keep working? (FTrace) ====

We can see if other functions that use i2c produce errors--for example, when we run our <code>"v4l2-ctl" ... "--stream-mmap"</code> command, we expect that <code>imx258_start_streaming</code> will get called, and this function also writes a load of registers over i2c.

Build and install a kernel with CONFIG_FTRACE (not hard, but outside the scope of this write-up) and enable function graph tracing:
<pre>
$ sudo su
# cd /sys/kernel/debug/tracing
# echo function_graph > current_tracer
# cat available_filter_functions | grep -E 'rkisp|imx258|v4l2' | awk '{print $1}' > set_graph_function
# echo 'vm_map_pages,__iommu_map,iommu_map_sg_atomic,rk_iommu_map,__alloc_pages,dma_alloc_attrs,dma_mmap_attrs,__vm_map_pages,vb2_mmap,clk_e,able,schedule_timeout,clk_disable,regmap_write,schedule,__i2c_tra,sfer,i2c_tra,sfer_buffer_flags,dma_free_attrs' | tr , '\n' > set_graph_notrace
# tee trace_pipe > ~/function-trace.log
</pre>

Now, in another terminal session, run the <code>"v4l2-ctl" ... "--stream-mmap"</code> command again. We end up with a bunch (but hopefully not *too* much) of output in our first terminal and the <code>~/function-trace.log</code> file.

A quick grep shows that <code>imx258_start_streaming</code> is indeed called, so '''i2c is working properly'''.

=== If i2c works, what doesn't? ===

Our program that asks for a video frame is still hung! We ask the camera to start streaming frames, it presumably does, but the v4l2 stack never tells us a frame has finished. Doing some digging in the v4l2 stack (and the rkisp1 driver), we find out that rkisp1 is notified about frame status via interrupts.

Perhaps the hardware is never actually emitting the interrupt that signals a frame being finished. Indeed, grepping our FTrace log shows that <code>rkisp1_isr</code> is never called. A quick look at <code>/proc/interrupts</code> shows that '''the only interrupt associated with the isp has never triggered''' (0 count on every CPU):
<pre>
# head -n1 /proc/interrupts; grep isp /proc/interrupts
CPU0 CPU1 CPU2 CPU3 CPU4 CPU5
57: 0 0 0 0 0 0 GICv3 75 Level ff914000.iommu, rkisp1
</pre>

=== <code>rkisp1</code> debugfs info ===

The <code>rkisp1</code> driver keeps some [https://github.com/megous/linux/blob/c5af9f4f874a66b65c73c450b79f6a86b1b46332/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h#L335-L363 error/debug counters] in debugfs, which can be viewed at <code>/sys/kernel/debug/ff910000.isp0</code>.

Sadly, the only one of these that comes up nonzero is <code>sp_stop_timeout</code> ("upon stream stop, the capture waits 1 second for the isr to stop the stream. This param is incremented in case of timeout"). I observe it to increment once per time I terminate the <code>"v4l2-ctl" ... "--stream-mmap"</code> command. This tells us what we already know: interrupts are not firing, but it also tells us that we aren't hitting any other errors that rkisp1 knows about.

== Next steps ==

Where do we go from here? I don't know--something may be wrong with the way the .dts specifies interrupts, some kind of firmware/GIC (the rk3399's interrupt controller) configuration issue, or something else. It would be good to try to determine:

* Whether the MIPI lines actually show traffic, if someone has a logic analyzer and a dissected PPP.
* Whether the DMA of frames to memory actually happens, regardless of (lack of) emission of the interrupt that tells us when said DMA finishes a frame.
* Whether the regulator messages we see during boot are actually significant. I don't know enough about the Linux regulator framework to say whether these being not-found means power is not correctly supplied, or if Linux just isn't being properly informed of the power supply requirements for precise tracking/power-saving when the camera is off.

== Megi saves the day ==

<pre>
[I] <megi> you might be interested in this https://megous.com/dl/tmp/0ae6ba03a17a3d53.png
[I] <megi> https://megous.com/git/linux/tag/?h=orange-pi-5.18-20220521-1759
</pre>

Turns out, the .dts had the wrong ISP connected to the sensor!

Apply [https://github.com/megous/linux/commit/9afd884f8b36121fb6097e77b6d35fe46ab48ad9 this commit] ([https://github.com/megous/linux/commit/9afd884f8b36121fb6097e77b6d35fe46ab48ad9.patch patch]) to your kernel tree, rebuild, and reinstall. Now, the following should produce an image (<code>frame.jpg</code>):

<pre>
"media-ctl" "-d" "platform:rkisp1" "-r"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"imx258 1-001a":0 [fmt:SGRBG10_1X10/1048x780]'
"media-ctl" "-d" "platform:rkisp1" "-l" "'imx258 1-001a':0 -> 'rkisp1_isp':0 [1]"
"media-ctl" "-d" "platform:rkisp1" "-l" "'rkisp1_isp':2 -> 'rkisp1_resizer_selfpath':0 [1]"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_isp":0 [fmt:SGRBG10_1X10/1048x780 crop:(0,0)/1048x780]'
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_isp":2 [fmt:YUYV8_2X8/1048x780 crop:(0,0)/1048x780]'
"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "width=1048,height=780,"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_resizer_selfpath":0 [fmt:YUYV8_2X8/1048x780 crop:(0,0)/1048x780]'

"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "width=1048,height=780,pixelformat=422P"
"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "--stream-mmap" "--stream-count" "1" "--stream-to=frame.raw"

ffmpeg -y -s:v 1048x780 -pix_fmt yuv422p -i frame.raw frame.jpg
</pre>

Similarly, you can take a photo from the front camera with the following:

<pre>
"media-ctl" "-d" "pltform:rkisp1" "-r"
"media-ctl" "-d" "pltform:rkisp1" "--set-v4l2" "'m00_f_ov8858 1-0036':0 [fmt:SBGGR10_1X10/1632x1224]"
"media-ctl" "-d" "pltform:rkisp1" "-l" "'m00_f_ov8858 1-0036':0 -> 'rkisp1_isp':0 [1]"
"media-ctl" "-d" "pltform:rkisp1" "-l" "'rkisp1_isp':2 -> 'rkisp1_resizer_selfpath':0 [1]"
"media-ctl" "-d" "pltform:rkisp1" "--set-v4l2" "'rkisp1_isp':0 [fmt:SBGGR10_1X10/1632x1224 crop:(0,0)/1632x1224]"
"media-ctl" "-d" "pltform:rkisp1" "--set-v4l2" "'rkisp1_isp':2 [fmt:YUYV8_2X8/1632x1224 crop:(0,0)/1632x1224]"
"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "width=1632,height=1224,"
"media-ctl" "-d" "pltform:rkisp1" "--set-v4l2" "'rkisp1_resizer_selfpath':0 [fmt:YUYV8_2X8/1632x1224 crop:(0,0)/1632x1224]"
"media-ctl" "-d" "pltform:rkisp1" "--set-v4l2" "'rkisp1_resizer_selfpath':1 [fmt:YUYV8_2X8/1632x1224]"

"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "width=1632,height=1224,pixelformat=422P"
"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "--stream-mmap" "--stream-count" "1" "--stream-to=frame.raw"

ffmpeg -y -s:v 1048x780 -pix_fmt yuv422p -i frame.raw frame.jpg
</pre>

In some cases, the isp exposes 2 devices nodes, and the nodes have separate topologies with a different sensor attached to each. In this situation, you may need to reference the isp using the correct device node for the commands to work.

=== What's left? ===

* libcamera and megapixels still don't work. Megapixels appears to need support for debayering YUYV, and both appear to be configuring the rkisp1 pipeline wrong.
* The images captured are pretty green and seem to have two pixels of garbage at their bottom edge. rkisp1 supports auto white-balance (AWB) using the params/stats pads, and libcamera has some smarts that may hook these up properly. In the meantime, raw images can be postprocessed to improve white balance and exposure.

== Useful links ==

* [https://files.pine64.org/doc/PinePhonePro/PinephonePro-Schematic-V1.0-20211127.pdf Pinephone Pro Schematic]
* [https://blog.krybot.com/a?ID=01650-cea27a80-a1ab-4da1-9cb5-3945be91e3e1 Background on the GIC]
* [https://elinux.org/images/9/94/ISP-presentation.pdf Background on the rkisp1 ISP]

PinePhone Pro/IMX258 Camera Debugging

2022-06-13T16:48:31Z

Kgmt0: /* Megi saves the day */ Add commands for front camera

{{Under construction}}

== Background ==

The Pinephone Pro's rear camera is a Sony IMX258. The camera driver is present upstream, and at least [https://gitlab.manjaro.org/manjaro-arm/packages/core/linux-pinephonepro Manjaro's kernel] (downstream of Megi's) enables the driver. The camera does not seem to work; this page catalogs work on figuring out why.

== Initial status ==

What do we see in dmesg?

<pre>
imx258 1-001a: supply vana not found, using dummy regulator
imx258 1-001a: supply vdig not found, using dummy regulator
</pre>

This tells us the kernel loads the driver, which is something.

Are these errors bad/fatal? Inspecting the [https://github.com/megous/linux/blob/orange-pi-5.17/drivers/media/i2c/imx258.c source] shows us that these supplies are defined in the <code>imx258_supply_names</code> array, and <code>imx258_probe</code> calls <code>devm_regulator_bulk_get</code> to set them up.
<code>imx258_probe</code> checks the return code from <code>devm_regulator_bulk_get</code>, emitting a fatal "Failed to get supplies" message if the return code is negative (indicating failure). Since we don't see this message (or any others from the driver), it looks like it loads successfully!

=== How can we test the camera? ===

==== Megapixels ====

[https://gitlab.com/postmarketOS/megapixels Megapixels] requires a per-device file to configure cameras. This one isn't complete (most of its numbers are complete nonsense), but it should let us see what happens when Megapixels tries to access the camera:
<code>pine64,pinephone-pro.ini</code>:
<pre>
[device]
make=PINE64
model=PinePhone Pro

[rear]
driver=imx258
media-driver=rkisp1
capture-width=4208
capture-height=3118
capture-rate=60
capture-fmt=GRBG10P
preview-width=1280
preview-height=960
preview-rate=60
#preview-fmt=BGGR8
capture-fmt=GRBG10P
rotate=90
mirrored=true
focallength=2.6
cropfactor=12.7
fnumber=2.8
flash-display=true
</pre>

This segfaulted Megapixels for me. After patching some trivial segfaults, I get fatal GTK4 issues and other tangential problems. Let's try something else.

==== Manually setting up a camera pipeline ====

We can roughly follow [https://www.kernel.org/doc/html/latest/admin-guide/media/rkisp1.html this guide]. Doing so gives us the following commands:
<pre>
"media-ctl" "-d" "platform:rkisp1" "-r"
"media-ctl" "-d" "platform:rkisp1" "-l" "'imx258 1-001a':0 -> 'rkisp1_isp':0 [1]"
"media-ctl" "-d" "platform:rkisp1" "-l" "'rkisp1_isp':2 -> 'rkisp1_resizer_selfpath':0 [1]"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_isp":0 [fmt:SGRBG10_1X10/1048x780]'
"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "width=1048,height=780,"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_resizer_selfpath":0 [fmt:SGRBG10_1X10/1048x780 crop:(0,0)/1048x780]'

"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "width=800,height=600,pixelformat=422P"
"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "--stream-mmap" "--stream-count" "1" "--verbose"
</pre>

Careful readers will note that this is setting the camera up with a low resolution compared to what the sensor can do. For some reason I can't get the rkisp1 node to configure its sink pad with the maximum resolution of the sensor (4208x3118). Instead, it sets itself to 4032x3024. This mismatch is forbidden and causes v4l2 to give us EPIPE when we try to enable the stream. Using the lower resolution gets us further--the last command (which attempts to save a single frame of video) gives output like this:
<pre>
VIDIOC_QUERYCAP: ok
VIDIOC_REQBUFS returned 0 (Success)
VIDIOC_QUERYBUF returned 0 (Success)
VIDIOC_QUERYBUF returned 0 (Success)
VIDIOC_QUERYBUF returned 0 (Success)
VIDIOC_QUERYBUF returned 0 (Success)
VIDIOC_QBUF returned 0 (Success)
VIDIOC_QBUF returned 0 (Success)
VIDIOC_QBUF returned 0 (Success)
VIDIOC_QBUF returned 0 (Success)
VIDIOC_STREAMON returned 0 (Success)
</pre>

But then it just hangs there indefinitely.

Running under strace, we see the program stuck in an ioctl:
<pre>
ioctl(3, VIDIOC_DQBUF, {type=V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE
</pre>

[https://www.kernel.org/doc/html/v4.13/media/uapi/v4l/vidioc-qbuf.html VIDIOC_DQBUF] corresponds to dequeuing a buffer once it has been filled with an image by the v4l2 device. Apparently, the device never gives us back a captured frame.

=== Now what? ===

==== Does i2c work at all? ====

The camera driver uses i2c; we can see the camera [https://github.com/megous/linux/blob/c5af9f4f874a66b65c73c450b79f6a86b1b46332/arch/arm64/boot/dts/rockchip/rk3399-pinephone-pro.dts#L890-L914 in the .dts] to determine that it's on i2c bus 1, address 0x1a.

The imx258 is controlled over i2c, and sends image data over MIPI. The first step is to see if the i2c control is working.

Looking back at the driver, it seems we should see errors in dmesg if i2c communication fails. The driver checks the chip id in its probe function, which uses i2c, so we can conclude that i2c gets set up properly initially. Does it keep working?

==== Does i2c keep working? (FTrace) ====

We can see if other functions that use i2c produce errors--for example, when we run our <code>"v4l2-ctl" ... "--stream-mmap"</code> command, we expect that <code>imx258_start_streaming</code> will get called, and this function also writes a load of registers over i2c.

Build and install a kernel with CONFIG_FTRACE (not hard, but outside the scope of this write-up) and enable function graph tracing:
<pre>
$ sudo su
# cd /sys/kernel/debug/tracing
# echo function_graph > current_tracer
# cat available_filter_functions | grep -E 'rkisp|imx258|v4l2' | awk '{print $1}' > set_graph_function
# echo 'vm_map_pages,__iommu_map,iommu_map_sg_atomic,rk_iommu_map,__alloc_pages,dma_alloc_attrs,dma_mmap_attrs,__vm_map_pages,vb2_mmap,clk_e,able,schedule_timeout,clk_disable,regmap_write,schedule,__i2c_tra,sfer,i2c_tra,sfer_buffer_flags,dma_free_attrs' | tr , '\n' > set_graph_notrace
# tee trace_pipe > ~/function-trace.log
</pre>

Now, in another terminal session, run the <code>"v4l2-ctl" ... "--stream-mmap"</code> command again. We end up with a bunch (but hopefully not *too* much) of output in our first terminal and the <code>~/function-trace.log</code> file.

A quick grep shows that <code>imx258_start_streaming</code> is indeed called, so '''i2c is working properly'''.

=== If i2c works, what doesn't? ===

Our program that asks for a video frame is still hung! We ask the camera to start streaming frames, it presumably does, but the v4l2 stack never tells us a frame has finished. Doing some digging in the v4l2 stack (and the rkisp1 driver), we find out that rkisp1 is notified about frame status via interrupts.

Perhaps the hardware is never actually emitting the interrupt that signals a frame being finished. Indeed, grepping our FTrace log shows that <code>rkisp1_isr</code> is never called. A quick look at <code>/proc/interrupts</code> shows that '''the only interrupt associated with the isp has never triggered''' (0 count on every CPU):
<pre>
# head -n1 /proc/interrupts; grep isp /proc/interrupts
CPU0 CPU1 CPU2 CPU3 CPU4 CPU5
57: 0 0 0 0 0 0 GICv3 75 Level ff914000.iommu, rkisp1
</pre>

=== <code>rkisp1</code> debugfs info ===

The <code>rkisp1</code> driver keeps some [https://github.com/megous/linux/blob/c5af9f4f874a66b65c73c450b79f6a86b1b46332/drivers/media/platform/rockchip/rkisp1/rkisp1-common.h#L335-L363 error/debug counters] in debugfs, which can be viewed at <code>/sys/kernel/debug/ff910000.isp0</code>.

Sadly, the only one of these that comes up nonzero is <code>sp_stop_timeout</code> ("upon stream stop, the capture waits 1 second for the isr to stop the stream. This param is incremented in case of timeout"). I observe it to increment once per time I terminate the <code>"v4l2-ctl" ... "--stream-mmap"</code> command. This tells us what we already know: interrupts are not firing, but it also tells us that we aren't hitting any other errors that rkisp1 knows about.

== Next steps ==

Where do we go from here? I don't know--something may be wrong with the way the .dts specifies interrupts, some kind of firmware/GIC (the rk3399's interrupt controller) configuration issue, or something else. It would be good to try to determine:

* Whether the MIPI lines actually show traffic, if someone has a logic analyzer and a dissected PPP.
* Whether the DMA of frames to memory actually happens, regardless of (lack of) emission of the interrupt that tells us when said DMA finishes a frame.
* Whether the regulator messages we see during boot are actually significant. I don't know enough about the Linux regulator framework to say whether these being not-found means power is not correctly supplied, or if Linux just isn't being properly informed of the power supply requirements for precise tracking/power-saving when the camera is off.

== Megi saves the day ==

<pre>
[I] <megi> you might be interested in this https://megous.com/dl/tmp/0ae6ba03a17a3d53.png
[I] <megi> https://megous.com/git/linux/tag/?h=orange-pi-5.18-20220521-1759
</pre>

Turns out, the .dts had the wrong ISP connected to the sensor!

Apply [https://github.com/megous/linux/commit/9afd884f8b36121fb6097e77b6d35fe46ab48ad9 this commit] ([https://github.com/megous/linux/commit/9afd884f8b36121fb6097e77b6d35fe46ab48ad9.patch patch]) to your kernel tree, rebuild, and reinstall. Now, the following should produce an image (<code>frame.jpg</code>):

<pre>
"media-ctl" "-d" "platform:rkisp1" "-r"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"imx258 1-001a":0 [fmt:SGRBG10_1X10/1048x780]'
"media-ctl" "-d" "platform:rkisp1" "-l" "'imx258 1-001a':0 -> 'rkisp1_isp':0 [1]"
"media-ctl" "-d" "platform:rkisp1" "-l" "'rkisp1_isp':2 -> 'rkisp1_resizer_selfpath':0 [1]"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_isp":0 [fmt:SGRBG10_1X10/1048x780 crop:(0,0)/1048x780]'
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_isp":2 [fmt:YUYV8_2X8/1048x780 crop:(0,0)/1048x780]'
"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "width=1048,height=780,"
"media-ctl" "-d" "platform:rkisp1" "--set-v4l2" '"rkisp1_resizer_selfpath":0 [fmt:YUYV8_2X8/1048x780 crop:(0,0)/1048x780]'

"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "width=1048,height=780,pixelformat=422P"
"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "--stream-mmap" "--stream-count" "1" "--stream-to=frame.raw"

ffmpeg -y -s:v 1048x780 -pix_fmt yuv422p -i frame.raw frame.jpg
</pre>

Similarly, you can take a photo from the front camera with the following:

<pre>
"media-ctl" "-d" "pltform:rkisp1" "-r"
"media-ctl" "-d" "pltform:rkisp1" "--set-v4l2" "'m00_f_ov8858 1-0036':0 [fmt:SBGGR10_1X10/1632x1224]"
"media-ctl" "-d" "pltform:rkisp1" "-l" "'m00_f_ov8858 1-0036':0 -> 'rkisp1_isp':0 [1]"
"media-ctl" "-d" "pltform:rkisp1" "-l" "'rkisp1_isp':2 -> 'rkisp1_resizer_selfpath':0 [1]"
"media-ctl" "-d" "pltform:rkisp1" "--set-v4l2" "'rkisp1_isp':0 [fmt:SBGGR10_1X10/1632x1224 crop:(0,0)/1632x1224]"
"media-ctl" "-d" "pltform:rkisp1" "--set-v4l2" "'rkisp1_isp':2 [fmt:YUYV8_2X8/1632x1224 crop:(0,0)/1632x1224]"
"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "width=1632,height=1224,"
"media-ctl" "-d" "pltform:rkisp1" "--set-v4l2" "'rkisp1_resizer_selfpath':0 [fmt:YUYV8_2X8/1632x1224 crop:(0,0)/1632x1224]"
"media-ctl" "-d" "pltform:rkisp1" "--set-v4l2" "'rkisp1_resizer_selfpath':1 [fmt:YUYV8_2X8/1632x1224]"

"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "-v" "width=1632,height=1224,pixelformat=422P"
"v4l2-ctl" "-z" "platform:rkisp1" "-d" "rkisp1_selfpath" "--stream-mmap" "--stream-count" "1" "--stream-to=frame.raw"

ffmpeg -y -s:v 1048x780 -pix_fmt yuv422p -i frame.raw frame.jpg
</pre>

=== What's left? ===

* libcamera and megapixels still don't work. Megapixels appears to need support for debayering YUYV, and both appear to be configuring the rkisp1 pipeline wrong.
* The images captured are pretty green and seem to have two pixels of garbage at their bottom edge. rkisp1 supports auto white-balance (AWB) using the params/stats pads, and libcamera has some smarts that may hook these up properly. In the meantime, raw images can be postprocessed to improve white balance and exposure.

== Useful links ==

* [https://files.pine64.org/doc/PinePhonePro/PinephonePro-Schematic-V1.0-20211127.pdf Pinephone Pro Schematic]
* [https://blog.krybot.com/a?ID=01650-cea27a80-a1ab-4da1-9cb5-3945be91e3e1 Background on the GIC]
* [https://elinux.org/images/9/94/ISP-presentation.pdf Background on the rkisp1 ISP]

PinePhone Pro Software State

2022-06-13T16:31:37Z

Kgmt0: Update front camera status

Presently the PinePhone Pro Explorer Edition is aimed at '''Linux developers with an extensive knowledge of embedded systems and/or experience with mobile Linux'''. It will take time for all the PinePhone Pro’s functionality to reach software parity with the original PinePhone and for mobile operating systems, in more general, to reach a higher degree of maturity.

Bear in mind that the software for these smartphones is still in a very early stage, with most of the software being in alpha or beta state. That's especially also the case for scalability of applications, their availability and practicability, any hardware function implementations and the firmware. The software is provided as is. There is no warranty for the software, not even for merchantability or fitness for a particular purpose.

The following table lists the feature functionality status of the unaltered pre-installed factory image of the current shipping batch and as comparison an up-to-date reference image (no responsibility is accepted for the accuracy of this information, the list is provided and updated by the community). If you have any questions regarding the current state of the software or of specific features working, please don't hesitate to ask in the [[Main Page#Community and Support|community chat]] '''before buying the device''':

* Discord: ''#pinephone'' under https://discord.gg/pine64
* IRC: ''#pinephone'' on ''irc.pine64.org''. Note: please consider Matrix, Discord or Telegram due to the volatile nature of IRC
* Matrix: https://app.element.io/#/room/#pinephone:matrix.org
* Telegram: https://t.me/pinephone

{{Hint|The software is '''written by the community''', any contributions towards the community projects are greatly appreciated! Please see "[[How to Contribute]]" to learn about how to contribute to the software projects and "[[Where to Report Bugs]]" to learn about where to report bugs.}}

{| class="wikitable plainrowheaders" border="1"
! scope="col" | Functionality
! scope="col" style="min-width: 260px;" | Component
! scope="col" style="width: 120px;" | Status (factory)¹
! scope="col" style="width: 120px;" | Status (updated)²
! scope="col" | Notes
|-
! scope="row" rowspan="3" | Bootloader
| <code>U-Boot</code>
| colspan="2" style="background:lightcoral; text-align:center;" | Critical bugs
| Some critical software bugs currently not fixed yet: A drained battery causes U-Boot to bootloop (cause is known and currently being worked on);[[PinePhone_Pro#The battery is fully drained|Workaround]] Occasionally the eMMC fails to initialize[https://gitlab.com/postmarketOS/pmaports/-/issues/1424 Report]
|-
| <code>SPI</code>
| colspan="2" style="background:lightcoral; text-align:center;" | Unfinished
| SPI memory is currently empty in the Explorer Edition batch, a corrupted installation on the eMMC has a higher boot priority than a fresh microSD card installation, leading to a involved recovery procedure[https://tow-boot.org/ Planned]
|-
| <code>Boot GUI</code>
| colspan="2" style="background:lightcoral; text-align:center;" | Not implemented
| Currently there is no graphical boot selection implemented[https://tow-boot.org/ Possibly planned]
|-
! scope="row" rowspan="3" | Operating System
| <code>Stability</code>
| style="background:lightcoral; text-align:center;" | Critical bugs
| style="background:LightYellow; text-align:center;" | WIP
| The pre-flashed and outdated operating system on the eMMC often gets corrupted after updating[https://forum.pine64.org/showthread.php?tid=15950 Example]
|-
| <code>Suspend</code>
| style="background:lightcoral; text-align:center;" | Not working
| style="background:LightYellow; text-align:center;" | Experimental
| '''Without suspend the phone will drain very fast!''' Suspend only available in specific more recent images[https://www.pine64.org/2022/03/15/march-update-introducing-the-quartzpro64/ News] Audio is often higher pitched after waking up from suspend due to a bug[https://github.com/dreemurrs-embedded/Pine64-Arch/issues/381 Report]
|-
| <code>Updates</code>
| style="background:lightcoral; text-align:center;" | Critical bugs
| style="background:LightYellow; text-align:center;" | WIP
| The pre-flashed and outdated operating system on the eMMC often gets corrupted after updating[https://forum.pine64.org/showthread.php?tid=15950 Example]; Pacman database lock preventing updates[https://wiki.archlinux.org/title/pacman#%22Failed_to_init_transaction_(unable_to_lock_database)%22_error Solution]; Keyring bug[Solution is to run "pinephonepro-post-install" script as root]
|-
! scope="row" rowspan="5" | Modem
| <code>General</code>
| colspan="2" style="background:lightcoral; text-align:center;" | Critical bugs
| The modem crashes frequently[https://gitlab.manjaro.org/manjaro-arm/issues/pinephone/phosh/-/issues/147 Report][https://gitlab.com/mobian1/devices/eg25-manager/-/issues/15 Report][https://github.com/Biktorgj/pinephone_modem_sdk Alternative firmware]; Slow wakeup[https://gitlab.com/mobian1/devices/eg25-manager/-/issues/34 Report]; Some carriers blocking specific TANs in their network[[PinePhone Carrier Support]]; '''Note:''' Proprietary firmware
|-
| <code>Phone</code>
| colspan="2" style="background:LightYellow; text-align:center;" | WIP
| The modem crashes frequently, which can lead to missed calls[https://gitlab.manjaro.org/manjaro-arm/issues/pinephone/phosh/-/issues/147 Report][https://gitlab.com/mobian1/devices/eg25-manager/-/issues/15 Report][https://github.com/Biktorgj/pinephone_modem_sdk Alternative firmware]; bad call audio quality[https://gitlab.manjaro.org/manjaro-arm/issues/pinephone/phosh/-/issues/249 Report];
|-
| <code>SMS</code>
| colspan="2" style="background:LightGreen; text-align:center;" | Working
| SMS functionality is expected to work. In certain cases the functionality might be blocked by a clogged modem[https://gitlab.manjaro.org/manjaro-arm/issues/pinephone/phosh/-/issues/203 Report]; Some bugs
|-
| <code>MMS</code>
| colspan="2" style="background:LightYellow; text-align:center;" | WIP
| MMS functionality is integrated into the application "Spacebar", some bugs remaining and expected
|-
| <code>Push notifications</code>
| colspan="2" style="background:lightcoral; text-align:center;" | Not implemented
| Receiving push notifications while the phone is suspended is not implemented
|-
! scope="row" rowspan="12" | Components
| <code>LCD</code>
| colspan="2" style="background:LightGreen; text-align:center;" | Working
| Under certain circumstances it is difficult to wake up the screen[Citation]
|-
| <code>Touch</code>
| colspan="2" style="background:LightGreen; text-align:center;" | Working
|
|-
| <code>Rear camera</code>
| style="background:lightcoral; text-align:center;" | Not working
| style="background:LightYellow; text-align:center;" | WIP
| Camera works with DTS fix; userspace still needs to do some catching up (see [[PinePhone Pro/IMX258 Camera Debugging|debugging article]])
|-
| <code>Front camera</code>
| style="background:lightcoral; text-align:center;" | Not working
| style="background:LightYellow; text-align:center;" | WIP
| Camera works with DTS fix; userspace still needs to do some catching up (see [[PinePhone Pro/IMX258 Camera Debugging|debugging article]])
|-
| <code>Camera flash</code>
| colspan="2" style="background:LightGreen; text-align:center;" | Working
| (tested on Manjaro; at <code>/sys/class/leds/white:flash</code>)
|
|-
| <code>WiFi</code>
| colspan="2" style="background:LightGreen; text-align:center;" | Working
| WiFi is expected to work. Some remaining tasks, such as "monitor mode". '''Note:''' Proprietary firmware
|-
| <code>Bluetooth</code>
| colspan="2" style="background:LightYellow; text-align:center;" | WIP
| Bluetooth not necessarily working for calls yet due to missing audio routing[Citation]; Bluetooth in general dodgy under Pulseaudio.[https://wiki.archlinux.org/title/bluetooth_headset#Headset_via_Pipewire Info] '''Note:''' Proprietary firmware
|-
| <code>GNSS/GPS</code>
| colspan="2" style="background:LightYellow; text-align:center;" | WIP
| aGPS to be implemented[Citation]; long loading times to get a GPS fix[Citation]; No preinstalled application[Citation]
|-
| <code>Sensors</code>
| colspan="2" style="background:LightYellow; text-align:center;" | WIP
| "3 Axis Magnetometer" (<code>lis3mdl</code>): Not present in hardware (it's an alternative chip to <code>af8133j</code> due to component shortages) 
"Geo Magnetic Sensor" (<code>af8133j</code>): Working (tested on Manjaro, at <code>/sys/bus/i2c/devices/4-001c/iio:device1</code>) 
"Accelerometer / Gyroscope" (<code>mpu6500</code>): Working (tested on Manjaro, at <code>/sys/bus/i2c/devices/4-0068/iio:device2</code>) 
"Ambient light / Proximity" (<code>stk3311</code>): Not working (tested on Manjaro 5.17.6-2 kernel; IRQ issues)
<div class="toccolours mw-collapsible mw-collapsed" style="overflow:auto;">
<div style="font-weight:bold;line-height:1.6;">stk3311 dmesg excerpt</div>
<div class="mw-collapsible-content"><pre>
[ 9.406478] genirq: Failed to request resources for stk3310_event (irq 99) on irqchip rockchip_gpio_irq
[ 9.407380] stk3310 3-0048: request irq 99 failed
[ 9.408360] stk3310: probe of 3-0048 failed with error -5
</pre></div>
</div>
|-
| <code>Vibration motor</code>
| colspan="2" style="background:LightGreen; text-align:center;" | Working
|
|-
| <code>Notification LED</code>
| colspan="2" style="background:LightGreen; text-align:center;" | Working
|
|-
| <code>Buttons</code>
| colspan="2" style="background:LightGreen; text-align:center;" | Working
| Power buttons and volume buttons are working.
|-
! scope="row" rowspan="5" | Accessory compatibility, spare parts
| <code>Keyboard Add-on</code>
| colspan="2" style="background:lightcoral; text-align:center;" | WIP and critical hardware issue
| The keyboard add-on compatibility is work-in-progress. '''Hardware issue: USB-C port of the phone must not be used while the current keyboard revision is connected'''[[PinePhone_(Pro)_Keyboard#Safety|Warning]]; Top row symbols not implemented as designed[Citation]; Keyboard hot-plugging not supported[Citation]; Battery state not exposed in the pre-flashed operating system state[Citation]; general driver bugs[Citation]
|-
| <code>LoRa Add-on</code>
| colspan="2" style="background:lightcoral; text-align:center;" | Not implemented
| No software support implemented
|-
| <code>Qi Wireless Charging Add-on</code>
| colspan="2" style="background:LightYellow; text-align:center;" | WIP
| Wireless charging with the add-on case is expected to work to some degree. Certain software functionality and a driver is currently missing[Citation]
|-
| <code>Fingerprint Reader Add-on</code>
| colspan="2" style="background:lightcoral; text-align:center;" | Not implemented
| No software support implemented
|-
| <code>Spare parts</code>
| colspan="2" style="background:lightcoral; text-align:center;" | Not available yet
| Spare parts currently not available yet in the store.[https://pine64.com/product-category/smartphone-spare-parts/ Store]
|-
! scope="row" rowspan="1" | Software notes
| <code>Waydroid</code>
| colspan="2" style="background:LightYellow; text-align:center;" | Software rendering only
| Waydroid is an Android container used to run Android applications. Currently can only be started using software rendering due to a bug[https://github.com/waydroid/waydroid/issues/291 Report][https://gitlab.freedesktop.org/mesa/mesa/-/issues/5928 Report]
|-
|}

¹ Status of the features at the time of the last factory installation without updates

² Status of the features with an up-to-date reference image