Difference between revisions of "Mainline Hardware Decoding"

From PINE64
Jump to navigation Jump to search
m (CounterPillow moved page Hardware acceleration to Mainline Hardware Decoding: More descriptive than "Hardware acceleration" for the contents of this page)
(improve page somewhat)
Line 1: Line 1:
{{hint|This page is incomplete, you're welcome to improve it.}}
{{hint|This page is incomplete, you're welcome to improve it.}}


Onboard most SoCs there is what is referred to as a VPU (Video Processing Unit). The VPU is responsible for efficient encoding and decoding of videos. Video decoding can be extremely useful for example if you wanted to watch or stream video from your device without a high CPU utilization (which results from software decoding). Below information can be found on the various drivers that the SoCs used by PINE64 use, and their state of video decoding as a result of the mainline Linux drivers and software.
'''Mainline Hardware Decoding''' refers to video decoding done using hardware accelerators on the mainline Linux kernel (i.e. what sits in Linus' tree).


=Cedrus and Hantro=
On most consumer-oriented SoCs, there is what is referred to as a VPU (Video Processing Unit). The VPU is responsible for power-efficient encoding and decoding of videos. Hardware-accelerated video decoding can be useful to get smoother video playback on your devices, lower power consumption, and lower CPU utilisation. Below is information regarding various hardware PINE64 uses and software that works with it.


In 2018 Bootlin launched a crowdfunding campaign to bring a open source Allwinner VPU driver to mainline Linux, which came to be called Cedrus. The Cedrus media driver (For Allwinner SOCs such as A64) supported by mainline Linux supports H.264 and H.265 video decoding as of Linux 5.10, and with 5.11 came VP8 decoding support and a H.264 stateless video decoder interface. For more information refer to the [https://linux-sunxi.org/Sunxi-Cedrus#Codec_Support Sunxi wiki].
= Hardware =
 
Here's a table of the current support for different hardware.
 
{| class="wikitable plainrowheaders" border="1"
! scope="col" {{Diagonal split header|Codec|SoC}}
! scope="col" | RK3328
! scope="col" | RK3399
! scope="col" | RK3566
! scope="col" | RK3588
|-
! scope="row" | JPEG
| style="text-align:center;"| ?
| style="background:PaleGreen; text-align:center;"| Yes
| style="background:LightYellow; text-align:center;"| In review
| style="background:#F99; text-align:center;"| No
|-
! scope="row" | MPEG-2
| style="background:PaleGreen; text-align:center;"| Yes
| style="background:PaleGreen; text-align:center;"| Yes
| style="background:LightYellow; text-align:center;"| In review
| style="background:#F99; text-align:center;"| No
|-
! scope="row" | VP-8
| style="background:PaleGreen; text-align:center;"| Yes
| style="background:PaleGreen; text-align:center;"| Yes
| style="text-align:center;"| N/A
| style="text-align:center;"| N/A
|-
! scope="row" | H.264/AVC
| style="background:LightYellow; text-align:center;"| In review
| style="background:PaleGreen; text-align:center;"| Yes
| style="background:#F99; text-align:center;"| No
| style="background:#F99; text-align:center;"| No
|-
! scope="row" | H.265/HEVC
| style="background:LightYellow; text-align:center;"| In review
| style="background:LightYellow; text-align:center;"| In review
| style="background:#F99; text-align:center;"| No
| style="background:#F99; text-align:center;"| No
|-
! scope="row" | VP-9
| style="background:LightYellow; text-align:center;"| In review
| style="background:LightYellow; text-align:center;"| In review
| style="background:#F99; text-align:center;"| No
| style="background:#F99; text-align:center;"| No
|-
! scope="row" | AV1
| style="text-align:center;"| N/A
| style="text-align:center;"| N/A
| style="text-align:center;"| N/A
| style="background:#F99; text-align:center;"| No
|}
 
== Cedrus ==
 
In 2018, Bootlin launched a crowdfunding campaign to bring a open source Allwinner VPU driver to mainline Linux, which came to be called Cedrus. The Cedrus media driver (For Allwinner SOCs such as A64) supported by mainline Linux supports H.264 and H.265 video decoding as of Linux 5.10, and with 5.11 came VP8 decoding support and a H.264 stateless video decoder interface. For more information refer to the [https://linux-sunxi.org/Sunxi-Cedrus#Codec_Support Sunxi wiki].
 
== Hantro ==


The Hantro media driver supports Rockchip and NXP SoCs including the RK3399 used in the Pinebook Pro and RockPro64. In November 2020 it [https://www.cnx-software.com/2020/11/24/hantro-h1-hardware-accelerated-video-encoding-support-in-mainline-linux/ was announced] that Bootlin was working on encoding support for the driver.
The Hantro media driver supports Rockchip and NXP SoCs including the RK3399 used in the Pinebook Pro and RockPro64. In November 2020 it [https://www.cnx-software.com/2020/11/24/hantro-h1-hardware-accelerated-video-encoding-support-in-mainline-linux/ was announced] that Bootlin was working on encoding support for the driver.


==GStreamer==
== rkvdec ==
H264 video decoding is possible when using GStreamer built from source, or an application utilizing it such as [https://github.com/Rafostar/clapper Clapper] or [https://flathub.org/apps/details/org.sigxcpu.Livi µPlayer]. µPlayer includes a indicator of when hardware acceleration is properly working and in use.
 
rkvdec is the video decoding hardware that's developed by Rockchip presumably in house. It's what Rockchip uses for decoding 4K H.264/AVC, VP-9 and H.265/HEVC content.


==FFmpeg==
RK3566, RK3568 and likely RK3588 use a second generation of rkvdec called rkvdec2. No mainline driver for this exists yet.


= Software =


==More Resources==
== GStreamer ==
H.264 video decoding is possible when using GStreamer built from source, or an application utilizing it such as [https://github.com/Rafostar/clapper Clapper] or [https://flathub.org/apps/details/org.sigxcpu.Livi µPlayer]. µPlayer includes a indicator of when hardware acceleration is properly working and in use.
 
== FFmpeg ==
 
Mainline FFmpeg currently lacks the necessary patches to use the v4l2-requests based API, but [https://github.com/jernejsk/FFmpeg a fork that can utilise it exists].
 
= More Resources =


[https://xnux.eu/devices/feature/cedrus-pp.html Megi's Cedrus Information]
[https://xnux.eu/devices/feature/cedrus-pp.html Megi's Cedrus Information]
Line 22: Line 89:


[https://briandaniels.me/2021/07/06/hardware-accelerated-video-playback-on-the-pinephone-with-clapper.html HW accelerated Clapper video player on the PinePhone]
[https://briandaniels.me/2021/07/06/hardware-accelerated-video-playback-on-the-pinephone-with-clapper.html HW accelerated Clapper video player on the PinePhone]
= See Also =
[[Mainline Hardware Encoding]]

Revision as of 17:24, 28 April 2022

This page is incomplete, you're welcome to improve it.

Mainline Hardware Decoding refers to video decoding done using hardware accelerators on the mainline Linux kernel (i.e. what sits in Linus' tree).

On most consumer-oriented SoCs, there is what is referred to as a VPU (Video Processing Unit). The VPU is responsible for power-efficient encoding and decoding of videos. Hardware-accelerated video decoding can be useful to get smoother video playback on your devices, lower power consumption, and lower CPU utilisation. Below is information regarding various hardware PINE64 uses and software that works with it.

Hardware

Here's a table of the current support for different hardware.

SoC
Codec
 RK3328 RK3399 RK3566 RK3588
JPEG ? Yes In review No
MPEG-2 Yes Yes In review No
VP-8 Yes Yes N/A N/A
H.264/AVC In review Yes No No
H.265/HEVC In review In review No No
VP-9 In review In review No No
AV1 N/A N/A N/A No

Cedrus

In 2018, Bootlin launched a crowdfunding campaign to bring a open source Allwinner VPU driver to mainline Linux, which came to be called Cedrus. The Cedrus media driver (For Allwinner SOCs such as A64) supported by mainline Linux supports H.264 and H.265 video decoding as of Linux 5.10, and with 5.11 came VP8 decoding support and a H.264 stateless video decoder interface. For more information refer to the Sunxi wiki.

Hantro

The Hantro media driver supports Rockchip and NXP SoCs including the RK3399 used in the Pinebook Pro and RockPro64. In November 2020 it was announced that Bootlin was working on encoding support for the driver.

rkvdec

rkvdec is the video decoding hardware that's developed by Rockchip presumably in house. It's what Rockchip uses for decoding 4K H.264/AVC, VP-9 and H.265/HEVC content.

RK3566, RK3568 and likely RK3588 use a second generation of rkvdec called rkvdec2. No mainline driver for this exists yet.

Software

GStreamer

H.264 video decoding is possible when using GStreamer built from source, or an application utilizing it such as Clapper or µPlayer. µPlayer includes a indicator of when hardware acceleration is properly working and in use.

FFmpeg

Mainline FFmpeg currently lacks the necessary patches to use the v4l2-requests based API, but a fork that can utilise it exists.

More Resources

Megi's Cedrus Information

HW accelerated GStreamer playback on the PinePhone

HW accelerated Clapper video player on the PinePhone

See Also

Mainline Hardware Encoding

Retrieved from "https://wiki.pine64.org/index.php?title=Mainline_Hardware_Decoding&oldid=12942"