View source for User:Vitali/Create a bootable Debian/Linux SDCard for the Pine64/Star64 from scratch

__TOC__

<span id="introduction"></span>
= Introduction =

This guide is for advance users only. In the process we will build every part necessary to boot the star64 until the kernel from '''source'''. For people interested to just get the star64 to boot I would recommend to search for [[#|a guide using a SDCard image]].

This guide is a compilation of mostly already available information but this guide tries to glue all the parts together to make them work. All original sources are linked for reference.

By following this guide you will compile <code>u-boot/spl</code>, <code>u-boot</code>, <code>opensbi</code> and the <code>linux kernel</code> for the pine64/star64 and create a bootable SDCard by creating and populating the necessary partitions on the SDCard. Finally the precompiled official unstable <code>Debian</code> riscv64 port will be installed to the rootfs partition on the SDCard. By using <code>qemu</code> and <code>systemd-nspawn</code> we will be able to execute <code>debootstrap</code> to install a foreign architecture (e.g. install <code>riscv64</code> while running <code>debootstrap</code> on a <code>amd64</code> host). Also minimal configuration of the rootfs is done on the host computer. All commands are documented in a way to be easy repeatable and will produce a ready to boot and configured SDCard image. You can of course modify configurations done on the host to your preference.

In contrast to methods using [[#|<code>buildroot</code>]] or [[#|<code>yocto</code>]] this guide is compiling each component separately. And the advantage of using <code>debian</code> is that you will have access to thousands of precompiled <code>debian</code> packages.

<span id="preparation"></span>
== Preparation ==

Install <code>qemu</code>, <code>systemd-nspawn</code> and a <code>riscv compiler</code>.

<syntaxhighlight lang="sh">sudo apt install gcc-riscv64-linux-gnu systemd-container qemu debootstrap pwgen</syntaxhighlight>
Optional: Create .env file with your environment settings. Or Just set them directly.

<syntaxhighlight lang="sh">nano .env</syntaxhighlight>
<syntaxhighlight lang="sh"># cross compiler
CROSS_COMPILE=riscv64-linux-gnu-
# where to store apt cache, for faster installs
APT_CACHE_DIR=/tmp/apt/cache
# arbitrary path where the rootfs will be mounted at
ROOTFS=/tmp/star64/rootfs
# SDCard dev path
USBDEV=/dev/sdX

# hostname of the target, can be left empty
MNAME=

# username you are logged in your host system this name will be also used to create a new user on the newly created rootfs.
USER=$USER
test "$USER" = root && { echo "USER can't be root"; exit; }

PASSWORD=$(pwgen -1)

AUTHORIZED_KEY=$(ssh-keygen -f /home/$USER/.ssh/id_ed25519 -y)

# define helper functions
function star64_mount() {
    mkdir -p ${ROOTFS?}/
    sudo -E mount ${USBDEV:?}4 ${ROOTFS?}/
    sudo mkdir -p ${ROOTFS?}/boot
    sudo -E mount -o umask=0000 ${USBDEV:?}3 ${ROOTFS?}/boot
}
function star64_unmount() {
    sudo umount ${ROOTFS?}/boot
    sudo umount ${ROOTFS?}/
}</syntaxhighlight>
<syntaxhighlight lang="sh">source .env</syntaxhighlight>
<span id="connect-serial-cable"></span>
=== Connect Serial Cable ===

It is a good idea to first make sure the serial connection is working. Make sure the boot switches are set to boot from UART.

<pre>PINE64 STAR64 BOOT UART
          ┌──┐
GOIO_0  1 │X │
GPIO_1  2 │X │
          └──┘
        H      L</pre>
Connect and monitor the serial port as stated in the [[#serial-uart|Troubleshooting Section: Serial / UART]]. You should see repeatably printing of ‘C’. After making sure the serial works we can change it to SDCard boot mode.

<pre>PINE64 STAR64 BOOT SDCARD
          ┌──┐
GOIO_0  1 │ X│
GPIO_1  2 │X │
          └──┘
        H      L</pre>
<span id="build"></span>
= Build =

Build each of the following projects. Make sure to navigate back into your base project directory (e.g. <code>/tmp/star64</code>) if you proceed to build the next project. After completing this guide it should at least contain the directories: <code>linux opensbi Tools u-boot</code>.

* [[#build-tools|Build tools]]
* [[#build-opensbi|Build OpenSBI]]
* [[#build-u-boot|Build U-Boot]]
* [[#partition-the-sdcard|Partition the SDCard]]
* [[#build-the-linux-kernel|Build the Linux kernel]]
* [[#debootstrap-the-debian-riscv64-rootfs|Debootstrap the Debian riscv64 rootfs]]
* [[#copy-the-linux-kernel-to-sdcard|Copy the Linux kernel to SDCard]]

<span id="build-tools"></span>
= Build tools =

Build [https://github.com/starfive-tech/Tools/blob/master/spl_tool/README.md spl_tool].

<syntaxhighlight lang="sh">git clone --depth 1 https://github.com/starfive-tech/Tools.git
make -C Tools/spl_tool</syntaxhighlight>
<span id="build-opensbi"></span>
= Build OpenSBI =

<syntaxhighlight lang="sh">git clone --depth 1 https://github.com/riscv/opensbi.git
cd opensbi
make CROSS_COMPILE=${CROSS_COMPILE?} PLATFORM=generic FW_TEXT_START=0x40000000 FW_OPTIONS=0 -j
</syntaxhighlight>
For more insight about the u-boot boot process you can find more useful information in the [https://github.com/u-boot/u-boot/blob/master/doc/board/starfive/visionfive2.rst u-boot visionfive2 docs].

<span id="build-u-boot"></span>
= Build U-Boot =

Make sure to [[#build-tools|build Tools]] and [[#build-opensbi|openSBI]] before proceeding.

The upsteam u-boot builds the <code>u-boot.itb</code> automatically by using openSBIs fw_dynamic.bin. But the upstream u-boot does not boot properly (Last checked: 26-05-23)… Therefore we use the fw_payload.bin method with the starfive/u-boot.

<syntaxhighlight lang="sh">git clone --depth 1 -b Star64 https://github.com/Fishwaldo/u-boot.git
cd u-boot

CROSS_COMPILE=${CROSS_COMPILE?} make pine64_star64_defconfig
CROSS_COMPILE=${CROSS_COMPILE?} make OPENSBI=../opensbi/build/platform/generic/firmware/fw_dynamic.bin -j

make -C ../opensbi CROSS_COMPILE=${CROSS_COMPILE?} PLATFORM=generic \
    FW_FDT_PATH=../u-boot/arch/riscv/dts/pine64_star64.dtb \
    FW_PAYLOAD_FDT_ADDR=0x40080000 \
    FW_PAYLOAD_OFFSET=0x00100000 \
    FW_PAYLOAD_PATH=../u-boot/u-boot.bin \
    FW_TEXT_START=0x40000000

ln -s ../../opensbi/build/platform/generic/firmware/fw_payload.bin ../Tools/uboot_its/
tools/mkimage -f ../Tools/uboot_its/visionfive2-uboot-fit-image.its -A riscv -O u-boot -T firmware u-boot.itb
</syntaxhighlight>
Create a SPL header as stated in [https://github.com/u-boot/u-boot/blob/master/doc/board/starfive/visionfive2.rst u-boot visionfive2 docs] and [https://github.com/starfive-tech/Tools/blob/master/spl_tool/README.md spl_tool]:

<syntaxhighlight lang="sh">../Tools/spl_tool/spl_tool -c -f spl/u-boot-spl.bin</syntaxhighlight>
<span id="update-with-linux"></span>
== Update with Linux ==

If you alreay have Linux running on the star64 and want to update <code>u-boot</code> you can do that like that:

<syntaxhighlight lang="sh">scp u-boot.itb ${MNAME?}:/tmp
ssh ${MNAME?} sudo dd if=/tmp/u-boot.itb of=/dev/mmcblk1p2</syntaxhighlight>
More infos on the [https://u-boot.readthedocs.io/en/latest/build/gcc.html u-boot build process with gcc]. I strongly recommend to read [https://doc-en.rvspace.org/VisionFive2/Developer_Guide/JH7110_Boot_UG.pdf JH7110Boot User Guide] for more information on the boot process.

<span id="partition-the-sdcard"></span>
= Partition the SDCard =

After build [[#build-opensbi|OpenSBI]] and [[#build-u-boot|U-Boot]] from above we can proceed to setup the partitions for the SDCard and write u-boot-spl into the <code>SPL</code>-partition and u-boot into the <code>UBOOT</code>-partition.

<syntaxhighlight lang="sh"># Set the SDCard dev path (content will be lost)
sudo su # continue with caution!
source .env
fdisk -l ${USBDEV:?} # make sure its in fact the right drive

lsblk ${USBDEV:?} -n | grep -q part && \
    { echo "delete partitions with 'wipefs --all ${USBDEV:?}' first"; exit; }

sfdisk ${USBDEV:?} <<EOF
label: gpt
: start=4096, size=2MiB, name=SPL, type=2E54B353-1271-4842-806F-E436D6AF6985
: size=4MiB, name=UBOOT, type=BC13C2FF-59E6-4262-A352-B275FD6F7172
: size=512MiB, name=BOOT, type=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7
: name=SYSTEM, type=0FC63DAF-8483-4772-8E79-3D69D8477DE4
EOF

mkfs.vfat -F 32 ${USBDEV:?}3
mkfs.ext4 -E lazy_itable_init=0,lazy_journal_init=0 ${USBDEV:?}4

# if you install on eMMC: execute:
# Tools/spl_tool/spl_tool --fix-imghdr -f ${USBDEV:?}

dd if=u-boot/spl/u-boot-spl.bin.normal.out of=${USBDEV:?}1
dd if=u-boot/u-boot.itb of=${USBDEV:?}2</syntaxhighlight>
You can find more about the partition layout on [https://doc-en.rvspace.org/VisionFive2/Developer_Guide/JH7110_Boot_UG.pdf JH7110_Boot_UG] Page 8 Table1-3 SD/eMMC Boot Address Allocation and [https://github.com/u-boot/u-boot/blob/master/doc/board/starfive/visionfive2.rst#flashing u-boot/visionfive2: create bootable image: flashing]

<span id="build-the-linux-kernel"></span>
= Build the Linux kernel =

<syntaxhighlight lang="sh"># alternative: https://github.com/starfive-tech/linux/tree/JH7110_VisionFive2_upstream
git clone --depth 1 -b Star64_devel https://github.com/Fishwaldo/Star64_linux.git linux
cd linux/

ARCH=riscv CROSS_COMPILE=${CROSS_COMPILE?} make pine64_star64_defconfig

# if you need any extra symbols for your kernel
ARCH=riscv CROSS_COMPILE=${CROSS_COMPILE?} make xconfig

# or by command line
ARCH=riscv CROSS_COMPILE=${CROSS_COMPILE?} scripts/config \
    -m ATA \
    -m SATA_AHCI \
    -m ATA_GENERIC

# set new defaults if changed by command line
ARCH=riscv CROSS_COMPILE=${CROSS_COMPILE?} make olddefconfig

ARCH=riscv CROSS_COMPILE=${CROSS_COMPILE?} make -j14
</syntaxhighlight>
<span id="copy-the-linux-kernel-to-sdcard"></span>
= Copy the Linux kernel to SDCard =

Make sure to [[#build-the-linux-kernel|build the Linux kernel]] and [[#partition-the-sdcard|Partition the SDCard]] before proceeding.

<syntaxhighlight lang="sh">
star64_mount

cp -v linux/arch/riscv/boot/Image.gz ${ROOTFS?}/boot
# cp initramfs.cpio.gz ${ROOTFS?}/boot # don't have one...
cp -v linux/arch/riscv/boot/dts/starfive/jh7110-pine64-star64.dtb ${ROOTFS?}/boot

sudo ARCH=riscv CROSS_COMPILE=${CROSS_COMPILE?} \
    INSTALL_MOD_PATH=${ROOTFS?}/usr \
    INSTALL_PATH=${ROOTFS?}/boot \
    INSTALL_HDR_PATH=${ROOTFS?}/usr \
    make -C linux install modules_install headers_install

star64_unmount</syntaxhighlight>
<span id="update-with-linux-1"></span>
== Update with Linux ==

If you already have Linux running on the star64 and want to update the <code>linux</code> kernel you can do that like that:

<syntaxhighlight lang="sh">rsync --rsync-path='sudo rsync' arch/riscv/boot/Image.gz ${MNAME?}:/boot
sudo ARCH=riscv CROSS_COMPILE=${CROSS_COMPILE?} \
    INSTALL_MOD_PATH=/tmp/modules \
    make modules_install
rsync -rv --rsync-path='sudo rsync' /tmp/modules/lib/modules/ ${MNAME?}:/lib/modules</syntaxhighlight>
<span id="debootstrap-the-debian-riscv64-rootfs"></span>
= Debootstrap the Debian riscv64 rootfs =

Make sure to setup [https://de.wikipedia.org/wiki/Binfmt_misc binfmt]. Check if it is available with <code>mount | grep binfmt_misc</code> and <code>cat /proc/sys/fs/binfmt_misc/qemu-riscv64</code>.

Copy and past the <code>star64_install_debian</code> function into bash and execute it or create a script file from that function. You can also execute them one by one if you prefer.

You can also just run the <code>debootstrap</code> command and setup Debian on your own. But at least set a root password so you can login over serial.

For reference read [https://wiki.debian.org/RISC-V#debootstrap debootstrap].

Alternatively if you dont want to use <code>qemu</code> and <code>systemd-nspawn</code> you can just follow the [https://forum.pine64.org/showthread.php?tid=18276 Star64: Armbian Installation] guide instead of <code>debootsraping</code> <code>Debian</code>.

<syntaxhighlight lang="sh">#switch to root
sudo su
source .env

function star64_install_debian() {
MNAME=${MNAME:-"star64-$(pwgen 4 -1 -A)"}
mkdir -p ${APT_CACHE_DIR:-/tmp/apt/cache}

debootstrap --foreign --cache-dir=/mnt/data/tmp/apt-riscv --arch=riscv64 --no-check-gpg --include=debian-ports-archive-keyring,usrmerge,openssh-server,htop,vim,nano,tcpdump,sudo,curl,wget,iputils-ping,fdisk,gdisk,pciutils,usbutils,screen,rsync,firmware-linux-free unstable ${ROOTFS?} http://deb.debian.org/debian-ports
cp /usr/bin/qemu-riscv64 ${ROOTFS?}/usr/bin/
chroot ${ROOTFS?} /debootstrap/debootstrap --second-stage

NSPAWN="systemd-nspawn -M $MNAME -D ${ROOTFS?} --quiet --no-pager"

# ${NSPAWN?} sed -i 's/main/main non-free/g' /etc/apt/sources.list
# ${NSPAWN?} apt update
# ${NSPAWN?} apt install -y firmware-linux-nonfree

PASSWORD=${PASSWORD:-$(pwgen 8 -1 -A)}
echo "Your root/$USER password is set to $PASSWORD"
${NSPAWN?} useradd --shell /bin/bash --create-home ${USER:-user}
${NSPAWN?} sh -c "echo root:$PASSWORD | chpasswd"
${NSPAWN?} sh -c "echo ${USER:-user}:$PASSWORD | chpasswd"
echo "$MNAME" > ${ROOTFS?}/etc/hostname
echo "127.0.0.1 $MNAME" >> ${ROOTFS?}/etc/hosts

echo "${USER:-user} ALL=(ALL) NOPASSWD:ALL" > ${ROOTFS?}/etc/sudoers.d/${USER:-user}
install -o 1000 -g 1000 -d -m 0700 ${ROOTFS?}/home/${USER:-user}/.ssh/
install -o 1000 -g 1000 -m 0644 <(echo "$AUTHORIZED_KEY") ${ROOTFS?}/home/${USER:-user}/.ssh/authorized_keys

mkdir -p ${ROOTFS?}/boot

test -b ${USBDEV:-''} && {
echo "PARTUUID=$(blkid ${USBDEV:?}4 --output value | tail -1)        /            ext4     defaults,noatime    0    0" >> ${ROOTFS?}/etc/fstab
echo "PARTUUID=$(blkid ${USBDEV:?}3 --output value | tail -1)        /boot        vfat     defaults,noatime    0    0" >> ${ROOTFS?}/etc/fstab
echo "none                                                 /tmp         tmpfs    defaults,noatime    0    0" >> ${ROOTFS?}/etc/fstab
}

${NSPAWN?} sh -c "
systemctl enable networking
systemctl enable systemd-networkd.service
systemctl enable ssh
"

cat > ${ROOTFS?}/etc/systemd/network/20-dhcp-all.network <<EOF
[Match]
Name=*

[Network]
DHCP=yes
EOF

echo "nameserver 1.1.1.1" > ${ROOTFS?}/etc/resolv.conf
echo "nameserver 8.8.8.8" >> ${ROOTFS?}/etc/resolv.conf

test -d ssh_host && {
sudo rsync --chown=root:root -vr ssh_host/etc/ssh/ ${ROOTFS?}/etc/ssh/
sudo chmod 600 ${ROOTFS?}/etc/ssh/ssh_host_*
sudo chmod +r ${ROOTFS?}/etc/ssh/ssh_host_*.pub
}
}

star64_mount
star64_install_debian

tee ${ROOTFS?}/boot/vf2_uEnv.txt <<'EOF'
bootargs=console=ttyS0,115200 rootwait earlycon=sbi root=/dev/mmcblk1p4 rw
distro_bootcmd=fatload mmc 1:3 0x60200000 Image.gz; \
    fatload mmc 1:3 ${fdt_addr_r} jh7110-pine64-star64.dtb \
    unzip 0x60200000 ${kernel_addr_r}; \
    booti ${kernel_addr_r} - ${fdt_addr_r}
EOF

star64_unmount</syntaxhighlight>
<span id="troubleshooting"></span>
= Troubleshooting =

<span id="project-versions-used-last-git-commit-hash"></span>
== Project versions used (last git commit hash) ==

* linux: 7943fd6427b0abd25e1af2756e061a112bde8711
* opensbi: d4c46e0ff1b0ead9d5a586e1a19a00a92160206d
* Tools: 693661d4ba314424f76c06da1bbb799e9b534c9f
* u-boot: 172b47f62039605d6806fa96bd403c21cda28996

<span id="serial-uart"></span>
== Serial / UART ==

<pre>       PINE64 STAR64 UART

             ┌─────┐
       3.3V  │ 1  2│ 5V
       SDA1  │ 3  4│ 5V
       SCL1  │ 5  6│ GND
GPIO55       │ 7  8│ TXD
       GND   │ 9 10│ RXD</pre>
Use <code>screen /dev/ttyUSB0 115200</code> to monitor serial port.

<span id="jtag"></span>
== JTAG ==

Connect a JTAG adapter to GPIO [https://files.pine64.org/doc/star64/Star64_Schematic_V1.1_20230504.pdf Star64 schematic] p. 18 J5 and Match it with the configuration as of in [https://github.com/starfive-tech/u-boot/blob/ac0ac696256abf412826d74ee918dd417e207d7b/board/starfive/visionfive2/starfive_visionfive2.c#L354 starfive-tech/u-boot: board: starfive_visionfive2.c#L354]. Also see: - [https://forum.rvspace.org/t/connecting-to-visionfive-s-jtag-port-a-short-guide/514 visionfive jtag port] - [https://doc-en.rvspace.org/JH7110/Datasheet/JH7110_DS/function_0.html JH7110 GPIO AF]

<pre>      PINE64 STAR64 JTAG

            │33 34│ GND
GPIO63 TMS  │35 36│ nTRST GPIO36
GPIO60 TCK  │37 38│ TDI   GPIO61
       GND  │39 40│ TDO   GPIO44
            └─────┘</pre>
<!-- RV_DEBUGGER:   TCK TDI TDO TMS -->
<!-- LOGICANALIZER: TCK TDI TDO TMS -->
<span id="openocd"></span>
=== Openocd ===

You can use the visionfive2 [https://forum.rvspace.org/t/openocd-config-for-visionfive-2/1452 openocd config].

<syntaxhighlight lang="sh">mkdir -p openocd/
nano openocd/starfive-visionfive2.cfg
openocd -f interface/ftdi/um232h.cfg -f openocd/starfive-visionfive2.cfg</syntaxhighlight>
<span id="boot-failerror-is-0xffffffff"></span>
=== BOOT fail,Error is 0xffffffff ===

If you get the <code>BOOT fail,Error is 0xffffffff</code> error just press the reset key again. This error keeps ocurring if powering the star64 on. But pressing reset will make the star64 boot normally.

[[Category:STAR64]]