path: root/README.md

<!--
SPDX-FileCopyrightText: 2024 Himbeer <himbeer@disroot.org>

SPDX-License-Identifier: CC-BY-NC-SA-4.0
-->

# srvre kernel

Simple RISC-V research environment kernel

This is the microkernel for the SRVRE riscv64 operating system.

Its purpose is learning RISC-V OS development and trying to experiment
with solutions to some of the shortcomings of existing operating systems.
The ultimate goal is to make it compile itself, maybe even to make it usable
for limited command-line or server usage.

See the [project page](https://himbeerserver.de/md/srvre/kernel.md)
for more information.

# Build a bootable image

## Boot flow

This kernel is mainly developed for qemu-virt64 and the Lichee Pi 4A.
The latter uses the following boot flow:

```
~ M-mode ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~|~ S-mode ~~~
+------+     +------------+     +--------+     +---------+     +--------+
| brom | --> | U-Boot SPL | --> | U-Boot | --> | OpenSBI | --> | Kernel |
+------+     +------------+     +--------+     +---------+     +--------+
```

QEMU can be adapted to use this boot flow as well (the default is different).

## Build the kernel

Required dependencies:

* zig (most recent release version at the time of the latest commit)
* init executable (see below)

To make a debug build of this kernel, run:

```
zig build -Dplatform=<PLATFORM>
```

Replace `<PLATFORM>` with the platform you want to build for.
Supported options include `qemu` and `lpi4a`.
See the `src/cfg/platform` directory for the full list.

You can also use any other Zig build mode, e.g. `--release=fast`.

This results in a `zig-out/bin/srvre_kernel.elf` file.
You may `strip(1)` this file if you want to.

### init executable

The init executable is expected to be a statically linked ELF
(with program headers) at `src/cfg/init`. It is embedded in the kernel
binary to avoid running (filesystem) drivers in S-mode, meaning that a kernel
rebuild is required to apply modifications.

Example programs are provided in the `examples/` directory.

## Build OpenSBI

Required dependencies:

* riscv64-linux-gnu-gcc

Clone the [OpenSBI repository](https://github.com/riscv-software-src/opensbi)
and run the following command. If building for the Lichee Pi 4A, use the
[T-Head OpenSBI fork](https://github.com/revyos/thead-opensbi) instead.

```
make CROSS_COMPILE=riscv64-linux-gnu- PLATFORM=generic FW_DYNAMIC=y -j $(nproc) all
```

This results in a `build/platform/generic/firmware/fw_dynamic.bin` file.
You might be able to use precompiled versions of this file from other sources.

## Build U-Boot

Required dependencies:

* riscv64-linux-gnu-gcc
* bison
* flex
* python-setuptools
* swig

The boot flow is not supported by upstream U-Boot.
There are several T-Head-specific forks that implement it for the Linux kernel,
but none that support regular ELF binaries.
Therefore a custom fork of U-Boot is required.

### QEMU

Clone the [U-Boot fork](https://codeberg.org/Himbeer/u-boot),
check out the `sbiboot` branch and run the following commands:

```
make qemu-riscv64_spl_defconfig
make -j$(nproc) menuconfig
```

This will bring up the build configuration tool.
Set the following options:

* Device Drivers > Serial > Base address of UART (CONFIG_DEBUG_UART_BASE): 0x10000000
* Device Drivers > Serial > Base address of UART for SPL (CONFIG_SPL_DEBUG_UART_BASE): 0x10000000
* Device Drivers > Serial > Select which UART will provide the debug UART (CONFIG_DEBUG_UART_NS16550=y): ns16550
* Device Drivers > Serial > Check if UART is enabled on output (CONFIG_DEBUG_UART_NS16550_CHECK_ENABLED): Yes
* RISC-V architecture > Symmetric Multi-Processing (CONFIG_SMP): Yes
* RISC-V architecture > Symmetric Multi-Processing in SPL (CONFIG_SPL_SMP): Yes
* RISC-V architecture > Run Mode (CONFIG_RISCV_MMODE=y): Machine
* RISC-V architecture > SPL Run Mode (CONFIG_SPL_RISCV_MMODE=y): Machine
* SPL configuration options > Show more information when something goes wrong (CONFIG_SPL_SHOW_ERRORS): Yes
* SPL configuration options > Offset to which the SPL should be padded before appending the SPL payload (CONFIG_SPL_PAD_TO): 0x1200000
* Boot options > Boot images > Flattened Image Tree (FIT) > Enable SPL loading U-Boot as a FIT (basic fitImage features) (CONFIG_SPL_LOAD_FIT): No
* Boot options > bootcmd value (CONFIG_BOOTCOMMAND): "virtio scan; fatload virtio 0 0x8f000000 uImage; bootm 0x8f000000"
* (General setup > Support for multiprocessor (?): No; Won't compile otherwise)

Then exit and select 'yes' to save your changes.

### Lichee Pi 4A

If building for the Lichee Pi 4A, use the
[T-Head-specific U-Boot fork](https://codeberg.org/Himbeer/thead-u-boot)
instead and run the following command:

```
make light_lpi4a_defconfig
```

For the 16 GB variant, use a different config:

```
make light_lpi4a_16g_defconfig
```

### Build the binaries

Finally, build the binaries:

```
make ARCH=riscv CROSS_COMPILE=riscv64-linux-gnu- -j $(nproc) u-boot-with-spl.bin
```

## Create multi-file kernel image

Create a legacy U-Boot image:

```
/path/to/u-boot/tools/mkimage -A riscv -O elf -T multi -C none -a 0x80400000 -e 0x80400000 -n SRVRE -d /path/to/srvre_kernel.elf:/path/to/fw_dynamic.bin:/path/to/devicetree.dtb /path/to/output/uImage
```

Write this `uImage` file to the first, FAT32 (LBA) partition
on an MBR partitioned drive. You should now be able to boot.

### Extract QEMU device tree

The default `arch/riscv/dts/qemu-virt64.dtb` tree is an image tree and doesn't help.
To extract the device tree, create an empty file on the disk image *and unmount it*.
Then boot the VM as explained above but hit a key before it autoboots.
Run the following command in the U-Boot shell:

```
env print fdtcontroladdr
```

Then use the printed value like so (without the angle brackets):

```
fdt addr 0x<ENV_OUTPUT>
fdt header
```

You should now see a `totalsize` value. Take note of the hexadecimal notation
and save it to disk:

```
virtio scan
fatwrite virtio 0 0x<ENV_OUTPUT> <EMPTY_FILE_NAME> <HEX_TOTALSIZE>
```

You can now `poweroff`, mount the partition again and build an image
using the commands above.

## Start QEMU

Run the following command:

```
qemu-system-riscv64 -M virt -m 2G -smp 4 -display none -serial stdio -bios /path/to/u-boot-with-spl.bin -drive file=/path/to/disk/image,if=virtio,media=disk,format=raw
```

## Flash to Lichee Pi 4A

Required dependencies:

* android-sdk-platform-tools

Hold down the `BOOT` button on the board, connect it to your computer
using USB-C and release the button. Create an image file, format it
as ext4 (using `dd(1)` and `mkfs.ext4(1)`) and mount it.
Copy the `uImage` file to it, unmount it and flush the changes to disk
(using `sync(1)`). Then run the following commands as root:

```
fastboot flash ram /path/to/thead-u-boot/u-boot-with-spl.bin
fastboot reboot
# The next command is going to wait for the device for a few seconds, be patient
fastboot flash uboot /path/to/thead-u-boot/u-boot-with-spl.bin
fastboot flash boot /path/to/image/file.img
```

You can now reboot the device and use the serial (UART0) console
to start the kernel by entering the U-Boot shell
(by pressing a key to stop autoboot or by waiting for it to fail)
and typing:

```
ext4load mmc 0:2 0x8f000000 uImage
bootm 0x8f000000
```

# Boot protocol

The kernel expects to be loaded in S-mode with the following register values:

* a0: Current Hart ID

Typically you'll want to load it directly from OpenSBI.
Depending on your platform you may use a slightly patched version of U-Boot
that runs in M-mode to boot a combined kernel / OpenSBI image.
This process is documented above.

Completely skipping U-Boot and using OpenSBI directly should work
if your platform supports it. However it is currently untested.

The advantage of this boot flow is that it doesn't require the use of U-Boot.
In theory, anything that sets the registers as described above
and runs the kernel in S-mode should work, so any SBI should work
if you can get it to run.

# Debugging

You can run QEMU with the `-s -S` flags and attach `riscv64-elf-gdb`:

```
add-symbol-file /path/to/srvre_kernel.elf
target remote localhost:1234
```

Be sure to confirm the prompt from the `add-symbol-file` command.
You can also use `riscv64-elf-gdb /path/to/srvre_kernel.elf` directly.

# Translating device trees to the HWI (Hardware Info) format

The HWI format was developed as a more robust and efficient alternative to
standard device trees. The `hwi` tool reads a text representation from stdin
and writes the corresponding binary representation to stdout.  The values can
be obtained from the device tree, but automated conversion is not implemented
due to the complexity of the device tree format.

## Build hwi

The `hwi` tool can be built easily:

```
zig build-exe src/hwi.zig
```

This produces a `hwi` binary in the current working directory.

## Use hwi tool

You can use the `hwi` command to convert from the text representation to the
binary representation:

```
hwi < src/cfg/platform/<PLATFORM>.txt > src/cfg/platform/<PLATFORM>.hwi
```

Omitting the stdin pipe allows you to type out the textual representation
manually:

```
hwi > src/cfg/platform/<PLATFORM>.hwi
```

Press Control+D after finishing the last line (and pressing Enter)
to close stdin, quitting the tool.

## Text and binary formats

The [text](https://himbeerserver.de/md/srvre/kernel/wiki/hwi.md#text-format)
and
[binary](https://himbeerserver.de/md/srvre/kernel/wiki/hwi.md#binary-format)
formats are documented in the wiki.