# srvre kernel Simple RISC-V research environment kernel This is the microkernel for the SRVRE riscv64 operating system. Its purpose is learning RISC-V OS development and trying to experiment with solutions to some of the shortcomings of existing operating systems. The ultimate goal is to make it compile itself, maybe even to make it usable for limited command-line or server usage. See the [project page](https://himbeerserver.de/md/srvre/kernel.md) for more information. # Build a bootable image ## Boot flow This kernel is mainly developed for qemu-virt64 and the Lichee Pi 4A. The latter uses the following boot flow: ``` ~ M-mode ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~|~ S-mode ~~~ +------+ +------------+ +--------+ +---------+ +--------+ | brom | --> | U-Boot SPL | --> | U-Boot | --> | OpenSBI | --> | Kernel | +------+ +------------+ +--------+ +---------+ +--------+ ``` QEMU can be adapted to use this boot flow as well (the default is different). ## Build the kernel Required dependencies: * zig ^0.12.0 To make a debug build of this kernel, run: ``` zig build -Dplatform= ``` Replace `` with the platform you want to build for. Supported options include `qemu` and `lpi4a`. See the `src/lib/cfg/platform` directory for the full list. You can also use any other Zig build mode, e.g. `--release=fast`. This results in a `zig-out/bin/srvre_kernel.elf` file. You may `strip(1)` this file if you want to. ## Build OpenSBI Required dependencies: * riscv64-linux-gnu-gcc Clone the [OpenSBI repository](https://github.com/riscv-software-src/opensbi) and run the following command. If building for the Lichee Pi 4A, use the [T-Head OpenSBI fork](https://github.com/revyos/thead-opensbi) instead. ``` make CROSS_COMPILE=riscv64-linux-gnu- PLATFORM=generic FW_DYNAMIC=y -j $(nproc) all ``` This results in a `build/platform/generic/firmware/fw_dynamic.bin` file. You might be able to use precompiled versions of this file from other sources. ## Build U-Boot Required dependencies: * riscv64-linux-gnu-gcc * bison * flex * python-setuptools * swig The boot flow is not supported by upstream U-Boot. There are several T-Head-specific forks that implement it for the Linux kernel, but none that support regular ELF binaries. Therefore a custom fork of U-Boot is required. ### QEMU Clone the [U-Boot fork](https://codeberg.org/Himbeer/u-boot), check out the `sbiboot` branch and run the following commands: ``` make qemu-riscv64_spl_defconfig make -j$(nproc) menuconfig ``` This will bring up the build configuration tool. Set the following options: * Device Drivers > Serial > Base address of UART (CONFIG_DEBUG_UART_BASE): 0x10000000 * Device Drivers > Serial > Base address of UART for SPL (CONFIG_SPL_DEBUG_UART_BASE): 0x10000000 * Device Drivers > Serial > Select which UART will provide the debug UART (CONFIG_DEBUG_UART_NS16550=y): ns16550 * Device Drivers > Serial > Check if UART is enabled on output (CONFIG_DEBUG_UART_NS16550_CHECK_ENABLED): Yes * RISC-V architecture > Symmetric Multi-Processing (CONFIG_SMP): No * RISC-V architecture > Symmetric Multi-Processing in SPL (CONFIG_SPL_SMP): No * RISC-V architecture > Run Mode (CONFIG_RISCV_MMODE=y): Machine * RISC-V architecture > SPL Run Mode (CONFIG_SPL_RISCV_MMODE=y): Machine * SPL configuration options > Show more information when something goes wrong (CONFIG_SPL_SHOW_ERRORS): Yes * SPL configuration options > Offset to which the SPL should be padded before appending the SPL payload (CONFIG_SPL_PAD_TO): 0x1200000 * Boot options > Boot images > Flattened Image Tree (FIT) > Enable SPL loading U-Boot as a FIT (basic fitImage features) (CONFIG_SPL_LOAD_FIT): No * Boot options > bootcmd value (CONFIG_BOOTCOMMAND): "virtio scan; fatload virtio 0 0x8f000000 uImage; bootm 0x8f000000" Then exit and select 'yes' to save your changes. ### Lichee Pi 4A If building for the Lichee Pi 4A, use the [T-Head-specific U-Boot fork](https://codeberg.org/Himbeer/thead-u-boot) instead and run the following command: ``` make light_lpi4a_defconfig ``` For the 16 GB variant, use a different config: ``` make light_lpi4a_16g_defconfig ``` ### Build the binaries Finally, build the binaries: ``` make ARCH=riscv CROSS_COMPILE=riscv64-linux-gnu- -j $(nproc) u-boot-with-spl.bin ``` ## Create multi-file kernel image Create a legacy U-Boot image: ``` /path/to/u-boot/tools/mkimage -A riscv -O elf -T multi -C none -a 0x80400000 -e 0x80400000 -n SRVRE -d /path/to/srvre_kernel.elf:/path/to/fw_dynamic.bin:/path/to/devicetree.dtb /path/to/output/uImage ``` Write this `uImage` file to the first, FAT32 (LBA) partition on an MBR partitioned drive. You should now be able to boot. ### Extract QEMU device tree The default `arch/riscv/dts/qemu-virt64.dtb` tree is an image tree and doesn't help. To extract the device tree, create an empty file on the disk image *and unmount it*. Then boot the VM as explained above but hit a key before it autoboots. Run the following command in the U-Boot shell: ``` env print fdtcontroladdr ``` Then use the printed value like so (without the angle brackets): ``` fdt addr 0x fdt header ``` You should now see a `totalsize` value. Take note of the hexadecimal notation and save it to disk: ``` virtio scan fatwrite virtio 0 0x ``` You can now `poweroff`, mount the partition again and build an image using the commands above. ## Start QEMU Run the following command: ``` qemu-system-riscv64 -M virt -m 2G -smp 4 -display none -serial stdio -bios /path/to/u-boot-with-spl.bin -drive file=/path/to/disk/image,if=virtio,media=disk,format=raw ``` ## Flash to Lichee Pi 4A Required dependencies: * android-sdk-platform-tools Hold down the `BOOT` button on the board, connect it to your computer using USB-C and release the button. Create an image file, format it as ext4 (using `dd(1)` and `mkfs.ext4(1)`) and mount it. Copy the `uImage` file to it, unmount it and flush the changes to disk (using `sync(1)`). Then run the following commands as root: ``` fastboot flash ram /path/to/thead-u-boot/u-boot-with-spl.bin fastboot reboot # The next command is going to wait for the device for a few seconds, be patient fastboot flash uboot /path/to/thead-u-boot/u-boot-with-spl.bin fastboot flash boot /path/to/image/file.img ``` You can now reboot the device and use the serial (UART0) console to start the kernel by entering the U-Boot shell (by pressing a key to stop autoboot or by waiting for it to fail) and typing: ``` ext4load mmc 0:2 0x8f000000 uImage bootm 0x8f000000 ``` # Boot protocol The kernel expects to be loaded in S-mode with the following register values: * a0: Current Hart ID Typically you'll want to load it directly from OpenSBI. Depending on your platform you may use a slightly patched version of U-Boot that runs in M-mode to boot a combined kernel / OpenSBI image. This process is documented above. Completely skipping U-Boot and using OpenSBI directly should work if your platform supports it. However it is currently untested. The advantage of this boot flow is that it doesn't require the use of U-Boot. In theory, anything that sets the registers as described above and runs the kernel in S-mode should work, so any SBI should work if you can get it to run. # Debugging You can run QEMU with the `-s -S` flags and attach `riscv64-elf-gdb`: ``` add-symbol-file /path/to/srvre_kernel.elf target remote localhost:1234 ``` Be sure to confirm the prompt from the `add-symbol-file` command. You can also use `riscv64-elf-gdb /path/to/srvre_kernel.elf` directly. # Translating device trees to the HWI (Hardware Info) format The HWI format was developed as a more robust and efficient alternative to standard device trees. The `hwi(1)` tool reads a textual representation from stdin and writes the corresponding binary representation to stdout. The values can be obtained from the device tree, but automated conversion is not implemented due to the complexity of the device tree format. ## Build hwi The `hwi(1)` tool can be built easily: ``` zig build-exe src/hwi.zig ``` This produces a `hwi` binary in the current working directory. ## The hwi tool You can use the `hwi(1)` command to convert from the textual representation to the binary representation: ``` hwi < src/lib/cfg/platform/.txt > src/lib/cfg/platform/.hwi ``` Omitting the stdin pipe allows you to type out the textual representation manually: ``` hwi > src/lib/cfg/platform/.hwi ``` Press Control+D after finishing the last line (and pressing Enter) to close stdin, quitting the tool. ## Textual format The textual HWI format is a list of entries where each entry is represented by a line of text. Separation is done using a single `\n` character. The entry format is as follows: ```

[VALUE] ``` * ``: The kind of the entry. Can be `cpus`, `plic`, `pcie` or `pci`. * `

`: The base address for memory-mapped I/O. Ignored (and preferably set to zero) if not applicable. * ``: The size of the memory-mapped I/O region. Ignored (and preferably set to zero) if not applicable. * `[VALUE]`: Optional additional (numeric) data. Defaults to zero if unspecified. Numeric values are represented as unsigned 64-bit integers and may be parsed from non-decimal notations. See [the documentation of std.fmt.parseUnsigned](https://ziglang.org/documentation/0.12.0/std/#std.fmt.parseUnsigned) for details on supported notations. ### cpus Gives information about the processors. MMIO is not applicable. The `VALUE` parameter holds the timebase frequency (used for interrupt timers). ### plic Gives information about the Platform-Level Interrupt Controller. MMIO is applicable. The `VALUE` parameter is ignored. ### pcie Gives information about an ECAM PCI(e) controller. MMIO is applicable. The `VALUE` parameter is ignored. ### pci Gives information about a legacy CAM PCI controller. MMIO is applicable. The `VALUE` parameter is ignored. ## Binary format The binary HWI format is a sequence of entry structures that isn't aware of endianness. The number of entries is `n/32` where `n` is the size of a valid HWI sequence in bytes. The entry format is as follows: ``` kind: enum(u32) { cpus, plic, pcie, pci } ALIGNMENT (u32) reg.addr: u64 reg.len: u64 value: u64 ```