srvre kernel
Simple RISC-V research environment kernel
This is the microkernel for the SRVRE riscv64 operating system.
Its purpose is learning RISC-V OS development and trying to experiment with solutions to some of the shortcomings of existing operating systems. The ultimate goal is to make it compile itself, maybe even to make it usable for limited command-line or server usage.
See the project page for more information.
Build a bootable image
Boot flow
This kernel is mainly developed for qemu-virt64 and the Lichee Pi 4A. The latter uses the following boot flow:
~ M-mode ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~|~ S-mode ~~~
+------+ +------------+ +--------+ +---------+ +--------+
| brom | --> | U-Boot SPL | --> | U-Boot | --> | OpenSBI | --> | Kernel |
+------+ +------------+ +--------+ +---------+ +--------+
QEMU can be adapted to use this boot flow as well (the default is different).
Build the kernel
Required dependencies:
- zig (most recent release version at the time of the latest commit)
- init executable (see below)
To make a debug build of this kernel, run:
zig build -Dplatform=<PLATFORM>
Replace <PLATFORM> with the platform you want to build for.
Supported options include qemu and lpi4a.
See the src/cfg/platform directory for the full list.
You can also use any other Zig build mode, e.g. --release=fast.
This results in a zig-out/bin/srvre_kernel.elf file.
You may strip(1) this file if you want to.
init executable
The init executable is expected to be a statically linked ELF
(with program headers) at src/cfg/init. It is embedded in the kernel
binary to avoid running (filesystem) drivers in S-mode, meaning that a kernel
rebuild is required to apply modifications.
Example programs are provided in the examples/ directory.
Build OpenSBI
Required dependencies:
- riscv64-linux-gnu-gcc
Clone the OpenSBI repository and run the following command. If building for the Lichee Pi 4A, use the T-Head OpenSBI fork instead.
make CROSS_COMPILE=riscv64-linux-gnu- PLATFORM=generic FW_DYNAMIC=y -j $(nproc) all
This results in a build/platform/generic/firmware/fw_dynamic.bin file.
You might be able to use precompiled versions of this file from other sources.
Build U-Boot
Required dependencies:
- riscv64-linux-gnu-gcc
- bison
- flex
- python-setuptools
- swig
The boot flow is not supported by upstream U-Boot. There are several T-Head-specific forks that implement it for the Linux kernel, but none that support regular ELF binaries. Therefore a custom fork of U-Boot is required.
QEMU
Clone the U-Boot fork,
check out the sbiboot branch and run the following commands:
make qemu-riscv64_spl_defconfig
make -j$(nproc) menuconfig
This will bring up the build configuration tool. Set the following options:
- Device Drivers > Serial > Base address of UART (CONFIG_DEBUG_UART_BASE): 0x10000000
- Device Drivers > Serial > Base address of UART for SPL (CONFIG_SPL_DEBUG_UART_BASE): 0x10000000
- Device Drivers > Serial > Select which UART will provide the debug UART (CONFIG_DEBUG_UART_NS16550=y): ns16550
- Device Drivers > Serial > Check if UART is enabled on output (CONFIG_DEBUG_UART_NS16550_CHECK_ENABLED): Yes
- RISC-V architecture > Symmetric Multi-Processing (CONFIG_SMP): Yes
- RISC-V architecture > Symmetric Multi-Processing in SPL (CONFIG_SPL_SMP): Yes
- RISC-V architecture > Run Mode (CONFIG_RISCV_MMODE=y): Machine
- RISC-V architecture > SPL Run Mode (CONFIG_SPL_RISCV_MMODE=y): Machine
- SPL configuration options > Show more information when something goes wrong (CONFIG_SPL_SHOW_ERRORS): Yes
- SPL configuration options > Offset to which the SPL should be padded before appending the SPL payload (CONFIG_SPL_PAD_TO): 0x1200000
- Boot options > Boot images > Flattened Image Tree (FIT) > Enable SPL loading U-Boot as a FIT (basic fitImage features) (CONFIG_SPL_LOAD_FIT): No
- Boot options > bootcmd value (CONFIG_BOOTCOMMAND): "virtio scan; fatload virtio 0 0x8f000000 uImage; bootm 0x8f000000"
- (General setup > Support for multiprocessor (?): No; Won't compile otherwise)
Then exit and select 'yes' to save your changes.
Lichee Pi 4A
If building for the Lichee Pi 4A, use the T-Head-specific U-Boot fork instead and run the following command:
make light_lpi4a_defconfig
For the 16 GB variant, use a different config:
make light_lpi4a_16g_defconfig
Build the binaries
Finally, build the binaries:
make ARCH=riscv CROSS_COMPILE=riscv64-linux-gnu- -j $(nproc) u-boot-with-spl.bin
Create multi-file kernel image
Create a legacy U-Boot image:
/path/to/u-boot/tools/mkimage -A riscv -O elf -T multi -C none -a 0x80400000 -e 0x80400000 -n SRVRE -d /path/to/srvre_kernel.elf:/path/to/fw_dynamic.bin:/path/to/devicetree.dtb /path/to/output/uImage
Write this uImage file to the first, FAT32 (LBA) partition
on an MBR partitioned drive. You should now be able to boot.
Extract QEMU device tree
The default arch/riscv/dts/qemu-virt64.dtb tree is an image tree and doesn't help.
To extract the device tree, create an empty file on the disk image and unmount it.
Then boot the VM as explained above but hit a key before it autoboots.
Run the following command in the U-Boot shell:
env print fdtcontroladdr
Then use the printed value like so (without the angle brackets):
fdt addr 0x<ENV_OUTPUT>
fdt header
You should now see a totalsize value. Take note of the hexadecimal notation
and save it to disk:
virtio scan
fatwrite virtio 0 0x<ENV_OUTPUT> <EMPTY_FILE_NAME> <HEX_TOTALSIZE>
You can now poweroff, mount the partition again and build an image
using the commands above.
Start QEMU
Run the following command:
qemu-system-riscv64 -M virt -m 2G -smp 4 -display none -serial stdio -bios /path/to/u-boot-with-spl.bin -drive file=/path/to/disk/image,if=virtio,media=disk,format=raw
Flash to Lichee Pi 4A
Required dependencies:
- android-sdk-platform-tools
Hold down the BOOT button on the board, connect it to your computer
using USB-C and release the button. Create an image file, format it
as ext4 (using dd(1) and mkfs.ext4(1)) and mount it.
Copy the uImage file to it, unmount it and flush the changes to disk
(using sync(1)). Then run the following commands as root:
fastboot flash ram /path/to/thead-u-boot/u-boot-with-spl.bin
fastboot reboot
# The next command is going to wait for the device for a few seconds, be patient
fastboot flash uboot /path/to/thead-u-boot/u-boot-with-spl.bin
fastboot flash boot /path/to/image/file.img
You can now reboot the device and use the serial (UART0) console to start the kernel by entering the U-Boot shell (by pressing a key to stop autoboot or by waiting for it to fail) and typing:
ext4load mmc 0:2 0x8f000000 uImage
bootm 0x8f000000
Boot protocol
The kernel expects to be loaded in S-mode with the following register values:
- a0: Current Hart ID
Typically you'll want to load it directly from OpenSBI. Depending on your platform you may use a slightly patched version of U-Boot that runs in M-mode to boot a combined kernel / OpenSBI image. This process is documented above.
Completely skipping U-Boot and using OpenSBI directly should work if your platform supports it. However it is currently untested.
The advantage of this boot flow is that it doesn't require the use of U-Boot. In theory, anything that sets the registers as described above and runs the kernel in S-mode should work, so any SBI should work if you can get it to run.
Debugging
You can run QEMU with the -s -S flags and attach riscv64-elf-gdb:
add-symbol-file /path/to/srvre_kernel.elf
target remote localhost:1234
Be sure to confirm the prompt from the add-symbol-file command.
You can also use riscv64-elf-gdb /path/to/srvre_kernel.elf directly.
Translating device trees to the HWI (Hardware Info) format
The HWI format was developed as a more robust and efficient alternative to
standard device trees. The hwi tool reads a text representation from stdin
and writes the corresponding binary representation to stdout. The values can
be obtained from the device tree, but automated conversion is not implemented
due to the complexity of the device tree format.
Build hwi
The hwi tool can be built easily:
zig build-exe src/hwi.zig
This produces a hwi binary in the current working directory.
Use hwi tool
You can use the hwi command to convert from the text representation to the
binary representation:
hwi < src/cfg/platform/<PLATFORM>.txt > src/cfg/platform/<PLATFORM>.hwi
Omitting the stdin pipe allows you to type out the textual representation manually:
hwi > src/cfg/platform/<PLATFORM>.hwi
Press Control+D after finishing the last line (and pressing Enter) to close stdin, quitting the tool.