3 min read

Buildroot Tarball on Docker ARM

Take a Linux root filesystem ‘tarball’ [1] built by cross-compiling for an ARM processor. Can a developer run processes from the filesystem using QEMU [2] on a non-ARM host? This assumes that the target root filesystem does not include an emulator; it contains the actual native ARM target root having no emulation because the target is an ARM processor.

Why useful? During an embedded development cycle, tests and checks might usefully run in an emulated environment. Continuous Integration could utilise such an emulated environment, using exactly the same root image as a firmware’s production deployment. The Buildroot [3] tools generate a rootfs.tar archive. This ‘flashes’ to an embedded system’s integrated flash memory and launches in user-land1 when the microprocessor boots.

Steps to Launch

The answer requires only two Docker [4] steps: one to extract the tarball, and another to launch with the extracted volume. Detailed explanation comes later.

Run tar to prepare a Docker volume. Assume that the current working directory contains the rootfs.tar unzipped tarball.

docker run -it --rm -v rootfs:/mnt -v .:/home alpine tar -C /mnt -xf /home/rootfs.tar

Run an ARM-based Alpine [5], mounting the prepared volume and ’change root’ing to the mounted volume.

docker run -it --rm -v rootfs:/mnt --privileged --platform linux/arm arm32v7/alpine chroot /mnt bash -l

The result is a connection to Bash running via QEMU on ARM32 v7 with the prepared root filesystem. Most things work as expected but not all.

root@4123ec6c14d3:/# mount proc /proc -t proc
root@4123ec6c14d3:/# ps ax
    PID TTY      STAT   TIME COMMAND
      1 ?        Ssl    0:00 /usr/bin/qemu-arm /usr/bin/bash bash -l
     19 ?        Rl+    0:00 ps ax

Explanation

Points to note about Docker volumes:

  • They exist within Docker as mountable ext4 filesystems.

  • Docker requires absolute mount paths.

  • The volume does not need to pre-exist. Docker creates a new empty volume if not found by name.

Preparing the volume (the first step) runs tar from a temporary Alpine container with the rootfs volume bound to mount point /mnt—for simplicity. The current directory . binds to /home, again to keep things simple. The tar process launches, changes the directory to /mnt and extracts the tarball at /home/rootfs.tar. Mission accomplished. Docker removes the container.

Running chroot from the extracted root filesystem volume executes Bash with -l so that it acts as a log-in shell. The container specifies linux/arm as the platform and arm32v7/alpine as the image.

Weaknesses

Not all things work as expected. The proc filesystem is not available. Hence, commands such as ps will fail. Mount as mount proc /proc -t proc which requires a --privileged container. Even when mounted, the proc system reflects the host not the target. Cat’ing /proc/cpuinfo gives Intel cores, not ARM, for example. Notice that qemu-arm appears in the process table. The emulation only propagates throughout user-land Linux not the kernel itself.

Nevertheless, /etc/init.d/rcK and /etc/init.d/rcS kernel and system startup scripts can launch albeit noisily since host hardware fails to match the target hardware. The host is an Intel-based Windows machine running Docker Desktop, not an embedded ARM.

[1]
U. of E. School of Informatics, “What’s a tarball?” Available: https://computing.help.inf.ed.ac.uk/FAQ/whats-tarball-or-how-do-i-unpack-or-create-tgz-or-targz-file
[2]
QEMU, “A generic and open source machine emulator and virtualizer.” Accessed: Oct. 07, 2023. [Online]. Available: https://www.qemu.org/
[3]
Buildroot, “Making Embedded Linux Easy.” Accessed: Oct. 07, 2023. [Online]. Available: https://buildroot.org/
[4]
Docker, “Docker desktop: The #1 containerization tool for developers.” Oct. 06, 2021. Available: https://www.docker.com/products/docker-desktop/
[5]
A. Linux, “Alpine linux is a security-oriented, lightweight linux distribution based on musl libc and busybox.” Accessed: Oct. 07, 2023. [Online]. Available: https://www.alpinelinux.org/

  1. User “land” where normal processes operate outside kernel “space”↩︎