Take a Linux root filesystem ‘tarball’ [1] built by cross-compiling for an ARM processor. Can a developer run processes from the filesystem using QEMU [2] on a non-ARM host? This assumes that the target root filesystem does not include an emulator; it contains the actual native ARM target root having no emulation because the target is an ARM processor.
Why useful? During an embedded development cycle, tests and checks might usefully run in an emulated environment. Continuous Integration could utilise such an emulated environment, using exactly the same root image as a firmware’s production deployment. The Buildroot [3] tools generate a rootfs.tar
archive. This ‘flashes’ to an embedded system’s integrated flash memory and launches in user-land1 when the microprocessor boots.
Steps to Launch
The answer requires only two Docker [4] steps: one to extract the tarball, and another to launch with the extracted volume. Detailed explanation comes later.
Run tar
to prepare a Docker volume. Assume that the current working directory contains the rootfs.tar
unzipped tarball.
docker run -it --rm -v rootfs:/mnt -v .:/home alpine tar -C /mnt -xf /home/rootfs.tar
Run an ARM-based Alpine [5], mounting the prepared volume and ’change root’ing to the mounted volume.
docker run -it --rm -v rootfs:/mnt --privileged --platform linux/arm arm32v7/alpine chroot /mnt bash -l
The result is a connection to Bash running via QEMU on ARM32 v7 with the prepared root filesystem. Most things work as expected but not all.
root@4123ec6c14d3:/# mount proc /proc -t proc
root@4123ec6c14d3:/# ps ax
PID TTY STAT TIME COMMAND
1 ? Ssl 0:00 /usr/bin/qemu-arm /usr/bin/bash bash -l
19 ? Rl+ 0:00 ps ax
Explanation
Points to note about Docker volumes:
They exist within Docker as mountable
ext4
filesystems.Docker requires absolute mount paths.
The volume does not need to pre-exist. Docker creates a new empty volume if not found by name.
Preparing the volume (the first step) runs tar
from a temporary Alpine container with the rootfs
volume bound to mount point /mnt
—for simplicity. The current directory .
binds to /home
, again to keep things simple. The tar
process launches, changes the directory to /mnt
and extracts the tarball at /home/rootfs.tar
. Mission accomplished. Docker removes the container.
Running chroot
from the extracted root filesystem volume executes Bash with -l
so that it acts as a log-in shell. The container specifies linux/arm
as the platform and arm32v7/alpine
as the image.
Weaknesses
Not all things work as expected. The proc
filesystem is not available. Hence, commands such as ps
will fail. Mount as mount proc /proc -t proc
which requires a --privileged
container. Even when mounted, the proc
system reflects the host not the target. Cat’ing /proc/cpuinfo
gives Intel cores, not ARM, for example. Notice that qemu-arm
appears in the process table. The emulation only propagates throughout user-land Linux not the kernel itself.
Nevertheless, /etc/init.d/rcK
and /etc/init.d/rcS
kernel and system startup scripts can launch albeit noisily since host hardware fails to match the target hardware. The host is an Intel-based Windows machine running Docker Desktop, not an embedded ARM.
User “land” where normal processes operate outside kernel “space”↩︎