[Bug 2133188] Re: Illegal instruction in memset under qemu-user for riscv64

[Heinrich Schuchardt <2133188@bugs.launchpad.net>]

I have created RISC-V containers for Ubuntu 25.04, 25.10, and 26.10 as
follows:

sudo -s

wget http://cdimage.ubuntu.com/ubuntu-base/daily/current/resolute-base-riscv64.tar.gz
cat > Dockerfile_resolute << eof
FROM scratch
ADD resolute-base-riscv64.tar.gz /
CMD ["bash"]
eof
docker build -t lp2133188_resolute -f Dockerfile_resolute .
docker run -ti lp2133188_resolute /bin/bash


wget http://cdimage.ubuntu.com/ubuntu-base/releases/questing/release/ubuntu-base-25.10-base-riscv64.tar.gz
cat > Dockerfile_questing << eof
FROM scratch
ADD ubuntu-base-25.10-base-riscv64.tar.gz /
CMD ["bash"]
eof
docker build -t lp2133188_questing -f Dockerfile_questing .
docker run -ti lp2133188_questing /bin/bash


wget http://cdimage.ubuntu.com/ubuntu-base/releases/plucky/release/ubuntu-base-25.04-base-riscv64.tar.gz
cat > Dockerfile_plucky << eof
FROM scratch
ADD ubuntu-base-25.04-base-riscv64.tar.gz /
CMD ["bash"]
eof
docker build -t lp2133188_plucky -f Dockerfile_plucky .
docker run -ti lp2133188_plucky /bin/bash


In container lp2133188_resolute and lp2133188_questing /root is empty.
No prompt is shown when entering.
export PS1=$ has not effect.

Container lp2133188_plucky shows a console prompt 'root@817acf060fbf:/#'
defined by PS1.

So it seems we removed something between Plucky and Questing that is
needed for a proper console in Docker.

This seems to be the explanation for the observed error

   bash: [15: 1 (255)] tcsetattr: Inappropriate ioctl for device
   
seen in cmake which is not correctly handled.

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/2133188

Title:
  Illegal instruction in memset under qemu-user for riscv64

Status in QEMU:
  New
Status in qemu package in Ubuntu:
  Confirmed

Bug description:
  # Title
  qemu-user (qemu-riscv64-static): intermittent Illegal instruction in memset (vse64.v) when running cmake in riscv64 container (Ubuntu 26.04)

  ## Summary
  While running cmake (and other build steps) inside a linux/riscv64 Ubuntu 26.04 container on an x86_64 host using qemu-user (qemu-riscv64-static) registered via binfmt_misc, cmake sometimes crashes with "Illegal instruction (core dumped)" or "died with signal 4". The illegal instruction is observed inside glibc's memset implementation at an instruction that uses RISC-V vector extension (vse64.v). The failure is intermittent (~50% reproducer rate). Using a scalar-only memset (libnovecmem.so via LD_PRELOAD) or running under gdb / enabling QEMU_STRACE significantly reduces or eliminates the failure, which strongly suggests a qemu-user/emulation bug (vector handling / code generation / state corruption), not a cmake bug.

  ## Affects
  - qemu-user qemu-riscv64-static (as packaged in Ubuntu qemu 10.1.0+ds-5ubuntu3)
  - Running in Docker container for riscv64 on x86_64 host via binfmt_misc qemu-user static interpreter

  ## Environment / Context
  - Host CPU: x86_64 (Docker multiarch running qemu-user for riscv64)
  - Host OS：multiple Ubuntu releases (22.04, 24.04, 25.10) 
  - Container image: ubuntu:26.04 for riscv64
  - qemu package used:
    - downloaded .deb from Launchpad: qemu-user_10.1.0+ds-5ubuntu3_amd64.deb and on several Debian qemu-user packages (qemu-user_10.2.0~rc1+ds-1, qemu-user_10.0.6+ds-0+deb13u2). 
    - copied qemu-riscv64 binary into /usr/bin/qemu-riscv64-static inside host and registered via /proc/sys/fs/binfmt_misc/register
  - CMake version used inside container (bootstrap/build may use system-provided cmake binary): cmake 3.x (bootstrapping cmake while building also triggers crash)
  - Reproduction frequency: intermittent, ~50% (can get large variance: several consecutive successes or failures)
  - Observed behavior changes when: LD_PRELOAD libnovecmem.so (scalar memset) — almost completely avoids crash; running under gdb or enabling QEMU_STRACE also makes it much harder to reproduce.
    

  ## Full reproduction steps
  1. On x86_64 host, fetch qemu-user .deb and extract the riscv static binary:
     wget https://launchpad.net/ubuntu/+source/qemu/1:10.1.0+ds-5ubuntu3/+build/31393935/+files/qemu-user_10.1.0+ds-5ubuntu3_amd64.deb
     dpkg-deb -x qemu-user_10.1.0+ds-5ubuntu3_amd64.deb qemu-user_10.1.0+ds-5ubuntu3_amd64
     sudo cp qemu-user_10.1.0+ds-5ubuntu3_amd64/usr/bin/qemu-riscv64 /usr/bin/qemu-riscv64-static

  2. Register qemu-riscv64 with binfmt_misc:
     echo -1 > /proc/sys/fs/binfmt_misc/qemu-riscv64
     echo ':qemu-riscv64:M:0:\x7f\x45\x4c\x46\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\xf3\x00:\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff:/usr/bin/qemu-riscv64-static:POCF' >/proc/sys/fs/binfmt_misc/register

  3. Start riscv64 ubuntu container:
     docker run --platform=linux/riscv64 --name ubuntu26 -itd ubuntu:26.04 bash
     docker exec -it ubuntu26 bash -i

  4. Inside container:
     apt update
     apt install -y build-essential cmake

  5. Reproducer 1:
     cmake --system-information
     -> Often fails with:
        bash: [15: 1 (255)] tcsetattr: Inappropriate ioctl for device
        Illegal instruction (core dumped)

  6. Reproducer 2 (minimal C project):
     Create test_cmake/CMakeLists.txt:
     cmake_minimum_required(VERSION 3.10)
     project(HelloCMake C)
     add_executable(hello main.c)

     Create test_cmake/main.c:
     #include <stdio.h>
     int main() {
         printf("Hello, CMake!\n");
         return 0;
     }

     cd test_cmake
     cmake .
     -> Crash with:
        -- Detecting C compiler ABI info
        bash: line 1:  8489 Illegal instruction        (core dumped) cmake .

  7. Reproducer 3 (rebuild cmake from source inside container):
     apt source cmake
     cd cmake
     apt-get build-dep .
     dpkg-buildpackage -us -uc -b
     -> Bootstrapping error:
        Illegal instruction (core dumped)
        Error when bootstrapping CMake:
        Problem while running initial CMake

  8. Observed crash location (from gdb/QEMU_STRACE when available):
     - Illegal instruction is in memset@@GLIBC_2.27+0x52
     - Faulting instruction: vse64.v v1,(a5)    (RISC-V vector store of 64-bit elements)


  ## Workarounds
  - LD_PRELOAD a scalar-only memset library (libnovecmem.so) to avoid glibc using vectorized memset.
  - Run the failing process under gdb (slower) or enable QEMU_STRACE=1 — both make the failure much less likely.

  Note: The same workload does not reproduce the crash when run under
  qemu-system (full-system emulation). The issue appears specific to
  qemu-user

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/2133188/+subscriptions