[Bug 2133188] Re: Illegal instruction in memset under qemu-user for riscv64

[Heinrich Schuchardt <2133188@bugs.launchpad.net>]

In the noble Docker container I have been running the following program:


#include <stdio.h>
#include <stdint.h>

#define VECTOR_SIZE 8  // Number of 64-bit elements

void store_vector_values(int64_t* buffer) {
    int64_t values[VECTOR_SIZE] = {0x1000000000000001, 0x2, 0x3, 0x4, 0x5, 0x6, 0x7, 0x8};

    // Inline assembly to load values into the vector register and store them
    asm volatile (
        "vsetvli t0, zero, e64, m2;"   // Set vector length to 2 elements of 64 bits
        "vle64.v v0, (%0);"            // Load values into vector register v0
        "vse64.v v0, (%1);"            // Store the contents of v0 into the buffer
        :
        : "r"(values), "r"(buffer)     // Input operands
        : "v0", "t0", "memory"         // Clobbered registers
    );
}

int main() {
    int64_t buffer[VECTOR_SIZE] = {0}; // Buffer to store 64-bit values

    for (int i = 0; i < VECTOR_SIZE; i++) {
        printf("buffer[%d] = 0x%lx\n", i, buffer[i]);
    }
    store_vector_values(buffer);

    // Output the stored values for verification
    printf("Stored vector values:\n");
    for (int i = 0; i < VECTOR_SIZE; i++) {
        printf("buffer[%d] = 0x%lx\n", i, buffer[i]);
    }

    return 0;
}

And received this output:

./test
buffer[0] = 0x0
buffer[1] = 0x0
buffer[2] = 0x0
buffer[3] = 0x0
buffer[4] = 0x0
buffer[5] = 0x0
buffer[6] = 0x0
buffer[7] = 0x0
Stored vector values:
buffer[0] = 0x1000000000000001
buffer[1] = 0x2
buffer[2] = 0x3
buffer[3] = 0x4
buffer[4] = 0x0
buffer[5] = 0x0
buffer[6] = 0x0
buffer[7] = 0x0

So it seems that QEMU can emulate the vse64.v instruction. It is not
clear to me why for "m2" four elements are transferred and not two.

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/2133188

Title:
  Illegal instruction in memset under qemu-user for riscv64

Status in QEMU:
  New
Status in qemu package in Ubuntu:
  Confirmed

Bug description:
  # Title
  qemu-user (qemu-riscv64-static): intermittent Illegal instruction in memset (vse64.v) when running cmake in riscv64 container (Ubuntu 26.04)

  ## Summary
  While running cmake (and other build steps) inside a linux/riscv64 Ubuntu 26.04 container on an x86_64 host using qemu-user (qemu-riscv64-static) registered via binfmt_misc, cmake sometimes crashes with "Illegal instruction (core dumped)" or "died with signal 4". The illegal instruction is observed inside glibc's memset implementation at an instruction that uses RISC-V vector extension (vse64.v). The failure is intermittent (~50% reproducer rate). Using a scalar-only memset (libnovecmem.so via LD_PRELOAD) or running under gdb / enabling QEMU_STRACE significantly reduces or eliminates the failure, which strongly suggests a qemu-user/emulation bug (vector handling / code generation / state corruption), not a cmake bug.

  ## Affects
  - qemu-user qemu-riscv64-static (as packaged in Ubuntu qemu 10.1.0+ds-5ubuntu3)
  - Running in Docker container for riscv64 on x86_64 host via binfmt_misc qemu-user static interpreter

  ## Environment / Context
  - Host CPU: x86_64 (Docker multiarch running qemu-user for riscv64)
  - Host OS：multiple Ubuntu releases (22.04, 24.04, 25.10) 
  - Container image: ubuntu:26.04 for riscv64
  - qemu package used:
    - downloaded .deb from Launchpad: qemu-user_10.1.0+ds-5ubuntu3_amd64.deb and on several Debian qemu-user packages (qemu-user_10.2.0~rc1+ds-1, qemu-user_10.0.6+ds-0+deb13u2). 
    - copied qemu-riscv64 binary into /usr/bin/qemu-riscv64-static inside host and registered via /proc/sys/fs/binfmt_misc/register
  - CMake version used inside container (bootstrap/build may use system-provided cmake binary): cmake 3.x (bootstrapping cmake while building also triggers crash)
  - Reproduction frequency: intermittent, ~50% (can get large variance: several consecutive successes or failures)
  - Observed behavior changes when: LD_PRELOAD libnovecmem.so (scalar memset) — almost completely avoids crash; running under gdb or enabling QEMU_STRACE also makes it much harder to reproduce.
    

  ## Full reproduction steps
  1. On x86_64 host, fetch qemu-user .deb and extract the riscv static binary:
     wget https://launchpad.net/ubuntu/+source/qemu/1:10.1.0+ds-5ubuntu3/+build/31393935/+files/qemu-user_10.1.0+ds-5ubuntu3_amd64.deb
     dpkg-deb -x qemu-user_10.1.0+ds-5ubuntu3_amd64.deb qemu-user_10.1.0+ds-5ubuntu3_amd64
     sudo cp qemu-user_10.1.0+ds-5ubuntu3_amd64/usr/bin/qemu-riscv64 /usr/bin/qemu-riscv64-static

  2. Register qemu-riscv64 with binfmt_misc:
     echo -1 > /proc/sys/fs/binfmt_misc/qemu-riscv64
     echo ':qemu-riscv64:M:0:\x7f\x45\x4c\x46\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\xf3\x00:\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff:/usr/bin/qemu-riscv64-static:POCF' >/proc/sys/fs/binfmt_misc/register

  3. Start riscv64 ubuntu container:
     docker run --platform=linux/riscv64 --name ubuntu26 -itd ubuntu:26.04 bash
     docker exec -it ubuntu26 bash -i

  4. Inside container:
     apt update
     apt install -y build-essential cmake

  5. Reproducer 1:
     cmake --system-information
     -> Often fails with:
        bash: [15: 1 (255)] tcsetattr: Inappropriate ioctl for device
        Illegal instruction (core dumped)

  6. Reproducer 2 (minimal C project):
     Create test_cmake/CMakeLists.txt:
     cmake_minimum_required(VERSION 3.10)
     project(HelloCMake C)
     add_executable(hello main.c)

     Create test_cmake/main.c:
     #include <stdio.h>
     int main() {
         printf("Hello, CMake!\n");
         return 0;
     }

     cd test_cmake
     cmake .
     -> Crash with:
        -- Detecting C compiler ABI info
        bash: line 1:  8489 Illegal instruction        (core dumped) cmake .

  7. Reproducer 3 (rebuild cmake from source inside container):
     apt source cmake
     cd cmake
     apt-get build-dep .
     dpkg-buildpackage -us -uc -b
     -> Bootstrapping error:
        Illegal instruction (core dumped)
        Error when bootstrapping CMake:
        Problem while running initial CMake

  8. Observed crash location (from gdb/QEMU_STRACE when available):
     - Illegal instruction is in memset@@GLIBC_2.27+0x52
     - Faulting instruction: vse64.v v1,(a5)    (RISC-V vector store of 64-bit elements)


  ## Workarounds
  - LD_PRELOAD a scalar-only memset library (libnovecmem.so) to avoid glibc using vectorized memset.
  - Run the failing process under gdb (slower) or enable QEMU_STRACE=1 — both make the failure much less likely.

  Note: The same workload does not reproduce the crash when run under
  qemu-system (full-system emulation). The issue appears specific to
  qemu-user

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/2133188/+subscriptions