Random corruption on SpacemiT K1 with RVV and THP

Random corruption on SpacemiT K1 with RVV and THP

[Aurelien Jarno]

Dear all,

For the last weeks, I have been tracking a random memory corruption and
relatively rare on SpacemiT K1 (Banana Pi F3 and Milk-V Jupiter). It 
started upgrading to glibc 2.43, which does memset() through vector 
instructions.  It is reproducible using the Debian 7.1.7-1~bpo13+1 
kernel, but I have also been able to reproduce it with a vanilla 7.2.2 
kernel, using a similar configuration to the Debian kernel. The board 
uses OpenSBI 1.9 and the vendor U-Boot.

Typically it manifests itself with the following kind of error, when 
running g++ from GCC 16 as part of building software (e.g. OpenJDK, 
Blender, Dolfin, Qt6):   

Assembler messages:
{standard input}:284588: Error: unknown pseudo-op: `.uleb1'
{standard input}:284588: Error: unrecognized opcode `ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿvl874'

The broken chars are 0xff and it seems there are always 240 (but with 
poor statistics). Sometimes it instead causes a GCC ICE instead.

Using glibc 2.44 instead of glibc 2.43, which does a lot of string 
operations through vector instructions, increases the probability to 
have corruption, makes it a bit more reproducible, but it always seems 
to manifest as a GCC ICE. It typically happens withing one hour when 
running on the 8 cores instead of every 1 or 2 days. From there I have 
been able to determine the following things:

- Disabling vector instructions (by patching the DTB to remove "v", 
  "zvhf" and "zvtk") fixes the issue

- Disabling THP (by setting /sys/kernel/mm/transparent_hugepage/enabled 
  to never instead of always) also seems to fix the issue. Keeping it 
  enabled with defrag=never or use_zero_page=0 doesn't change anything.

- The issue is reproducible with or without swap enabled.

I have not been able to reproduce the issue on other non RVV boards 
(Unmatched, VF2) nor on a SpacemiT K3 board.

I am not really sure how to debug that further. I tried a few ways to 
reproduce the issue with a small C code around the glibc memset code 
(including triggering unaligned accesses and page faults), but failed to 
do so. I therefore welcome any idea about the issue or how to debug it 
further.

Thanks
Aurelien

-- 
Aurelien Jarno                          GPG: 4096R/1DDD8C9B
aurelien@aurel32.net                     http://aurel32.net

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv