[PATCH] MIPS: Make cpu_has_mips64r6 constant on 64-bit r6 kernels

[PATCH] MIPS: Make cpu_has_mips64r6 constant on 64-bit r6 kernels

[Tzu-Ching Yang]

On 64-bit r6 kernels, __fls() and __ffs() never use dclz: they always
run the C binary search, 37 and 39 instructions with GCC instead of 4
and 6. __ffs() calls __fls(), which uses dclz only if cpu_has_mips64
is a compile-time constant:

	if (BITS_PER_LONG == 64 && !__builtin_constant_p(word) &&
	    __builtin_constant_p(cpu_has_mips64) && cpu_has_mips64) {

cpu_has_mips64 ORs cpu_has_mips64r1/r2/r5/r6. On r6 kernels the r1, r2
and r5 terms fold to 0, but the r6 term tests cpu_data[0].isa_level, so
64-bit r6 kernels have never used dclz here.

The r1, r2 and r6 macros were run-time tests until commit 1aeba347b3a9
("MIPS: Hardcode cpu_has_mips* where target ISA allows") made r1 and r2
constant in 2018; it kept for r6 the flag test that tells MIPS64r6 from
MIPS32r6 CPUs. Commit a2ac81c6ef40 ("mips: Fix cpu_has_mips64r1/2
activation for MIPS32 CPUs") later added cpu_has_64bits to r1 and r2,
and left r6 alone. Every other cpu_has_mips32rN and cpu_has_mips64rN
macro is a constant on the kernels built for its ISA (the 64-bit ones
for cpu_has_mips64rN); only cpu_has_mips64r6 still tests the flag at run
time on 64-bit r6 kernels.

A 64-bit r6 kernel only runs on a MIPS64r6 CPU, so define
cpu_has_mips64r6 as cpu_has_64bits && MIPS_ISA_REV >= 6, like r1/r2/r5
without the flag test. It folds to 1 on 64-bit r6 kernels, and pre-r6
kernels build to identical code. On 32-bit r6 kernels, cpu_has_64bits
tests isa_level & MIPS_CPU_ISA_64BIT, and set_isa() gives r6 CPUs no
such flag except MIPS_CPU_ISA_M64R6, so the run-time result is
unchanged. No __isa_*_flag() helper fits: a pre-r6 kernel cannot run on
an r6 CPU and vice versa, so the flag test adds nothing, yet a helper
keeps it as a run-time load in r6 kernels (__isa_ge_and_flag(), the
current form) or in pre-r6 ones (__isa_ge_or_flag()). The r1/r2/r5-like
cpu_has_64bits && __isa_ge_or_flag(6, ...) grows 32r2el by 3352 bytes
and 64r2el by 264 bytes with GCC 16.

Because __fls() and __ffs() are inlined into find_*_bit(), cpumask
iteration and ilog2(), vmlinux of the 64-bit r6 defconfigs shrinks by
105-108 KB with GCC 16 and 124-147 KB with Clang 23
(scripts/bloat-o-meter -t). In the 64r6el and eyeq6 builds, the other
users of cpu_has_mips64r6 change 3 functions, by 148-248 bytes in all;
the rest comes through these two functions. __fls(0) now returns ~0UL
instead of 0 on 64-bit r6 kernels, as its comment says and as on 64-bit
r2 kernels.

Signed-off-by: Tzu-Ching Yang <shingekinocore@gmail.com>
---
No 64-bit r6 hardware was available: every run-time test below ran on
QEMU. A run of the ffs and bitops KUnit tests and find_bit_benchmark
on an EyeQ or Boston board, before and after, would be welcome.

Instructions of a function that only returns __fls(x) or __ffs(x)
(return included), compiled alone with the lib/find_bit.o command of
64r6el, before -> after:

               __fls     __ffs
  GCC 8-16     37 -> 4   39 -> 6
  Clang 17-19  62 -> 4   64 -> 6
  Clang 20     61 -> 4   63 -> 6
  Clang 21-23  55 -> 4   57 -> 6

The new __fls() with GCC 16:

	li      v0,63
	dclz    a0,a0
	jr      ra
	subu    v0,v0,a0

find_bit_benchmark built into 64r6el on QEMU boston (I6400 model),
random-filled bitmap, executed instructions per iteration counted by a
TCG plugin, before -> after:

                                  GCC 16           Clang 23
  find_next_bit                   53.1 -> 28.4     86.0 -> 34.5
  find_next_zero_bit              55.1 -> 29.4     87.1 -> 35.5
  find_last_bit                   54.1 -> 29.4     76.9 -> 27.4
  find_next_and_bit               60.5 -> 34.7     90.4 -> 38.8
  bitmap_find_next_zero_area_off  3839.7 -> 2176.4 6045.6 -> 2625.9

The other three tests (find_first_bit, find_first_and_bit and
find_nth_bit) spend their time scanning words or counting bits rather
than in __fls(), and change by 4% or less. The helpers themselves shrink
alike: _find_next_bit() goes from 61 to 31 instructions with GCC 16 and
from 85 to 34 with Clang 23.

bloat-o-meter -t on vmlinux of the 64r6, 64r6el, eyeq5, eyeq6 and
eyeq6lplus defconfigs, bfe97f12a0a6 before and after with the same
.config, kernel.org GCC 16.2.0 and Clang 23.1.3: -105340 to -108352
bytes with GCC 16, -124340 to -146692 bytes with Clang 23. Of the 105 KB
of 64r6el with GCC 16, inlined bit search (find_*_bit(), mostly cpumask
iteration) accounts for 74.4 KB, log2 math (ilog2(), get_order(),
roundup_pow_of_two()) for 21.6 KB and direct __ffs()/__fls() calls
(lib/find_bit.c among them) for 7.0 KB, and secondary inlining for most
of the rest. With GCC 16, 13 symbols grow and one function is added
(1712 bytes in all): 608 bytes are the alignment gap that bloat-o-meter
assigns to the __unlikely_text_end marker, 64 bytes are two loops that
GCC lays out differently, and the rest are callers into which a now
smaller callee with a bit search is inlined, or changes from GCC's
inline growth limit and IPA-RA; the ten largest growers with Clang 23
are callers into which a callee with a cpumask loop or a bit search is
inlined.

__isa_ge_and_flag() has no user left after this; I left it next to
the other __isa_*() helpers, but can drop it if you prefer.

Switching MIPS to the generic builtin helpers would fix this too, but
was dropped in 2019 because the builtins become libgcc calls on pre-r1
ISAs:
https://lore.kernel.org/all/20191001215249.4157062-1-paul.burton@mips.com/

Testing: built before and after (W=1, no new warnings) with GCC 8.1
to 16.2 and Clang 17 to 23 for 64r6el, eyeq5, eyeq6, 32r6el and
64r2el, and with GCC 16 and Clang 23 for 64r6, eyeq6lplus,
malta_qemu_32r6 and a 64-bit r6 allmodconfig; Clang 17 to 20 cannot
assemble genex.S before or after, so they were compared on objects.
Pre-r6 kernels build to identical code (64r2el and 32r2el with every
compiler; malta, ip27, loongson3 and cavium_octeon with GCC 16).
64r6el on QEMU boston (I6400) and malta_qemu_32r6 on QEMU malta
(mips32r6-generic and I6400), built with GCC 16 and Clang 23, pass
the ffs and bitops KUnit suites and test_bitmap and print the same
"isa" line in /proc/cpuinfo before and after.

Against the mips-next branch of mips/linux.git (bfe97f12a0a6); also
applies to next-20261029.

 arch/mips/include/asm/cpu-features.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/mips/include/asm/cpu-features.h b/arch/mips/include/asm/cpu-features.h
index 3f11e5218e6c..2842e22c69ea 100644
--- a/arch/mips/include/asm/cpu-features.h
+++ b/arch/mips/include/asm/cpu-features.h
@@ -317,7 +317,7 @@
 				 __isa_range_or_flag(5, 6, MIPS_CPU_ISA_M64R5))
 #endif
 #ifndef cpu_has_mips64r6
-# define cpu_has_mips64r6	__isa_ge_and_flag(6, MIPS_CPU_ISA_M64R6)
+# define cpu_has_mips64r6	(cpu_has_64bits && MIPS_ISA_REV >= 6)
 #endif
 
 /*

base-commit: bfe97f12a0a64c3682be21a02404238e3b91e945
-- 
2.34.1

Re: [PATCH] MIPS: Make cpu_has_mips64r6 constant on 64-bit r6 kernels

[Maciej W. Rozycki]

On Fri, 30 Oct 2026, Tzu-Ching Yang wrote:

> diff --git a/arch/mips/include/asm/cpu-features.h b/arch/mips/include/asm/cpu-features.h
> index 3f11e5218e6c..2842e22c69ea 100644
> --- a/arch/mips/include/asm/cpu-features.h
> +++ b/arch/mips/include/asm/cpu-features.h
> @@ -317,7 +317,7 @@
>  				 __isa_range_or_flag(5, 6, MIPS_CPU_ISA_M64R5))
>  #endif
>  #ifndef cpu_has_mips64r6
> -# define cpu_has_mips64r6	__isa_ge_and_flag(6, MIPS_CPU_ISA_M64R6)
> +# define cpu_has_mips64r6	(cpu_has_64bits && MIPS_ISA_REV >= 6)

 Umm, the wrong fix.  You want __isa_ge_or_flag() instead, plus obviously 
`cpu_has_64bits' is also needed (and __isa_ge_and_flag() can be discarded; 
I guess it was meant to take care of what `cpu_has_64bits' does and broke 
the whole idea by mistake).  Your proposal precludes the use of R6 stuff 
with pre-R6 compilations -- not sure if that makes sense, but let's stay 
consistent with these macro definitions.

  Maciej