Re: [PATCH v2 6/8] arm64: alternatives: have callbacks take a cap

Re: [PATCH v2 6/8] arm64: alternatives: have callbacks take a cap

[Jon Hunter]

Hi Mark,

On 12/09/2022 17:22, Mark Rutland wrote:
> Today, callback alternatives are special-cased within
> __apply_alternatives(), and are applied alongside patching for system
> capabilities as ARM64_NCAPS is not part of the boot_capabilities feature
> mask.
> 
> This special-casing is less than ideal. Giving special meaning to
> ARM64_NCAPS for this requires some structures and loops to use
> ARM64_NCAPS + 1 (AKA ARM64_NPATCHABLE), while others use ARM64_NCAPS.
> It's also not immediately clear callback alternatives are only applied
> when applying alternatives for system-wide features.
> 
> To make this a bit clearer, changes the way that callback alternatives
> are identified to remove the special-casing of ARM64_NCAPS, and to allow
> callback alternatives to be associated with a cpucap as with all other
> alternatives.
> 
> New cpucaps, ARM64_ALWAYS_BOOT and ARM64_ALWAYS_SYSTEM are added which
> are always detected alongside boot cpu capabilities and system
> capabilities respectively. All existing callback alternatives are made
> to use ARM64_ALWAYS_SYSTEM, and so will be patched at the same point
> during the boot flow as before.
> 
> Subsequent patches will make more use of these new cpucaps.
> 
> There should be no functional change as a result of this patch.
> 
> Signed-off-by: Mark Rutland <mark.rutland@arm.com>
> Cc: Ard Biesheuvel <ardb@kernel.org>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: James Morse <james.morse@arm.com>
> Cc: Joey Gouly <joey.gouly@arm.com>
> Cc: Marc Zyngier <maz@kernel.org>
> Cc: Will Deacon <will@kernel.org>
> ---
>   arch/arm64/include/asm/alternative-macros.h | 18 +++++++++-----
>   arch/arm64/include/asm/assembler.h          | 10 ++++----
>   arch/arm64/include/asm/cpufeature.h         |  4 +---
>   arch/arm64/include/asm/kvm_mmu.h            |  5 ++--
>   arch/arm64/kernel/alternative.c             | 26 +++++++++++----------
>   arch/arm64/kernel/cpufeature.c              | 19 +++++++++++++--
>   arch/arm64/kernel/entry.S                   |  8 +++----
>   arch/arm64/kvm/hyp/hyp-entry.S              |  4 ++--
>   arch/arm64/tools/cpucaps                    |  2 ++
>   9 files changed, 60 insertions(+), 36 deletions(-)
> 
> diff --git a/arch/arm64/include/asm/alternative-macros.h b/arch/arm64/include/asm/alternative-macros.h
> index 7e157ab6cd505..189c31be163ce 100644
> --- a/arch/arm64/include/asm/alternative-macros.h
> +++ b/arch/arm64/include/asm/alternative-macros.h
> @@ -2,10 +2,16 @@
>   #ifndef __ASM_ALTERNATIVE_MACROS_H
>   #define __ASM_ALTERNATIVE_MACROS_H
>   
> +#include <linux/const.h>
> +
>   #include <asm/cpucaps.h>
>   #include <asm/insn-def.h>
>   
> -#define ARM64_CB_PATCH ARM64_NCAPS
> +#define ARM64_CB_BIT	(UL(1) << 15)
> +
> +#if ARM64_NCAPS >= ARM64_CB_BIT
> +#error "cpucaps have overflown ARM64_CB_BIT"
> +#endif


Some of our builders are failing and bisect is pointing to this commit. 
Looks like they don't like the above and I see the following errors ...

   CC      arch/arm64/kvm/hyp/vhe/debug-sr.o
/tmp/ccY3kbki.s: Assembler messages:
/tmp/ccY3kbki.s:1600: Error: found 'L', expected: ')'
/tmp/ccY3kbki.s:1600: Error: found 'L', expected: ')'
/tmp/ccY3kbki.s:1600: Error: found 'L', expected: ')'
/tmp/ccY3kbki.s:1600: Error: found 'L', expected: ')'
/tmp/ccY3kbki.s:1600: Error: junk at end of line, first unrecognized 
character is `L'
/tmp/ccY3kbki.s:1723: Error: found 'L', expected: ')'
/tmp/ccY3kbki.s:1723: Error: found 'L', expected: ')'
/tmp/ccY3kbki.s:1723: Error: found 'L', expected: ')'
/tmp/ccY3kbki.s:1723: Error: found 'L', expected: ')'
/tmp/ccY3kbki.s:1723: Error: junk at end of line, first unrecognized 
character is `L'
scripts/Makefile.build:249: recipe for target 
'arch/arm64/kvm/hyp/vhe/debug-sr.o' failed

Seems that it does not like the 'UL' macro for some reason. Any thoughts?

Thanks
Jon

-- 
nvpublic

Re: [PATCH v2 6/8] arm64: alternatives: have callbacks take a cap

[Jon Hunter]

On 27/09/2022 10:31, Jon Hunter wrote:

...

>> diff --git a/arch/arm64/include/asm/alternative-macros.h 
>> b/arch/arm64/include/asm/alternative-macros.h
>> index 7e157ab6cd505..189c31be163ce 100644
>> --- a/arch/arm64/include/asm/alternative-macros.h
>> +++ b/arch/arm64/include/asm/alternative-macros.h
>> @@ -2,10 +2,16 @@
>>   #ifndef __ASM_ALTERNATIVE_MACROS_H
>>   #define __ASM_ALTERNATIVE_MACROS_H
>> +#include <linux/const.h>
>> +
>>   #include <asm/cpucaps.h>
>>   #include <asm/insn-def.h>
>> -#define ARM64_CB_PATCH ARM64_NCAPS
>> +#define ARM64_CB_BIT    (UL(1) << 15)
>> +
>> +#if ARM64_NCAPS >= ARM64_CB_BIT
>> +#error "cpucaps have overflown ARM64_CB_BIT"
>> +#endif
> 
> 
> Some of our builders are failing and bisect is pointing to this commit. 
> Looks like they don't like the above and I see the following errors ...
> 
>    CC      arch/arm64/kvm/hyp/vhe/debug-sr.o
> /tmp/ccY3kbki.s: Assembler messages:
> /tmp/ccY3kbki.s:1600: Error: found 'L', expected: ')'
> /tmp/ccY3kbki.s:1600: Error: found 'L', expected: ')'
> /tmp/ccY3kbki.s:1600: Error: found 'L', expected: ')'
> /tmp/ccY3kbki.s:1600: Error: found 'L', expected: ')'
> /tmp/ccY3kbki.s:1600: Error: junk at end of line, first unrecognized 
> character is `L'
> /tmp/ccY3kbki.s:1723: Error: found 'L', expected: ')'
> /tmp/ccY3kbki.s:1723: Error: found 'L', expected: ')'
> /tmp/ccY3kbki.s:1723: Error: found 'L', expected: ')'
> /tmp/ccY3kbki.s:1723: Error: found 'L', expected: ')'
> /tmp/ccY3kbki.s:1723: Error: junk at end of line, first unrecognized 
> character is `L'
> scripts/Makefile.build:249: recipe for target 
> 'arch/arm64/kvm/hyp/vhe/debug-sr.o' failed
> 
> Seems that it does not like the 'UL' macro for some reason. Any thoughts?


FYI, this issue is seen with GCC6 but GCC7 and beyond appear to work fine.

Jon

-- 
nvpublic

Re: [PATCH v2 6/8] arm64: alternatives: have callbacks take a cap

[Ard Biesheuvel]

On Thu, 29 Sept 2022 at 11:54, Jon Hunter <jonathanh@nvidia.com> wrote:
>
>
> On 27/09/2022 10:31, Jon Hunter wrote:
>
> ...
>
> >> diff --git a/arch/arm64/include/asm/alternative-macros.h
> >> b/arch/arm64/include/asm/alternative-macros.h
> >> index 7e157ab6cd505..189c31be163ce 100644
> >> --- a/arch/arm64/include/asm/alternative-macros.h
> >> +++ b/arch/arm64/include/asm/alternative-macros.h
> >> @@ -2,10 +2,16 @@
> >>   #ifndef __ASM_ALTERNATIVE_MACROS_H
> >>   #define __ASM_ALTERNATIVE_MACROS_H
> >> +#include <linux/const.h>
> >> +
> >>   #include <asm/cpucaps.h>
> >>   #include <asm/insn-def.h>
> >> -#define ARM64_CB_PATCH ARM64_NCAPS
> >> +#define ARM64_CB_BIT    (UL(1) << 15)
> >> +
> >> +#if ARM64_NCAPS >= ARM64_CB_BIT
> >> +#error "cpucaps have overflown ARM64_CB_BIT"
> >> +#endif
> >
> >
> > Some of our builders are failing and bisect is pointing to this commit.
> > Looks like they don't like the above and I see the following errors ...
> >
> >    CC      arch/arm64/kvm/hyp/vhe/debug-sr.o
> > /tmp/ccY3kbki.s: Assembler messages:
> > /tmp/ccY3kbki.s:1600: Error: found 'L', expected: ')'
> > /tmp/ccY3kbki.s:1600: Error: found 'L', expected: ')'
> > /tmp/ccY3kbki.s:1600: Error: found 'L', expected: ')'
> > /tmp/ccY3kbki.s:1600: Error: found 'L', expected: ')'
> > /tmp/ccY3kbki.s:1600: Error: junk at end of line, first unrecognized
> > character is `L'
> > /tmp/ccY3kbki.s:1723: Error: found 'L', expected: ')'
> > /tmp/ccY3kbki.s:1723: Error: found 'L', expected: ')'
> > /tmp/ccY3kbki.s:1723: Error: found 'L', expected: ')'
> > /tmp/ccY3kbki.s:1723: Error: found 'L', expected: ')'
> > /tmp/ccY3kbki.s:1723: Error: junk at end of line, first unrecognized
> > character is `L'
> > scripts/Makefile.build:249: recipe for target
> > 'arch/arm64/kvm/hyp/vhe/debug-sr.o' failed
> >
> > Seems that it does not like the 'UL' macro for some reason. Any thoughts?
>
>
> FYI, this issue is seen with GCC6 but GCC7 and beyond appear to work fine.
>

Are you using the same version of binutils with those different
compilers? And which is/are the exact version(s)?

Re: [PATCH v2 6/8] arm64: alternatives: have callbacks take a cap

[Mark Rutland]

On Thu, Sep 29, 2022 at 10:53:56AM +0100, Jon Hunter wrote:
> 
> On 27/09/2022 10:31, Jon Hunter wrote:
> 
> ...
> 
> > > diff --git a/arch/arm64/include/asm/alternative-macros.h
> > > b/arch/arm64/include/asm/alternative-macros.h
> > > index 7e157ab6cd505..189c31be163ce 100644
> > > --- a/arch/arm64/include/asm/alternative-macros.h
> > > +++ b/arch/arm64/include/asm/alternative-macros.h
> > > @@ -2,10 +2,16 @@
> > >   #ifndef __ASM_ALTERNATIVE_MACROS_H
> > >   #define __ASM_ALTERNATIVE_MACROS_H
> > > +#include <linux/const.h>
> > > +
> > >   #include <asm/cpucaps.h>
> > >   #include <asm/insn-def.h>
> > > -#define ARM64_CB_PATCH ARM64_NCAPS
> > > +#define ARM64_CB_BIT    (UL(1) << 15)
> > > +
> > > +#if ARM64_NCAPS >= ARM64_CB_BIT
> > > +#error "cpucaps have overflown ARM64_CB_BIT"
> > > +#endif
> > 
> > 
> > Some of our builders are failing and bisect is pointing to this commit.
> > Looks like they don't like the above and I see the following errors ...
> > 
> >    CC      arch/arm64/kvm/hyp/vhe/debug-sr.o
> > /tmp/ccY3kbki.s: Assembler messages:
> > /tmp/ccY3kbki.s:1600: Error: found 'L', expected: ')'
> > /tmp/ccY3kbki.s:1600: Error: found 'L', expected: ')'
> > /tmp/ccY3kbki.s:1600: Error: found 'L', expected: ')'
> > /tmp/ccY3kbki.s:1600: Error: found 'L', expected: ')'
> > /tmp/ccY3kbki.s:1600: Error: junk at end of line, first unrecognized
> > character is `L'
> > /tmp/ccY3kbki.s:1723: Error: found 'L', expected: ')'
> > /tmp/ccY3kbki.s:1723: Error: found 'L', expected: ')'
> > /tmp/ccY3kbki.s:1723: Error: found 'L', expected: ')'
> > /tmp/ccY3kbki.s:1723: Error: found 'L', expected: ')'
> > /tmp/ccY3kbki.s:1723: Error: junk at end of line, first unrecognized
> > character is `L'
> > scripts/Makefile.build:249: recipe for target
> > 'arch/arm64/kvm/hyp/vhe/debug-sr.o' failed
> > 
> > Seems that it does not like the 'UL' macro for some reason. Any thoughts?
> 
> 
> FYI, this issue is seen with GCC6 but GCC7 and beyond appear to work fine.

Hmm... IIRC there was an issue with some older binutils here not liking the UL
suffix, but I thought we'd moved beyond those versions now; can you tell me
exactly which binutils version you're using?

I currently can't run the kernel.org crosstool GCC 5.5.0 release on my machine
since something's going wrong looking for an older version of libisl.so than my
system provides; I'll see if I can get that going and test locally.

I suspect we can bodge around this with something like the diff below.

Thanks,
Mark.

---->8----
diff --git a/arch/arm64/include/asm/alternative-macros.h b/arch/arm64/include/asm/alternative-macros.h
index 966767debaa3..4dd23bdbfb9e 100644
--- a/arch/arm64/include/asm/alternative-macros.h
+++ b/arch/arm64/include/asm/alternative-macros.h
@@ -2,12 +2,14 @@
 #ifndef __ASM_ALTERNATIVE_MACROS_H
 #define __ASM_ALTERNATIVE_MACROS_H
 
+#include <linux/bits.h>
 #include <linux/const.h>
 
 #include <asm/cpucaps.h>
 #include <asm/insn-def.h>
 
-#define ARM64_CB_BIT	(UL(1) << 15)
+#define ARM64_CB_SHIFT	15
+#define ARM64_CB_BIT	BIT(ARM64_CB_SHIFT)
 
 #if ARM64_NCAPS >= ARM64_CB_BIT
 #error "cpucaps have overflown ARM64_CB_BIT"
@@ -80,7 +82,7 @@
 	__ALTERNATIVE_CFG(oldinstr, newinstr, feature, IS_ENABLED(cfg))
 
 #define ALTERNATIVE_CB(oldinstr, feature, cb) \
-	__ALTERNATIVE_CFG_CB(oldinstr, ARM64_CB_BIT | (feature), 1, cb)
+	__ALTERNATIVE_CFG_CB(oldinstr, (1 << ARM64_CB_SHIFT) | (feature), 1, cb)
 #else
 
 #include <asm/assembler.h>
@@ -150,7 +152,7 @@
 .macro alternative_cb cap, cb
 	.set .Lasm_alt_mode, 0
 	.pushsection .altinstructions, "a"
-	altinstruction_entry 661f, \cb, ARM64_CB_BIT | \cap, 662f-661f, 0
+	altinstruction_entry 661f, \cb, (1 << ARM64_CB_SHIFT) | \cap, 662f-661f, 0
 	.popsection
 661:
 .endm

Re: [PATCH v2 6/8] arm64: alternatives: have callbacks take a cap

[Jon Hunter]

On 29/09/2022 11:10, Ard Biesheuvel wrote:
> On Thu, 29 Sept 2022 at 11:54, Jon Hunter <jonathanh@nvidia.com> wrote:
>>
>>
>> On 27/09/2022 10:31, Jon Hunter wrote:
>>
>> ...
>>
>>>> diff --git a/arch/arm64/include/asm/alternative-macros.h
>>>> b/arch/arm64/include/asm/alternative-macros.h
>>>> index 7e157ab6cd505..189c31be163ce 100644
>>>> --- a/arch/arm64/include/asm/alternative-macros.h
>>>> +++ b/arch/arm64/include/asm/alternative-macros.h
>>>> @@ -2,10 +2,16 @@
>>>>    #ifndef __ASM_ALTERNATIVE_MACROS_H
>>>>    #define __ASM_ALTERNATIVE_MACROS_H
>>>> +#include <linux/const.h>
>>>> +
>>>>    #include <asm/cpucaps.h>
>>>>    #include <asm/insn-def.h>
>>>> -#define ARM64_CB_PATCH ARM64_NCAPS
>>>> +#define ARM64_CB_BIT    (UL(1) << 15)
>>>> +
>>>> +#if ARM64_NCAPS >= ARM64_CB_BIT
>>>> +#error "cpucaps have overflown ARM64_CB_BIT"
>>>> +#endif
>>>
>>>
>>> Some of our builders are failing and bisect is pointing to this commit.
>>> Looks like they don't like the above and I see the following errors ...
>>>
>>>     CC      arch/arm64/kvm/hyp/vhe/debug-sr.o
>>> /tmp/ccY3kbki.s: Assembler messages:
>>> /tmp/ccY3kbki.s:1600: Error: found 'L', expected: ')'
>>> /tmp/ccY3kbki.s:1600: Error: found 'L', expected: ')'
>>> /tmp/ccY3kbki.s:1600: Error: found 'L', expected: ')'
>>> /tmp/ccY3kbki.s:1600: Error: found 'L', expected: ')'
>>> /tmp/ccY3kbki.s:1600: Error: junk at end of line, first unrecognized
>>> character is `L'
>>> /tmp/ccY3kbki.s:1723: Error: found 'L', expected: ')'
>>> /tmp/ccY3kbki.s:1723: Error: found 'L', expected: ')'
>>> /tmp/ccY3kbki.s:1723: Error: found 'L', expected: ')'
>>> /tmp/ccY3kbki.s:1723: Error: found 'L', expected: ')'
>>> /tmp/ccY3kbki.s:1723: Error: junk at end of line, first unrecognized
>>> character is `L'
>>> scripts/Makefile.build:249: recipe for target
>>> 'arch/arm64/kvm/hyp/vhe/debug-sr.o' failed
>>>
>>> Seems that it does not like the 'UL' macro for some reason. Any thoughts?
>>
>>
>> FYI, this issue is seen with GCC6 but GCC7 and beyond appear to work fine.
>>
> 
> Are you using the same version of binutils with those different
> compilers? And which is/are the exact version(s)?

Looks like they are built with different versions ...

GCC6: binutils 2.27.0.20161019
GCC7: binutils 2.28.2.20170706

Jon

-- 
nvpublic

Re: [PATCH v2 6/8] arm64: alternatives: have callbacks take a cap

[Jon Hunter]

On 29/09/2022 11:47, Mark Rutland wrote:
> On Thu, Sep 29, 2022 at 10:53:56AM +0100, Jon Hunter wrote:
>>
>> On 27/09/2022 10:31, Jon Hunter wrote:
>>
>> ...
>>
>>>> diff --git a/arch/arm64/include/asm/alternative-macros.h
>>>> b/arch/arm64/include/asm/alternative-macros.h
>>>> index 7e157ab6cd505..189c31be163ce 100644
>>>> --- a/arch/arm64/include/asm/alternative-macros.h
>>>> +++ b/arch/arm64/include/asm/alternative-macros.h
>>>> @@ -2,10 +2,16 @@
>>>>    #ifndef __ASM_ALTERNATIVE_MACROS_H
>>>>    #define __ASM_ALTERNATIVE_MACROS_H
>>>> +#include <linux/const.h>
>>>> +
>>>>    #include <asm/cpucaps.h>
>>>>    #include <asm/insn-def.h>
>>>> -#define ARM64_CB_PATCH ARM64_NCAPS
>>>> +#define ARM64_CB_BIT    (UL(1) << 15)
>>>> +
>>>> +#if ARM64_NCAPS >= ARM64_CB_BIT
>>>> +#error "cpucaps have overflown ARM64_CB_BIT"
>>>> +#endif
>>>
>>>
>>> Some of our builders are failing and bisect is pointing to this commit.
>>> Looks like they don't like the above and I see the following errors ...
>>>
>>>     CC      arch/arm64/kvm/hyp/vhe/debug-sr.o
>>> /tmp/ccY3kbki.s: Assembler messages:
>>> /tmp/ccY3kbki.s:1600: Error: found 'L', expected: ')'
>>> /tmp/ccY3kbki.s:1600: Error: found 'L', expected: ')'
>>> /tmp/ccY3kbki.s:1600: Error: found 'L', expected: ')'
>>> /tmp/ccY3kbki.s:1600: Error: found 'L', expected: ')'
>>> /tmp/ccY3kbki.s:1600: Error: junk at end of line, first unrecognized
>>> character is `L'
>>> /tmp/ccY3kbki.s:1723: Error: found 'L', expected: ')'
>>> /tmp/ccY3kbki.s:1723: Error: found 'L', expected: ')'
>>> /tmp/ccY3kbki.s:1723: Error: found 'L', expected: ')'
>>> /tmp/ccY3kbki.s:1723: Error: found 'L', expected: ')'
>>> /tmp/ccY3kbki.s:1723: Error: junk at end of line, first unrecognized
>>> character is `L'
>>> scripts/Makefile.build:249: recipe for target
>>> 'arch/arm64/kvm/hyp/vhe/debug-sr.o' failed
>>>
>>> Seems that it does not like the 'UL' macro for some reason. Any thoughts?
>>
>>
>> FYI, this issue is seen with GCC6 but GCC7 and beyond appear to work fine.
> 
> Hmm... IIRC there was an issue with some older binutils here not liking the UL
> suffix, but I thought we'd moved beyond those versions now; can you tell me
> exactly which binutils version you're using?
> 
> I currently can't run the kernel.org crosstool GCC 5.5.0 release on my machine
> since something's going wrong looking for an older version of libisl.so than my
> system provides; I'll see if I can get that going and test locally.
> 
> I suspect we can bodge around this with something like the diff below.
> 
> Thanks,
> Mark.
> 
> ---->8----
> diff --git a/arch/arm64/include/asm/alternative-macros.h b/arch/arm64/include/asm/alternative-macros.h
> index 966767debaa3..4dd23bdbfb9e 100644
> --- a/arch/arm64/include/asm/alternative-macros.h
> +++ b/arch/arm64/include/asm/alternative-macros.h
> @@ -2,12 +2,14 @@
>   #ifndef __ASM_ALTERNATIVE_MACROS_H
>   #define __ASM_ALTERNATIVE_MACROS_H
>   
> +#include <linux/bits.h>
>   #include <linux/const.h>
>   
>   #include <asm/cpucaps.h>
>   #include <asm/insn-def.h>
>   
> -#define ARM64_CB_BIT	(UL(1) << 15)
> +#define ARM64_CB_SHIFT	15
> +#define ARM64_CB_BIT	BIT(ARM64_CB_SHIFT)
>   
>   #if ARM64_NCAPS >= ARM64_CB_BIT
>   #error "cpucaps have overflown ARM64_CB_BIT"
> @@ -80,7 +82,7 @@
>   	__ALTERNATIVE_CFG(oldinstr, newinstr, feature, IS_ENABLED(cfg))
>   
>   #define ALTERNATIVE_CB(oldinstr, feature, cb) \
> -	__ALTERNATIVE_CFG_CB(oldinstr, ARM64_CB_BIT | (feature), 1, cb)
> +	__ALTERNATIVE_CFG_CB(oldinstr, (1 << ARM64_CB_SHIFT) | (feature), 1, cb)
>   #else
>   
>   #include <asm/assembler.h>
> @@ -150,7 +152,7 @@
>   .macro alternative_cb cap, cb
>   	.set .Lasm_alt_mode, 0
>   	.pushsection .altinstructions, "a"
> -	altinstruction_entry 661f, \cb, ARM64_CB_BIT | \cap, 662f-661f, 0
> +	altinstruction_entry 661f, \cb, (1 << ARM64_CB_SHIFT) | \cap, 662f-661f, 0
>   	.popsection
>   661:
>   .endm


Yes that fixes it.

Tested-by: Jon Hunter <jonathanh@nvidia.com>

Cheers
Jon

-- 
nvpublic

Re: [PATCH v2 6/8] arm64: alternatives: have callbacks take a cap

[Mark Rutland]

On Thu, Sep 29, 2022 at 12:01:24PM +0100, Jon Hunter wrote:
> 
> On 29/09/2022 11:47, Mark Rutland wrote:
> > On Thu, Sep 29, 2022 at 10:53:56AM +0100, Jon Hunter wrote:
> > > 
> > > On 27/09/2022 10:31, Jon Hunter wrote:
> > > 
> > > ...
> > > 
> > > > > diff --git a/arch/arm64/include/asm/alternative-macros.h
> > > > > b/arch/arm64/include/asm/alternative-macros.h
> > > > > index 7e157ab6cd505..189c31be163ce 100644
> > > > > --- a/arch/arm64/include/asm/alternative-macros.h
> > > > > +++ b/arch/arm64/include/asm/alternative-macros.h
> > > > > @@ -2,10 +2,16 @@
> > > > >    #ifndef __ASM_ALTERNATIVE_MACROS_H
> > > > >    #define __ASM_ALTERNATIVE_MACROS_H
> > > > > +#include <linux/const.h>
> > > > > +
> > > > >    #include <asm/cpucaps.h>
> > > > >    #include <asm/insn-def.h>
> > > > > -#define ARM64_CB_PATCH ARM64_NCAPS
> > > > > +#define ARM64_CB_BIT    (UL(1) << 15)
> > > > > +
> > > > > +#if ARM64_NCAPS >= ARM64_CB_BIT
> > > > > +#error "cpucaps have overflown ARM64_CB_BIT"
> > > > > +#endif
> > > > 
> > > > 
> > > > Some of our builders are failing and bisect is pointing to this commit.
> > > > Looks like they don't like the above and I see the following errors ...
> > > > 
> > > >     CC      arch/arm64/kvm/hyp/vhe/debug-sr.o
> > > > /tmp/ccY3kbki.s: Assembler messages:
> > > > /tmp/ccY3kbki.s:1600: Error: found 'L', expected: ')'
> > > > /tmp/ccY3kbki.s:1600: Error: found 'L', expected: ')'
> > > > /tmp/ccY3kbki.s:1600: Error: found 'L', expected: ')'
> > > > /tmp/ccY3kbki.s:1600: Error: found 'L', expected: ')'
> > > > /tmp/ccY3kbki.s:1600: Error: junk at end of line, first unrecognized
> > > > character is `L'
> > > > /tmp/ccY3kbki.s:1723: Error: found 'L', expected: ')'
> > > > /tmp/ccY3kbki.s:1723: Error: found 'L', expected: ')'
> > > > /tmp/ccY3kbki.s:1723: Error: found 'L', expected: ')'
> > > > /tmp/ccY3kbki.s:1723: Error: found 'L', expected: ')'
> > > > /tmp/ccY3kbki.s:1723: Error: junk at end of line, first unrecognized
> > > > character is `L'
> > > > scripts/Makefile.build:249: recipe for target
> > > > 'arch/arm64/kvm/hyp/vhe/debug-sr.o' failed
> > > > 
> > > > Seems that it does not like the 'UL' macro for some reason. Any thoughts?
> > > 
> > > 
> > > FYI, this issue is seen with GCC6 but GCC7 and beyond appear to work fine.
> > 
> > Hmm... IIRC there was an issue with some older binutils here not liking the UL
> > suffix, but I thought we'd moved beyond those versions now; can you tell me
> > exactly which binutils version you're using?
> > 
> > I currently can't run the kernel.org crosstool GCC 5.5.0 release on my machine
> > since something's going wrong looking for an older version of libisl.so than my
> > system provides; I'll see if I can get that going and test locally.
> > 
> > I suspect we can bodge around this with something like the diff below.
> > 
> > Thanks,
> > Mark.
> > 
> > ---->8----
> > diff --git a/arch/arm64/include/asm/alternative-macros.h b/arch/arm64/include/asm/alternative-macros.h
> > index 966767debaa3..4dd23bdbfb9e 100644
> > --- a/arch/arm64/include/asm/alternative-macros.h
> > +++ b/arch/arm64/include/asm/alternative-macros.h
> > @@ -2,12 +2,14 @@
> >   #ifndef __ASM_ALTERNATIVE_MACROS_H
> >   #define __ASM_ALTERNATIVE_MACROS_H
> > +#include <linux/bits.h>
> >   #include <linux/const.h>
> >   #include <asm/cpucaps.h>
> >   #include <asm/insn-def.h>
> > -#define ARM64_CB_BIT	(UL(1) << 15)
> > +#define ARM64_CB_SHIFT	15
> > +#define ARM64_CB_BIT	BIT(ARM64_CB_SHIFT)
> >   #if ARM64_NCAPS >= ARM64_CB_BIT
> >   #error "cpucaps have overflown ARM64_CB_BIT"
> > @@ -80,7 +82,7 @@
> >   	__ALTERNATIVE_CFG(oldinstr, newinstr, feature, IS_ENABLED(cfg))
> >   #define ALTERNATIVE_CB(oldinstr, feature, cb) \
> > -	__ALTERNATIVE_CFG_CB(oldinstr, ARM64_CB_BIT | (feature), 1, cb)
> > +	__ALTERNATIVE_CFG_CB(oldinstr, (1 << ARM64_CB_SHIFT) | (feature), 1, cb)
> >   #else
> >   #include <asm/assembler.h>
> > @@ -150,7 +152,7 @@
> >   .macro alternative_cb cap, cb
> >   	.set .Lasm_alt_mode, 0
> >   	.pushsection .altinstructions, "a"
> > -	altinstruction_entry 661f, \cb, ARM64_CB_BIT | \cap, 662f-661f, 0
> > +	altinstruction_entry 661f, \cb, (1 << ARM64_CB_SHIFT) | \cap, 662f-661f, 0
> >   	.popsection
> >   661:
> >   .endm
> 
> 
> Yes that fixes it.
> 
> Tested-by: Jon Hunter <jonathanh@nvidia.com>

Great!

Could you please let me know which version of binutils, so that we can add
something regarding that in a comment and in the commit message?

The output of ${CROSS_COMPILE}as --version would suffice.

With that, I can clean this up and send as a proper patch.

Thanks,
Mark.

Re: [PATCH v2 6/8] arm64: alternatives: have callbacks take a cap

[Jon Hunter]

On 29/09/2022 12:09, Mark Rutland wrote:

...

>> Yes that fixes it.
>>
>> Tested-by: Jon Hunter <jonathanh@nvidia.com>
> 
> Great!
> 
> Could you please let me know which version of binutils, so that we can add
> something regarding that in a comment and in the commit message?
> 
> The output of ${CROSS_COMPILE}as --version would suffice.
> 
> With that, I can clean this up and send as a proper patch.


Yes it is ...

GNU ld (Linaro_Binutils-2017.08) 2.27.0.20161019


Cheers
Jon

-- 
nvpublic

Re: [PATCH v2 6/8] arm64: alternatives: have callbacks take a cap

[Mark Rutland]

On Thu, Sep 29, 2022 at 02:37:54PM +0100, Jon Hunter wrote:
> 
> On 29/09/2022 12:09, Mark Rutland wrote:
> 
> ...
> 
> > > Yes that fixes it.
> > > 
> > > Tested-by: Jon Hunter <jonathanh@nvidia.com>
> > 
> > Great!
> > 
> > Could you please let me know which version of binutils, so that we can add
> > something regarding that in a comment and in the commit message?
> > 
> > The output of ${CROSS_COMPILE}as --version would suffice.
> > 
> > With that, I can clean this up and send as a proper patch.
> 
> 
> Yes it is ...
> 
> GNU ld (Linaro_Binutils-2017.08) 2.27.0.20161019

Thanks for that!

FWIW< I can reproduce that with the Linaro 17.05 toolchain release:

| [mark@lakrids:~/src/linux]% uselinaro 17.05 aarch64-linux-gnu-as --version
| GNU assembler (Linaro_Binutils-2017.05) 2.27.0.20161019
| Copyright (C) 2016 Free Software Foundation, Inc.
| This program is free software; you may redistribute it under the terms of
| the GNU General Public License version 3 or later.
| This program has absolutely no warranty.
| This assembler was configured for a target of `aarch64-linux-gnu'.
| [mark@lakrids:~/src/linux]% uselinaro 17.05 make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -s defconfig
| [mark@lakrids:~/src/linux]% uselinaro 17.05 make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -s Image    
| /tmp/ccAy74PK.s: Assembler messages:
| /tmp/ccAy74PK.s:2528: Error: found 'L', expected: ')'
| /tmp/ccAy74PK.s:2528: Error: found 'L', expected: ')'
| /tmp/ccAy74PK.s:2528: Error: found 'L', expected: ')'
| /tmp/ccAy74PK.s:2528: Error: found 'L', expected: ')'
| /tmp/ccAy74PK.s:2528: Error: junk at end of line, first unrecognized character is `L'
| /tmp/ccAy74PK.s:3562: Error: found 'L', expected: ')'
| /tmp/ccAy74PK.s:3562: Error: found 'L', expected: ')'
| /tmp/ccAy74PK.s:3562: Error: found 'L', expected: ')'
| /tmp/ccAy74PK.s:3562: Error: found 'L', expected: ')'
| /tmp/ccAy74PK.s:3562: Error: junk at end of line, first unrecognized character is `L'
| make[1]: *** [scripts/Makefile.build:249: init/main.o] Error 1
| make: *** [Makefile:1853: init] Error 2

... but curiously the 17.08 release seems to have a new, working version of
binutils:

| [mark@lakrids:~/src/linux]% uselinaro 17.08 aarch64-linux-gnu-as --version
| GNU assembler (Linaro_Binutils-2017.08) 2.28.2.20170706
| Copyright (C) 2017 Free Software Foundation, Inc.
| This program is free software; you may redistribute it under the terms of
| the GNU General Public License version 3 or later.
| This program has absolutely no warranty.
| This assembler was configured for a target of `aarch64-linux-gnu'.
| [mark@lakrids:~/src/linux]% uselinaro 17.08 make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -s defconfig
| [mark@lakrids:~/src/linux]% uselinaro 17.08 make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -j50 -s Image
| [mark@lakrids:~/src/linux]% echo $?
| 0

... so I'm not sure why your copy has an older binutils.

Regardless, I'll go prep that patch with a real commit message, and add your
Tested-by.

Thanks,
Mark.