Re: [PATCH v3 1/3] arm64: kvm: support kvmtool to detect RAS extension feature

Re: [PATCH v3 1/3] arm64: kvm: support kvmtool to detect RAS extension feature

[gengdongjiu]

Hi Christoffer,
   thanks for your review and comments.

On 2017/5/2 15:56, Christoffer Dall wrote:
> Hi Dongjiu,
> 
> Please send a cover letter for patch series with more than a single
> patch.
 OK, got it.

> 
> The subject and description of these patches are also misleading.
> Hopefully this is in no way tied to kvmtool, but to userspace
> generically, for example also to be used by QEMU?
> 
> On Sun, Apr 30, 2017 at 01:37:55PM +0800, Dongjiu Geng wrote:
>> Handle kvmtool's detection for RAS extension, because sometimes
>> the APP needs to know the CPU's capacity
> 
> the APP ?
> 
> the CPU's capacity?
I will fix it.

> 
>>
>> Signed-off-by: Dongjiu Geng <gengdongjiu@huawei.com>
>> ---
>>  arch/arm64/kvm/reset.c   | 11 +++++++++++
>>  include/uapi/linux/kvm.h |  1 +
>>  2 files changed, 12 insertions(+)
>>
>> diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c
>> index d9e9697..1004039 100644
>> --- a/arch/arm64/kvm/reset.c
>> +++ b/arch/arm64/kvm/reset.c
>> @@ -64,6 +64,14 @@ static bool cpu_has_32bit_el1(void)
>>  	return !!(pfr0 & 0x20);
>>  }
>>  
>> +static bool kvm_arm_support_ras_extension(void)
>> +{
>> +	u64 pfr0;
>> +
>> +	pfr0 = read_system_reg(SYS_ID_AA64PFR0_EL1);
>> +	return !!(pfr0 & 0x10000000);
>> +}
>> +
>>  /**
>>   * kvm_arch_dev_ioctl_check_extension
>>   *
>> @@ -87,6 +95,9 @@ int kvm_arch_dev_ioctl_check_extension(struct kvm *kvm, long ext)
>>  	case KVM_CAP_ARM_PMU_V3:
>>  		r = kvm_arm_support_pmu_v3();
>>  		break;
>> +	case KVM_CAP_ARM_RAS_EXTENSION:
>> +		r = kvm_arm_support_ras_extension();
>> +		break;
> 
> You need to document this capability and API in
> Documentation/virtual/kvm/api.txt and explain how this works.
 Ok, thanks for your suggestion.

> 
> 
> 
>>  	case KVM_CAP_SET_GUEST_DEBUG:
>>  	case KVM_CAP_VCPU_ATTRIBUTES:
>>  		r = 1;
>> diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
>> index f51d508..27fe556 100644
>> --- a/include/uapi/linux/kvm.h
>> +++ b/include/uapi/linux/kvm.h
>> @@ -883,6 +883,7 @@ struct kvm_ppc_resize_hpt {
>>  #define KVM_CAP_PPC_MMU_RADIX 134
>>  #define KVM_CAP_PPC_MMU_HASH_V3 135
>>  #define KVM_CAP_IMMEDIATE_EXIT 136
>> +#define KVM_CAP_ARM_RAS_EXTENSION 137
>>  
>>  #ifdef KVM_CAP_IRQ_ROUTING
>>  
>> -- 
>> 2.10.1
>>
> 
> Thanks,
> -Christoffer
> 
> .
> 

Re: [PATCH v3 1/3] arm64: kvm: support kvmtool to detect RAS extension feature

[gengdongjiu]

Dear James,

>
>  Hi Dongjiu Geng,
>
> On 30/04/17 06:37, Dongjiu Geng wrote:
>> Handle kvmtool's detection for RAS extension, because sometimes
>> the APP needs to know the CPU's capacity
>
>> diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c
>> index d9e9697..1004039 100644
>> --- a/arch/arm64/kvm/reset.c
>> +++ b/arch/arm64/kvm/reset.c
>> @@ -64,6 +64,14 @@ static bool cpu_has_32bit_el1(void)
>>   return !!(pfr0 & 0x20);
>>  }
>>
>> +static bool kvm_arm_support_ras_extension(void)
>> +{
>> + u64 pfr0;
>> +
>> + pfr0 = read_system_reg(SYS_ID_AA64PFR0_EL1);
>> + return !!(pfr0 & 0x10000000);
>> +}
>
> Why are we telling user-space that the CPU has RAS extensions? EL0 can't do
> anything with this and the guest EL1 can detect it from the id registers.
>
>
> Are you using this to decide whether or not to generate a HEST for the guest?

James, yes, it is.  my current user-space qemu EL0 patches indeed will
check the RAS  extensions.
if has the RAS extensions. for SEA, userspace qemu will generate the
CPER and inject the SEA to guest;
for SEI,  userspace qemu sets the virtual SEI with the specified
Syndrome(set the HCR_EL2.VSE and vsesr_el2 );
if not have RAS extensions, Qemu does nothing


>
> If Qemu/kvmtool supports handling memory-failure notifications from signals you
> should always generate a HEST. The GHES notification method could be anything
> Qemu can deliver to the guest using the KVM APIs. Notifications from Qemu to the
> guest don't depend on the RAS extensions. KVM has APIs for IRQ and SEA (you can
> use KVM_SET_ONE_REG).

I will consider your suggestion to  always generate a CPER instead of
relying on the RAS extensions, thanks
For this comments "SEA (you can use KVM_SET_ONE_REG)",  may be it will
be duplicated with the existed KVM API  as I mentioned in another mail
thread.I am considering that whether it is necessary to change the
vcpu registers in the Qemu/KVMTools

>
>
> I think we need a new API for injecting SError for SEI from Qemu/kvmtool, but it
> shouldn't be related to the RAS extensions. All v8.0 CPUs have HCR_EL2.VSE, so
> we need to know KVM supports this API.
>
> Your later patch adds code to set VSESR to make virtual RAS SErrors work, I
> think we need to expose that to user-space.
>
>
> Thanks,
>
> James

Re: [PATCH v3 1/3] arm64: kvm: support kvmtool to detect RAS extension feature

[James Morse]

Hi gengdongjiu,

On 04/05/17 18:20, gengdongjiu wrote:
>> On 30/04/17 06:37, Dongjiu Geng wrote:
>>> Handle kvmtool's detection for RAS extension, because sometimes
>>> the APP needs to know the CPU's capacity
>>
>>> diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c
>>> index d9e9697..1004039 100644
>>> --- a/arch/arm64/kvm/reset.c
>>> +++ b/arch/arm64/kvm/reset.c
>>> @@ -64,6 +64,14 @@ static bool cpu_has_32bit_el1(void)
>>>   return !!(pfr0 & 0x20);
>>>  }
>>>
>>> +static bool kvm_arm_support_ras_extension(void)
>>> +{
>>> + u64 pfr0;
>>> +
>>> + pfr0 = read_system_reg(SYS_ID_AA64PFR0_EL1);
>>> + return !!(pfr0 & 0x10000000);
>>> +}
>>
>> Why are we telling user-space that the CPU has RAS extensions? EL0 can't do
>> anything with this and the guest EL1 can detect it from the id registers.
>>
>>
>> Are you using this to decide whether or not to generate a HEST for the guest?
> 
> James, yes, it is.  my current user-space qemu EL0 patches indeed will
> check the RAS  extensions.
> if has the RAS extensions. for SEA, userspace qemu will generate the
> CPER and inject the SEA to guest;
> for SEI,  userspace qemu sets the virtual SEI with the specified
> Syndrome(set the HCR_EL2.VSE and vsesr_el2 );
> if not have RAS extensions, Qemu does nothing

But you can use APEI in a guest on CPUs without the RAS extensions: the host may
signal memory errors to Qemu for any number of reasons, user-space shouldn't
care how it knows. Examples are PCI-AER, any APEI event notified by polling or
one of the flavours of irq.

I would expect Qemu to generate a HEST based on its abilities, i.e. if it
supports any mechanism of notifying the guest about errors. Choosing the
mechanism then depends on the type of error.

Ideally the Qemu code for HEST/GHES/CPER generation code using some of the irqs
and polling could be shared with x86, as these should be possible using common
KVM APIs.


>> If Qemu/kvmtool supports handling memory-failure notifications from signals you
>> should always generate a HEST. The GHES notification method could be anything
>> Qemu can deliver to the guest using the KVM APIs. Notifications from Qemu to the
>> guest don't depend on the RAS extensions. KVM has APIs for IRQ and SEA (you can
>> use KVM_SET_ONE_REG).
> 
> I will consider your suggestion to  always generate a CPER instead of

(generate a HEST, CPER are the runtime records. There are too many acronyms in
this space!)

> relying on the RAS extensions, thanks


Thanks,

James

Re: [PATCH v3 1/3] arm64: kvm: support kvmtool to detect RAS extension feature

[gengdongjiu]

Dear, James

On 2017/5/9 1:31, James Morse wrote:
> Hi gengdongjiu,
> 
> On 04/05/17 18:20, gengdongjiu wrote:
>>> On 30/04/17 06:37, Dongjiu Geng wrote:
>>>> Handle kvmtool's detection for RAS extension, because sometimes
>>>> the APP needs to know the CPU's capacity
>>>
>>>> diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c
>>>> index d9e9697..1004039 100644
>>>> --- a/arch/arm64/kvm/reset.c
>>>> +++ b/arch/arm64/kvm/reset.c
>>>> @@ -64,6 +64,14 @@ static bool cpu_has_32bit_el1(void)
>>>>   return !!(pfr0 & 0x20);
>>>>  }
>>>>
>>>> +static bool kvm_arm_support_ras_extension(void)
>>>> +{
>>>> + u64 pfr0;
>>>> +
>>>> + pfr0 = read_system_reg(SYS_ID_AA64PFR0_EL1);
>>>> + return !!(pfr0 & 0x10000000);
>>>> +}
>>>
>>> Why are we telling user-space that the CPU has RAS extensions? EL0 can't do
>>> anything with this and the guest EL1 can detect it from the id registers.
>>>
>>>
>>> Are you using this to decide whether or not to generate a HEST for the guest?
>>
>> James, yes, it is.  my current user-space qemu EL0 patches indeed will
>> check the RAS  extensions.
>> if has the RAS extensions. for SEA, userspace qemu will generate the
>> CPER and inject the SEA to guest;
>> for SEI,  userspace qemu sets the virtual SEI with the specified
>> Syndrome(set the HCR_EL2.VSE and vsesr_el2 );
>> if not have RAS extensions, Qemu does nothing
> 
> But you can use APEI in a guest on CPUs without the RAS extensions: the host may
> signal memory errors to Qemu for any number of reasons, user-space shouldn't
> care how it knows. Examples are PCI-AER, any APEI event notified by polling or
> one of the flavours of irq.
> 
> I would expect Qemu to generate a HEST based on its abilities, i.e. if it
> supports any mechanism of notifying the guest about errors. Choosing the
> mechanism then depends on the type of error.
> 
> Ideally the Qemu code for HEST/GHES/CPER generation code using some of the irqs
> and polling could be shared with x86, as these should be possible using common
> KVM APIs.
Ok, got it.

> 
> 
>>> If Qemu/kvmtool supports handling memory-failure notifications from signals you
>>> should always generate a HEST. The GHES notification method could be anything
>>> Qemu can deliver to the guest using the KVM APIs. Notifications from Qemu to the
>>> guest don't depend on the RAS extensions. KVM has APIs for IRQ and SEA (you can
>>> use KVM_SET_ONE_REG).
>>
>> I will consider your suggestion to  always generate a CPER instead of
> 
> (generate a HEST, CPER are the runtime records. There are too many acronyms in
> this space!)
  thanks James's correction.

> 
>> relying on the RAS extensions, thanks
> 
> 
> Thanks,
> 
> James
> 
> 
> .
>