[question] hw/arm/virt: about the default gic-version in accelerated mode

[question] hw/arm/virt: about the default gic-version in accelerated mode

[Auger Eric]

Hi Peter,

When arm virt machine is run in accelerated mode with "-cpu host
-machine virt", the default gic version is 2.

I understand the rationale with TCG where we don't have MSI ITS
emulation along with GICv3 so we need to choose GICv2 to get GICv2M
functionality.

However in KVM mode, I would have expected to see the host GIC probed to
set the same version on guest. Indeed most of our HW now have GICv3
without GICv2 compat mode so our default values lead to weird traces:

"
qemu-system-aarch64: PMU: KVM_SET_DEVICE_ATTR: Invalid argument
qemu-system-aarch64: failed to set irq for PMU
"

I would like to propose a patch to improve those errors and also suggest
a hint. But I also wanted to know whether you would accept to change the
default value with KVM and choose the host version instead of 2. For TCG
we would keep v2.

Thanks

Eric

Re: [question] hw/arm/virt: about the default gic-version in accelerated mode

[Peter Maydell]

On Tue, 28 Jan 2020 at 10:47, Auger Eric <eric.auger@redhat.com> wrote:
> When arm virt machine is run in accelerated mode with "-cpu host
> -machine virt", the default gic version is 2.
>
> I understand the rationale with TCG where we don't have MSI ITS
> emulation along with GICv3 so we need to choose GICv2 to get GICv2M
> functionality.
>
> However in KVM mode, I would have expected to see the host GIC probed to
> set the same version on guest. Indeed most of our HW now have GICv3
> without GICv2 compat mode so our default values lead to weird traces:
>
> "
> qemu-system-aarch64: PMU: KVM_SET_DEVICE_ATTR: Invalid argument
> qemu-system-aarch64: failed to set irq for PMU
> "
>
> I would like to propose a patch to improve those errors and also suggest
> a hint. But I also wanted to know whether you would accept to change the
> default value with KVM and choose the host version instead of 2. For TCG
> we would keep v2.

As with the -cpu option, the default is there for command
line backward compatibility primarily. Even if we had
better support for MSI ITS emulation we'd still leave
the default at GICv2.

If you want "do the best you can, regardless of accelerator"
that is "-cpu max -machine gic-version=max".

thanks
-- PMM

Re: [question] hw/arm/virt: about the default gic-version in accelerated mode

[Dr. David Alan Gilbert]

* Peter Maydell (peter.maydell@linaro.org) wrote:
> On Tue, 28 Jan 2020 at 10:47, Auger Eric <eric.auger@redhat.com> wrote:
> > When arm virt machine is run in accelerated mode with "-cpu host
> > -machine virt", the default gic version is 2.
> >
> > I understand the rationale with TCG where we don't have MSI ITS
> > emulation along with GICv3 so we need to choose GICv2 to get GICv2M
> > functionality.
> >
> > However in KVM mode, I would have expected to see the host GIC probed to
> > set the same version on guest. Indeed most of our HW now have GICv3
> > without GICv2 compat mode so our default values lead to weird traces:
> >
> > "
> > qemu-system-aarch64: PMU: KVM_SET_DEVICE_ATTR: Invalid argument
> > qemu-system-aarch64: failed to set irq for PMU
> > "
> >
> > I would like to propose a patch to improve those errors and also suggest
> > a hint. But I also wanted to know whether you would accept to change the
> > default value with KVM and choose the host version instead of 2. For TCG
> > we would keep v2.
> 
> As with the -cpu option, the default is there for command
> line backward compatibility primarily. Even if we had
> better support for MSI ITS emulation we'd still leave
> the default at GICv2.

I'm curious why; 'virt' is a versioned machine type, the whole point of
versioned machine types is you're allowed to change the behaviour of
them between versions of qemu.   If people want their command lines
to stay stable they can specify a version, i.e. -M virt-4.2 and
know that it won't change.

> If you want "do the best you can, regardless of accelerator"
> that is "-cpu max -machine gic-version=max".

Dave

> thanks
> -- PMM
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [question] hw/arm/virt: about the default gic-version in accelerated mode

[Auger Eric]

Hi Peter,

On 1/28/20 11:52 AM, Peter Maydell wrote:
> On Tue, 28 Jan 2020 at 10:47, Auger Eric <eric.auger@redhat.com> wrote:
>> When arm virt machine is run in accelerated mode with "-cpu host
>> -machine virt", the default gic version is 2.
>>
>> I understand the rationale with TCG where we don't have MSI ITS
>> emulation along with GICv3 so we need to choose GICv2 to get GICv2M
>> functionality.
>>
>> However in KVM mode, I would have expected to see the host GIC probed to
>> set the same version on guest. Indeed most of our HW now have GICv3
>> without GICv2 compat mode so our default values lead to weird traces:
>>
>> "
>> qemu-system-aarch64: PMU: KVM_SET_DEVICE_ATTR: Invalid argument
>> qemu-system-aarch64: failed to set irq for PMU
>> "
>>
>> I would like to propose a patch to improve those errors and also suggest
>> a hint. But I also wanted to know whether you would accept to change the
>> default value with KVM and choose the host version instead of 2. For TCG
>> we would keep v2.
> 
> As with the -cpu option, the default is there for command
> line backward compatibility primarily. Even if we had
> better support for MSI ITS emulation we'd still leave
> the default at GICv2.
> 
> If you want "do the best you can, regardless of accelerator"
> that is "-cpu max -machine gic-version=max".

OK that's understood.

So I will just try to improve the above traces.

Thanks

Eric
> 
> thanks
> -- PMM
> 

Re: [question] hw/arm/virt: about the default gic-version in accelerated mode

[Andrew Jones]

On Tue, Jan 28, 2020 at 10:52:50AM +0000, Peter Maydell wrote:
> On Tue, 28 Jan 2020 at 10:47, Auger Eric <eric.auger@redhat.com> wrote:
> > When arm virt machine is run in accelerated mode with "-cpu host
> > -machine virt", the default gic version is 2.
> >
> > I understand the rationale with TCG where we don't have MSI ITS
> > emulation along with GICv3 so we need to choose GICv2 to get GICv2M
> > functionality.
> >
> > However in KVM mode, I would have expected to see the host GIC probed to
> > set the same version on guest. Indeed most of our HW now have GICv3
> > without GICv2 compat mode so our default values lead to weird traces:
> >
> > "
> > qemu-system-aarch64: PMU: KVM_SET_DEVICE_ATTR: Invalid argument
> > qemu-system-aarch64: failed to set irq for PMU
> > "
> >
> > I would like to propose a patch to improve those errors and also suggest
> > a hint. But I also wanted to know whether you would accept to change the
> > default value with KVM and choose the host version instead of 2. For TCG
> > we would keep v2.
> 
> As with the -cpu option, the default is there for command
> line backward compatibility primarily. Even if we had
> better support for MSI ITS emulation we'd still leave
> the default at GICv2.
> 
> If you want "do the best you can, regardless of accelerator"
> that is "-cpu max -machine gic-version=max".
>

There is a case where we can probe without breaking backward
compatibility. That case is kvm-enabled and no gic-version
specified. The reason it would be safe to probe the GIC version
is because unless the host was a gicv2 host, then that command
line wouldn't have worked anyway. And, if it is a gicv2 host,
then the probing will come to the same conclusion as the
default and nothing will have changed.

Thanks,
drew

Re: [question] hw/arm/virt: about the default gic-version in accelerated mode

[Auger Eric]

Hi Drew,

On 1/28/20 1:29 PM, Andrew Jones wrote:
> On Tue, Jan 28, 2020 at 10:52:50AM +0000, Peter Maydell wrote:
>> On Tue, 28 Jan 2020 at 10:47, Auger Eric <eric.auger@redhat.com> wrote:
>>> When arm virt machine is run in accelerated mode with "-cpu host
>>> -machine virt", the default gic version is 2.
>>>
>>> I understand the rationale with TCG where we don't have MSI ITS
>>> emulation along with GICv3 so we need to choose GICv2 to get GICv2M
>>> functionality.
>>>
>>> However in KVM mode, I would have expected to see the host GIC probed to
>>> set the same version on guest. Indeed most of our HW now have GICv3
>>> without GICv2 compat mode so our default values lead to weird traces:
>>>
>>> "
>>> qemu-system-aarch64: PMU: KVM_SET_DEVICE_ATTR: Invalid argument
>>> qemu-system-aarch64: failed to set irq for PMU
>>> "
>>>
>>> I would like to propose a patch to improve those errors and also suggest
>>> a hint. But I also wanted to know whether you would accept to change the
>>> default value with KVM and choose the host version instead of 2. For TCG
>>> we would keep v2.
>>
>> As with the -cpu option, the default is there for command
>> line backward compatibility primarily. Even if we had
>> better support for MSI ITS emulation we'd still leave
>> the default at GICv2.
>>
>> If you want "do the best you can, regardless of accelerator"
>> that is "-cpu max -machine gic-version=max".
>>
> 
> There is a case where we can probe without breaking backward
> compatibility. That case is kvm-enabled and no gic-version
> specified. The reason it would be safe to probe the GIC version
> is because unless the host was a gicv2 host, then that command
> line wouldn't have worked anyway.
Except if the host GICv3 has a GICv2 compat (which is pretty unlikely)?

Eric

 And, if it is a gicv2 host,
> then the probing will come to the same conclusion as the
> default and nothing will have changed.
> 
> Thanks,
> drew
> 
> 

Re: [question] hw/arm/virt: about the default gic-version in accelerated mode

[Andrew Jones]

On Tue, Jan 28, 2020 at 01:34:06PM +0100, Auger Eric wrote:
> Hi Drew,
> 
> On 1/28/20 1:29 PM, Andrew Jones wrote:
> > On Tue, Jan 28, 2020 at 10:52:50AM +0000, Peter Maydell wrote:
> >> On Tue, 28 Jan 2020 at 10:47, Auger Eric <eric.auger@redhat.com> wrote:
> >>> When arm virt machine is run in accelerated mode with "-cpu host
> >>> -machine virt", the default gic version is 2.
> >>>
> >>> I understand the rationale with TCG where we don't have MSI ITS
> >>> emulation along with GICv3 so we need to choose GICv2 to get GICv2M
> >>> functionality.
> >>>
> >>> However in KVM mode, I would have expected to see the host GIC probed to
> >>> set the same version on guest. Indeed most of our HW now have GICv3
> >>> without GICv2 compat mode so our default values lead to weird traces:
> >>>
> >>> "
> >>> qemu-system-aarch64: PMU: KVM_SET_DEVICE_ATTR: Invalid argument
> >>> qemu-system-aarch64: failed to set irq for PMU
> >>> "
> >>>
> >>> I would like to propose a patch to improve those errors and also suggest
> >>> a hint. But I also wanted to know whether you would accept to change the
> >>> default value with KVM and choose the host version instead of 2. For TCG
> >>> we would keep v2.
> >>
> >> As with the -cpu option, the default is there for command
> >> line backward compatibility primarily. Even if we had
> >> better support for MSI ITS emulation we'd still leave
> >> the default at GICv2.
> >>
> >> If you want "do the best you can, regardless of accelerator"
> >> that is "-cpu max -machine gic-version=max".
> >>
> > 
> > There is a case where we can probe without breaking backward
> > compatibility. That case is kvm-enabled and no gic-version
> > specified. The reason it would be safe to probe the GIC version
> > is because unless the host was a gicv2 host, then that command
> > line wouldn't have worked anyway.
> Except if the host GICv3 has a GICv2 compat (which is pretty unlikely)?

Is there a way to probe that? If so, and the setting up of gicv2 on
a gicv3 host with the gicv2-compat is the same as setting up gicv2,
then we can just choose gicv2 to keep the command line compatibility.

Thanks,
drew

Re: [question] hw/arm/virt: about the default gic-version in accelerated mode

[Auger Eric]

Hi,

On 1/28/20 1:41 PM, Andrew Jones wrote:
> On Tue, Jan 28, 2020 at 01:34:06PM +0100, Auger Eric wrote:
>> Hi Drew,
>>
>> On 1/28/20 1:29 PM, Andrew Jones wrote:
>>> On Tue, Jan 28, 2020 at 10:52:50AM +0000, Peter Maydell wrote:
>>>> On Tue, 28 Jan 2020 at 10:47, Auger Eric <eric.auger@redhat.com> wrote:
>>>>> When arm virt machine is run in accelerated mode with "-cpu host
>>>>> -machine virt", the default gic version is 2.
>>>>>
>>>>> I understand the rationale with TCG where we don't have MSI ITS
>>>>> emulation along with GICv3 so we need to choose GICv2 to get GICv2M
>>>>> functionality.
>>>>>
>>>>> However in KVM mode, I would have expected to see the host GIC probed to
>>>>> set the same version on guest. Indeed most of our HW now have GICv3
>>>>> without GICv2 compat mode so our default values lead to weird traces:
>>>>>
>>>>> "
>>>>> qemu-system-aarch64: PMU: KVM_SET_DEVICE_ATTR: Invalid argument
>>>>> qemu-system-aarch64: failed to set irq for PMU
>>>>> "
>>>>>
>>>>> I would like to propose a patch to improve those errors and also suggest
>>>>> a hint. But I also wanted to know whether you would accept to change the
>>>>> default value with KVM and choose the host version instead of 2. For TCG
>>>>> we would keep v2.
>>>>
>>>> As with the -cpu option, the default is there for command
>>>> line backward compatibility primarily. Even if we had
>>>> better support for MSI ITS emulation we'd still leave
>>>> the default at GICv2.
>>>>
>>>> If you want "do the best you can, regardless of accelerator"
>>>> that is "-cpu max -machine gic-version=max".
>>>>
>>>
>>> There is a case where we can probe without breaking backward
>>> compatibility. That case is kvm-enabled and no gic-version
>>> specified. The reason it would be safe to probe the GIC version
>>> is because unless the host was a gicv2 host, then that command
>>> line wouldn't have worked anyway.
>> Except if the host GICv3 has a GICv2 compat (which is pretty unlikely)?
> 
> Is there a way to probe that? If so, and the setting up of gicv2 on
> a gicv3 host with the gicv2-compat is the same as setting up gicv2,
> then we can just choose gicv2 to keep the command line compatibility.
I think that if the host GICv3 is GICv2 compatible then you can create
both a KVM_DEV_TYPE_ARM_VGIC_V2 device and a KVM_DEV_TYPE_ARM_VGIC_V3
device. Otherwise you can only create a KVM_DEV_TYPE_ARM_VGIC_V3 KVM device.

Thanks

Eric

> 
> Thanks,
> drew
> 
> 

Re: [question] hw/arm/virt: about the default gic-version in accelerated mode

[Andrew Jones]

On Tue, Jan 28, 2020 at 02:53:25PM +0100, Auger Eric wrote:
> Hi,
> 
> On 1/28/20 1:41 PM, Andrew Jones wrote:
> > On Tue, Jan 28, 2020 at 01:34:06PM +0100, Auger Eric wrote:
> >> Hi Drew,
> >>
> >> On 1/28/20 1:29 PM, Andrew Jones wrote:
> >>> On Tue, Jan 28, 2020 at 10:52:50AM +0000, Peter Maydell wrote:
> >>>> On Tue, 28 Jan 2020 at 10:47, Auger Eric <eric.auger@redhat.com> wrote:
> >>>>> When arm virt machine is run in accelerated mode with "-cpu host
> >>>>> -machine virt", the default gic version is 2.
> >>>>>
> >>>>> I understand the rationale with TCG where we don't have MSI ITS
> >>>>> emulation along with GICv3 so we need to choose GICv2 to get GICv2M
> >>>>> functionality.
> >>>>>
> >>>>> However in KVM mode, I would have expected to see the host GIC probed to
> >>>>> set the same version on guest. Indeed most of our HW now have GICv3
> >>>>> without GICv2 compat mode so our default values lead to weird traces:
> >>>>>
> >>>>> "
> >>>>> qemu-system-aarch64: PMU: KVM_SET_DEVICE_ATTR: Invalid argument
> >>>>> qemu-system-aarch64: failed to set irq for PMU
> >>>>> "
> >>>>>
> >>>>> I would like to propose a patch to improve those errors and also suggest
> >>>>> a hint. But I also wanted to know whether you would accept to change the
> >>>>> default value with KVM and choose the host version instead of 2. For TCG
> >>>>> we would keep v2.
> >>>>
> >>>> As with the -cpu option, the default is there for command
> >>>> line backward compatibility primarily. Even if we had
> >>>> better support for MSI ITS emulation we'd still leave
> >>>> the default at GICv2.
> >>>>
> >>>> If you want "do the best you can, regardless of accelerator"
> >>>> that is "-cpu max -machine gic-version=max".
> >>>>
> >>>
> >>> There is a case where we can probe without breaking backward
> >>> compatibility. That case is kvm-enabled and no gic-version
> >>> specified. The reason it would be safe to probe the GIC version
> >>> is because unless the host was a gicv2 host, then that command
> >>> line wouldn't have worked anyway.
> >> Except if the host GICv3 has a GICv2 compat (which is pretty unlikely)?
> > 
> > Is there a way to probe that? If so, and the setting up of gicv2 on
> > a gicv3 host with the gicv2-compat is the same as setting up gicv2,
> > then we can just choose gicv2 to keep the command line compatibility.
> I think that if the host GICv3 is GICv2 compatible then you can create
> both a KVM_DEV_TYPE_ARM_VGIC_V2 device and a KVM_DEV_TYPE_ARM_VGIC_V3
> device. Otherwise you can only create a KVM_DEV_TYPE_ARM_VGIC_V3 KVM device.

So it should be backward compatible to switch to probing. Just try gicv2
first. If it works, then use it.

Thanks,
drew