Re: [Qemu-devel] [PATCH v2 5/5] s390x/cpumodel: Set up CPU model for AP device support

[Pierre Morel <pmorel@linux.vnet.ibm.com>]

On 06/03/2018 18:15, David Hildenbrand wrote:
>>>> 1) ap=on is a guest ABI feature saying to the guest you can use AP
>>>> instructions
>>> Indeed, that's what belongs into the CPU model.
>>>
>>>> 2) How we provide AP instructions to the guest can be done in three
>>>> different ways:
>>>>     - SIE Interpretation
>> AP instructions executed on the guest will be interpreted and passed
>> directly
>> through to a real AP device installed on the host according to the APQN
>> specified in the instruction, so the AP instructions must be available
>> on the
>> host.
>>>>     - interception with VFIO
>> AP instructions executed on the guest will be intercepted, and
>> interpreted by
>> QEMU then forwarded to a real AP device installed on the host according
>> to how
>> the AP devices are configured in userspace - i.e., whether there is
>> remapping,
>> multiplexing or sharing of adapters/domains etc. This also requires that AP
>> instructions be available on the host.
> See my other mail: I think this is a conflict with vSIE.
>
>>>>     - interception with emulation
>> AP instructions executed on the guest will be intercepted, interpreted
>> by QEMU
>> and emulated. This will not require AP instructions be available on the
>> host.
> See my other mail: I think this is a conflict with vSIE.
>
>> In all cases above, the need to set ECA_APIE is dependent upon the device
>> type configured for the guest.
>>> Due to bad AP design we can't handle this like zPCI - completely in
>>> QEMU. That's what should control it.
>>>
>>>> 3) We implement this with a device in QEMU and a certain level kernel
>>>> support.
>>>>
>>>> It seems possible to set or not ECA.28 , based on the type of kernel device:
>>>> - SIE interpretation -> MATRIX KVM device -> ECA.28
>>>> - Interception with VFIO and virtualization -> no ECA.28
>>>> - interception with emulation -> no ECA.28
>>>>
>>>> I understand the concern with the vCPU but I think we can handle it with
>>>> an indirect variable
>>>> like:
>>>>
>>>> SIE interpretation Device + KVM_S390_VM_CPU_FEAT_AP -> set the variable
>>>> ap_to_be_sie_interpreted=1
>>>> Then in vCPU initialization set ECA.28 based on this variable.
>>> That's exactly why we have the cpu feature interface. E.g. CMMA -> if
>>> not enabled, not interpreted by HW (however also not intercepted to user
>>> space - no sense in doing that right now).
>> There are two factors at play here, the device type (i.e., -device
>> vfio_ap) and
>> the KVM_S390_VM_CPU_FEAT_AP feature. Setting ECA_APIE makes no sense if the
>> vfio_ap device is not configured. For the passthrough implementation, this
>> means that the AP bus will successfully load on the guest, but there will
>> be no AP devices detected. If the expectation is that ap=on will allow
>> access
>> to AP devices on the guest, that would be a mistaken assumption.
>>
>> If down the road a new guest AP device is introduced that allows
>> multiplexing,
>> device remapping etc., then it will be necessary to intercept AP
>> instructions
>> executed on the guest. In this case, if ap=on results in setting ECA_APIE,
>> then the instructions will be interpreted and passed through to a device
>> that does not exist because it won't be defined in the guest's CRYCB.
> Again, see my other mail, this discussion is superfluous if we can't get
> vSIE to work with emulated devices. And it smells like this is the case.
> But I don't have access to documentation.
>
>> Based on these two scenarios, I think what we are really saying with ap=on
>> is that the guest will require use of the AP instructions installed on the
>> host. Whether those instructions are executed as a result of interpretation
>> by the hardware or because they are executed by the device driver is a
>> separate matter. So, I am inclined to agree with Pierre for that reason.
>> ECA_APIE should be set only if ap=on (i.e., AP instructions are available
>> on the host) and the user of those instructions (i.e., the driver) indicate
>> an intent to use them.
> ap=on -> set ECA_APIE is what I proposed.

True if we only support SIE interpretation, what you propose.
but
It is not what I meant.
What I mean is the reverse implication

ECA_APIE => ap=on

But you can have ap = on and ECA_APIE = off
This is interception or emulation.

and the second thing is that we need two QEMU cpu features
AP : guest API to say we provide AP instructions to the guest (what ever 
we do to provide it)
ECA_APIE : kernel will setup the SIE with interpretation

other said:
if( !ap)
     return -ENODEVICE
if(ECA_API)
     set_interpretation()
else
     set_interception()




-- 
Pierre Morel
Linux/KVM/QEMU in Böblingen - Germany