Re: [Qemu-devel] [PATCH v3 6/7] s390x/kvm: handle AP instruction interception

[Cornelia Huck <cohuck@redhat.com>]

On Fri, 6 Apr 2018 18:07:56 +0200
Halil Pasic <pasic@linux.vnet.ibm.com> wrote:

> On 04/05/2018 06:38 PM, Tony Krowiak wrote:
> > On 04/03/2018 05:36 AM, Cornelia Huck wrote:  
> >> On Mon, 2 Apr 2018 12:36:27 -0400
> >> Tony Krowiak <akrowiak@linux.vnet.ibm.com> wrote:
> >>  
> >>> On 03/26/2018 05:03 AM, Pierre Morel wrote:  
> >>>> On 26/03/2018 10:32, David Hildenbrand wrote:  
> >>>>> On 16.03.2018 00:24, Tony Krowiak wrote:  
> >>>>>> +    /*
> >>>>>> +     * The Query Configuration Information (QCI) function (fc == 4)
> >>>>>> does not
> >>>>>> +     * set a response code in reg 1, so check for that along with the
> >>>>>> +     * AP feature.
> >>>>>> +     */
> >>>>>> +    if ((fc != 4) && s390_has_feat(S390_FEAT_AP)) {
> >>>>>> +        env->regs[1] = 0x10000;
> >>>>>> +
> >>>>>> +        return 0;
> >>>>>> +    }  
> >>>>> This would imply an operation exception in case fc==4, which sounds very
> >>>>> wrong.  
> >>>> It depends but I think that the S390_FEAT_AP_QUERY_CONFIG_INFO must be
> >>>> tested
> >>>> to know what to answer.
> >>>> If the feature is there, QCI must be answered correctly.  
> >>> This is an interesting proposition which raises several issues that will
> >>> need to
> >>> be addressed. The only time the PQAP(QCI) instruction is intercepted is
> >>> when:
> >>> * A vfio-ap device is NOT defined for the guest because the vfio_ap
> >>> device driver
> >>>     will set ECA.28 and the PQAP(QCI) instruction will be interpreted
> >>> * STFLE.12 is set for the guest
> >>>
> >>> You say that the QCI must be answered correctly, but what is the correct
> >>> response?
> >>> If we return the matrix - i.e., APM, ADM and AQM - configured via the
> >>> mediated
> >>> matrix device's sysfs attributes files, then if any APQNs are defined in
> >>> the matrix,
> >>> we will have to handle *ALL* AP instructions by executing them on behalf
> >>> of the
> >>> guest. I suppose we could return an empty matrix in which case the AP
> >>> bus will come
> >>> up without any devices on the guest, but what is the expectation of an
> >>> admin who
> >>> deliberately configures the mediated matrix device? Should we forego
> >>> handling interception
> >>> of AP instructions and consider a guest configuration that turns on
> >>> S390_FEAT_AP but
> >>> does not define a vfio-ap device to be erroneous and terminate starting
> >>> of the guest?
> >>> Anybody have any thoughts?  
> >> Hard to really give good advice without access to the documentation, but:
> >> - If we tell the guest that the feature is available, but it does not
> >>    get any cards to use, returning an empty matrix makes the most sense
> >>    to me.
> >> - I would not tie starting the guest to the presence of a vfio-ap
> >>    device. Having the feature available in theory but without any
> >>    devices actually being usable by the guest does not really sound
> >>    wrong (can we hotplug this later?)  
> > For this phase of development, it is my opinion that introducing AP instruction
> > interception handlers is superfluous for the following reasons:
> > 
> > 1. Interception handling was introduced solely to ensure an operation exception would
> >    not be injected into the guest when CPU model feature for AP (i.e., ap=on)
> >    is specified but a VFIO AP device (i.e., -device vfio-ap,sysfsdev=$path)
> >    is not.  
> 
> We can kind of (i.e. modulo EECA.28) ensure this in a different fashion I think. How
> about proclaiming a 'has ap instructions, but nothing to see here' in the
> SIE interpreted flavor (ECA.28 set) the default way of having ap instructions
> under KVM. This should be easily accomplished with an all zero CRYCB and eca.28
> set. The for the guest to actually get real work done with AP we would
> still require some sort of driver to either provide a non-zero matrix by
> altering the CRYCB or unsettling ECA.28 and doing the intercepted flavor.
> 
> Please notice, the cpu facility ap would still keep it's semantic
> 'has ap instructions' (opposed to 'has ap instructions implemented in
> SIE interpreted flavor). And give us all the flexibility.
> 
> Yet implementing what we want to have in absence of a driver would become
> much easier (under the assumption that ECA.28 equals EECA.28).
> 
> How about this?

Unfortunately, this is really hard to follow without the AR... let me
summarize it to check whether I got the gist of it :)

- If the "ap" cpu feature is specified, set a bit that indicates "hey,
  we basically have have AP support" and create the basics, but don't
  enable actual SIE handling. This means the guest gets exceptions from
  the SIE already and we don't need to emulate them.
- Actually enable the missing pieces if a vfio device is created. This
  would enable processing by the SIE, and we would not need to do
  emulation, either (for most of it, IIRC).

I may be all wrong, though... can we at least have a translation of
ECA.28 and EECA.28 (the "ap is there" bit and the "ap instructions are
interpreted" bit?)