Re: [PATCH v5 11/13] KVM: s390: implement mediated device open callback

[Tony Krowiak <akrowiak@linux.vnet.ibm.com>]

On 06/07/2018 11:20 AM, Pierre Morel wrote:
> On 07/06/2018 15:54, Tony Krowiak wrote:
>> On 06/06/2018 01:40 PM, Pierre Morel wrote:
>>> On 06/06/2018 18:08, Pierre Morel wrote:
>>>> On 06/06/2018 16:28, Tony Krowiak wrote:
>>>>> On 06/05/2018 08:19 AM, Pierre Morel wrote:
>>>>>> On 30/05/2018 16:33, Tony Krowiak wrote:
>>>>>>> On 05/24/2018 05:08 AM, Pierre Morel wrote:
>>>>>>>> On 23/05/2018 16:45, Tony Krowiak wrote:
>>>>>>>>> On 05/16/2018 04:03 AM, Pierre Morel wrote:
>>>>>>>>>> On 07/05/2018 17:11, Tony Krowiak wrote:
>>>>>>>>>>> Implements the open callback on the mediated matrix device.
>>>>>>>>>>> The function registers a group notifier to receive notification
>>>>>>>>>>> of the VFIO_GROUP_NOTIFY_SET_KVM event. When notified,
>>>>>>>>>>> the vfio_ap device driver will get access to the guest's
>>>>>>>>>>> kvm structure. With access to this structure the driver will:
>>>>>>>>>>>
>>>>>>>>>>> 1. Ensure that only one mediated device is opened for the guest
>>>>>>>>
>>>>>>>> You should explain why.
>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> 2. Configure access to the AP devices for the guest.
>>>>>>>>>>>
>>>>>>>> ...snip...
>>>>>>>>>>> +void kvm_ap_refcount_inc(struct kvm *kvm)
>>>>>>>>>>> +{
>>>>>>>>>>> + atomic_inc(&kvm->arch.crypto.aprefs);
>>>>>>>>>>> +}
>>>>>>>>>>> +EXPORT_SYMBOL(kvm_ap_refcount_inc);
>>>>>>>>>>> +
>>>>>>>>>>> +void kvm_ap_refcount_dec(struct kvm *kvm)
>>>>>>>>>>> +{
>>>>>>>>>>> + atomic_dec(&kvm->arch.crypto.aprefs);
>>>>>>>>>>> +}
>>>>>>>>>>> +EXPORT_SYMBOL(kvm_ap_refcount_dec);
>>>>>>>>>>
>>>>>>>>>> Why are these functions inside kvm-ap ?
>>>>>>>>>> Will anyone use this outer of vfio-ap ?
>>>>>>>>>
>>>>>>>>> As I've stated before, I made the choice to contain all 
>>>>>>>>> interfaces that
>>>>>>>>> access KVM in kvm-ap because I don't think it is appropriate 
>>>>>>>>> for the device
>>>>>>>>> driver to have to have "knowledge" of the inner workings of 
>>>>>>>>> KVM. Why does
>>>>>>>>> it matter whether any entity outside of the vfio_ap device 
>>>>>>>>> driver calls
>>>>>>>>> these functions? I could ask a similar question if the 
>>>>>>>>> interfaces were
>>>>>>>>> contained in vfio-ap; what if another device driver needs 
>>>>>>>>> access to these
>>>>>>>>> interfaces?
>>>>>>>>
>>>>>>>> This is very driver specific and only used during initialization.
>>>>>>>> It is not a common property of the cryptographic interface.
>>>>>>>>
>>>>>>>> I really think you should handle this inside the driver.
>>>>>>>
>>>>>>> We are going to have to agree to disagree on this one. Is it not 
>>>>>>> possible
>>>>>>> that future drivers - e.g., when full virtualization is 
>>>>>>> implemented - will
>>>>>>> require access to KVM?
>>>>>>
>>>>>> I do not think that an access to KVM is required for full 
>>>>>> virtualization.
>>>>>
>>>>> You may be right, but at this point, there is no guarantee. I 
>>>>> stand by my
>>>>> design on this one.
>>>>
>>>> I really regret that we abandoned the initial design with the 
>>>> matrix bus and one
>>>> single parent matrix device per guest.
>>>> We would not have the problem of these KVM dependencies.
>>>>
>>>> It had the advantage of taking care of having only one device per 
>>>> guest
>>>> (available_instance = 1), could take care of provisioning as you have
>>>> sysfs entries available for a matrix without having a guest and a 
>>>> mediated
>>>> device.
>>>>
>>>> it also had advantage for virtualization to keep host side and 
>>>> guest side matrix
>>>> separate inside parent (host side) and mediated device (guest side).
>>>>
>>>> Shouldn't we treat this problem with a design using standard 
>>>> interfaces
>>>> Instead of adding new dedicated interfaces?
>>>>
>>>> Regards,
>>>>
>>>> Pierre
>>>>
>>>>
>>>
>>> Forget it.
>>>
>>> I am not happy with the design but the design I was speaking of may 
>>> not be the solution either.
>>
>> The AP architecture makes virtualization of AP devices complex. We 
>> tried the solution you
>> described and found it to be sorely lacking which is why we ended up 
>> where we are now.
>
> I did not see any explanation on why between v1 and v2 as it was 
> abandoned.
>
>
> We have internal structures like the ap_matrix and kvm_ap_matrix
> which look like the bus/devices we had previously but are differently
> or not at all integrated with the LDD.

What is LDD? Are you talking about dependencies between the vfio_ap device
driver and KVM? If so, see my arguments below.

>
>
> Also I think that with a little data structure refactoring you can 
> avoid most of
> the code in the arch/s390/kvm.

How will structure refactoring help us avoid the code for updating the CRYCB
in the guest's SIE state description.

>
>
> For example, storing the kvm pointer inside the kvm_ap_matrix and
> maintaining a list of the kvm_ap_matrix structures allows to easily know
> if a guest already has an associated mediated device.

How is that easier than storing the kvm pointer inside of the mediated 
matrix
device (i.e., struct ap_matrix_mdev) which also contains the struct 
kvm_ap_matrix?
How does that allow us to avoid the code in arch/s390/kvm? We still need 
the code
to update the CRYCB in the SIE block. I can obviously avoid placing that 
code in
kvm-ap.c and move it to the driver, but I already explained my reasoning 
for
keeping it in KVM. Let's face it, there is no way around the dependency 
between
the vfio_ap device driver and KVM unless guest matrix configuration is 
managed
solely by KVM through KVM interfaces.

Why maintain a list of kvm_ap_matrix structures if we don't have to; it 
is stored
with the mediated matrix device which is passed in to all of the vfio_ap 
driver
callbacks.

>
>
> Pierre
>
>>
>>>
>>>
>>> Sorry for the noise.
>>>
>>> Regards,
>>>
>>> Pierre
>>>
>>>
>>
>