Re: [PATCH v4 3/7] s390: ap: associate a ap_vfio_queue and a matrix mdev

Tony Krowiak
2019-03-05T22:17:36Z

On 3/3/19 9:09 PM, Halil Pasic wrote:
> On Fri, 22 Feb 2019 16:29:56 +0100
> Pierre Morel <pmorel@linux.ibm.com> wrote:
>
>> We need to associate the ap_vfio_queue, which will hold the
>> per queue information for interrupt with a matrix mediated device
>> which hold the configuration and the way to the CRYCB.
> [..]
>> +static int vfio_ap_get_all_domains(struct ap_matrix_mdev *matrix_mdev, int apid)
>> +{
>> + int apqi, apqn;
>> + int ret = 0;
>> + struct vfio_ap_queue *q;
>> + struct list_head q_list;
>> +
>> + INIT_LIST_HEAD(&q_list);
>> +
>> + for_each_set_bit_inv(apqi, matrix_mdev->matrix.aqm, AP_DOMAINS) {
>> + apqn = AP_MKQID(apid, apqi);
>> + q = vfio_ap_get_queue(apqn, &matrix_dev->free_list);
>> + if (!q) {
>> + ret = -EADDRNOTAVAIL;
>> + goto rewind;
>> + }
>> + if (q->matrix_mdev) {
>> + ret = -EADDRINUSE;
>
> You tried to get the q from matrix_dev->free_list thus modulo races
> q->matrix_mdev should be 0. This change breaks the error codes in a
> sense that it becomes impossible to provoke EADDRINUSE (the proper
> error code for taken by another matrix_mdev).

I don't understand what you are saying here. AFIU, the idea here is to
pull the q from the free list. If there is no q for the apqn on the free
list, then that would indicate the queue has not been bound to a driver
in which case the appropriate rc is EADDRNOTAVAIL. If the queue has
been bound, then a check is done to see whether the queue has been
associated with an mdev device. If so, the rc is -EADDRINUSE, which is
also appropriate. What am I missing?

>
>> + goto rewind;
>> + }
>> + list_move(&q->list, &q_list);
>> + }
>> + move_and_set(&q_list, &matrix_mdev->qlist, matrix_mdev);
>> return 0;
>> +rewind:
>> + move_and_set(&q_list, &matrix_dev->free_list, NULL);
>> + return ret;
>> }
>> -
>> /**
>> - * vfio_ap_mdev_verify_no_sharing
>> + * vfio_ap_get_all_cards:
>> *
>> - * Verifies that the APQNs derived from the cross product of the AP adapter IDs
>> - * and AP queue indexes comprising the AP matrix are not configured for another
>> - * mediated device. AP queue sharing is not allowed.
>> + * @matrix_mdev: the matrix mediated device for which we want to associate
>> + * all available queues with a given apqi.
>> + * @apqi: The apqi which associated with all defined APID of the
>> + * mediated device will define a AP queue.
>> *
>> - * @matrix_mdev: the mediated matrix device
>> + * We define a local list to put all queues we find on the matrix device
>> + * free list when associating the apqi with all already defined apid for
>> + * this matrix mediated device.
>> *
>> - * Returns 0 if the APQNs are not shared, otherwise; returns -EADDRINUSE.
>> + * If we can get all the devices we roll them to the mediated device list
>> + * If we get errors we unroll them to the free list.
>> */
>> -static int vfio_ap_mdev_verify_no_sharing(struct ap_matrix_mdev *matrix_mdev)
>> +static int vfio_ap_get_all_cards(struct ap_matrix_mdev *matrix_mdev, int apqi)
>> {
>> - struct ap_matrix_mdev *lstdev;
>> - DECLARE_BITMAP(apm, AP_DEVICES);
>> - DECLARE_BITMAP(aqm, AP_DOMAINS);
>> -
>> - list_for_each_entry(lstdev, &matrix_dev->mdev_list, node) {
>> - if (matrix_mdev == lstdev)
>> - continue;
>> -
>> - memset(apm, 0, sizeof(apm));
>> - memset(aqm, 0, sizeof(aqm));
>> -
>> - /*
>> - * We work on full longs, as we can only exclude the leftover
>> - * bits in non-inverse order. The leftover is all zeros.
>> - */
>> - if (!bitmap_and(apm, matrix_mdev->matrix.apm,
>> - lstdev->matrix.apm, AP_DEVICES))
>> - continue;
>> -
>> - if (!bitmap_and(aqm, matrix_mdev->matrix.aqm,
>> - lstdev->matrix.aqm, AP_DOMAINS))
>> - continue;
>> -
>> - return -EADDRINUSE;
>> + int apid, apqn;
>> + int ret = 0;
>> + struct vfio_ap_queue *q;
>> + struct list_head q_list;
>> + struct ap_matrix_mdev *tmp = NULL;
>> +
>> + INIT_LIST_HEAD(&q_list);
>> +
>> + for_each_set_bit_inv(apid, matrix_mdev->matrix.apm, AP_DEVICES) {
>> + apqn = AP_MKQID(apid, apqi);
>> + q = vfio_ap_get_queue(apqn, &matrix_dev->free_list);
>> + if (!q) {
>> + ret = -EADDRNOTAVAIL;
>> + goto rewind;
>> + }
>> + if (q->matrix_mdev) {
>> + ret = -EADDRINUSE;
>
> Same here!
>
> Regards,
> Halil
>
>> + goto rewind;
>> + }
>> + list_move(&q->list, &q_list);
>> }
>
> [..]
>