Re: [RFC PATCH v9 3/4] hw/vfio/ap: Storing event information for an AP configuration change event

[Anthony Krowiak <akrowiak@linux.ibm.com>]

On 5/22/25 9:30 AM, Cédric Le Goater wrote:
> On 5/12/25 20:02, Rorie Reyes wrote:
>> These functions can be invoked by the function that handles interception
>> of the CHSC SEI instruction for requests indicating the accessibility of
>> one or more adjunct processors has changed.
>>
>> Signed-off-by: Rorie Reyes <rreyes@linux.ibm.com>
>> ---
>>   hw/vfio/ap.c                 | 39 ++++++++++++++++++++++++++++++++++++
>>   include/hw/s390x/ap-bridge.h | 22 ++++++++++++++++++++
>>   2 files changed, 61 insertions(+)
>>
>> diff --git a/hw/vfio/ap.c b/hw/vfio/ap.c
>> index 5ea5dd9cca..4f88f80c54 100644
>> --- a/hw/vfio/ap.c
>> +++ b/hw/vfio/ap.c
>> @@ -96,6 +96,45 @@ static void vfio_ap_cfg_chg_notifier_handler(void 
>> *opaque)
>>     }
>>   +int ap_chsc_sei_nt0_get_event(void *res)
>> +{
>> +    ChscSeiNt0Res *nt0_res  = (ChscSeiNt0Res *)res;
>> +    APConfigChgEvent *cfg_chg_event;
>> +
>> +    if (!ap_chsc_sei_nt0_have_event()) {
>> +        return 1;
>> +    }
>> +
>> +    cfg_chg_event = QTAILQ_FIRST(&cfg_chg_events);
>> +    memset(nt0_res, 0, sizeof(*nt0_res));
>> +
>> +    QTAILQ_REMOVE(&cfg_chg_events, cfg_chg_event, next);
>
> btw, I don't know if this was discussed. Are we OK to manipulate the
> 'cfg_chg_events' construct withou locking ?

This has never been discussed, but it's an interesting question. The
ap_chsc_sei_nt0_get_event and ap_chsc_sei_nt0_have_event functions
are called as a result of a SIE exit to handle interception of a
CHSC SEI instruction. Handling of the intercepted instructions is
done under the Big QEMU Lock (see kvm_arch_handle_exit in 
target/s390x/kvm/kvm.c),
so presumably no other processes will get access to these functions
until the instruction is handled.

On the other hand, the vfio_cfg_chg_notifier_handler function that 
handles the eventfd
indicating the guest's AP configuration has been changed by the host 
device driver
adds  APConfigChgEvent objects to this queue. If, however, you think 
about the flow,
when the notifier handler gets called to handle an AP config changed 
event, it
queues a channel request word (CRW) indicating there is an SEI pending. 
Consequently,
the ap_chsc_sei_nt0_get_event and ap_chsc_sei_nt0_have_event functions 
will get called
only after the guest receives the CRW event and executes the CHSC SEI 
instruction. Since
the Big QEMU Lock is taken when the CHSC SE instruction is intercepted, 
it can not proceed
until whatever the holding process releases it; so for that flow, it 
seems highly likely if not
impossible for conflict given the event will always be added to the 
queue before an attempt
can be made to retrieve it.

Having gone through this dissertation, I don't see how it can hurt to 
lock the queue when
it is being accessed and would certainly make things bullet proof. What 
is your opinion?

>
>> +    g_free(cfg_chg_event);
>> +
>> +    /*
>> +     * If there are any AP configuration change events in the queue,
>> +     * indicate to the caller that there is pending event info in
>> +     * the response block
>> +     */
>> +    if (ap_chsc_sei_nt0_have_event()) {
>> +        nt0_res->flags |= PENDING_EVENT_INFO_BITMASK;
>> +    }
>> +
>> +    nt0_res->length = sizeof(ChscSeiNt0Res);
>> +    nt0_res->code = NT0_RES_RESPONSE_CODE;
>> +    nt0_res->nt = NT0_RES_NT_DEFAULT;
>> +    nt0_res->rs = NT0_RES_RS_AP_CHANGE;
>> +    nt0_res->cc = NT0_RES_CC_AP_CHANGE;
>> +
>> +    return 0;
>> +
>> +}
>> +
>> +int ap_chsc_sei_nt0_have_event(void)
>> +{
>> +    return !QTAILQ_EMPTY(&cfg_chg_events);
>> +}
>> +
>>   static bool vfio_ap_register_irq_notifier(VFIOAPDevice *vapdev,
>>                                             unsigned int irq, Error 
>> **errp)
>>   {
>> diff --git a/include/hw/s390x/ap-bridge.h b/include/hw/s390x/ap-bridge.h
>> index 470e439a98..f4d838bf99 100644
>> --- a/include/hw/s390x/ap-bridge.h
>> +++ b/include/hw/s390x/ap-bridge.h
>> @@ -16,4 +16,26 @@
>>     void s390_init_ap(void);
>>   +typedef struct ChscSeiNt0Res {
>> +    uint16_t length;
>> +    uint16_t code;
>> +    uint8_t reserved1;
>> +    uint16_t reserved2;
>> +    uint8_t nt;
>> +#define PENDING_EVENT_INFO_BITMASK 0x80;
>> +    uint8_t flags;
>> +    uint8_t reserved3;
>> +    uint8_t rs;
>> +    uint8_t cc;
>> +} QEMU_PACKED ChscSeiNt0Res;
>> +
>> +#define NT0_RES_RESPONSE_CODE 1;
>> +#define NT0_RES_NT_DEFAULT    0;
>> +#define NT0_RES_RS_AP_CHANGE  5;
>> +#define NT0_RES_CC_AP_CHANGE  3;
>
>
>
> please drop the ending ';'
>
>
> Thanks,
>
> C.
>
>
>> +int ap_chsc_sei_nt0_get_event(void *res);
>> +
>> +int ap_chsc_sei_nt0_have_event(void);
>> +
>>   #endif
>