s390/vfio-ap: NIB unpin when the IR bit does not clear

[Yuho Choi <oss.patchbox@gmail.com>]

Hi Tony, Halil, Jason,

I have a question about how vfio_ap turns off AP interrupts before it
unpins the guest page that holds the NIB
(drivers/s390/crypto/vfio_ap_ops.c, v7.4-rc1).

vfio_ap_irq_disable() issues PQAP(AQIC) to disable interrupts and then
calls vfio_ap_wait_for_irqclear(), which checks the IR bit with TAPQ
five times, 20ms apart. If the bit is still set after that, or if AQIC
keeps returning BUSY or RESET_IN_PROGRESS, the code only warns and
unpins the page anyway:

	WARN_ONCE(1, "%s: tapq rc %02x: %04x could not clear IR bit\n",
		  __func__, status.response_code, apqn);
	...
end_free:
	vfio_ap_free_aqic_resources(q);

This path is reached both when the guest turns interrupts off with
PQAP(AQIC) (handle_pqap()) and when user space unmaps the IOVA that
holds the NIB (vfio_ap_mdev_dma_unmap()). In the second case, user
space may free the page right after the unmap.

My questions are:

1. While TAPQ still reports the IR bit as set, can the adapter still
   store into the NIB?

2. Can that state last longer than about 100ms on working hardware,
   for example while long-running requests are still outstanding on
   the queue? Or does it only happen when something is broken?

If both are possible, the adapter could write into a page that is no
longer pinned.

I found this by reading the code and have not reproduced it.

Thanks,
Yuho