Re: [PATCH v5 08/14] iommufd/viommu: Add iommufd_viommu_report_event helper

[Nicolin Chen <nicolinc@nvidia.com>]

On Fri, Jan 10, 2025 at 03:51:14PM -0400, Jason Gunthorpe wrote:
> On Fri, Jan 10, 2025 at 10:38:42AM -0800, Nicolin Chen wrote:
> > > The virtual event queue should behave the same as if the physical
> > > event queue overflows, and that logic should be in the smmu driver -
> > > this should return some Exxx to indicate the queue is filled.
> > 
> > Hmm, the driver only screams...
> > 
> > static irqreturn_t arm_smmu_evtq_thread(int irq, void *dev)
> > {
> > [...]
> > 		/*
> > 		 * Not much we can do on overflow, so scream and pretend we're
> > 		 * trying harder.
> > 		 */
> > 		if (queue_sync_prod_in(q) == -EOVERFLOW)
> > 			dev_err(smmu->dev, "EVTQ overflow detected -- events lost\n");
> 
> Well it must know from the HW somehow that the overflow has happened??
> 
> > > I supposed we will need a way to indicate lost events to userspace on
> > > top of this?
> > 
> > Perhaps another u32 flag in the arm_smmuv3_vevent struct to report
> > an overflow. That said, what userspace/VMM will need to do with it?
> 
> Trigger the above code in the VM somehow?

I found two ways of forwarding an overflow flag:

1. Return -EOVERFLOW to read(). But it cannot return the read bytes
any more:
--------------------------------------------------
@@ -95,2 +95,3 @@ int iommufd_viommu_report_event(struct iommufd_viommu *viommu,
        if (atomic_read(&veventq->num_events) == veventq->depth) {
+               set_bit(IOMMUFD_VEVENTQ_ERROR_OVERFLOW, veventq->errors);
                rc = -EOVERFLOW;

[..]

@@ -386,2 +386,5 @@ static ssize_t iommufd_veventq_fops_read(struct file *filep, char __user *buf,
 
+       if (test_bit(IOMMUFD_VEVENTQ_ERROR_OVERFLOW, veventq->errors))
+               rc = -EOVERFLOW;
+
        mutex_lock(&eventq->mutex);
@@ -398,2 +401,3 @@ static ssize_t iommufd_veventq_fops_read(struct file *filep, char __user *buf,
                }
+               clear_bit(IOMMUFD_VEVENTQ_ERROR_OVERFLOW, veventq->errors);
                atomic_dec(&veventq->num_events);
@@ -405,2 +409,4 @@ static ssize_t iommufd_veventq_fops_read(struct file *filep, char __user *buf,
 
+       if (rc == -EOVERFLOW)
+               return rc;
        return done == 0 ? rc : done;

[..]

@@ -554,2 +554,4 @@ struct iommufd_veventq {
        atomic_t num_events;
+#define IOMMUFD_VEVENTQ_ERROR_OVERFLOW 0
+       DECLARE_BITMAP(errors, 32);
 };
--------------------------------------------------

2. Return EPOLLERR via pollfd.revents. But it cannot use POLLERR
for other errors any more:
--------------------------------------------------
@@ -420,2 +421,4 @@ static __poll_t iommufd_eventq_fops_poll(struct file *filep,
        poll_wait(filep, &eventq->wait_queue, wait);
+       if (test_bit(IOMMUFD_VEVENTQ_ERROR_OVERFLOW, veventq->errors))
+               return EPOLLERR;
        mutex_lock(&eventq->mutex);

[..]

@@ -1001,2 +1001,5 @@ static int _test_cmd_trigger_vevent(int fd, __u32 dev_id, __u32 event_fd,
 
+       if (pollfd.revents & POLLERR)
+               return -1;
+
        return event.virt_id == virt_id ? 0 : -EINVAL
--------------------------------------------------

It feels that returning at read() might be slightly nicer?

Thanks
Nicolin