Re: [PATCH v5 08/14] iommufd/viommu: Add iommufd_viommu_report_event helper

Nicolin Chen
2025-01-13T05:38:00Z

On Fri, Jan 10, 2025 at 03:51:14PM -0400, Jason Gunthorpe wrote:
> On Fri, Jan 10, 2025 at 10:38:42AM -0800, Nicolin Chen wrote:
> > > The virtual event queue should behave the same as if the physical
> > > event queue overflows, and that logic should be in the smmu driver -
> > > this should return some Exxx to indicate the queue is filled.
> >
> > Hmm, the driver only screams...
> >
> > static irqreturn_t arm_smmu_evtq_thread(int irq, void *dev)
> > {
> > [...]
> > /*
> > * Not much we can do on overflow, so scream and pretend we're
> > * trying harder.
> > */
> > if (queue_sync_prod_in(q) == -EOVERFLOW)
> > dev_err(smmu->dev, "EVTQ overflow detected -- events lost\n");
>
> Well it must know from the HW somehow that the overflow has happened??
>
> > > I supposed we will need a way to indicate lost events to userspace on
> > > top of this?
> >
> > Perhaps another u32 flag in the arm_smmuv3_vevent struct to report
> > an overflow. That said, what userspace/VMM will need to do with it?
>
> Trigger the above code in the VM somehow?

I found two ways of forwarding an overflow flag:

1. Return -EOVERFLOW to read(). But it cannot return the read bytes
any more:
--------------------------------------------------
@@ -95,2 +95,3 @@ int iommufd_viommu_report_event(struct iommufd_viommu *viommu,
if (atomic_read(&veventq->num_events) == veventq->depth) {
+ set_bit(IOMMUFD_VEVENTQ_ERROR_OVERFLOW, veventq->errors);
rc = -EOVERFLOW;

[..]

@@ -386,2 +386,5 @@ static ssize_t iommufd_veventq_fops_read(struct file *filep, char __user *buf,

+ if (test_bit(IOMMUFD_VEVENTQ_ERROR_OVERFLOW, veventq->errors))
+ rc = -EOVERFLOW;
+
mutex_lock(&eventq->mutex);
@@ -398,2 +401,3 @@ static ssize_t iommufd_veventq_fops_read(struct file *filep, char __user *buf,
}
+ clear_bit(IOMMUFD_VEVENTQ_ERROR_OVERFLOW, veventq->errors);
atomic_dec(&veventq->num_events);
@@ -405,2 +409,4 @@ static ssize_t iommufd_veventq_fops_read(struct file *filep, char __user *buf,

+ if (rc == -EOVERFLOW)
+ return rc;
return done == 0 ? rc : done;

[..]

@@ -554,2 +554,4 @@ struct iommufd_veventq {
atomic_t num_events;
+#define IOMMUFD_VEVENTQ_ERROR_OVERFLOW 0
+ DECLARE_BITMAP(errors, 32);
};
--------------------------------------------------

2. Return EPOLLERR via pollfd.revents. But it cannot use POLLERR
for other errors any more:
--------------------------------------------------
@@ -420,2 +421,4 @@ static __poll_t iommufd_eventq_fops_poll(struct file *filep,
poll_wait(filep, &eventq->wait_queue, wait);
+ if (test_bit(IOMMUFD_VEVENTQ_ERROR_OVERFLOW, veventq->errors))
+ return EPOLLERR;
mutex_lock(&eventq->mutex);

[..]

@@ -1001,2 +1001,5 @@ static int _test_cmd_trigger_vevent(int fd, __u32 dev_id, __u32 event_fd,

+ if (pollfd.revents & POLLERR)
+ return -1;
+
return event.virt_id == virt_id ? 0 : -EINVAL
--------------------------------------------------

It feels that returning at read() might be slightly nicer?

Thanks
Nicolin