On Fri, Jan 10, 2025 at 03:51:14PM -0400, Jason Gunthorpe wrote:
On Fri, Jan 10, 2025 at 10:38:42AM -0800, Nicolin Chen wrote:
The virtual event queue should behave the same as if the physical event queue overflows, and that logic should be in the smmu driver - this should return some Exxx to indicate the queue is filled.
Hmm, the driver only screams...
static irqreturn_t arm_smmu_evtq_thread(int irq, void *dev) { [...] /* * Not much we can do on overflow, so scream and pretend we're * trying harder. */ if (queue_sync_prod_in(q) == -EOVERFLOW) dev_err(smmu->dev, "EVTQ overflow detected -- events lost\n");
Well it must know from the HW somehow that the overflow has happened??
I supposed we will need a way to indicate lost events to userspace on top of this?
Perhaps another u32 flag in the arm_smmuv3_vevent struct to report an overflow. That said, what userspace/VMM will need to do with it?
Trigger the above code in the VM somehow?
I found two ways of forwarding an overflow flag:
1. Return -EOVERFLOW to read(). But it cannot return the read bytes any more: -------------------------------------------------- @@ -95,2 +95,3 @@ int iommufd_viommu_report_event(struct iommufd_viommu *viommu, if (atomic_read(&veventq->num_events) == veventq->depth) { + set_bit(IOMMUFD_VEVENTQ_ERROR_OVERFLOW, veventq->errors); rc = -EOVERFLOW;
[..]
@@ -386,2 +386,5 @@ static ssize_t iommufd_veventq_fops_read(struct file *filep, char __user *buf,
+ if (test_bit(IOMMUFD_VEVENTQ_ERROR_OVERFLOW, veventq->errors)) + rc = -EOVERFLOW; + mutex_lock(&eventq->mutex); @@ -398,2 +401,3 @@ static ssize_t iommufd_veventq_fops_read(struct file *filep, char __user *buf, } + clear_bit(IOMMUFD_VEVENTQ_ERROR_OVERFLOW, veventq->errors); atomic_dec(&veventq->num_events); @@ -405,2 +409,4 @@ static ssize_t iommufd_veventq_fops_read(struct file *filep, char __user *buf,
+ if (rc == -EOVERFLOW) + return rc; return done == 0 ? rc : done;
[..]
@@ -554,2 +554,4 @@ struct iommufd_veventq { atomic_t num_events; +#define IOMMUFD_VEVENTQ_ERROR_OVERFLOW 0 + DECLARE_BITMAP(errors, 32); }; --------------------------------------------------
2. Return EPOLLERR via pollfd.revents. But it cannot use POLLERR for other errors any more: -------------------------------------------------- @@ -420,2 +421,4 @@ static __poll_t iommufd_eventq_fops_poll(struct file *filep, poll_wait(filep, &eventq->wait_queue, wait); + if (test_bit(IOMMUFD_VEVENTQ_ERROR_OVERFLOW, veventq->errors)) + return EPOLLERR; mutex_lock(&eventq->mutex);
[..]
@@ -1001,2 +1001,5 @@ static int _test_cmd_trigger_vevent(int fd, __u32 dev_id, __u32 event_fd,
+ if (pollfd.revents & POLLERR) + return -1; + return event.virt_id == virt_id ? 0 : -EINVAL --------------------------------------------------
It feels that returning at read() might be slightly nicer?
Thanks Nicolin