On Mon, Jan 13, 2025 at 03:54:33PM -0400, Jason Gunthorpe wrote:
On Mon, Jan 13, 2025 at 11:47:52AM -0800, Nicolin Chen wrote:
The other approach would be to add a sequence number to each event and let userspace detect the non-montonicity. It would require adding a header to the native ARM evt.
Yea, I thought about that. The tricky thing is that the header will be a core-level header pairing with a driver-level vEVENTQ type and can never change its length, though we can define a 64-bit flag that can reserve the other 63 bits for future use?
The header format could be revised by changing the driver specific format tag.
Yea, we need a header format (or "header type") when the vEVENTQ is allocated.
You'd want to push a special event when the first overflow happens and probably also report a counter so userspace can know how many events got lost.
How about this:
enum iommufd_veventq_header_type { IOMMU_VEVENTQ_HEADER_TYPE_V1, };
enum iommu_hwpt_pgfault_flags { IOMMU_VEVENT_HEADER_FLAGS_OVERFLOW = (1 << 0), };
struct iommufd_vevent_header_v1 { __u64 flags; __u32 num_events; __u32 num_overflows; // valid if flag_overflow is set };
This seems most robust and simplest to implement..
I think I'd implement it by having a static overflow list entry so no memory allocation is needed and just keep moving that entry to the back of the list every time an event is lost. This way it will cover lost events due to memory outages too
Could double-adding the same static node to the list happen and corrupt the list?
I think the vevent_header doesn't need to be exactly match with the driver event. So, maybe a vEVENTQ object could hold a header structure during iommu_veventq_alloc?
struct iommufd_veventq { ... atomic_t num_events; atomic_t num_overflows; DECLARE_BITMAP(errors, 32); struct iommufd_vevent_header_v1 *header; };
The header is a bit redundant to its upper three members though.
For old formats like the fault queue you could return EOVERFLOW whenever the sequence number becomes discontiguous or it sees the overflow event..
So, IOMMU_OBJ_FAULT is safe to return EOVERFLOW via read(), as you mentioned that it is self-limited right?
Thanks Nicolin