On Thu, 24 Sep 2026 16:21:43 +0100 Matt Evans matt@ozlabs.org wrote:
Dear Reviewers,
Along the way several related issues came up that warrant more eyes, and I'd be grateful for your input:
If VFIO fd is opened O_RDONLY, it currently can't be mmap()ed (because PROT_WRITE is rejected in do_mmap(), and PROT_READ alone drops the VM_SHARED so VFIO's mmap rejects it). BUT it seems we can export a DMABUF from it, and then pass the resulting fd around for P2P writes.
I don't know if this is intentional/relied on/a known limitation, or a bug?
Seems like a bug. In practice it's probably not very meaningful, the user can still potentially change the device power state and trigger a reset, but being able to source a writable dmabuf to a region on the device fd that isn't itself writable seems semantically wrong.
a) We could reject export w/ -EPERM unless the device fd's f_mode has O_RDWR, to reflect the RW abilities of P2P
This seems sufficient...
If we agree it's a bug, I want to do this fix (a), as we can now export a DMABUF RW from an O_RDONLY device fd and then succeed to mmap() the DMABUF with RW. (That said, even with an O_RDONLY device fd, the device state can still be changed/reset. But it feels cleaner to prevent export for a O_RDONLY device fd, and match the device fd mmap() behaviour.)
In future, we could consider finer-grained RD/WR if there's a future goal to tie DMABUF permissions to, say, iommufd IOMMU_READ/IOMMU_WRITE permissions:
b) Instead of just failing if !O_RDWR, we could limit the get_dma_buf.open_flags to the VFIO device fd's f_mode, such as:
VFIO device fd perms: Export flags: Result: O_RDWR O_RDWR, O_RDONLY OK O_RDONLY O_RDONLY OK O_RDONLY O_RDWR -EPERM O_WRONLY * -EPERM * O_WRONLY -EPERM (Skipping WRONLY because a PROT_WRITE-only mmap() won't work, though it probably should be included for P2P.)
Certainly more complete, but I'm not sure there's a use case here that really warrants the effort.
Another angle to the dmabuf permissions is the region permissions themselves. We don't currently hit this since we're only exporting PCI BARs, but for instance Manish wants to protect the HDM decoder range in the vfio-cxl series[1]. Again, there's probably a simple solution to simply consult the excluded ranges list, added in that series, and reject dmabuf exports overlapping it. I don't expect any action item for this series though.
[1]https://lore.kernel.org/all/20260916183540.3813685-1-mhonap@nvidia.com/
The mmap fault handler takes a bunch of locks non-interruptibly, and potentially depends on a lot of DMABUF-related activities completing. I'd had a go at converting them to interruptible/killable forms, but that revealed there seems to be a wider issue if move/revoke doesn't complete in a timely fashion (due to buggy importers). Where I got to was that just updating the fault handler won't fix the user experience of an unkillable task, and move()/revocation will need thought too. I don't intend to fix this here but wanted to start discussion so we can address it in a follow up. There's now a dependency between mmap_lock in the fault handler and the DMABUF resv (which might take a while to resolve), though revocation will be rare in practice.
vfio_basic_config_write() has an error path if vfio_default_config_write() fails that releases memory_lock but doesn't un-revoke BARs in the case of PCI_COMMAND.MSE being cleared. When can the write fail, in practice, perhaps surprise removal?
The effect on this series would be: a write of MSE=0 revokes BARs, vconfig[PCI_COMMAND]'s MSE becomes 0, but if the physical write fails then the physical MSE remains 1 and BAR VMAs stay revoked.
This seemed a mess; fixing isn't as simple as un-revoking on the error path since vfio_default_config_write() has already trampled vconfig so that'd need unwinding. It felt like a catastrophic scenario where BARs staying revoked isn't a bad outcome, but want to hear your experience of the likelihood of this issue.
Our vfio-pci story around surprise removal and DPC is pretty weak, I'm hoping that some of our parallel error handling work will shut down the device when this occurs. For now, leaving the dmabuf revoked seems like a reasonable thing to do. In practice, I don't really see this coming up other than in testing surprise removal of NVMe drives.
- The exchange of a VFIO fd mmap()'s vma->vm_file with an implicitly created DMABUF's file has implications on LSM. For example, an mmap will be checked against the policy for a VFIO fd, but a subsequent mprotect() relates to the policy of the DMABUF file (which is anon/unique to the mapping). This is pretty confusing.
Maybe I'm not seeing the issue, but this seems like correct behavior to me. Policy decisions are made at creation time and later changing protections on one object doesn't affect the other. Thanks,
Alex