On Tue, Jul 29, 2025 at 08:44:21PM +0100, Robin Murphy wrote:
In this case with just one single contiguous mapping, it is clearly objectively worse to have to bounce in and out of the IOMMU layer 3 separate times and store a dma_map_state,
The non-contiguous mappings are comming back, it was in earlier drafts of this. Regardless, the point is to show how to use the general API that we would want to bring into the DRM drivers that don't have contiguity even though VFIO is a bit special.
Oh yeah, and mapping MMIO with regular memory attributes (IOMMU_CACHE) rather than appropriate ones (IOMMU_MMIO), as this will end up doing, isn't guaranteed not to end badly either (e.g. if the system interconnect ends up merging consecutive write bursts and exceeding the target root port's MPS.)
Yes, I recently noticed this too, it should be fixed..
But so we are all on the same page, alot of the PCI P2P systems are setup so P2P does not transit through the iommu. It either takes the ACS path through a switch or it uses ATS and takes a different ACS path through a switch. It only transits through the iommu in misconfigured systems or in the rarer case of P2P between root ports.
And again, if the IOMMU is in bypass (the idea of P2P with vfio-noiommu simply isn't worth entertaining)
Not quite. DMABUF is sort of upside down.
For example if we are exporting a DMABUF from VFIO and importing it to RDMA then RDMA will call VFIO to make an attachment and the above VFIO code will perform the DMA map to the RDMA struct device. DMABUF returns a dma mapped scatterlist back to the RDMA driver.
The above dma_map_phys(rdma_dev,...) can be in bypass because the rdma device can legitimately be in bypass, or not have a iommu, or whatever.
AFAICS you're *depending* on this call being an effective no-op, and thus only demonstrating that the dma_map_phys() idea is still entirely unnecessary.
It should not be a full no-op, and it should be closer to dma map resource to avoid the mmio issues.
It should be failing for cases where it is not supported (ie swiotlb=force), it should still be calling the legacy dma_ops, and it should be undoing any CC mangling with the address. (also the pci_p2pdma_bus_addr_map() needs to deal with any CC issues too)
Jason