On Thu, Jan 16, 2025 at 06:33:48AM +0100, Christoph Hellwig wrote:
On Wed, Jan 15, 2025 at 09:34:19AM -0400, Jason Gunthorpe wrote:
Or do you mean some that don't have pages associated with them, and thus have pfn_valid fail on them? They still have a PFN, just not one that is valid to use in most of the Linux MM.
He is talking about private interconnect hidden inside clusters of devices.
Ie the system may have many GPUs and those GPUs have their own private interconnect between them. It is not PCI, and packets don't transit through the CPU SOC at all, so the IOMMU is not involved.
DMA can happen on that private interconnect, but from a Linux perspective it is not DMA API DMA, and the addresses used to describe it are not part of the CPU address space. The initiating device will have a way to choose which path the DMA goes through when setting up the DMA.
So how is this in any way relevant to dma_buf which operates on a dma_addr_t right now and thus by definition can't be used for these?
Oh, well since this private stuff exists the DRM folks implemented it and used dmabuf to hook it together tough the uAPI. To make it work it abuses scatterlist and dma_addr_t to carry this other information.
Thus the pushback in this thread we can't naively fixup dmabuf because this non-dma_addr_t abuse exists and is uAPI. So it also needs some improved architecture to move forward :\
Basically, the scatterlist in dmabuf API does not follow any of the normal rules scatterlist should follow. It is not using the semantics of dma_addr_t even though that is the type. It is really just an array of addr/len pairs - we can't reason about it in the normal way :(
Jason