Linaro-mm-sig

linaro-mm-sig@lists.linaro.org

61 participants
6177 discussions

Re: [PATCH v8 06/11] dma-buf: provide phys_vec to scatter-gather mapping routine

by Leon Romanovsky

On Wed, Nov 19, 2025 at 05:54:55AM +0000, Tian, Kevin wrote: > > From: Leon Romanovsky <leon(a)kernel.org> > > Sent: Tuesday, November 11, 2025 5:58 PM > > + > > + if (dma->state && dma_use_iova(dma->state)) { > > + WARN_ON_ONCE(mapped_len != size); > > then "goto err_unmap_dma". It never should happen, there is no need to provide error unwind to something that you won't get. > > Reviewed-by: Kevin Tian <kevin.tian(a)intel.com> Thanks

4 months

Re: [PATCH v8 06/11] dma-buf: provide phys_vec to scatter-gather mapping routine

by Leon Romanovsky

On Tue, Nov 18, 2025 at 04:06:11PM -0800, Nicolin Chen wrote: > On Tue, Nov 11, 2025 at 11:57:48AM +0200, Leon Romanovsky wrote: > > From: Leon Romanovsky <leonro(a)nvidia.com> > > > > Add dma_buf_map() and dma_buf_unmap() helpers to convert an array of > > MMIO physical address ranges into scatter-gather tables with proper > > DMA mapping. > > > > These common functions are a starting point and support any PCI > > drivers creating mappings from their BAR's MMIO addresses. VFIO is one > > case, as shortly will be RDMA. We can review existing DRM drivers to > > refactor them separately. We hope this will evolve into routines to > > help common DRM that include mixed CPU and MMIO mappings. > > > > Compared to the dma_map_resource() abuse this implementation handles > > the complicated PCI P2P scenarios properly, especially when an IOMMU > > is enabled: > > > > - Direct bus address mapping without IOVA allocation for > > PCI_P2PDMA_MAP_BUS_ADDR, using pci_p2pdma_bus_addr_map(). This > > happens if the IOMMU is enabled but the PCIe switch ACS flags allow > > transactions to avoid the host bridge. > > > > Further, this handles the slightly obscure, case of MMIO with a > > phys_addr_t that is different from the physical BAR programming > > (bus offset). The phys_addr_t is converted to a dma_addr_t and > > accommodates this effect. This enables certain real systems to > > work, especially on ARM platforms. > > > > - Mapping through host bridge with IOVA allocation and DMA_ATTR_MMIO > > attribute for MMIO memory regions (PCI_P2PDMA_MAP_THRU_HOST_BRIDGE). > > This happens when the IOMMU is enabled and the ACS flags are forcing > > all traffic to the IOMMU - ie for virtualization systems. > > > > - Cases where P2P is not supported through the host bridge/CPU. The > > P2P subsystem is the proper place to detect this and block it. > > > > Helper functions fill_sg_entry() and calc_sg_nents() handle the > > scatter-gather table construction, splitting large regions into > > UINT_MAX-sized chunks to fit within sg->length field limits. > > > > Since the physical address based DMA API forbids use of the CPU list > > of the scatterlist this will produce a mangled scatterlist that has > > a fully zero-length and NULL'd CPU list. The list is 0 length, > > all the struct page pointers are NULL and zero sized. This is stronger > > and more robust than the existing mangle_sg_table() technique. It is > > a future project to migrate DMABUF as a subsystem away from using > > scatterlist for this data structure. > > > > Tested-by: Alex Mastro <amastro(a)fb.com> > > Tested-by: Nicolin Chen <nicolinc(a)nvidia.com> > > Signed-off-by: Leon Romanovsky <leonro(a)nvidia.com> > > Reviewed-by: Nicolin Chen <nicolinc(a)nvidia.com> > > With a nit: > > > +err_unmap_dma: > > + if (!i || !dma->state) { > > + ; /* Do nothing */ > > + } else if (dma_use_iova(dma->state)) { > > + dma_iova_destroy(attach->dev, dma->state, mapped_len, dir, > > + DMA_ATTR_MMIO); > > + } else { > > + for_each_sgtable_dma_sg(&dma->sgt, sgl, i) > > + dma_unmap_phys(attach->dev, sg_dma_address(sgl), > > + sg_dma_len(sgl), dir, DMA_ATTR_MMIO); > > Would it be safer to skip dma_unmap_phys() the range [i, nents)? [i, nents) is not supposed to be in SG list which we are iterating. Thanks

4 months

Re: [PATCH 8/9] iommufd: Accept a DMABUF through IOMMU_IOAS_MAP_FILE

by Jason Gunthorpe

On Thu, Nov 13, 2025 at 04:05:09PM -0800, Nicolin Chen wrote: > > -struct iopt_pages *iopt_alloc_file_pages(struct file *file, unsigned long start, > > +struct iopt_pages *iopt_alloc_file_pages(struct file *file, > > + unsigned long start_byte, > > + unsigned long start, > > unsigned long length, bool writable); > > Passing in start_byte looks like a cleanup to me, aligning with > what iopt_map_common() has. > Since we are doing this cleanup, maybe we could follow the same > sequence: xxx, start, length, start_byte, writable? ?? static int iopt_map_common(struct iommufd_ctx *ictx, struct io_pagetable *iopt, struct iopt_pages *pages, unsigned long *iova, unsigned long length, unsigned long start_byte, int iommu_prot, unsigned int flags) Not the same arguments, we don't pass start and start_byte there? Jason

4 months

Re: [PATCH 6/9] iommufd: Have pfn_reader process DMABUF iopt_pages

by Jason Gunthorpe

On Thu, Nov 13, 2025 at 03:39:46PM -0800, Nicolin Chen wrote: > > @@ -1687,6 +1737,12 @@ static void __iopt_area_unfill_domain(struct iopt_area *area, > > > > lockdep_assert_held(&pages->mutex); > > > > + if (iopt_is_dmabuf(pages)) { > > + iopt_area_unmap_domain_range(area, domain, start_index, > > + last_index); > > + return; > > + } > > Should it be: > if (iopt_is_dmabuf(pages) && !iopt_dmabuf_revoked(pages)) { > ? All callers have already done it, let's add an assertion though.. @@ -1873,6 +1873,8 @@ static void __iopt_area_unfill_domain(struct iopt_area *area, lockdep_assert_held(&pages->mutex); if (iopt_is_dmabuf(pages)) { + if (WARN_ON(iopt_dmabuf_revoked(pages))) + return; iopt_area_unmap_domain_range(area, domain, start_index, last_index); return; Jason

4 months

Re: [PATCH 9/9] iommufd/selftest: Add some tests for the dmabuf flow

by Jason Gunthorpe

On Fri, Nov 07, 2025 at 11:43:56AM -0800, Nicolin Chen wrote: > > +static void iommufd_test_dma_buf_release(struct dma_buf *dmabuf) > > +{ > > + struct iommufd_test_dma_buf *priv = dmabuf->priv; > > + > > + kfree(priv); > > +} > > Missing > kfree(priv->memory); > ? Yes, thanks Jason

4 months

Re: [PATCH 0/9] Initial DMABUF support for iommufd

by Jason Gunthorpe

On Tue, Nov 18, 2025 at 05:37:59AM +0000, Kasireddy, Vivek wrote: > Hi Jason, > > > Subject: Re: [PATCH 0/9] Initial DMABUF support for iommufd > > > > On Thu, Nov 13, 2025 at 11:37:12AM -0700, Alex Williamson wrote: > > > > The latest series for interconnect negotation to exchange a phys_addr is: > > > > https://lore.kernel.org/r/20251027044712.1676175-1- > > vivek.kasireddy(a)intel.com > > > > > > If this is in development, why are we pursuing a vfio specific > > > temporary "private interconnect" here rather than building on that > > > work? What are the gaps/barriers/timeline? > > > > I broadly don't expect to see an agreement on the above for probably > Are you planning to post your SGT mapping type patches soon, so that we > can start discussion on the design? It is on my list, but probably not soon enough :\ I wanted to address the remarks given and I still have to conclude some urgent things for this merge window. > I went ahead and tested your patches and did not notice any regressions > with my test-cases (after adding some minor fixups). I have also added/tested > support for IOV mapping type based on your design: > https://gitlab.freedesktop.org/Vivek/drm-tip/-/commits/dmabuf_iov_v1 Wow, that's great! Jason

4 months

Re: [PATCH v7 00/11] vfio/pci: Allow MMIO regions to be exported through dma-buf

by Jason Gunthorpe

On Mon, Nov 17, 2025 at 08:36:20AM -0700, Alex Williamson wrote: > On Tue, 11 Nov 2025 09:54:22 +0100 > Christian König <christian.koenig(a)amd.com> wrote: > > > On 11/10/25 21:42, Alex Williamson wrote: > > > On Thu, 6 Nov 2025 16:16:45 +0200 > > > Leon Romanovsky <leon(a)kernel.org> wrote: > > > > > >> Changelog: > > >> v7: > > >> * Dropped restore_revoke flag and added vfio_pci_dma_buf_move > > >> to reverse loop. > > >> * Fixed spelling errors in documentation patch. > > >> * Rebased on top of v6.18-rc3. > > >> * Added include to stddef.h to vfio.h, to keep uapi header file independent. > > > > > > I think we're winding down on review comments. It'd be great to get > > > p2pdma and dma-buf acks on this series. Otherwise it's been posted > > > enough that we'll assume no objections. Thanks, > > > > Already have it on my TODO list to take a closer look, but no idea when that will be. > > > > This patch set is on place 4 or 5 on a rather long list of stuff to review/finish. > > Hi Christian, > > Gentle nudge. Leon posted v8[1] last week, which is not drawing any > new comments. Do you foresee having time for review that I should > still hold off merging for v6.19 a bit longer? Thanks, I really want this merged this cycle, along with the iommufd part, which means it needs to go into your tree by very early next week on a shared branch so I can do the iommufd part on top. It is the last blocking kernel piece to conclude the viommu support roll out into qemu for iommufd which quite a lot of people have been working on for years now. IMHO there is nothing profound in the dmabuf patch, it was written by the expert in the new DMA API operation, and doesn't form any troublesome API contracts. It is also the same basic code as from the v1 in July just moved into dmabuf .c files instead of vfio .c files at Christoph's request. My hope is DRM folks will pick up the baton and continue to improve this to move other drivers away from dma_map_resource(). Simona told me people have wanted DMA API improvements for ages, now we have them, now is the time! Any remarks after the fact can be addressed incrementally. If there are no concrete technical remarks please take it. 6 months is long enough to wait for feedback. Thanks, Jason

4 months

Re: [PATCH v8 11/11] vfio/nvgrace: Support get_dmabuf_phys

by Jason Gunthorpe

On Tue, Nov 18, 2025 at 07:59:20AM +0000, Ankit Agrawal wrote: > + if (nvdev->resmem.memlength && region_index == RESMEM_REGION_INDEX) { > + /* > + * The P2P properties of the non-BAR memory is the same as the > + * BAR memory, so just use the provider for index 0. Someday > + * when CXL gets P2P support we could create CXLish providers > + * for the non-BAR memory. > + */ > + mem_region = &nvdev->resmem; > + } else if (region_index == USEMEM_REGION_INDEX) { > + /* > + * This is actually cachable memory and isn't treated as P2P in > + * the chip. For now we have no way to push cachable memory > + * through everything and the Grace HW doesn't care what caching > + * attribute is programmed into the SMMU. So use BAR 0. > + */ > + mem_region = &nvdev->usemem; > + } > + > > Can we replace this with nvgrace_gpu_memregion()? Yes, looks like But we need to preserve the comments above as well somehow. Jason

4 months

Re: [PATCH v8 10/11] vfio/pci: Add dma-buf export support for MMIO regions

by Jason Gunthorpe

On Tue, Nov 18, 2025 at 07:33:23AM +0000, Tian, Kevin wrote: > > From: Leon Romanovsky <leon(a)kernel.org> > > Sent: Tuesday, November 11, 2025 5:58 PM > > > > - if (!new_mem) > > + if (!new_mem) { > > vfio_pci_zap_and_down_write_memory_lock(vdev); > > - else > > + vfio_pci_dma_buf_move(vdev, true); > > + } else { > > down_write(&vdev->memory_lock); > > + } > > shouldn't we notify move before zapping the bars? otherwise there is > still a small window in between where the exporter already has the > mapping cleared while the importer still keeps it... zapping the VMA and moving/revoking the DMABUF are independent operations that can happen in any order. They effect different kinds of users. The VMA zap prevents CPU access from userspace, the DMABUF move prevents DMA access from devices. The order has to be like the above because vfio_pci_dma_buf_move() must be called under the memory lock and vfio_pci_zap_and_down_write_memory_lock() gets the memory lock.. > > +static void vfio_pci_dma_buf_release(struct dma_buf *dmabuf) > > +{ > > + struct vfio_pci_dma_buf *priv = dmabuf->priv; > > + > > + /* > > + * Either this or vfio_pci_dma_buf_cleanup() will remove from the list. > > + * The refcount prevents both. > > which refcount? I thought it's vdev->memory_lock preventing the race... Refcount on the dmabuf > > +int vfio_pci_core_fill_phys_vec(struct dma_buf_phys_vec *phys_vec, > > + struct vfio_region_dma_range *dma_ranges, > > + size_t nr_ranges, phys_addr_t start, > > + phys_addr_t len) > > +{ > > + phys_addr_t max_addr; > > + unsigned int i; > > + > > + max_addr = start + len; > > + for (i = 0; i < nr_ranges; i++) { > > + phys_addr_t end; > > + > > + if (!dma_ranges[i].length) > > + return -EINVAL; > > Looks redundant as there is already a check in validate_dmabuf_input(). Agree > > +int vfio_pci_core_feature_dma_buf(struct vfio_pci_core_device *vdev, u32 > > flags, > > + struct vfio_device_feature_dma_buf __user > > *arg, > > + size_t argsz) > > +{ > > + struct vfio_device_feature_dma_buf get_dma_buf = {}; > > + struct vfio_region_dma_range *dma_ranges; > > + DEFINE_DMA_BUF_EXPORT_INFO(exp_info); > > + struct vfio_pci_dma_buf *priv; > > + size_t length; > > + int ret; > > + > > + if (!vdev->pci_ops || !vdev->pci_ops->get_dmabuf_phys) > > + return -EOPNOTSUPP; > > + > > + ret = vfio_check_feature(flags, argsz, VFIO_DEVICE_FEATURE_GET, > > + sizeof(get_dma_buf)); > > + if (ret != 1) > > + return ret; > > + > > + if (copy_from_user(&get_dma_buf, arg, sizeof(get_dma_buf))) > > + return -EFAULT; > > + > > + if (!get_dma_buf.nr_ranges || get_dma_buf.flags) > > + return -EINVAL; > > unknown flag bits get -EOPNOTSUPP. Agree > > + > > +void vfio_pci_dma_buf_cleanup(struct vfio_pci_core_device *vdev) > > +{ > > + struct vfio_pci_dma_buf *priv; > > + struct vfio_pci_dma_buf *tmp; > > + > > + down_write(&vdev->memory_lock); > > + list_for_each_entry_safe(priv, tmp, &vdev->dmabufs, dmabufs_elm) > > { > > + if (!get_file_active(&priv->dmabuf->file)) > > + continue; > > + > > + dma_resv_lock(priv->dmabuf->resv, NULL); > > + list_del_init(&priv->dmabufs_elm); > > + priv->vdev = NULL; > > + priv->revoked = true; > > + dma_buf_move_notify(priv->dmabuf); > > + dma_resv_unlock(priv->dmabuf->resv); > > + vfio_device_put_registration(&vdev->vdev); > > + fput(priv->dmabuf->file); > > dma_buf_put(priv->dmabuf), consistent with other places. Someone else said this, I don't agree, the above got the get via get_file_active() instead of a dma_buf version.. So we should pair with get_file_active() vs fput(). Christian rejected the idea of adding a dmabuf wrapper for get_file_active(), oh well. > > +struct vfio_device_feature_dma_buf { > > + __u32 region_index; > > + __u32 open_flags; > > + __u32 flags; > > Usually the 'flags' field is put in the start (following argsz if existing). Yeah, but doesn't really matter. Thanks, Jason

4 months

Re: [PATCH 02/18] dma-buf: protected fence ops by RCU v3

by Christian König

On 11/14/25 11:50, Tvrtko Ursulin wrote: >> @@ -569,12 +577,12 @@ void dma_fence_release(struct kref *kref) >> spin_unlock_irqrestore(fence->lock, flags); >> } >> - rcu_read_unlock(); >> - >> - if (fence->ops->release) >> - fence->ops->release(fence); >> + ops = rcu_dereference(fence->ops); >> + if (ops->release) >> + ops->release(fence); >> else >> dma_fence_free(fence); >> + rcu_read_unlock(); > > Risk being a spin lock in the release callback will trigger a warning on PREEMPT_RT. But at least the current code base does not have anything like that AFAICS so I guess it is okay. I don't think that this is a problem. When PREEMPT_RT is enabled both RCU and spinlocks become preemptible. So as far as I know it is perfectly valid to grab a spinlock under an rcu read side critical section. >> diff --git a/include/linux/dma-fence.h b/include/linux/dma-fence.h >> index 64639e104110..77f07735f556 100644 >> --- a/include/linux/dma-fence.h >> +++ b/include/linux/dma-fence.h >> @@ -66,7 +66,7 @@ struct seq_file; >> */ >> struct dma_fence { >> spinlock_t *lock; >> - const struct dma_fence_ops *ops; >> + const struct dma_fence_ops __rcu *ops; >> /* >> * We clear the callback list on kref_put so that by the time we >> * release the fence it is unused. No one should be adding to the >> @@ -218,6 +218,10 @@ struct dma_fence_ops { >> * timed out. Can also return other error values on custom implementations, >> * which should be treated as if the fence is signaled. For example a hardware >> * lockup could be reported like that. >> + * >> + * Implementing this callback prevents the BO from detaching after > > s/BO/fence/ > >> + * signaling and so it is mandatory for the module providing the >> + * dma_fence_ops to stay loaded as long as the dma_fence exists. >> */ >> signed long (*wait)(struct dma_fence *fence, >> bool intr, signed long timeout); >> @@ -229,6 +233,13 @@ struct dma_fence_ops { >> * Can be called from irq context. This callback is optional. If it is >> * NULL, then dma_fence_free() is instead called as the default >> * implementation. >> + * >> + * Implementing this callback prevents the BO from detaching after > > Ditto. Both fixed, thanks. > >> + * signaling and so it is mandatory for the module providing the >> + * dma_fence_ops to stay loaded as long as the dma_fence exists. >> + * >> + * If the callback is implemented the memory backing the dma_fence >> + * object must be freed RCU safe. >> */ >> void (*release)(struct dma_fence *fence); >> @@ -418,13 +429,19 @@ const char __rcu *dma_fence_timeline_name(struct dma_fence *fence); >> static inline bool >> dma_fence_is_signaled_locked(struct dma_fence *fence) >> { >> + const struct dma_fence_ops *ops; >> + >> if (test_bit(DMA_FENCE_FLAG_SIGNALED_BIT, &fence->flags)) >> return true; >> - if (fence->ops->signaled && fence->ops->signaled(fence)) { >> + rcu_read_lock(); >> + ops = rcu_dereference(fence->ops); >> + if (ops->signaled && ops->signaled(fence)) { >> + rcu_read_unlock(); >> dma_fence_signal_locked(fence); >> return true; >> } >> + rcu_read_unlock(); >> return false; >> } >> @@ -448,13 +465,19 @@ dma_fence_is_signaled_locked(struct dma_fence *fence) >> static inline bool >> dma_fence_is_signaled(struct dma_fence *fence) >> { >> + const struct dma_fence_ops *ops; >> + >> if (test_bit(DMA_FENCE_FLAG_SIGNALED_BIT, &fence->flags)) >> return true; >> - if (fence->ops->signaled && fence->ops->signaled(fence)) { >> + rcu_read_lock(); >> + ops = rcu_dereference(fence->ops); >> + if (ops->signaled && ops->signaled(fence)) { >> + rcu_read_unlock(); > > With the unlocked version two threads could race and one could make the fence->lock go away just around here, before the dma_fence_signal below will take it. It seems it is only safe to rcu_read_unlock before signaling if using the embedded fence (later in the series). Can you think of a downside to holding the rcu read lock to after signaling? that would make it safe I think. Well it's good to talk about it but I think that it is not necessary to protect the lock in this particular case. See the RCU protection is only for the fence->ops pointer, but the lock can be taken way after the fence is already signaled. That's why I came up with the patch to move the lock into the fence in the first place. Regards, Christian. > > Regards, > > Tvrtko > >> dma_fence_signal(fence); >> return true; >> } >> + rcu_read_unlock(); >> return false; >> } >

4 months

Jump to page:

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

Linaro-mm-sig