This patch series introduces dma-buf export support for RDMA/InfiniBand
devices, enabling userspace applications to export RDMA PCI-backed
memory regions (such as device memory or mlx5 UAR pages) as dma-buf file
descriptors.
This allows PCI device memory to be shared with other kernel subsystems
(e.g., graphics or media) or between userspace processes, via the
standard dma-buf interface, avoiding unnecessary copies and enabling
efficient peer-to-peer (P2P) DMA transfers. See [1] for background on
dma-buf.
As part of this series, we introduce a new uverbs object of type FD for
dma-buf export, along with the corresponding APIs for allocation and
teardown. This object encapsulates all attributes required to export a
dma-buf.
The implementation enforces P2P-only mappings and properly manages
resource lifecycle, including:
- Cleanup during driver removal or RDMA context destruction.
- Revocation via dma_buf_move_notify() when the underlying mmap entries
are removed.
- Refactors common cleanup logic for reuse across FD uobject types.
The infrastructure is generic within uverbs, allowing individual drivers
to easily integrate and supply their vendor-specific implementation.
The mlx5 driver is the first consumer of this new API, providing:
- Initialization of PCI peer-to-peer DMA support.
- mlx5-specific implementations of the mmap_get_pfns and
pgoff_to_mmap_entry device operations required for dma-buf export.
[1] https://docs.kernel.org/driver-api/dma-buf.html
Signed-off-by: Yishai Hadas <yishaih(a)nvidia.com>
Signed-off-by: Edward Srouji <edwards(a)nvidia.com>
---
Yishai Hadas (2):
RDMA/uverbs: Add DMABUF object type and operations
RDMA/mlx5: Implement DMABUF export ops
drivers/infiniband/core/Makefile | 1 +
drivers/infiniband/core/device.c | 2 +
drivers/infiniband/core/ib_core_uverbs.c | 19 +++
drivers/infiniband/core/rdma_core.c | 63 ++++----
drivers/infiniband/core/rdma_core.h | 1 +
drivers/infiniband/core/uverbs.h | 10 ++
drivers/infiniband/core/uverbs_std_types_dmabuf.c | 172 ++++++++++++++++++++++
drivers/infiniband/core/uverbs_uapi.c | 1 +
drivers/infiniband/hw/mlx5/main.c | 72 +++++++++
include/rdma/ib_verbs.h | 9 ++
include/rdma/uverbs_types.h | 1 +
include/uapi/rdma/ib_user_ioctl_cmds.h | 10 ++
12 files changed, 335 insertions(+), 26 deletions(-)
---
base-commit: 325e3b5431ddd27c5f93156b36838a351e3b2f72
change-id: 20260108-dmabuf-export-0d598058dd1e
Best regards,
--
Edward Srouji <edwards(a)nvidia.com>
Changelog:
v4:
* Changed DMA_RESV_USAGE_KERNEL to DMA_RESV_USAGE_BOOKKEEP.
* Made .invalidate_mapping() truly optional.
* Added patch which renames dma_buf_move_notify() to be
dma_buf_invalidate_mappings().
* Restored dma_buf_attachment_is_dynamic() function.
v3: https://lore.kernel.org/all/20260120-dmabuf-revoke-v3-0-b7e0b07b8214@nvidia…
* Used Jason's wordings for commits and cover letter.
* Removed IOMMUFD patch.
* Renamed dma_buf_attachment_is_revoke() to be dma_buf_attach_revocable().
* Added patch to remove CONFIG_DMABUF_MOVE_NOTIFY.
* Added Reviewed-by tags.
* Called to dma_resv_wait_timeout() after dma_buf_move_notify() in VFIO.
* Added dma_buf_attach_revocable() check to VFIO DMABUF attach function.
* Slightly changed commit messages.
v2: https://patch.msgid.link/20260118-dmabuf-revoke-v2-0-a03bb27c0875@nvidia.com
* Changed series to document the revoke semantics instead of
implementing it.
v1: https://patch.msgid.link/20260111-dmabuf-revoke-v1-0-fb4bcc8c259b@nvidia.com
-------------------------------------------------------------------------
This series documents a dma-buf “revoke” mechanism: to allow a dma-buf
exporter to explicitly invalidate (“kill”) a shared buffer after it has
been distributed to importers, so that further CPU and device access is
prevented and importers reliably observe failure.
The change in this series is to properly document and use existing core
“revoked” state on the dma-buf object and a corresponding exporter-triggered
revoke operation.
dma-buf has quietly allowed calling move_notify on pinned dma-bufs, even
though legacy importers using dma_buf_attach() would simply ignore
these calls.
RDMA saw this and needed to use allow_peer2peer=true, so implemented a
new-style pinned importer with an explicitly non-working move_notify()
callback.
This has been tolerable because the existing exporters are thought to
only call move_notify() on a pinned DMABUF under RAS events and we
have been willing to tolerate the UAF that results by allowing the
importer to continue to use the mapping in this rare case.
VFIO wants to implement a pin supporting exporter that will issue a
revoking move_notify() around FLRs and a few other user triggerable
operations. Since this is much more common we are not willing to
tolerate the security UAF caused by interworking with non-move_notify()
supporting drivers. Thus till now VFIO has required dynamic importers,
even though it never actually moves the buffer location.
To allow VFIO to work with pinned importers, according to how dma-buf
was intended, we need to allow VFIO to detect if an importer is legacy
or RDMA and does not actually implement move_notify().
In theory all exporters that call move_notify() on pinned dma-buf's
should call this function, however that would break a number of widely
used NIC/GPU flows. Thus for now do not spread this further than VFIO
until we can understand how much of RDMA can implement the full
semantic.
In the process clarify how move_notify is intended to be used with
pinned dma-bufs.
Thanks
Signed-off-by: Leon Romanovsky <leonro(a)nvidia.com>
---
Leon Romanovsky (8):
dma-buf: Rename .move_notify() callback to a clearer identifier
dma-buf: Rename dma_buf_move_notify() to dma_buf_invalidate_mappings()
dma-buf: Always build with DMABUF_MOVE_NOTIFY
dma-buf: Make .invalidate_mapping() truly optional
dma-buf: Add check function for revoke semantics
iommufd: Pin dma-buf importer for revoke semantics
vfio: Wait for dma-buf invalidation to complete
vfio: Validate dma-buf revocation semantics
drivers/dma-buf/Kconfig | 12 -------
drivers/dma-buf/dma-buf.c | 53 ++++++++++++++++++++++-------
drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c | 14 +++-----
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 2 +-
drivers/gpu/drm/amd/amdkfd/Kconfig | 2 +-
drivers/gpu/drm/virtio/virtgpu_prime.c | 2 +-
drivers/gpu/drm/xe/tests/xe_dma_buf.c | 7 ++--
drivers/gpu/drm/xe/xe_bo.c | 2 +-
drivers/gpu/drm/xe/xe_dma_buf.c | 14 +++-----
drivers/infiniband/core/umem_dmabuf.c | 13 -------
drivers/infiniband/hw/mlx5/mr.c | 2 +-
drivers/iommu/iommufd/pages.c | 11 ++++--
drivers/iommu/iommufd/selftest.c | 2 +-
drivers/vfio/pci/vfio_pci_dmabuf.c | 13 +++++--
include/linux/dma-buf.h | 9 ++---
15 files changed, 84 insertions(+), 74 deletions(-)
---
base-commit: 9ace4753a5202b02191d54e9fdf7f9e3d02b85eb
change-id: 20251221-dmabuf-revoke-b90ef16e4236
Best regards,
--
Leon Romanovsky <leonro(a)nvidia.com>
This series adds a new DRM/Accel driver that supports the C7x DSPs
inside some Texas Instruments SoCs such as the J722S. These can be used
as accelerators for various workloads, including machine learning
inference.
This driver controls the power state of the hardware via remoteproc and
communicates with the firmware running on the DSP via rpmsg_virtio. The
kernel driver itself allocates buffers, manages contexts, and submits
jobs to the DSP firmware. Buffers are mapped by the DSP itself using its
MMU, providing memory isolation among different clients.
The source code for the firmware running on the DSP is available at:
https://gitlab.freedesktop.org/tomeu/thames_firmware/.
Everything else is done in userspace, as a Gallium driver (also called
thames) that is part of the Mesa3D project: https://docs.mesa3d.org/teflon.html
If there is more than one core that advertises the same rpmsg_virtio
service name, the driver will load balance jobs between them with
drm-gpu-scheduler.
Userspace portion of the driver: https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39298
Signed-off-by: Tomeu Vizoso <tomeu(a)tomeuvizoso.net>
---
Changes in v2:
- Add thames_accel.h UAPI header (Robert Nelson).
- Link to v1: https://lore.kernel.org/r/20260113-thames-v1-0-99390026937c@tomeuvizoso.net
---
Tomeu Vizoso (5):
arm64: dts: ti: k3-j722s-ti-ipc-firmware: Add memory pool for DSP i/o buffers
accel/thames: Add driver for the C7x DSPs in TI SoCs
accel/thames: Add IOCTLs for BO creation and mapping
accel/thames: Add IOCTL for job submission
accel/thames: Add IOCTL for memory synchronization
Documentation/accel/thames/index.rst | 28 ++
MAINTAINERS | 9 +
.../boot/dts/ti/k3-j722s-ti-ipc-firmware.dtsi | 11 +-
drivers/accel/Kconfig | 1 +
drivers/accel/Makefile | 3 +-
drivers/accel/thames/Kconfig | 26 ++
drivers/accel/thames/Makefile | 11 +
drivers/accel/thames/thames_core.c | 161 +++++++
drivers/accel/thames/thames_core.h | 53 +++
drivers/accel/thames/thames_device.c | 93 +++++
drivers/accel/thames/thames_device.h | 46 ++
drivers/accel/thames/thames_drv.c | 180 ++++++++
drivers/accel/thames/thames_drv.h | 21 +
drivers/accel/thames/thames_gem.c | 405 ++++++++++++++++++
drivers/accel/thames/thames_gem.h | 45 ++
drivers/accel/thames/thames_ipc.h | 204 +++++++++
drivers/accel/thames/thames_job.c | 463 +++++++++++++++++++++
drivers/accel/thames/thames_job.h | 51 +++
drivers/accel/thames/thames_rpmsg.c | 276 ++++++++++++
drivers/accel/thames/thames_rpmsg.h | 27 ++
include/uapi/drm/thames_accel.h | 189 +++++++++
21 files changed, 2300 insertions(+), 3 deletions(-)
---
base-commit: 27927a79b3c6aebd18f38507a8160294243763dc
change-id: 20260113-thames-334127a2d91d
Best regards,
--
Tomeu Vizoso <tomeu(a)tomeuvizoso.net>
Capture dmabuf system heap allocations in memcg following prior
conversations[1][2]. Disable this behavior by default unless configured
by "dma_heap.mem_accounting" module parameter.
[1] https://lore.kernel.org/dri-devel/Z-5GZ3kJDbhgVBPG@phenom.ffwll.local/
[2] https://lore.kernel.org/all/CABdmKX2_UOENujpW0dXe0Z0x+4V3onfGDmHf1DMOXfDha6…
Changes in v3:
- Declare mem_accounting in include/linux/dma-heap.h, since it's
intended for heap implementations. Make it __read_mostly while at it.
- Link to v2: https://lore.kernel.org/r/20260113-dmabuf-heap-system-memcg-v2-0-e85722cc2f…
Changes in v2:
- Add a module parameter to enable dma-buf cgroup accounting, disabled
by default.
- Split system_heap logic in its own commit.
- Link to v1: https://lore.kernel.org/lkml/20251211193106.755485-2-echanude@redhat.com/
Signed-off-by: Eric Chanudet <echanude(a)redhat.com>
---
Eric Chanudet (2):
dma-buf: heaps: add parameter to account allocations using cgroup
dma-buf: system_heap: account for system heap allocation in memcg
drivers/dma-buf/dma-heap.c | 5 +++++
drivers/dma-buf/heaps/system_heap.c | 7 +++++--
include/linux/dma-heap.h | 2 ++
3 files changed, 12 insertions(+), 2 deletions(-)
---
base-commit: 983d014aafb14ee5e4915465bf8948e8f3a723b5
change-id: 20260102-dmabuf-heap-system-memcg-c86a381d663a
Best regards,
--
Eric Chanudet <echanude(a)redhat.com>
Changelog:
v2:
* Changed series to document the revoke semantics instead of
implementing it.
v1: https://patch.msgid.link/20260111-dmabuf-revoke-v1-0-fb4bcc8c259b@nvidia.com
-------------------------------------------------------------------------
This series documents a dma-buf “revoke” mechanism: to allow a dma-buf
exporter to explicitly invalidate (“kill”) a shared buffer after it has
been distributed to importers, so that further CPU and device access is
prevented and importers reliably observe failure.
The change in this series is to properly document and use existing core
“revoked” state on the dma-buf object and a corresponding exporter-triggered
revoke operation. Once a dma-buf is revoked, new access paths are blocked so
that attempts to DMA-map, vmap, or mmap the buffer fail in a consistent way.
Thanks
Cc: linux-media(a)vger.kernel.org
Cc: dri-devel(a)lists.freedesktop.org
Cc: linaro-mm-sig(a)lists.linaro.org
Cc: linux-kernel(a)vger.kernel.org
Cc: amd-gfx(a)lists.freedesktop.org
Cc: virtualization(a)lists.linux.dev
Cc: intel-xe(a)lists.freedesktop.org
Cc: linux-rdma(a)vger.kernel.org
Cc: iommu(a)lists.linux.dev
Cc: kvm(a)vger.kernel.org
To: Sumit Semwal <sumit.semwal(a)linaro.org>
To: Christian König <christian.koenig(a)amd.com>
To: Alex Deucher <alexander.deucher(a)amd.com>
To: David Airlie <airlied(a)gmail.com>
To: Simona Vetter <simona(a)ffwll.ch>
To: Gerd Hoffmann <kraxel(a)redhat.com>
To: Dmitry Osipenko <dmitry.osipenko(a)collabora.com>
To: Gurchetan Singh <gurchetansingh(a)chromium.org>
To: Chia-I Wu <olvaffe(a)gmail.com>
To: Maarten Lankhorst <maarten.lankhorst(a)linux.intel.com>
To: Maxime Ripard <mripard(a)kernel.org>
To: Thomas Zimmermann <tzimmermann(a)suse.de>
To: Lucas De Marchi <lucas.demarchi(a)intel.com>
To: Thomas Hellström <thomas.hellstrom(a)linux.intel.com>
To: Rodrigo Vivi <rodrigo.vivi(a)intel.com>
To: Jason Gunthorpe <jgg(a)ziepe.ca>
To: Leon Romanovsky <leon(a)kernel.org>
To: Kevin Tian <kevin.tian(a)intel.com>
To: Joerg Roedel <joro(a)8bytes.org>
To: Will Deacon <will(a)kernel.org>
To: Robin Murphy <robin.murphy(a)arm.com>
To: Alex Williamson <alex(a)shazbot.org>
---
Leon Romanovsky (4):
dma-buf: Rename .move_notify() callback to a clearer identifier
dma-buf: Document revoke semantics
iommufd: Require DMABUF revoke semantics
vfio: Add pinned interface to perform revoke semantics
drivers/dma-buf/dma-buf.c | 6 +++---
drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c | 4 ++--
drivers/gpu/drm/virtio/virtgpu_prime.c | 2 +-
drivers/gpu/drm/xe/tests/xe_dma_buf.c | 6 +++---
drivers/gpu/drm/xe/xe_dma_buf.c | 2 +-
drivers/infiniband/core/umem_dmabuf.c | 4 ++--
drivers/infiniband/hw/mlx5/mr.c | 2 +-
drivers/iommu/iommufd/pages.c | 11 +++++++++--
drivers/vfio/pci/vfio_pci_dmabuf.c | 16 ++++++++++++++++
include/linux/dma-buf.h | 25 ++++++++++++++++++++++---
10 files changed, 60 insertions(+), 18 deletions(-)
---
base-commit: 9ace4753a5202b02191d54e9fdf7f9e3d02b85eb
change-id: 20251221-dmabuf-revoke-b90ef16e4236
Best regards,
--
Leon Romanovsky <leonro(a)nvidia.com>
On 1/16/26 10:14 AM, Alex Williamson wrote:
> On Wed, 14 Jan 2026 14:18:19 +0200
> Leon Romanovsky <leon(a)kernel.org> wrote:
>
>> On Wed, Jan 07, 2026 at 11:14:14AM +0200, Leon Romanovsky wrote:
>>> From: Leon Romanovsky <leonro(a)nvidia.com>
>>>
>>> After commit fcf463b92a08 ("types: move phys_vec definition to common header"),
>>> we can use the shared phys_vec type instead of the DMABUF?specific
>>> dma_buf_phys_vec, which duplicated the same structure and semantics.
>>>
>>> Signed-off-by: Leon Romanovsky <leonro(a)nvidia.com>
>>> ---
>>> Alex,
>>>
>>> According to diffstat, VFIO is the subsystem with the largest set of changes,
>>> so it would be great if you could take it through your tree.
>>>
>>> The series is based on the for-7.0/blk-pvec shared branch from Jens:
>>> https://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux.git/log/?h=for-…
>>>
>>> Thanks
>>> ---
>>
>> Alex,
>>
>> Could you please move this patch forward? We have the RDMA series [1] that
>> depends on this rename, and I would like to base it on the shared branch.
>>
>> [1] https://lore.kernel.org/all/20260108-dmabuf-export-v1-0-6d47d46580d3@nvidia…
>
> I tried to ping Jens regarding why the branch with this code hasn't
> been merged into their for-next branch, maybe you have more traction.
> Thanks,
I get a lot of emails, and pings inside existing series don't always get
seen... Usually better to do a forward for a ping, then it's a lot more
likely to get seen sooner.
--
Jens Axboe
On Fri, Jan 16, 2026 at 10:14:55AM -0700, Alex Williamson wrote:
> On Wed, 14 Jan 2026 14:18:19 +0200
> Leon Romanovsky <leon(a)kernel.org> wrote:
>
> > On Wed, Jan 07, 2026 at 11:14:14AM +0200, Leon Romanovsky wrote:
> > > From: Leon Romanovsky <leonro(a)nvidia.com>
> > >
> > > After commit fcf463b92a08 ("types: move phys_vec definition to common header"),
> > > we can use the shared phys_vec type instead of the DMABUF‑specific
> > > dma_buf_phys_vec, which duplicated the same structure and semantics.
> > >
> > > Signed-off-by: Leon Romanovsky <leonro(a)nvidia.com>
> > > ---
> > > Alex,
> > >
> > > According to diffstat, VFIO is the subsystem with the largest set of changes,
> > > so it would be great if you could take it through your tree.
> > >
> > > The series is based on the for-7.0/blk-pvec shared branch from Jens:
> > > https://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux.git/log/?h=for-…
> > >
> > > Thanks
> > > ---
> >
> > Alex,
> >
> > Could you please move this patch forward? We have the RDMA series [1] that
> > depends on this rename, and I would like to base it on the shared branch.
> >
> > [1] https://lore.kernel.org/all/20260108-dmabuf-export-v1-0-6d47d46580d3@nvidia…
>
> I tried to ping Jens regarding why the branch with this code hasn't
> been merged into their for-next branch, maybe you have more traction.
It is in block/for-next now, commit df73d3c618b4 ("Merge branch
'for-7.0/blk-pvec' into for-7.0/block".
Thanks
> Thanks,
>
> Alex
>
Hi everyone,
dma_fences have ever lived under the tyranny dictated by the module
lifetime of their issuer, leading to crashes should anybody still holding
a reference to a dma_fence when the module of the issuer was unloaded.
The basic problem is that when buffer are shared between drivers
dma_fence objects can leak into external drivers and stay there even
after they are signaled. The dma_resv object for example only lazy releases
dma_fences.
So what happens is that when the module who originally created the dma_fence
unloads the dma_fence_ops function table becomes unavailable as well and so
any attempt to release the fence crashes the system.
Previously various approaches have been discussed, including changing the
locking semantics of the dma_fence callbacks (by me) as well as using the
drm scheduler as intermediate layer (by Sima) to disconnect dma_fences
from their actual users, but none of them are actually solving all problems.
Tvrtko did some really nice prerequisite work by protecting the returned
strings of the dma_fence_ops by RCU. This way dma_fence creators where
able to just wait for an RCU grace period after fence signaling before
they could be save to free those data structures.
Now this patch set here goes a step further and protects the whole
dma_fence_ops structure by RCU, so that after the fence signals the
pointer to the dma_fence_ops is set to NULL when there is no wait nor
release callback given. All functionality which use the dma_fence_ops
reference are put inside an RCU critical section, except for the
deprecated issuer specific wait and of course the optional release
callback.
Additional to the RCU changes the lock protecting the dma_fence state
previously had to be allocated external. This set here now changes the
functionality to make that external lock optional and allows dma_fences
to use an inline lock and be self contained.
v4:
Rebases the whole set on upstream changes, especially the cleanup
from Philip in patch "drm/amdgpu: independence for the amdkfd_fence!".
Adding two patches which brings the DMA-fence self tests up to date.
The first selftest changes removes the mock_wait and so actually starts
testing the default behavior instead of some hacky implementation in the
test. This one got upstreamed independent of this set.
The second drops the mock_fence as well and tests the new RCU and inline
spinlock functionality.
v5:
Rebase on top of drm-misc-next instead of drm-tip, leave out all driver
changes for now since those should go through the driver specific paths
anyway.
Address a few more review comments, especially some rebase mess and
typos. And finally fix one more bug found by AMDs CI system.
Especially the first patch still needs a Reviewed-by, apart from that I
think I've addressed all review comments and problems.
Please review and comment,
Christian.