The patch set allows to register a dmabuf to an io_uring instance for
a specified file and use it with io_uring read / write requests. The
infrastructure is not tied to io_uring and there could be more users
in the future. A similar idea was attempted some years ago by Keith [1],
from where I borrowed a good number of changes, and later was brough up
by Tushar and Vishal from Intel.
It's an opt-in feature for files, and they need to implement a new
file operation to use it. Only NVMe block devices are supported in this
series. The user API is built on top of io_uring's "registered buffers",
where a dmabuf is registered in a special way, but after it can be used
as any other "registered buffer" with IORING_OP_{READ,WRITE}_FIXED
requests. It's created via a new file operation and the resulted map is
then passed through the I/O stack in a new iterator type. There is some
additional infrastructure to bind it all, which also counts requests
using a dmabuf map and managing lifetimes, which is used to implement
map invalidation.
It was tested for GPU <-> NVMe transfers. Also, as it maintains a
long-term dma mapping, it helps with the IOMMU cost. The numbers
below are for udmabuf reads previously run by Anuj for different
IOMMU modes:
- STRICT: before = 570 KIOPS, after = 5.01 MIOPS
- LAZY: before = 1.93 MIOPS, after = 5.01 MIOPS
- PASSTHROUGH: before = 5.01 MIOPS, after = 5.01 MIOPS
There are some liburing tests that can serve as an example:
git: https://github.com/isilence/liburing.git rw-dmabuf-tests-v3
url: https://github.com/isilence/liburing/tree/rw-dmabuf-tests-v3
[1] https://lore.kernel.org/io-uring/20220805162444.3985535-1-kbusch@fb.com/
v3: - Rework io_uring registration
- Move token/map infrastructure code out of blk-mq
- Simplify callbacks: remove a separate blk-mq table, which was
mostly just forwarding calls (to nvme).
- Don't skip dma sync depending on request direction
- Fix a couple of hangs
- Rename s/dma/dmabuf/
- Other small changes
v2: - Don't pass raw dma addresses, wrap it into a driver specific object
- Split into two objects: token and map
- Implement move_notify
Pavel Begunkov (10):
file: add callback for creating long-term dmabuf maps
iov_iter: add iterator type for dmabuf maps
block: move bvec init into __bio_clone
block: introduce dma map backed bio type
lib: add dmabuf token infrastructure
block: forward create_dmabuf_token to drivers
nvme-pci: implement dma_token backed requests
io_uring/rsrc: introduce buf registration structure
io_uring/rsrc: extend buffer update
io_uring/rsrc: add dmabuf backed registered buffers
block/bio.c | 28 +++-
block/blk-merge.c | 14 ++
block/blk.h | 3 +-
block/fops.c | 16 ++
drivers/nvme/host/pci.c | 282 ++++++++++++++++++++++++++++++++
include/linux/bio.h | 19 ++-
include/linux/blk-mq.h | 9 +
include/linux/blk_types.h | 8 +-
include/linux/fs.h | 2 +
include/linux/io_dmabuf_token.h | 92 +++++++++++
include/linux/io_uring_types.h | 5 +
include/linux/uio.h | 11 ++
include/uapi/linux/io_uring.h | 31 +++-
io_uring/io_uring.c | 3 +-
io_uring/rsrc.c | 266 +++++++++++++++++++++++++-----
io_uring/rsrc.h | 30 +++-
io_uring/rw.c | 4 +-
lib/Kconfig | 4 +
lib/Makefile | 2 +
lib/io_dmabuf_token.c | 272 ++++++++++++++++++++++++++++++
lib/iov_iter.c | 29 +++-
21 files changed, 1071 insertions(+), 59 deletions(-)
create mode 100644 include/linux/io_dmabuf_token.h
create mode 100644 lib/io_dmabuf_token.c
--
2.53.0
Hi everyone,
dma_fences have ever lived under the tyranny dictated by the module
lifetime of their issuer, leading to crashes should anybody still holding
a reference to a dma_fence when the module of the issuer was unloaded.
The basic problem is that when buffer are shared between drivers
dma_fence objects can leak into external drivers and stay there even
after they are signaled. The dma_resv object for example only lazy releases
dma_fences.
So what happens is that when the module who originally created the dma_fence
unloads the dma_fence_ops function table becomes unavailable as well and so
any attempt to release the fence crashes the system.
Previously various approaches have been discussed, including changing the
locking semantics of the dma_fence callbacks (by me) as well as using the
drm scheduler as intermediate layer (by Sima) to disconnect dma_fences
from their actual users, but none of them are actually solving all problems.
Tvrtko did some really nice prerequisite work by protecting the returned
strings of the dma_fence_ops by RCU. This way dma_fence creators where
able to just wait for an RCU grace period after fence signaling before
they could be save to free those data structures.
Now this patch set here goes a step further and protects the whole
dma_fence_ops structure by RCU, so that after the fence signals the
pointer to the dma_fence_ops is set to NULL when there is no wait nor
release callback given. All functionality which use the dma_fence_ops
reference are put inside an RCU critical section, except for the
deprecated issuer specific wait and of course the optional release
callback.
Additional to the RCU changes the lock protecting the dma_fence state
previously had to be allocated external. This set here now changes the
functionality to make that external lock optional and allows dma_fences
to use an inline lock and be self contained.
v4:
Rebases the whole set on upstream changes, especially the cleanup
from Philip in patch "drm/amdgpu: independence for the amdkfd_fence!".
Adding two patches which brings the DMA-fence self tests up to date.
The first selftest changes removes the mock_wait and so actually starts
testing the default behavior instead of some hacky implementation in the
test. This one got upstreamed independent of this set.
The second drops the mock_fence as well and tests the new RCU and inline
spinlock functionality.
v5:
Rebase on top of drm-misc-next instead of drm-tip, leave out all driver
changes for now since those should go through the driver specific paths
anyway.
Address a few more review comments, especially some rebase mess and
typos. And finally fix one more bug found by AMDs CI system.
v6:
Minor style changes, re-ordered patch #1, dropped the scheduler fence
change for now
Please review and comment,
Christian.
As udmabuf increasingly enjoys popularity - being used in projects like
libcamera, Gstreamer, Mesa and KWin - users more frequently encounter
cases where the current default size limit of 64MB is too low. Examples
include allocating video buffers at a 8K resolution - and even 4K is
affected when using non-subsampled video formats or high bit depths.
While the limit can already be changed via the kernel command line,
exposing it as a kernel config makes that easier and more discoverable
for distros. Thus let's do that.
Signed-off-by: Robert Mader <robert.mader(a)collabora.com>
---
drivers/dma-buf/Kconfig | 6 ++++++
drivers/dma-buf/udmabuf.c | 4 ++++
2 files changed, 10 insertions(+)
diff --git a/drivers/dma-buf/Kconfig b/drivers/dma-buf/Kconfig
index 7efc0f0d0712..35f0779cdc80 100644
--- a/drivers/dma-buf/Kconfig
+++ b/drivers/dma-buf/Kconfig
@@ -40,6 +40,12 @@ config UDMABUF
A driver to let userspace turn memfd regions into dma-bufs.
Qemu can use this to create host dmabufs for guest framebuffers.
+config UDMABUF_SIZE_LIMIT_MBYTES
+ int "Size limit in Mega Bytes"
+ default 64
+ help
+ Maximum size of a udmabuf, in megabytes. Default is 64.
+
config DMABUF_DEBUG
bool "DMA-BUF debug checks"
depends on DMA_SHARED_BUFFER
diff --git a/drivers/dma-buf/udmabuf.c b/drivers/dma-buf/udmabuf.c
index bced421c0d65..a83153326362 100644
--- a/drivers/dma-buf/udmabuf.c
+++ b/drivers/dma-buf/udmabuf.c
@@ -20,7 +20,11 @@ static int list_limit = 1024;
module_param(list_limit, int, 0644);
MODULE_PARM_DESC(list_limit, "udmabuf_create_list->count limit. Default is 1024.");
+#ifdef CONFIG_UDMABUF_SIZE_LIMIT_MBYTES
+static int size_limit_mb = CONFIG_UDMABUF_SIZE_LIMIT_MBYTES;
+#else
static int size_limit_mb = 64;
+#endif
module_param(size_limit_mb, int, 0644);
MODULE_PARM_DESC(size_limit_mb, "Max size of a dmabuf, in megabytes. Default is 64.");
--
2.55.0
On Thu, May 29, 2025 at 01:34:53PM +0800, Xu Yilun wrote:
> Export vfio dma-buf specific info by attaching vfio_dma_buf_data in
> struct dma_buf::priv. Provide a helper vfio_dma_buf_get_data() for
> importers to fetch these data. Exporters identify VFIO dma-buf by
> successfully getting these data.
>
> VFIO dma-buf supports disabling host access to these exported MMIO
> regions when the device is converted to private. Exporters like KVM
> need to identify this type of dma-buf to decide if it is good to use.
> KVM only allows host unaccessible MMIO regions been mapped in private
> roots.
>
> Export struct kvm * handler attached to the vfio device. This
> allows KVM to do another sanity check. MMIO should only be assigned to
> a CoCo VM if its owner device is already assigned to the same VM.
This doesn't seem right, it should be encapsulated into the standard
DMABUF API in some way.
Jason
Thank you for sharing such insightful information on Caluanie Muelear Oxidize! It's fascinating how this product can impact various industries. I appreciate the detailed approach and look forward to learning more about its applications in the future. https://food-le.com
I actually just stumbled upon <a href="https://deerfootcasino-online.com">Deerfoot Casino</a> a few days back when I was hunting for a solid Canadian slots platform that doesn’t hit you with conversion fees. Since I’ve been to their real venue in Calgary, I figured their online casino setup would be legit, and it’s a pretty decent little hub tbh. They have a proper selection of games and the Interac cashouts are surprisingly fast compared to other sites that leave your money pending for ages. Handy page to keep in tabs if you want a reliable setup to play.
Commit 9b836641d3bf ("rust: helpers: Add bindings/wrappers for
dma_resv_lock") added rust_helper_dma_resv_lock() and
rust_helper_dma_resv_unlock() unconditionally. However, the dma-resv
functionality is only available when CONFIG_DMA_SHARED_BUFFER is
enabled, resulting in the following link error when it is not:
LD .tmp_vmlinux1
ld.lld: error: undefined symbol: dma_resv_reset_max_fences
>>> referenced by dma-resv.h:463
>>> rust/helpers/helpers.o:(rust_helper_dma_resv_unlock) in archive vmlinux.a
The dma_resv_unlock() inline in <linux/dma-resv.h> calls
dma_resv_reset_max_fences(), which is only compiled into
drivers/dma-buf/dma-resv.c when CONFIG_DMA_SHARED_BUFFER is set. With
the option disabled the symbol is never defined, so the helper fails
to link.
Fix this by guarding the helper definitions with `#ifdef
CONFIG_DMA_SHARED_BUFFER`.
The only user of this helper is the DRM shmem abstractions, which already
depend on `CONFIG_DMA_SHARED_BUFFER`.
Cc: stable(a)vger.kernel.org
Fixes: 9b836641d3bf ("rust: helpers: Add bindings/wrappers for dma_resv_lock")
Signed-off-by: Andreas Hindborg <a.hindborg(a)kernel.org>
---
rust/helpers/dma-resv.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/rust/helpers/dma-resv.c b/rust/helpers/dma-resv.c
index 71914d8241e2..1698092b9317 100644
--- a/rust/helpers/dma-resv.c
+++ b/rust/helpers/dma-resv.c
@@ -1,5 +1,7 @@
// SPDX-License-Identifier: GPL-2.0
+#ifdef CONFIG_DMA_SHARED_BUFFER
+
#include <linux/dma-resv.h>
__rust_helper
@@ -12,3 +14,5 @@ __rust_helper void rust_helper_dma_resv_unlock(struct dma_resv *obj)
{
dma_resv_unlock(obj);
}
+
+#endif
---
base-commit: dc59e4fea9d83f03bad6bddf3fa2e52491777482
change-id: 20260708-dma-shared-buffer-config-bb135fdb61f4
Best regards,
--
Andreas Hindborg <a.hindborg(a)kernel.org>
DMA_HEAP_IOCTL_ALLOC allocates a dma-buf and installs an fd into the
caller's fd table via fd_install() before dma_heap_ioctl() copies the
result back to userspace. If the trailing copy_to_user() fails, the
ioctl returns -EFAULT and userspace never learns the fd number, but
the fd (and the underlying dma-buf reference) remain in the caller's
fd table and are leaked for the lifetime of the process.
The failure is easily reachable from userspace: pass a struct
dma_heap_allocation_data that lives in a page whose protection is
flipped to PROT_READ between copy_from_user() and copy_to_user()
(e.g. via mprotect()). Each such ioctl leaks one dmabuf fd; repeating
the call quickly fills /proc/<pid>/fd with anonymous "/dmabuf:"
entries that only go away when the process exits.
Fix it by closing the installed fd (and clearing the fd field of the
kernel-side copy) when copy_to_user() fails after a successful
allocation, so the error path matches what userspace observes: no fd
was returned, therefore no fd is left behind.
Fixes: c02a81fba74f ("dma-buf: Add dma-buf heaps framework")
Cc: stable(a)vger.kernel.org
Signed-off-by: Baineng Shou <shoubaineng(a)gmail.com>
---
Reproducer (full source, gcc -o poc poc.c; run as root):
// poc.c -- leak one dma-buf fd per DMA_HEAP_IOCTL_ALLOC
// when copy_to_user() fails
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <sys/ioctl.h>
#include <sys/mman.h>
#include <linux/dma-heap.h>
int main(int argc, char **argv)
{
int n = argc > 1 ? atoi(argv[1]) : 100;
long ps = sysconf(_SC_PAGESIZE);
int heap = open("/dev/dma_heap/system", O_RDWR | O_CLOEXEC);
if (heap < 0)
return perror("open"), 1;
for (int i = 0; i < n; i++) {
/* Put a valid request in a page, then make the page
* read-only: copy_from_user() still succeeds and the
* dma-buf is allocated and fd_install()'d, but the
* trailing copy_to_user() fails and the fd, never
* returned to us, is leaked.
*/
struct dma_heap_allocation_data *req =
mmap(NULL, ps, PROT_READ | PROT_WRITE,
MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
memset(req, 0, sizeof(*req));
req->len = ps;
req->fd_flags = O_RDWR | O_CLOEXEC;
mprotect(req, ps, PROT_READ);
ioctl(heap, DMA_HEAP_IOCTL_ALLOC, req); /* -EFAULT */
munmap(req, ps);
}
printf("done: check ls -l /proc/%d/fd for %d leaked fds\n",
getpid(), n);
pause();
return 0;
}
Before the fix, ./poc 10 leaves 10 anonymous dmabuf fds in the
caller's fd table:
# ls -l /proc/$(pgrep poc)/fd
lrwx------ 1 root root 64 Jan 1 00:03 3 -> /dev/dma_heap/system
lrwx------ 1 root root 64 Jan 1 00:03 4 -> /dmabuf:
lrwx------ 1 root root 64 Jan 1 00:03 5 -> /dmabuf:
lrwx------ 1 root root 64 Jan 1 00:03 6 -> /dmabuf:
lrwx------ 1 root root 64 Jan 1 00:03 7 -> /dmabuf:
lrwx------ 1 root root 64 Jan 1 00:03 8 -> /dmabuf:
lrwx------ 1 root root 64 Jan 1 00:03 9 -> /dmabuf:
lrwx------ 1 root root 64 Jan 1 00:03 10 -> /dmabuf:
lrwx------ 1 root root 64 Jan 1 00:03 11 -> /dmabuf:
lrwx------ 1 root root 64 Jan 1 00:03 12 -> /dmabuf:
lrwx------ 1 root root 64 Jan 1 00:03 13 -> /dmabuf:
After the fix, only /dev/dma_heap/system remains open; the
anonymous "/dmabuf:" entries are gone.
drivers/dma-buf/dma-heap.c | 11 ++++++++++-
1 file changed, 10 insertions(+), 1 deletion(-)
diff --git a/drivers/dma-buf/dma-heap.c b/drivers/dma-buf/dma-heap.c
index a76bf3f8b071..0dd7a84b06bf 100644
--- a/drivers/dma-buf/dma-heap.c
+++ b/drivers/dma-buf/dma-heap.c
@@ -18,6 +18,7 @@
#include <linux/uaccess.h>
#include <linux/xarray.h>
#include <uapi/linux/dma-heap.h>
+#include <linux/fdtable.h>
#define DEVNAME "dma_heap"
@@ -181,8 +182,16 @@ static long dma_heap_ioctl(struct file *file, unsigned int ucmd,
goto err;
}
- if (copy_to_user((void __user *)arg, kdata, out_size) != 0)
+ if (copy_to_user((void __user *)arg, kdata, out_size) != 0) {
+ if (kcmd == DMA_HEAP_IOCTL_ALLOC && ret == 0) {
+ struct dma_heap_allocation_data *h = (void *)kdata;
+
+ close_fd(h->fd);
+ h->fd = -1;
+ }
ret = -EFAULT;
+ }
+
err:
if (kdata != stack_kdata)
kfree(kdata);
--
2.34.1