Shared-memory allocator services (e.g., Android's gralloc) allocate dma-bufs on behalf of client processes in different cgroups. Today those allocations are charged to the allocator's own cgroup, leaving client cgroups with no visibility into their actual memory consumption and defeating resource limits.
This series adds memory cgroup accounting for dma-buf allocations, building on T.J. Mercier's earlier series [1] and incorporating feedback from previous versions and LPC discussions [2].
Patch 1 introduces the MEMCG_DMABUF stat and reserve/commit/cancel and uncharge helpers. Rather than charging at the page level via __GFP_ACCOUNT (which would require every dma-buf exporter to implement charging, and makes cross-cgroup transfer impractical without access to backing pages), we track charges externally on struct dma_buf via a MEMCG_DMABUF stat counter and route all accounting through a single path, removing the __GFP_ACCOUNT flag from system-heap allocations to avoid double counting.
Patch 2 adds a charge_fd field to DMA_HEAP_IOCTL_ALLOC, accepting either a pidfd or a cgroupfd to identify the target cgroup at allocation time. If the target cgroup cannot accommodate the charge, the allocation returns -ENOMEM.
Patches 3-6 add access control for cross-cgroup charging. Letting userspace decide who to charge needs to be a privileged operation because it has material impact on OOM decission and resource limits. We gate it via LSM hooks for the two charge_fd flavours: security_dma_buf_charge_task() gates the pidfd path (target identified by task credentials) and security_dma_buf_charge_cgroup() gates the cgroupfd path (target identified by cgroup directory label), with SELinux implementations under a dma_buf object class.
Patch 7 adds DMA_BUF_IOCTL_XFER_CHARGE, a general ioctl on any dma-buf fd to transfer charges post-allocation. This covers the double-hop allocator case (SurfaceFlinger -> gralloc -> kernel) and works for buffers from any exporter (DRM, V4L2, etc.). A new memcg_account flag is added for dma_buf_export() to ensure that transfer/charge is only applied for buffers backed by reclaimable system memory. If the receiving cgroup lacks capacity for the charge, the ioctl returns -ENOMEM and the original charge is preserved.
Patch 8 adds selftests.
Known limitations:
- Single-owner model: a dma-buf carries one memcg charge regardless of how many processes share it. - Only memcg accounting. CMA heaps would additionally need dmem controller integration. - If a transfer of an uncharged buffer fails, it remains uncharged. The allocator (userspace) is expected to handle this (retry, free, ...). - The LSM hooks are intentionally narrow. Broader cgroup access controls (e.g. for the memfd/kexec handover case [2]) could be a separate effort reusing the same pattern. - Only system heap sets memcg_account today. Other exporters (DRM, V4L2, etc.) can opt in by setting the flag, but are not covered by this series.
[1] https://lore.kernel.org/cgroups/20230109213809.418135-1-tjmercier@google.com... [2] https://lpc.events/event/20/contributions/2416/
Signed-off-by: Albert Esteve aesteve@redhat.com --- Changes in v3: - Split mem_cgroup_charge_dmabuf() into reserve/commit/cancel three-phase API - Extract mem_cgroup_get_from_fd() helper shared by allocate and xfer paths - Rename __padding to flags in struct dma_buf_transfer_charge - Fix UAF race: move mod_memcg_state() before cmpxchg in transfer - Add memcg_account opt-in flag; xfer returns -EOPNOTSUPP if unset - Wrap dma_buf_charge_cgroup hook in #ifdef CONFIG_CGROUPS - Fix TOCTOU in SELinux charge_cgroup xattr length handling - Link to v2: https://lore.kernel.org/r/20261021-v2_20230123_tjmercier_google_com-v2-0-ef6...
Changes in v2: - Move charge_pid_fd to charge_fd, accepting cgroupfd in addition to pidfd per Christian König. - New DMA_BUF_IOCTL_XFER_CHARGE ioctl for post-allocation charge transfer on any dma-buf, not just dma-heap buffers per Christian König and T.J. Mercier. - Moved __GFP_ACCOUNT removal to patch 1 per Barry Song. - dma-buf charge/uncharge helpers take struct dma_buf * directly to avoid the need for caller to manage size calculations. - Added test_xfer_charge_cgroupfd_self to dmabuf-heap selftests. - Added LSM and SELinux hooks for cgroupfd-based charging. - Rebased to v7.3-rc7. - Link to v1: https://lore.kernel.org/r/20260512-v2_20230123_tjmercier_google_com-v1-0-632...
--- Albert Esteve (7): dma-heap: charge dma-buf memory via explicit memcg security: dma-buf: Add dma_buf_charge_task LSM hook security: dma-buf: Add dma_buf_charge_cgroup LSM hook selinux: Restrict cross-cgroup dma-buf charging selinux: Restrict dma-buf direct cgroup charging dma-buf: transfer memcg charges via DMA_BUF_IOCTL_XFER_CHARGE selftests/dmabuf-heaps: Add dma-buf memcg accounting tests
T.J. Mercier (1): memcg: Add MEMCG_DMABUF accounting infrastructure
Documentation/admin-guide/cgroup-v2.rst | 5 + drivers/dma-buf/dma-buf.c | 46 ++++ drivers/dma-buf/dma-heap.c | 28 +- drivers/dma-buf/heaps/cma_heap.c | 3 +- drivers/dma-buf/heaps/system_heap.c | 22 +- drivers/tee/tee_heap.c | 3 +- include/linux/dma-buf.h | 9 + include/linux/dma-heap.h | 3 +- include/linux/lsm_hook_defs.h | 4 + include/linux/memcontrol.h | 52 ++++ include/linux/security.h | 18 ++ include/uapi/linux/dma-buf.h | 26 ++ include/uapi/linux/dma-heap.h | 7 + mm/memcontrol.c | 192 ++++++++++++++ security/security.c | 34 +++ security/selinux/hooks.c | 44 ++++ security/selinux/include/classmap.h | 1 + tools/testing/selftests/cgroup/Makefile | 2 +- tools/testing/selftests/cgroup/test_memcontrol.c | 290 ++++++++++++++++++++- tools/testing/selftests/dmabuf-heaps/config | 1 + tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c | 227 +++++++++++++++- tools/testing/selftests/dmabuf-heaps/vmtest.sh | 226 ++++++++++++++++ 22 files changed, 1228 insertions(+), 15 deletions(-) --- base-commit: 24a19b953e2a3d0ea5a8c4951a0ab8f32720c907 change-id: 20260508-v2_20230123_tjmercier_google_com-f44fcfb16530
Best regards,
From: "T.J. Mercier" tjmercier@google.com
Introduce a MEMCG_DMABUF stat counter and helpers for reserve/commit/cancel and uncharging dma-buf memory against a memory cgroup:
The charge is tracked on struct dma_buf via a new memcg pointer. The page count is derived internally from PAGE_ALIGN(dmabuf->size), and the memcg reference is stored into (or cleared from) dmabuf->memcg by the helpers themselves, removing the need for callers to manage those details. The release path in dma_buf_release() is reduced to a single mem_cgroup_uncharge_dmabuf() call.
This decouples accounting from the backing page allocator, avoiding the need for every exporter to pass __GFP_ACCOUNT and enabling cross-cgroup charge transfer, which is impractical with page-level accounting.
The next patch wires up the charging policy in the dma-heap ioctl.
Signed-off-by: T.J. Mercier tjmercier@google.com Signed-off-by: Albert Esteve aesteve@redhat.com --- Documentation/admin-guide/cgroup-v2.rst | 4 ++ drivers/dma-buf/dma-buf.c | 3 ++ include/linux/dma-buf.h | 4 ++ include/linux/memcontrol.h | 36 ++++++++++++++ mm/memcontrol.c | 87 +++++++++++++++++++++++++++++++++ 5 files changed, 134 insertions(+)
diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst index 86a2a0099178..a3265a4a953e 100644 --- a/Documentation/admin-guide/cgroup-v2.rst +++ b/Documentation/admin-guide/cgroup-v2.rst @@ -1662,6 +1662,10 @@ The following nested keys are defined. Amount of memory used for storing in-kernel data structures.
+ dmabuf (npn) + Amount of memory used for exported DMA buffers allocated by the cgroup. + Stays with the allocating cgroup regardless of how the buffer is shared. + workingset_refault_anon Number of refaults of previously evicted anonymous pages.
diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c index 4c9add51f9ef..f31f49ebd0af 100644 --- a/drivers/dma-buf/dma-buf.c +++ b/drivers/dma-buf/dma-buf.c @@ -14,6 +14,7 @@ #include <linux/fs.h> #include <linux/slab.h> #include <linux/dma-buf.h> +#include <linux/memcontrol.h> #include <linux/dma-fence.h> #include <linux/dma-fence-unwrap.h> #include <linux/anon_inodes.h> @@ -180,6 +181,8 @@ static void dma_buf_release(struct dentry *dentry) */ BUG_ON(dmabuf->cb_in.active || dmabuf->cb_out.active);
+ mem_cgroup_uncharge_dmabuf(dmabuf); + dmabuf->ops->release(dmabuf);
if (dmabuf->resv == (struct dma_resv *)&dmabuf[1]) diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h index d15b2b31d3c9..23ec0ff23fba 100644 --- a/include/linux/dma-buf.h +++ b/include/linux/dma-buf.h @@ -27,6 +27,7 @@ struct device; struct dma_buf; struct dma_buf_attachment; +struct mem_cgroup;
/** * struct dma_buf_ops - operations possible on struct dma_buf @@ -429,6 +430,9 @@ struct dma_buf {
__poll_t active; } cb_in, cb_out; + + /** @memcg: the cgroup to which this buffer is currently attributed */ + struct mem_cgroup *memcg; };
/** diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 7d1c0ce189a8..4c8711fcf4fd 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -10,6 +10,9 @@
#ifndef _LINUX_MEMCONTROL_H #define _LINUX_MEMCONTROL_H + +struct dma_buf; + #include <linux/cgroup.h> #include <linux/vm_event_item.h> #include <linux/hardirq.h> @@ -40,6 +43,7 @@ enum memcg_stat_item { MEMCG_ZSWAP_B, MEMCG_ZSWAPPED, MEMCG_ZSWAP_INCOMP, + MEMCG_DMABUF, MEMCG_NR_STAT, };
@@ -667,6 +671,12 @@ int mem_cgroup_charge_hugetlb(struct folio* folio, gfp_t gfp); int mem_cgroup_swapin_charge_folio(struct folio *folio, unsigned short id, struct mm_struct *mm, gfp_t gfp);
+bool mem_cgroup_reserve_dmabuf(struct mem_cgroup *memcg, gfp_t gfp, + unsigned int nr_pages); + +void mem_cgroup_commit_dmabuf(struct dma_buf *dmabuf, struct mem_cgroup *memcg, + unsigned int nr_pages); + void __mem_cgroup_uncharge(struct folio *folio);
/** @@ -682,6 +692,10 @@ static inline void mem_cgroup_uncharge(struct folio *folio) __mem_cgroup_uncharge(folio); }
+void mem_cgroup_cancel_dmabuf(struct mem_cgroup *memcg, unsigned int nr_pages); + +void mem_cgroup_uncharge_dmabuf(struct dma_buf *dmabuf); + void __mem_cgroup_uncharge_folios(struct folio_batch *folios); static inline void mem_cgroup_uncharge_folios(struct folio_batch *folios) { @@ -1167,10 +1181,32 @@ static inline int mem_cgroup_swapin_charge_folio(struct folio *folio, return 0; }
+static inline bool mem_cgroup_reserve_dmabuf(struct mem_cgroup *memcg, + gfp_t gfp, + unsigned int nr_pages) +{ + return true; +} + +static inline void mem_cgroup_commit_dmabuf(struct dma_buf *dmabuf, + struct mem_cgroup *memcg, + unsigned int nr_pages) +{ +} + +static inline void mem_cgroup_cancel_dmabuf(struct mem_cgroup *memcg, + unsigned int nr_pages) +{ +} + static inline void mem_cgroup_uncharge(struct folio *folio) { }
+static inline void mem_cgroup_uncharge_dmabuf(struct dma_buf *dmabuf) +{ +} + static inline void mem_cgroup_uncharge_folios(struct folio_batch *folios) { } diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 856a7d07586c..bbd46ba40350 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -69,6 +69,7 @@ #include <net/ip.h> #include "slab.h" #include "memcontrol-v1.h" +#include <linux/dma-buf.h>
#include <linux/uaccess.h>
@@ -442,6 +443,7 @@ static const unsigned int memcg_stat_items[] = { MEMCG_ZSWAP_B, MEMCG_ZSWAPPED, MEMCG_ZSWAP_INCOMP, + MEMCG_DMABUF, };
#define NR_MEMCG_NODE_STAT_ITEMS ARRAY_SIZE(memcg_node_stat_items) @@ -1625,6 +1627,7 @@ static const struct memory_stat memory_stats[] = { #ifdef CONFIG_HUGETLB_PAGE { "hugetlb", NR_HUGETLB }, #endif + { "dmabuf", MEMCG_DMABUF },
/* The memory events */ { "workingset_refault_anon", WORKINGSET_REFAULT_ANON }, @@ -5658,6 +5661,90 @@ void mem_cgroup_flush_workqueue(void) flush_workqueue(memcg_wq); }
+/** + * mem_cgroup_reserve_dmabuf - charge a memcg for a dma-buf allocation + * @memcg: memcg to charge (may be %NULL) + * @gfp_mask: reclaim mode + * @nr_pages: number of pages to charge + * + * Charges @nr_pages against @memcg. Call this before allocating pages so + * the cgroup limit is enforced up front. On success the caller must either + * call mem_cgroup_commit_dmabuf() to finalize the charge, or + * mem_cgroup_cancel_dmabuf() to undo it. + * + * Returns %true on success, %false if the charge would exceed the limit. + */ +bool mem_cgroup_reserve_dmabuf(struct mem_cgroup *memcg, gfp_t gfp_mask, + unsigned int nr_pages) +{ + if (!memcg || mem_cgroup_disabled()) + return true; + + if (try_charge(memcg, gfp_mask, nr_pages)) + return false; + + return true; +} + +/** + * mem_cgroup_commit_dmabuf - finalize a dma-buf charge after export + * @dmabuf: the exported dma-buf + * @memcg: memcg previously charged via mem_cgroup_reserve_dmabuf() + * @nr_pages: same value passed to mem_cgroup_reserve_dmabuf() + * + * Updates the MEMCG_DMABUF counter and stores @memcg in @dmabuf->memcg, + * transferring the caller's reference. + */ +void mem_cgroup_commit_dmabuf(struct dma_buf *dmabuf, struct mem_cgroup *memcg, + unsigned int nr_pages) +{ + if (!memcg || mem_cgroup_disabled()) + return; + + mod_memcg_state(memcg, MEMCG_DMABUF, nr_pages); + dmabuf->memcg = memcg; +} + +/** + * mem_cgroup_cancel_dmabuf - undo a dma-buf charge + * @memcg: memcg to uncharge (may be %NULL) + * @nr_pages: same value passed to mem_cgroup_reserve_dmabuf() + * + * Returns @nr_pages of charge to @memcg. Use when the allocation fails + * after mem_cgroup_reserve_dmabuf() but before mem_cgroup_commit_dmabuf(). + */ +void mem_cgroup_cancel_dmabuf(struct mem_cgroup *memcg, unsigned int nr_pages) +{ + if (!memcg) + return; + + if (!mem_cgroup_is_root(memcg)) + refill_stock(memcg, nr_pages); +} + +/** + * mem_cgroup_uncharge_dmabuf - uncharge a dma-buf from its memcg + * @dmabuf: buffer to uncharge + * + * Reverses a successful mem_cgroup_commit_dmabuf(). Drops the memcg + * reference stored in @dmabuf->memcg and clears the pointer. + * Safe to call when @dmabuf->memcg is %NULL. + */ +void mem_cgroup_uncharge_dmabuf(struct dma_buf *dmabuf) +{ + unsigned int nr_pages; + struct mem_cgroup *memcg = dmabuf->memcg; + + if (!memcg) + return; + + nr_pages = PAGE_ALIGN(dmabuf->size) >> PAGE_SHIFT; + mod_memcg_state(memcg, MEMCG_DMABUF, -nr_pages); + mem_cgroup_cancel_dmabuf(memcg, nr_pages); + mem_cgroup_put(memcg); + dmabuf->memcg = NULL; +} + static int __init cgroup_memory(char *s) { char *token;
On embedded platforms a central process often allocates dma-buf memory on behalf of client applications. Without a way to attribute the charge to the requesting client's cgroup, the cost lands on the allocator, making per-cgroup memory limits ineffective for the actual consumers.
Add charge_fd to struct dma_heap_allocation_data. When set to a valid pidfd or cgroupfd, DMA_HEAP_IOCTL_ALLOC resolves the target task's or cgroup's memcg and charges the buffer there via mem_cgroup_reserve_dmabuf()/mem_cgroup_commit_dmabuf() inside dma_heap_ioctl_allocate(). Without charge_fd, and with the mem_accounting module parameter enabled, the buffer is charged to the allocator's own cgroup.
Usage examples:
1. Central allocator charging to a client at allocation time. The allocator knows the client's PID (e.g., from binder's sender_pid) and uses pidfd to attribute the charge:
pid_t client_pid = txn->sender_pid; int pidfd = pidfd_open(client_pid, 0);
struct dma_heap_allocation_data alloc = { .len = buffer_size, .fd_flags = O_RDWR | O_CLOEXEC, .charge_fd = pidfd, }; ioctl(heap_fd, DMA_HEAP_IOCTL_ALLOC, &alloc); close(pidfd); /* alloc.fd is now charged to client's cgroup */
2. Default allocation (no pidfd, mem_accounting=1). When charge_fd is not set and the mem_accounting module parameter is enabled, the buffer is charged to the allocator's own cgroup:
struct dma_heap_allocation_data alloc = { .len = buffer_size, .fd_flags = O_RDWR | O_CLOEXEC, }; ioctl(heap_fd, DMA_HEAP_IOCTL_ALLOC, &alloc); /* charged to current process's cgroup */
Current limitations:
- Single-owner model: a dma-buf carries one memcg charge regardless of how many processes share it. Means only the first owner (and exporter) of the shared buffer bears the charge. - Only memcg accounting supported. While this makes sense for system heap buffers, other heaps (e.g., CMA heaps) will require selectively charging also for the dmem controller.
Signed-off-by: Albert Esteve aesteve@redhat.com --- Documentation/admin-guide/cgroup-v2.rst | 5 ++-- drivers/dma-buf/dma-heap.c | 28 +++++++++++++++++--- drivers/dma-buf/heaps/cma_heap.c | 3 ++- drivers/dma-buf/heaps/system_heap.c | 21 +++++++++++---- drivers/tee/tee_heap.c | 3 ++- include/linux/dma-heap.h | 3 ++- include/linux/memcontrol.h | 7 +++++ include/uapi/linux/dma-heap.h | 7 +++++ mm/memcontrol.c | 47 +++++++++++++++++++++++++++++++++ 9 files changed, 111 insertions(+), 13 deletions(-)
diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst index a3265a4a953e..66ddba678784 100644 --- a/Documentation/admin-guide/cgroup-v2.rst +++ b/Documentation/admin-guide/cgroup-v2.rst @@ -1663,8 +1663,9 @@ The following nested keys are defined. structures.
dmabuf (npn) - Amount of memory used for exported DMA buffers allocated by the cgroup. - Stays with the allocating cgroup regardless of how the buffer is shared. + Amount of memory used for exported DMA buffers allocated by or on + behalf of the cgroup. Stays with the allocating cgroup regardless + of how the buffer is shared.
workingset_refault_anon Number of refaults of previously evicted anonymous pages. diff --git a/drivers/dma-buf/dma-heap.c b/drivers/dma-buf/dma-heap.c index 8a6c2b6e0cc6..7bd416ae2c23 100644 --- a/drivers/dma-buf/dma-heap.c +++ b/drivers/dma-buf/dma-heap.c @@ -10,6 +10,8 @@ #include <linux/device.h> #include <linux/dma-buf.h> #include <linux/dma-heap.h> +#include <linux/memcontrol.h> +#include <linux/sched/mm.h> #include <linux/err.h> #include <linux/export.h> #include <linux/list.h> @@ -78,6 +80,7 @@ static struct dma_buf *dma_heap_ioctl_allocate(struct file *file, void *data) struct dma_heap_allocation_data *heap_allocation = data; struct dma_heap *heap = file->private_data; struct dma_buf *dmabuf; + struct mem_cgroup *memcg = NULL; int fd; size_t len;
@@ -90,15 +93,34 @@ static struct dma_buf *dma_heap_ioctl_allocate(struct file *file, void *data) if (heap_allocation->heap_flags & ~DMA_HEAP_VALID_HEAP_FLAGS) return ERR_PTR(-EINVAL);
+ if (heap_allocation->__padding != 0) + return ERR_PTR(-EINVAL); + + if (heap_allocation->charge_fd) { + memcg = mem_cgroup_get_from_fd(heap_allocation->charge_fd); + if (IS_ERR(memcg)) + return ERR_CAST(memcg); + } + + if (!memcg && mem_accounting) + memcg = get_mem_cgroup_from_mm(current->mm); + len = PAGE_ALIGN(heap_allocation->len); - if (!len) + if (!len) { + mem_cgroup_put(memcg); return ERR_PTR(-EINVAL); + }
dmabuf = heap->ops->allocate(heap, len, heap_allocation->fd_flags, - heap_allocation->heap_flags); + heap_allocation->heap_flags, memcg);
- if (IS_ERR(dmabuf)) + if (IS_ERR(dmabuf)) { + mem_cgroup_put(memcg); return dmabuf; + } + + if (memcg && !dmabuf->memcg) + mem_cgroup_put(memcg);
fd = get_unused_fd_flags(heap_allocation->fd_flags); if (fd < 0) { diff --git a/drivers/dma-buf/heaps/cma_heap.c b/drivers/dma-buf/heaps/cma_heap.c index 3fb4b946c91a..04de88b468a8 100644 --- a/drivers/dma-buf/heaps/cma_heap.c +++ b/drivers/dma-buf/heaps/cma_heap.c @@ -281,7 +281,8 @@ static const struct dma_buf_ops cma_heap_buf_ops = { static struct dma_buf *cma_heap_allocate(struct dma_heap *heap, unsigned long len, u32 fd_flags, - u64 heap_flags) + u64 heap_flags, + struct mem_cgroup *memcg) { struct cma_heap *cma_heap = dma_heap_get_drvdata(heap); struct cma_heap_buffer *buffer; diff --git a/drivers/dma-buf/heaps/system_heap.c b/drivers/dma-buf/heaps/system_heap.c index c8959eadc71d..adeea8cd328b 100644 --- a/drivers/dma-buf/heaps/system_heap.c +++ b/drivers/dma-buf/heaps/system_heap.c @@ -17,6 +17,7 @@ #include <linux/err.h> #include <linux/highmem.h> #include <linux/mem_encrypt.h> +#include <linux/memcontrol.h> #include <linux/mm.h> #include <linux/set_memory.h> #include <linux/module.h> @@ -388,8 +389,6 @@ static struct page *alloc_largest_available(unsigned long size, if (max_order < orders[i]) continue; flags = order_flags[i]; - if (mem_accounting) - flags |= __GFP_ACCOUNT; page = alloc_pages(flags, orders[i]); if (!page) continue; @@ -401,11 +400,13 @@ static struct page *alloc_largest_available(unsigned long size, static struct dma_buf *system_heap_allocate(struct dma_heap *heap, unsigned long len, u32 fd_flags, - u64 heap_flags) + u64 heap_flags, + struct mem_cgroup *memcg) { struct system_heap_buffer *buffer; DEFINE_DMA_BUF_EXPORT_INFO(exp_info); unsigned long size_remaining = len; + unsigned int nr_pages = PAGE_ALIGN(len) >> PAGE_SHIFT; unsigned int max_order = orders[0]; struct system_heap_priv *priv = dma_heap_get_drvdata(heap); bool cc_shared = priv->cc_shared; @@ -416,10 +417,15 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap, struct page *page, *tmp_page; int i, ret = -ENOMEM;
- buffer = kzalloc_obj(*buffer); - if (!buffer) + if (!mem_cgroup_reserve_dmabuf(memcg, GFP_KERNEL, nr_pages)) return ERR_PTR(-ENOMEM);
+ buffer = kzalloc_obj(*buffer); + if (!buffer) { + ret = -ENOMEM; + goto cancel_charge; + } + INIT_LIST_HEAD(&buffer->attachments); mutex_init(&buffer->lock); buffer->heap = heap; @@ -478,6 +484,9 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap, ret = PTR_ERR(dmabuf); goto free_pages; } + + mem_cgroup_commit_dmabuf(dmabuf, memcg, nr_pages); + return dmabuf;
free_pages: @@ -498,6 +507,8 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap, list_for_each_entry_safe(page, tmp_page, &pages, lru) __free_pages(page, compound_order(page)); kfree(buffer); +cancel_charge: + mem_cgroup_cancel_dmabuf(memcg, nr_pages);
return ERR_PTR(ret); } diff --git a/drivers/tee/tee_heap.c b/drivers/tee/tee_heap.c index 007c00156520..83e2fbd6960b 100644 --- a/drivers/tee/tee_heap.c +++ b/drivers/tee/tee_heap.c @@ -167,7 +167,8 @@ static const struct dma_buf_ops tee_heap_buf_ops = {
static struct dma_buf *tee_dma_heap_alloc(struct dma_heap *heap, unsigned long len, u32 fd_flags, - u64 heap_flags) + u64 heap_flags, + struct mem_cgroup *memcg) { struct tee_dma_heap *h = dma_heap_get_drvdata(heap); DEFINE_DMA_BUF_EXPORT_INFO(exp_info); diff --git a/include/linux/dma-heap.h b/include/linux/dma-heap.h index 648328a64b27..2d69f8432bd2 100644 --- a/include/linux/dma-heap.h +++ b/include/linux/dma-heap.h @@ -23,7 +23,8 @@ struct dma_heap_ops { struct dma_buf *(*allocate)(struct dma_heap *heap, unsigned long len, u32 fd_flags, - u64 heap_flags); + u64 heap_flags, + struct mem_cgroup *memcg); };
/** diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 4c8711fcf4fd..6b30baa79fa9 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -671,6 +671,8 @@ int mem_cgroup_charge_hugetlb(struct folio* folio, gfp_t gfp); int mem_cgroup_swapin_charge_folio(struct folio *folio, unsigned short id, struct mm_struct *mm, gfp_t gfp);
+struct mem_cgroup *mem_cgroup_get_from_fd(int fd); + bool mem_cgroup_reserve_dmabuf(struct mem_cgroup *memcg, gfp_t gfp, unsigned int nr_pages);
@@ -1181,6 +1183,11 @@ static inline int mem_cgroup_swapin_charge_folio(struct folio *folio, return 0; }
+static inline struct mem_cgroup *mem_cgroup_get_from_fd(int fd) +{ + return ERR_PTR(-EOPNOTSUPP); +} + static inline bool mem_cgroup_reserve_dmabuf(struct mem_cgroup *memcg, gfp_t gfp, unsigned int nr_pages) diff --git a/include/uapi/linux/dma-heap.h b/include/uapi/linux/dma-heap.h index a4cf716a49fa..bfcd1c25bc7e 100644 --- a/include/uapi/linux/dma-heap.h +++ b/include/uapi/linux/dma-heap.h @@ -29,6 +29,11 @@ * handle to the allocated dma-buf * @fd_flags: file descriptor flags used when allocating * @heap_flags: flags passed to heap + * @charge_fd: optional pidfd or cgroupfd whose cgroup should be + * charged for this allocation; 0 means charge the calling + * process's cgroup when the mem_accounting module parameter + * is enabled, or skip charging otherwise + * @__padding: reserved, must be zero * * Provided by userspace as an argument to the ioctl */ @@ -37,6 +42,8 @@ struct dma_heap_allocation_data { __u32 fd; __u32 fd_flags; __u64 heap_flags; + __u32 charge_fd; + __u32 __padding; };
#define DMA_HEAP_IOC_MAGIC 'H' diff --git a/mm/memcontrol.c b/mm/memcontrol.c index bbd46ba40350..f66254f52b99 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -5661,6 +5661,53 @@ void mem_cgroup_flush_workqueue(void) flush_workqueue(memcg_wq); }
+/** + * mem_cgroup_get_from_fd - resolve a memcg from a cgroupfd or pidfd + * @fd: file descriptor (cgroupfd or pidfd) + * + * Tries @fd as a cgroupfd first; if that fails, tries it as a pidfd. + * On success returns a referenced memcg pointer. On failure returns + * an ERR_PTR. + */ +struct mem_cgroup *mem_cgroup_get_from_fd(int fd) +{ + struct cgroup_subsys_state *css; + struct mem_cgroup *memcg; + struct cgroup *cgrp; + struct task_struct *task; + struct mm_struct *mm; + unsigned int pidfd_flags; + + cgrp = cgroup_get_from_fd(fd); + if (!IS_ERR(cgrp)) { + rcu_read_lock(); + css = cgroup_css(cgrp, &memory_cgrp_subsys); + if (!css || !css_tryget_online(css)) { + rcu_read_unlock(); + cgroup_put(cgrp); + return ERR_PTR(-ENOENT); + } + + memcg = mem_cgroup_from_css(css); + rcu_read_unlock(); + cgroup_put(cgrp); + return memcg; + } + + task = pidfd_get_task(fd, &pidfd_flags); + if (IS_ERR(task)) + return ERR_CAST(task); + + mm = get_task_mm(task); + put_task_struct(task); + if (!mm) + return ERR_PTR(-ESRCH); + + memcg = get_mem_cgroup_from_mm(mm); + mmput(mm); + return memcg; +} + /** * mem_cgroup_reserve_dmabuf - charge a memcg for a dma-buf allocation * @memcg: memcg to charge (may be %NULL)
DMA_HEAP_IOCTL_ALLOC accepts a charge_fd field that, when set, causes the allocation to be charged to an arbitrary process's cgroup rather than the caller's.
Without an access-control point, any process that holds a handle to a dma-heap device node can charge unlimited memory to any other process's cgroup, potentially exhausting that cgroup's limit and triggering OOM kills independent of the victim's own activity or privileges.
Add security_dma_buf_charge_task(), called in mem_cgroup_get_from_fd() when charge_fd refers to another process. The hook receives the credentials of the allocating process (from) and the credentials of the process whose cgroup will be charged (to), giving security modules a controlled enforcement point for cross-cgroup dma-buf attribution policy.
When CONFIG_SECURITY is not set the hook compiles to an inline returning 0, adding no overhead to the fast path.
Signed-off-by: Albert Esteve aesteve@redhat.com --- include/linux/lsm_hook_defs.h | 1 + include/linux/security.h | 7 +++++++ mm/memcontrol.c | 11 +++++++++++ security/security.c | 16 ++++++++++++++++ 4 files changed, 35 insertions(+)
diff --git a/include/linux/lsm_hook_defs.h b/include/linux/lsm_hook_defs.h index 65c9609ec207..0f1138e16dbe 100644 --- a/include/linux/lsm_hook_defs.h +++ b/include/linux/lsm_hook_defs.h @@ -43,6 +43,7 @@ LSM_HOOK(int, 0, capset, struct cred *new, const struct cred *old, const kernel_cap_t *permitted) LSM_HOOK(int, 0, capable, const struct cred *cred, struct user_namespace *ns, int cap, unsigned int opts) +LSM_HOOK(int, 0, dma_buf_charge_task, const struct cred *from, const struct cred *to) LSM_HOOK(int, 0, quotactl, int cmds, int type, int id, const struct super_block *sb) LSM_HOOK(int, 0, quota_on, struct dentry *dentry) LSM_HOOK(int, 0, syslog, int type) diff --git a/include/linux/security.h b/include/linux/security.h index 153e9043058f..89a35eb07b8c 100644 --- a/include/linux/security.h +++ b/include/linux/security.h @@ -350,6 +350,7 @@ int security_capable(const struct cred *cred, struct user_namespace *ns, int cap, unsigned int opts); +int security_dma_buf_charge_task(const struct cred *from, const struct cred *to); int security_quotactl(int cmds, int type, int id, const struct super_block *sb); int security_quota_on(struct dentry *dentry); int security_syslog(int type); @@ -701,6 +702,12 @@ static inline int security_capable(const struct cred *cred, return cap_capable(cred, ns, cap, opts); }
+static inline int security_dma_buf_charge_task(const struct cred *from, + const struct cred *to) +{ + return 0; +} + static inline int security_quotactl(int cmds, int type, int id, const struct super_block *sb) { diff --git a/mm/memcontrol.c b/mm/memcontrol.c index f66254f52b99..ad195cfd1b5f 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -59,6 +59,7 @@ #include <linux/lockdep.h> #include <linux/resume_user_mode.h> #include <linux/psi.h> +#include <linux/security.h> #include <linux/seq_buf.h> #include <linux/sched/isolation.h> #include <linux/kmemleak.h> @@ -5672,11 +5673,13 @@ void mem_cgroup_flush_workqueue(void) struct mem_cgroup *mem_cgroup_get_from_fd(int fd) { struct cgroup_subsys_state *css; + const struct cred *tcred; struct mem_cgroup *memcg; struct cgroup *cgrp; struct task_struct *task; struct mm_struct *mm; unsigned int pidfd_flags; + int ret;
cgrp = cgroup_get_from_fd(fd); if (!IS_ERR(cgrp)) { @@ -5698,6 +5701,14 @@ struct mem_cgroup *mem_cgroup_get_from_fd(int fd) if (IS_ERR(task)) return ERR_CAST(task);
+ tcred = get_task_cred(task); + ret = security_dma_buf_charge_task(current_cred(), tcred); + put_cred(tcred); + if (ret) { + put_task_struct(task); + return ERR_PTR(ret); + } + mm = get_task_mm(task); put_task_struct(task); if (!mm) diff --git a/security/security.c b/security/security.c index 2ee276ab15c5..4a16e4076aab 100644 --- a/security/security.c +++ b/security/security.c @@ -660,6 +660,22 @@ int security_capable(const struct cred *cred, return call_int_hook(capable, cred, ns, cap, opts); }
+/** + * security_dma_buf_charge_task() - Check if cross-cgroup dma-heap charging is allowed + * @from: credentials of the allocating process + * @to: credentials of the process to charge + * + * Check whether the process with credentials @from is allowed to allocate + * dma-heap memory and charge it to the cgroup of the process with credentials + * @to. + * + * Return: Returns 0 if permission is granted. + */ +int security_dma_buf_charge_task(const struct cred *from, const struct cred *to) +{ + return call_int_hook(dma_buf_charge_task, from, to); +} + /** * security_quotactl() - Check if a quotactl() syscall is allowed for this fs * @cmds: commands
Similar to how security_dma_buf_charge_task adds an access-control point for pidfd for the charge_fd of a DMA_HEAP_IOCTL_ALLOC ioctl, we need a new access-control point for the case where charge_fd represents a cgroupfd.
Add security_dma_buf_charge_cgroup(), called in mem_cgroup_get_from_fd() when charge_fd refers to a cgroup. The hook receives the credentials of the requesting process (cred) and the target cgroup to charge (cgrp), giving security modules a controlled enforcement point for cross-cgroup dma-buf attribution policy.
When CONFIG_SECURITY is not set the hook compiles to an inline returning 0, adding no overhead to the fast path.
Signed-off-by: Albert Esteve aesteve@redhat.com --- include/linux/lsm_hook_defs.h | 3 +++ include/linux/security.h | 11 +++++++++++ mm/memcontrol.c | 6 ++++++ security/security.c | 18 ++++++++++++++++++ 4 files changed, 38 insertions(+)
diff --git a/include/linux/lsm_hook_defs.h b/include/linux/lsm_hook_defs.h index 0f1138e16dbe..fbf7f17127db 100644 --- a/include/linux/lsm_hook_defs.h +++ b/include/linux/lsm_hook_defs.h @@ -44,6 +44,9 @@ LSM_HOOK(int, 0, capset, struct cred *new, const struct cred *old, LSM_HOOK(int, 0, capable, const struct cred *cred, struct user_namespace *ns, int cap, unsigned int opts) LSM_HOOK(int, 0, dma_buf_charge_task, const struct cred *from, const struct cred *to) +#ifdef CONFIG_CGROUPS +LSM_HOOK(int, 0, dma_buf_charge_cgroup, const struct cred *cred, struct cgroup *cgrp) +#endif LSM_HOOK(int, 0, quotactl, int cmds, int type, int id, const struct super_block *sb) LSM_HOOK(int, 0, quota_on, struct dentry *dentry) LSM_HOOK(int, 0, syslog, int type) diff --git a/include/linux/security.h b/include/linux/security.h index 89a35eb07b8c..8fb1539156b5 100644 --- a/include/linux/security.h +++ b/include/linux/security.h @@ -351,6 +351,9 @@ int security_capable(const struct cred *cred, int cap, unsigned int opts); int security_dma_buf_charge_task(const struct cred *from, const struct cred *to); +#ifdef CONFIG_CGROUPS +int security_dma_buf_charge_cgroup(const struct cred *cred, struct cgroup *cgrp); +#endif int security_quotactl(int cmds, int type, int id, const struct super_block *sb); int security_quota_on(struct dentry *dentry); int security_syslog(int type); @@ -708,6 +711,14 @@ static inline int security_dma_buf_charge_task(const struct cred *from, return 0; }
+#ifdef CONFIG_CGROUPS +static inline int security_dma_buf_charge_cgroup(const struct cred *cred, + struct cgroup *cgrp) +{ + return 0; +} +#endif + static inline int security_quotactl(int cmds, int type, int id, const struct super_block *sb) { diff --git a/mm/memcontrol.c b/mm/memcontrol.c index ad195cfd1b5f..7b841b905759 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -5693,7 +5693,13 @@ struct mem_cgroup *mem_cgroup_get_from_fd(int fd)
memcg = mem_cgroup_from_css(css); rcu_read_unlock(); + + ret = security_dma_buf_charge_cgroup(current_cred(), cgrp); cgroup_put(cgrp); + if (ret) { + mem_cgroup_put(memcg); + return ERR_PTR(ret); + } return memcg; }
diff --git a/security/security.c b/security/security.c index 4a16e4076aab..a5dcf9dc4963 100644 --- a/security/security.c +++ b/security/security.c @@ -676,6 +676,24 @@ int security_dma_buf_charge_task(const struct cred *from, const struct cred *to) return call_int_hook(dma_buf_charge_task, from, to); }
+#ifdef CONFIG_CGROUPS +/** + * security_dma_buf_charge_cgroup() - Check if charging dma-buf memory to a cgroup is allowed + * @cred: credentials of the process requesting the charge + * @cgrp: target cgroup to charge + * + * Check whether the process with credentials @cred is allowed to charge + * dma-buf memory to @cgrp. Called when userspace passes a cgroupfd to + * identify the charge target instead of a pidfd. + * + * Return: Returns 0 if permission is granted. + */ +int security_dma_buf_charge_cgroup(const struct cred *cred, struct cgroup *cgrp) +{ + return call_int_hook(dma_buf_charge_cgroup, cred, cgrp); +} +#endif + /** * security_quotactl() - Check if a quotactl() syscall is allowed for this fs * @cmds: commands
The security_dma_buf_charge_task() hook allows security modules to control which processes may charge dma-buf allocations to another process's cgroup via the charge_fd field of DMA_HEAP_IOCTL_ALLOC. Without a policy implementation, the hook is a no-op and the restriction is not enforced.
On SELinux-managed systems any domain with access to a dma-heap device node can therefore exhaust another cgroup's memory budget without restriction.
Implement selinux_dma_buf_charge_task() using avc_has_perm() with a new dma_buf object class and a charge_task permission. Policy authors can then grant cross-cgroup charging selectively, for example:
allow allocator_app_t client_app_t:dma_buf charge_task;
Signed-off-by: Albert Esteve aesteve@redhat.com --- security/selinux/hooks.c | 7 +++++++ security/selinux/include/classmap.h | 1 + 2 files changed, 8 insertions(+)
diff --git a/security/selinux/hooks.c b/security/selinux/hooks.c index 30776b0107f2..8f39d91eabc9 100644 --- a/security/selinux/hooks.c +++ b/security/selinux/hooks.c @@ -2204,6 +2204,12 @@ static int selinux_capable(const struct cred *cred, struct user_namespace *ns, return cred_has_capability(cred, cap, opts, ns == &init_user_ns); }
+static int selinux_dma_buf_charge_task(const struct cred *from, const struct cred *to) +{ + return avc_has_perm(cred_sid(from), cred_sid(to), + SECCLASS_DMA_BUF, DMA_BUF__CHARGE_TASK, NULL); +} + static int selinux_quotactl(int cmds, int type, int id, const struct super_block *sb) { const struct cred *cred = current_cred(); @@ -7674,6 +7680,7 @@ static struct security_hook_list selinux_hooks[] __ro_after_init = { LSM_HOOK_INIT(capget, selinux_capget), LSM_HOOK_INIT(capset, selinux_capset), LSM_HOOK_INIT(capable, selinux_capable), + LSM_HOOK_INIT(dma_buf_charge_task, selinux_dma_buf_charge_task), LSM_HOOK_INIT(quotactl, selinux_quotactl), LSM_HOOK_INIT(quota_on, selinux_quota_on), LSM_HOOK_INIT(syslog, selinux_syslog), diff --git a/security/selinux/include/classmap.h b/security/selinux/include/classmap.h index 90cb61b16425..b4ebedc6e424 100644 --- a/security/selinux/include/classmap.h +++ b/security/selinux/include/classmap.h @@ -181,6 +181,7 @@ const struct security_class_mapping secclass_map[] = { { "user_namespace", { "create", NULL } }, { "memfd_file", { COMMON_FILE_PERMS, "execute_no_trans", "entrypoint", NULL } }, + { "dma_buf", { "charge_task", NULL } }, /* last one */ { NULL, {} } };
The security_dma_buf_charge_cgroup() hook allows security modules to control which processes may charge dma-buf to an arbitrary cgroup via cgroupfd passed in charge_fd. Without a policy implementation, the hook is a no-op and any process that can open a cgroup directory (effectively any process) can charge memory to it.
Implement selinux_dma_buf_charge_cgroup() using avc_has_perm() with the dma_buf object class and a charge_cgroup permission. The target SID is derived from the SELinux label on the cgroup's kernfs directory, so access control follows the cgroup filesystem labeling. Policy authors can then grant cross-cgroup charging selectively, for example:
allow allocator_app_t cgroup_t:dma_buf charge_cgroup;
The hook matches the fail-open semantics of selinux_kernfs_init_security(). This means unlabeled cgroups are unrestricted by default.
Signed-off-by: Albert Esteve aesteve@redhat.com --- security/selinux/hooks.c | 37 +++++++++++++++++++++++++++++++++++++ security/selinux/include/classmap.h | 2 +- 2 files changed, 38 insertions(+), 1 deletion(-)
diff --git a/security/selinux/hooks.c b/security/selinux/hooks.c index 8f39d91eabc9..2f99eaa910af 100644 --- a/security/selinux/hooks.c +++ b/security/selinux/hooks.c @@ -2210,6 +2210,40 @@ static int selinux_dma_buf_charge_task(const struct cred *from, const struct cre SECCLASS_DMA_BUF, DMA_BUF__CHARGE_TASK, NULL); }
+#ifdef CONFIG_CGROUPS +static int selinux_dma_buf_charge_cgroup(const struct cred *cred, struct cgroup *cgrp) +{ + u32 cgrp_sid, clen; + int rc; + char *context; + + rc = kernfs_xattr_get(cgrp->kn, XATTR_NAME_SELINUX, NULL, 0); + if (rc == -ENODATA) + return 0; + else if (rc < 0) + return rc; + + clen = (u32)rc; + context = kmalloc(clen, GFP_KERNEL); + if (!context) + return -ENOMEM; + + rc = kernfs_xattr_get(cgrp->kn, XATTR_NAME_SELINUX, context, clen); + if (rc < 0) { + kfree(context); + return rc; + } + + rc = security_context_to_sid(context, rc, &cgrp_sid, GFP_KERNEL); + kfree(context); + if (rc) + return rc; + + return avc_has_perm(cred_sid(cred), cgrp_sid, + SECCLASS_DMA_BUF, DMA_BUF__CHARGE_CGROUP, NULL); +} +#endif + static int selinux_quotactl(int cmds, int type, int id, const struct super_block *sb) { const struct cred *cred = current_cred(); @@ -7681,6 +7715,9 @@ static struct security_hook_list selinux_hooks[] __ro_after_init = { LSM_HOOK_INIT(capset, selinux_capset), LSM_HOOK_INIT(capable, selinux_capable), LSM_HOOK_INIT(dma_buf_charge_task, selinux_dma_buf_charge_task), +#ifdef CONFIG_CGROUPS + LSM_HOOK_INIT(dma_buf_charge_cgroup, selinux_dma_buf_charge_cgroup), +#endif LSM_HOOK_INIT(quotactl, selinux_quotactl), LSM_HOOK_INIT(quota_on, selinux_quota_on), LSM_HOOK_INIT(syslog, selinux_syslog), diff --git a/security/selinux/include/classmap.h b/security/selinux/include/classmap.h index b4ebedc6e424..90d1e04c3d27 100644 --- a/security/selinux/include/classmap.h +++ b/security/selinux/include/classmap.h @@ -181,7 +181,7 @@ const struct security_class_mapping secclass_map[] = { { "user_namespace", { "create", NULL } }, { "memfd_file", { COMMON_FILE_PERMS, "execute_no_trans", "entrypoint", NULL } }, - { "dma_buf", { "charge_task", NULL } }, + { "dma_buf", { "charge_task", "charge_cgroup", NULL } }, /* last one */ { NULL, {} } };
dma-heap already supports charging a specific cgroup at allocation time via charge_fd in DMA_HEAP_IOCTL_ALLOC. However, dma-bufs can originate from any exporter (DRM, V4L2, etc.), and the allocator may not know the final owner at allocation time.
Add DMA_BUF_IOCTL_XFER_CHARGE to transfer a dma-buf's memcg charge to a different cgroup after allocation. The target is identified by charge_fd, which may be a cgroupfd or a pidfd.
The transfer charges the new cgroup before uncharging the old one, preserving the original charge on failure. An atomic cmpxchg on dmabuf->memcg prevents races with concurrent transfers.
Not all dma-buf exporters back their buffers with reclaimable system memory. Some represent device registers, VRAM, or carved-out pools where size is not a meaningful memory cost. Add a memcg_account flag to dma_buf_export_info that exporters set when their buffers should participate in memcg accounting. The xfer ioctl returns -EOPNOTSUPP for buffers that do not set the flag. Currently only the system heap sets it, other exporters can opt in as needed.
Usage example: int cg_fd = open("/sys/fs/cgroup/clients/app1", O_RDONLY | O_DIRECTORY);
struct dma_buf_transfer_charge xfer = { .charge_fd = cg_fd, }; ioctl(dmabuf_fd, DMA_BUF_IOCTL_XFER_CHARGE, &xfer); /* charge now on app1's cgroup */ close(cg_fd);
Signed-off-by: Albert Esteve aesteve@redhat.com --- drivers/dma-buf/dma-buf.c | 43 +++++++++++++++++++++++++++++++++++++ drivers/dma-buf/heaps/system_heap.c | 1 + include/linux/dma-buf.h | 5 +++++ include/linux/memcontrol.h | 9 ++++++++ include/uapi/linux/dma-buf.h | 26 ++++++++++++++++++++++ mm/memcontrol.c | 41 +++++++++++++++++++++++++++++++++++ 6 files changed, 125 insertions(+)
diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c index f31f49ebd0af..07513ecedc10 100644 --- a/drivers/dma-buf/dma-buf.c +++ b/drivers/dma-buf/dma-buf.c @@ -540,6 +540,45 @@ static long dma_buf_import_sync_file(struct dma_buf *dmabuf, } #endif
+/* Transfer a dma-buf's memcg charge; see struct dma_buf_transfer_charge */ +static long dma_buf_xfer_charge(struct dma_buf *dmabuf, + const void __user *user_data) +{ + struct dma_buf_transfer_charge arg; + struct mem_cgroup *memcg; + int ret = 0; + + if (copy_from_user(&arg, user_data, sizeof(arg))) + return -EFAULT; + + if (!IS_ENABLED(CONFIG_MEMCG)) + return 0; + + if (arg.flags != 0) + return -EINVAL; + + if (!dmabuf->memcg_account) + return -EOPNOTSUPP; + + memcg = mem_cgroup_get_from_fd(arg.charge_fd); + if (IS_ERR(memcg)) + return PTR_ERR(memcg); + + if (dmabuf->memcg == memcg) + goto skip_transfer; + + if (!mem_cgroup_transfer_dmabuf_charge(dmabuf, memcg)) { + ret = -ENOMEM; + goto skip_transfer; + } + + return 0; + +skip_transfer: + mem_cgroup_put(memcg); + return ret; +} + static long dma_buf_ioctl(struct file *file, unsigned int cmd, unsigned long arg) { @@ -590,6 +629,9 @@ static long dma_buf_ioctl(struct file *file, return dma_buf_import_sync_file(dmabuf, (const void __user *)arg); #endif
+ case DMA_BUF_IOCTL_XFER_CHARGE: + return dma_buf_xfer_charge(dmabuf, (const void __user *)arg); + default: return -ENOTTY; } @@ -750,6 +792,7 @@ struct dma_buf *dma_buf_export(const struct dma_buf_export_info *exp_info) dmabuf->size = exp_info->size; dmabuf->exp_name = exp_info->exp_name; dmabuf->owner = exp_info->owner; + dmabuf->memcg_account = exp_info->memcg_account; spin_lock_init(&dmabuf->name_lock); init_waitqueue_head(&dmabuf->poll); dmabuf->cb_in.poll = dmabuf->cb_out.poll = &dmabuf->poll; diff --git a/drivers/dma-buf/heaps/system_heap.c b/drivers/dma-buf/heaps/system_heap.c index adeea8cd328b..a764167167ba 100644 --- a/drivers/dma-buf/heaps/system_heap.c +++ b/drivers/dma-buf/heaps/system_heap.c @@ -479,6 +479,7 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap, exp_info.size = buffer->len; exp_info.flags = fd_flags; exp_info.priv = buffer; + exp_info.memcg_account = true; dmabuf = dma_buf_export(&exp_info); if (IS_ERR(dmabuf)) { ret = PTR_ERR(dmabuf); diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h index 23ec0ff23fba..ac0ec88cab93 100644 --- a/include/linux/dma-buf.h +++ b/include/linux/dma-buf.h @@ -433,6 +433,9 @@ struct dma_buf {
/** @memcg: the cgroup to which this buffer is currently attributed */ struct mem_cgroup *memcg; + + /** @memcg_account: account buffer memory to cgroup */ + bool memcg_account; };
/** @@ -505,6 +508,7 @@ struct dma_buf_attachment { * @flags: mode flags for the file * @resv: reservation-object, NULL to allocate default one * @priv: Attach private data of allocator to this buffer + * @memcg_account: account buffer memory to cgroup * * This structure holds the information required to export the buffer. Used * with dma_buf_export() only. @@ -517,6 +521,7 @@ struct dma_buf_export_info { int flags; struct dma_resv *resv; void *priv; + bool memcg_account; };
/** diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 6b30baa79fa9..36fddafbef4e 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -679,6 +679,9 @@ bool mem_cgroup_reserve_dmabuf(struct mem_cgroup *memcg, gfp_t gfp, void mem_cgroup_commit_dmabuf(struct dma_buf *dmabuf, struct mem_cgroup *memcg, unsigned int nr_pages);
+bool mem_cgroup_transfer_dmabuf_charge(struct dma_buf *dmabuf, + struct mem_cgroup *new_memcg); + void __mem_cgroup_uncharge(struct folio *folio);
/** @@ -1206,6 +1209,12 @@ static inline void mem_cgroup_cancel_dmabuf(struct mem_cgroup *memcg, { }
+static inline bool mem_cgroup_transfer_dmabuf_charge(struct dma_buf *dmabuf, + struct mem_cgroup *new_memcg) +{ + return true; +} + static inline void mem_cgroup_uncharge(struct folio *folio) { } diff --git a/include/uapi/linux/dma-buf.h b/include/uapi/linux/dma-buf.h index e827c9d20c5d..ea957f4b1d38 100644 --- a/include/uapi/linux/dma-buf.h +++ b/include/uapi/linux/dma-buf.h @@ -168,6 +168,31 @@ struct dma_buf_import_sync_file { __s32 fd; };
+/** + * struct dma_buf_transfer_charge - Transfer a dma-buf's memcg charge + * + * Any holder of a dma-buf file descriptor can perform a + * DMA_BUF_IOCTL_XFER_CHARGE to attribute the buffer's memory + * cost to a specific cgroup, or move it from one cgroup to + * another. The ioctl charges the new cgroup before uncharging + * the old one atomically. On failure the original charge is + * preserved. If the buffer has no existing charge, a new + * charge is established. + */ +struct dma_buf_transfer_charge { + /** @flags: Reserved, must be zero */ + __u32 flags; /* unused for now */ + /** + * @charge_fd: File descriptor identifying the target cgroup. + * + * May be either a pidfd, in which case the charge moves to the + * memory cgroup of the referenced process, or a cgroupfd obtained + * by opening a cgroup v2 directory, in which case the charge moves + * directly to that cgroup's memory controller. + */ + __u32 charge_fd; +}; + #define DMA_BUF_BASE 'b' #define DMA_BUF_IOCTL_SYNC _IOW(DMA_BUF_BASE, 0, struct dma_buf_sync)
@@ -179,5 +204,6 @@ struct dma_buf_import_sync_file { #define DMA_BUF_SET_NAME_B _IOW(DMA_BUF_BASE, 1, __u64) #define DMA_BUF_IOCTL_EXPORT_SYNC_FILE _IOWR(DMA_BUF_BASE, 2, struct dma_buf_export_sync_file) #define DMA_BUF_IOCTL_IMPORT_SYNC_FILE _IOW(DMA_BUF_BASE, 3, struct dma_buf_import_sync_file) +#define DMA_BUF_IOCTL_XFER_CHARGE _IOW(DMA_BUF_BASE, 4, struct dma_buf_transfer_charge)
#endif diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 7b841b905759..3adc84fe6668 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -5786,6 +5786,47 @@ void mem_cgroup_cancel_dmabuf(struct mem_cgroup *memcg, unsigned int nr_pages) refill_stock(memcg, nr_pages); }
+/** + * mem_cgroup_transfer_dmabuf_charge - move a dma-buf charge + * @dmabuf: buffer to transfer charge + * @new_memcg: target memcg; caller must hold a reference + * + * Transfers the charge for @dmabuf to @new_memcg and, on + * success, stores @new_memcg in @dmabuf->memcg. + * Returns %true on success, %false if the charge against @new_memcg + * failed or a concurrent transfer raced on @dmabuf->memcg. + * On failure the caller retains its reference to @new_memcg. + */ +bool mem_cgroup_transfer_dmabuf_charge(struct dma_buf *dmabuf, + struct mem_cgroup *new_memcg) +{ + struct mem_cgroup *old_memcg = READ_ONCE(dmabuf->memcg); + unsigned int nr_pages = PAGE_ALIGN(dmabuf->size) >> PAGE_SHIFT; + + if (mem_cgroup_disabled()) + return true; + + if (!mem_cgroup_reserve_dmabuf(new_memcg, GFP_KERNEL, nr_pages)) + return false; + + mod_memcg_state(new_memcg, MEMCG_DMABUF, nr_pages); + + if (cmpxchg(&dmabuf->memcg, old_memcg, new_memcg) != old_memcg) { + mod_memcg_state(new_memcg, MEMCG_DMABUF, -nr_pages); + mem_cgroup_cancel_dmabuf(new_memcg, nr_pages); + return false; + } + + if (old_memcg) { + /* Uncharge old cgroup */ + mod_memcg_state(old_memcg, MEMCG_DMABUF, -nr_pages); + mem_cgroup_cancel_dmabuf(old_memcg, nr_pages); + mem_cgroup_put(old_memcg); + } + + return true; +} + /** * mem_cgroup_uncharge_dmabuf - uncharge a dma-buf from its memcg * @dmabuf: buffer to uncharge
On 10/27/26 16:02, Albert Esteve wrote:
dma-heap already supports charging a specific cgroup at allocation time via charge_fd in DMA_HEAP_IOCTL_ALLOC. However, dma-bufs can originate from any exporter (DRM, V4L2, etc.), and the allocator may not know the final owner at allocation time.
Add DMA_BUF_IOCTL_XFER_CHARGE to transfer a dma-buf's memcg charge to a different cgroup after allocation. The target is identified by charge_fd, which may be a cgroupfd or a pidfd.
The transfer charges the new cgroup before uncharging the old one, preserving the original charge on failure. An atomic cmpxchg on dmabuf->memcg prevents races with concurrent transfers.
Not all dma-buf exporters back their buffers with reclaimable system memory. Some represent device registers, VRAM, or carved-out pools where size is not a meaningful memory cost. Add a memcg_account flag to dma_buf_export_info that exporters set when their buffers should participate in memcg accounting. The xfer ioctl returns -EOPNOTSUPP for buffers that do not set the flag. Currently only the system heap sets it, other exporters can opt in as needed.
Usage example: int cg_fd = open("/sys/fs/cgroup/clients/app1", O_RDONLY | O_DIRECTORY);
struct dma_buf_transfer_charge xfer = { .charge_fd = cg_fd, }; ioctl(dmabuf_fd, DMA_BUF_IOCTL_XFER_CHARGE, &xfer); /* charge now on app1's cgroup */ close(cg_fd);
Signed-off-by: Albert Esteve aesteve@redhat.com
drivers/dma-buf/dma-buf.c | 43 +++++++++++++++++++++++++++++++++++++ drivers/dma-buf/heaps/system_heap.c | 1 + include/linux/dma-buf.h | 5 +++++ include/linux/memcontrol.h | 9 ++++++++ include/uapi/linux/dma-buf.h | 26 ++++++++++++++++++++++ mm/memcontrol.c | 41 +++++++++++++++++++++++++++++++++++ 6 files changed, 125 insertions(+)
diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c index f31f49ebd0af..07513ecedc10 100644 --- a/drivers/dma-buf/dma-buf.c +++ b/drivers/dma-buf/dma-buf.c @@ -540,6 +540,45 @@ static long dma_buf_import_sync_file(struct dma_buf *dmabuf, } #endif +/* Transfer a dma-buf's memcg charge; see struct dma_buf_transfer_charge */ +static long dma_buf_xfer_charge(struct dma_buf *dmabuf,
const void __user *user_data)+{
- struct dma_buf_transfer_charge arg;
- struct mem_cgroup *memcg;
- int ret = 0;
- if (copy_from_user(&arg, user_data, sizeof(arg)))
return -EFAULT;- if (!IS_ENABLED(CONFIG_MEMCG))
return 0;- if (arg.flags != 0)
return -EINVAL;- if (!dmabuf->memcg_account)
return -EOPNOTSUPP;- memcg = mem_cgroup_get_from_fd(arg.charge_fd);
- if (IS_ERR(memcg))
return PTR_ERR(memcg);- if (dmabuf->memcg == memcg)
goto skip_transfer;- if (!mem_cgroup_transfer_dmabuf_charge(dmabuf, memcg)) {
ret = -ENOMEM;goto skip_transfer;- }
- return 0;
+skip_transfer:
- mem_cgroup_put(memcg);
- return ret;
+}
static long dma_buf_ioctl(struct file *file, unsigned int cmd, unsigned long arg) { @@ -590,6 +629,9 @@ static long dma_buf_ioctl(struct file *file, return dma_buf_import_sync_file(dmabuf, (const void __user *)arg); #endif
- case DMA_BUF_IOCTL_XFER_CHARGE:
return dma_buf_xfer_charge(dmabuf, (const void __user *)arg);- default: return -ENOTTY; }
@@ -750,6 +792,7 @@ struct dma_buf *dma_buf_export(const struct dma_buf_export_info *exp_info) dmabuf->size = exp_info->size; dmabuf->exp_name = exp_info->exp_name; dmabuf->owner = exp_info->owner;
- dmabuf->memcg_account = exp_info->memcg_account;
The initial charging should probably be done here as well.
This way DMA-buf heaps only needs to pass in the charge_fd and we can re-use the functionality for other DMA-buf exporters as well.
spin_lock_init(&dmabuf->name_lock); init_waitqueue_head(&dmabuf->poll); dmabuf->cb_in.poll = dmabuf->cb_out.poll = &dmabuf->poll; diff --git a/drivers/dma-buf/heaps/system_heap.c b/drivers/dma-buf/heaps/system_heap.c index adeea8cd328b..a764167167ba 100644 --- a/drivers/dma-buf/heaps/system_heap.c +++ b/drivers/dma-buf/heaps/system_heap.c @@ -479,6 +479,7 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap, exp_info.size = buffer->len; exp_info.flags = fd_flags; exp_info.priv = buffer;
- exp_info.memcg_account = true; dmabuf = dma_buf_export(&exp_info); if (IS_ERR(dmabuf)) { ret = PTR_ERR(dmabuf);
diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h index 23ec0ff23fba..ac0ec88cab93 100644 --- a/include/linux/dma-buf.h +++ b/include/linux/dma-buf.h @@ -433,6 +433,9 @@ struct dma_buf { /** @memcg: the cgroup to which this buffer is currently attributed */ struct mem_cgroup *memcg;
- /** @memcg_account: account buffer memory to cgroup */
- bool memcg_account;
}; /** @@ -505,6 +508,7 @@ struct dma_buf_attachment {
- @flags: mode flags for the file
- @resv: reservation-object, NULL to allocate default one
- @priv: Attach private data of allocator to this buffer
- @memcg_account: account buffer memory to cgroup
- This structure holds the information required to export the buffer. Used
- with dma_buf_export() only.
@@ -517,6 +521,7 @@ struct dma_buf_export_info { int flags; struct dma_resv *resv; void *priv;
- bool memcg_account;
}; /** diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 6b30baa79fa9..36fddafbef4e 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -679,6 +679,9 @@ bool mem_cgroup_reserve_dmabuf(struct mem_cgroup *memcg, gfp_t gfp, void mem_cgroup_commit_dmabuf(struct dma_buf *dmabuf, struct mem_cgroup *memcg, unsigned int nr_pages); +bool mem_cgroup_transfer_dmabuf_charge(struct dma_buf *dmabuf,
struct mem_cgroup *new_memcg);void __mem_cgroup_uncharge(struct folio *folio); /** @@ -1206,6 +1209,12 @@ static inline void mem_cgroup_cancel_dmabuf(struct mem_cgroup *memcg, { } +static inline bool mem_cgroup_transfer_dmabuf_charge(struct dma_buf *dmabuf,
struct mem_cgroup *new_memcg)+{
- return true;
+}
static inline void mem_cgroup_uncharge(struct folio *folio) { } diff --git a/include/uapi/linux/dma-buf.h b/include/uapi/linux/dma-buf.h index e827c9d20c5d..ea957f4b1d38 100644 --- a/include/uapi/linux/dma-buf.h +++ b/include/uapi/linux/dma-buf.h @@ -168,6 +168,31 @@ struct dma_buf_import_sync_file { __s32 fd; }; +/**
- struct dma_buf_transfer_charge - Transfer a dma-buf's memcg charge
- Any holder of a dma-buf file descriptor can perform a
- DMA_BUF_IOCTL_XFER_CHARGE to attribute the buffer's memory
- cost to a specific cgroup, or move it from one cgroup to
- another. The ioctl charges the new cgroup before uncharging
- the old one atomically. On failure the original charge is
- preserved. If the buffer has no existing charge, a new
- charge is established.
- */
+struct dma_buf_transfer_charge {
- /** @flags: Reserved, must be zero */
- __u32 flags; /* unused for now */
- /**
* @charge_fd: File descriptor identifying the target cgroup.** May be either a pidfd, in which case the charge moves to the* memory cgroup of the referenced process, or a cgroupfd obtained* by opening a cgroup v2 directory, in which case the charge moves* directly to that cgroup's memory controller.*/- __u32 charge_fd;
+};
#define DMA_BUF_BASE 'b' #define DMA_BUF_IOCTL_SYNC _IOW(DMA_BUF_BASE, 0, struct dma_buf_sync) @@ -179,5 +204,6 @@ struct dma_buf_import_sync_file { #define DMA_BUF_SET_NAME_B _IOW(DMA_BUF_BASE, 1, __u64) #define DMA_BUF_IOCTL_EXPORT_SYNC_FILE _IOWR(DMA_BUF_BASE, 2, struct dma_buf_export_sync_file) #define DMA_BUF_IOCTL_IMPORT_SYNC_FILE _IOW(DMA_BUF_BASE, 3, struct dma_buf_import_sync_file) +#define DMA_BUF_IOCTL_XFER_CHARGE _IOW(DMA_BUF_BASE, 4, struct dma_buf_transfer_charge) #endif diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 7b841b905759..3adc84fe6668 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -5786,6 +5786,47 @@ void mem_cgroup_cancel_dmabuf(struct mem_cgroup *memcg, unsigned int nr_pages) refill_stock(memcg, nr_pages); } +/**
- mem_cgroup_transfer_dmabuf_charge - move a dma-buf charge
- @dmabuf: buffer to transfer charge
- @new_memcg: target memcg; caller must hold a reference
- Transfers the charge for @dmabuf to @new_memcg and, on
- success, stores @new_memcg in @dmabuf->memcg.
- Returns %true on success, %false if the charge against @new_memcg
- failed or a concurrent transfer raced on @dmabuf->memcg.
- On failure the caller retains its reference to @new_memcg.
- */
+bool mem_cgroup_transfer_dmabuf_charge(struct dma_buf *dmabuf,
struct mem_cgroup *new_memcg)
I would make that more generic instead of dmabuf specific. E.g. something like this:
bool mem_cgroup_transfer_charge(struct struct mem_cgroup **ptr_memcg, struct mem_cgroup *new_memcg, enum memcg_stat_item item, unsigned int nr_pages)
+{
- struct mem_cgroup *old_memcg = READ_ONCE(dmabuf->memcg);
- unsigned int nr_pages = PAGE_ALIGN(dmabuf->size) >> PAGE_SHIFT;
This way we can also document in the DMA-buf caller that this conversation must be ok when memcg_account is true.
Regards, Christian.
- if (mem_cgroup_disabled())
return true;- if (!mem_cgroup_reserve_dmabuf(new_memcg, GFP_KERNEL, nr_pages))
return false;- mod_memcg_state(new_memcg, MEMCG_DMABUF, nr_pages);
- if (cmpxchg(&dmabuf->memcg, old_memcg, new_memcg) != old_memcg) {
mod_memcg_state(new_memcg, MEMCG_DMABUF, -nr_pages);mem_cgroup_cancel_dmabuf(new_memcg, nr_pages);return false;- }
- if (old_memcg) {
/* Uncharge old cgroup */mod_memcg_state(old_memcg, MEMCG_DMABUF, -nr_pages);mem_cgroup_cancel_dmabuf(old_memcg, nr_pages);mem_cgroup_put(old_memcg);- }
- return true;
+}
/**
- mem_cgroup_uncharge_dmabuf - uncharge a dma-buf from its memcg
- @dmabuf: buffer to uncharge
On Wed, Oct 28, 2026 at 10:08:19AM +0100, Christian König wrote:
On 10/27/26 16:02, Albert Esteve wrote:
dma-heap already supports charging a specific cgroup at allocation time via charge_fd in DMA_HEAP_IOCTL_ALLOC. However, dma-bufs can originate from any exporter (DRM, V4L2, etc.), and the allocator may not know the final owner at allocation time.
Add DMA_BUF_IOCTL_XFER_CHARGE to transfer a dma-buf's memcg charge to a different cgroup after allocation. The target is identified by charge_fd, which may be a cgroupfd or a pidfd.
The transfer charges the new cgroup before uncharging the old one, preserving the original charge on failure. An atomic cmpxchg on dmabuf->memcg prevents races with concurrent transfers.
Not all dma-buf exporters back their buffers with reclaimable system memory. Some represent device registers, VRAM, or carved-out pools where size is not a meaningful memory cost. Add a memcg_account flag to dma_buf_export_info that exporters set when their buffers should participate in memcg accounting. The xfer ioctl returns -EOPNOTSUPP for buffers that do not set the flag. Currently only the system heap sets it, other exporters can opt in as needed.
Usage example: int cg_fd = open("/sys/fs/cgroup/clients/app1", O_RDONLY | O_DIRECTORY);
struct dma_buf_transfer_charge xfer = { .charge_fd = cg_fd, }; ioctl(dmabuf_fd, DMA_BUF_IOCTL_XFER_CHARGE, &xfer); /* charge now on app1's cgroup */ close(cg_fd);
Signed-off-by: Albert Esteve aesteve@redhat.com
drivers/dma-buf/dma-buf.c | 43 +++++++++++++++++++++++++++++++++++++ drivers/dma-buf/heaps/system_heap.c | 1 + include/linux/dma-buf.h | 5 +++++ include/linux/memcontrol.h | 9 ++++++++ include/uapi/linux/dma-buf.h | 26 ++++++++++++++++++++++ mm/memcontrol.c | 41 +++++++++++++++++++++++++++++++++++ 6 files changed, 125 insertions(+)
diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c index f31f49ebd0af..07513ecedc10 100644 --- a/drivers/dma-buf/dma-buf.c +++ b/drivers/dma-buf/dma-buf.c @@ -540,6 +540,45 @@ static long dma_buf_import_sync_file(struct dma_buf *dmabuf, } #endif +/* Transfer a dma-buf's memcg charge; see struct dma_buf_transfer_charge */ +static long dma_buf_xfer_charge(struct dma_buf *dmabuf,
const void __user *user_data)+{
- struct dma_buf_transfer_charge arg;
- struct mem_cgroup *memcg;
- int ret = 0;
- if (copy_from_user(&arg, user_data, sizeof(arg)))
return -EFAULT;- if (!IS_ENABLED(CONFIG_MEMCG))
return 0;- if (arg.flags != 0)
return -EINVAL;- if (!dmabuf->memcg_account)
return -EOPNOTSUPP;- memcg = mem_cgroup_get_from_fd(arg.charge_fd);
- if (IS_ERR(memcg))
return PTR_ERR(memcg);- if (dmabuf->memcg == memcg)
goto skip_transfer;- if (!mem_cgroup_transfer_dmabuf_charge(dmabuf, memcg)) {
ret = -ENOMEM;goto skip_transfer;- }
- return 0;
+skip_transfer:
- mem_cgroup_put(memcg);
- return ret;
+}
static long dma_buf_ioctl(struct file *file, unsigned int cmd, unsigned long arg) { @@ -590,6 +629,9 @@ static long dma_buf_ioctl(struct file *file, return dma_buf_import_sync_file(dmabuf, (const void __user *)arg); #endif
- case DMA_BUF_IOCTL_XFER_CHARGE:
return dma_buf_xfer_charge(dmabuf, (const void __user *)arg);- default: return -ENOTTY; }
@@ -750,6 +792,7 @@ struct dma_buf *dma_buf_export(const struct dma_buf_export_info *exp_info) dmabuf->size = exp_info->size; dmabuf->exp_name = exp_info->exp_name; dmabuf->owner = exp_info->owner;
- dmabuf->memcg_account = exp_info->memcg_account;
The initial charging should probably be done here as well.
This way DMA-buf heaps only needs to pass in the charge_fd and we can re-use the functionality for other DMA-buf exporters as well.
The problem with doing the charge here is that it happens too late. That is the reasoning for this proposal to move all the charging logic into heap allocators before pages are allocated.
What we could do is fill the dmabuf->memcg field when the flag is set (i.e., committing into the dmabuf in the patch semantics). But for that we'd need adding the memcg we want to charge it to into exp_info in order to support charging to cgroups other than the allocator's (for the common case would be simpler as we can just look at current). Maybe that's what you meant?
spin_lock_init(&dmabuf->name_lock); init_waitqueue_head(&dmabuf->poll); dmabuf->cb_in.poll = dmabuf->cb_out.poll = &dmabuf->poll; diff --git a/drivers/dma-buf/heaps/system_heap.c b/drivers/dma-buf/heaps/system_heap.c index adeea8cd328b..a764167167ba 100644 --- a/drivers/dma-buf/heaps/system_heap.c +++ b/drivers/dma-buf/heaps/system_heap.c @@ -479,6 +479,7 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap, exp_info.size = buffer->len; exp_info.flags = fd_flags; exp_info.priv = buffer;
- exp_info.memcg_account = true; dmabuf = dma_buf_export(&exp_info); if (IS_ERR(dmabuf)) { ret = PTR_ERR(dmabuf);
diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h index 23ec0ff23fba..ac0ec88cab93 100644 --- a/include/linux/dma-buf.h +++ b/include/linux/dma-buf.h @@ -433,6 +433,9 @@ struct dma_buf { /** @memcg: the cgroup to which this buffer is currently attributed */ struct mem_cgroup *memcg;
- /** @memcg_account: account buffer memory to cgroup */
- bool memcg_account;
}; /** @@ -505,6 +508,7 @@ struct dma_buf_attachment {
- @flags: mode flags for the file
- @resv: reservation-object, NULL to allocate default one
- @priv: Attach private data of allocator to this buffer
- @memcg_account: account buffer memory to cgroup
- This structure holds the information required to export the buffer. Used
- with dma_buf_export() only.
@@ -517,6 +521,7 @@ struct dma_buf_export_info { int flags; struct dma_resv *resv; void *priv;
- bool memcg_account;
}; /** diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 6b30baa79fa9..36fddafbef4e 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -679,6 +679,9 @@ bool mem_cgroup_reserve_dmabuf(struct mem_cgroup *memcg, gfp_t gfp, void mem_cgroup_commit_dmabuf(struct dma_buf *dmabuf, struct mem_cgroup *memcg, unsigned int nr_pages); +bool mem_cgroup_transfer_dmabuf_charge(struct dma_buf *dmabuf,
struct mem_cgroup *new_memcg);void __mem_cgroup_uncharge(struct folio *folio); /** @@ -1206,6 +1209,12 @@ static inline void mem_cgroup_cancel_dmabuf(struct mem_cgroup *memcg, { } +static inline bool mem_cgroup_transfer_dmabuf_charge(struct dma_buf *dmabuf,
struct mem_cgroup *new_memcg)+{
- return true;
+}
static inline void mem_cgroup_uncharge(struct folio *folio) { } diff --git a/include/uapi/linux/dma-buf.h b/include/uapi/linux/dma-buf.h index e827c9d20c5d..ea957f4b1d38 100644 --- a/include/uapi/linux/dma-buf.h +++ b/include/uapi/linux/dma-buf.h @@ -168,6 +168,31 @@ struct dma_buf_import_sync_file { __s32 fd; }; +/**
- struct dma_buf_transfer_charge - Transfer a dma-buf's memcg charge
- Any holder of a dma-buf file descriptor can perform a
- DMA_BUF_IOCTL_XFER_CHARGE to attribute the buffer's memory
- cost to a specific cgroup, or move it from one cgroup to
- another. The ioctl charges the new cgroup before uncharging
- the old one atomically. On failure the original charge is
- preserved. If the buffer has no existing charge, a new
- charge is established.
- */
+struct dma_buf_transfer_charge {
- /** @flags: Reserved, must be zero */
- __u32 flags; /* unused for now */
- /**
* @charge_fd: File descriptor identifying the target cgroup.** May be either a pidfd, in which case the charge moves to the* memory cgroup of the referenced process, or a cgroupfd obtained* by opening a cgroup v2 directory, in which case the charge moves* directly to that cgroup's memory controller.*/- __u32 charge_fd;
+};
#define DMA_BUF_BASE 'b' #define DMA_BUF_IOCTL_SYNC _IOW(DMA_BUF_BASE, 0, struct dma_buf_sync) @@ -179,5 +204,6 @@ struct dma_buf_import_sync_file { #define DMA_BUF_SET_NAME_B _IOW(DMA_BUF_BASE, 1, __u64) #define DMA_BUF_IOCTL_EXPORT_SYNC_FILE _IOWR(DMA_BUF_BASE, 2, struct dma_buf_export_sync_file) #define DMA_BUF_IOCTL_IMPORT_SYNC_FILE _IOW(DMA_BUF_BASE, 3, struct dma_buf_import_sync_file) +#define DMA_BUF_IOCTL_XFER_CHARGE _IOW(DMA_BUF_BASE, 4, struct dma_buf_transfer_charge) #endif diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 7b841b905759..3adc84fe6668 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -5786,6 +5786,47 @@ void mem_cgroup_cancel_dmabuf(struct mem_cgroup *memcg, unsigned int nr_pages) refill_stock(memcg, nr_pages); } +/**
- mem_cgroup_transfer_dmabuf_charge - move a dma-buf charge
- @dmabuf: buffer to transfer charge
- @new_memcg: target memcg; caller must hold a reference
- Transfers the charge for @dmabuf to @new_memcg and, on
- success, stores @new_memcg in @dmabuf->memcg.
- Returns %true on success, %false if the charge against @new_memcg
- failed or a concurrent transfer raced on @dmabuf->memcg.
- On failure the caller retains its reference to @new_memcg.
- */
+bool mem_cgroup_transfer_dmabuf_charge(struct dma_buf *dmabuf,
struct mem_cgroup *new_memcg)I would make that more generic instead of dmabuf specific. E.g. something like this:
bool mem_cgroup_transfer_charge(struct struct mem_cgroup **ptr_memcg, struct mem_cgroup *new_memcg, enum memcg_stat_item item, unsigned int nr_pages)
Ah yes, I like this generalization. And the implementation remains mostly the same. Thanks!
+{
- struct mem_cgroup *old_memcg = READ_ONCE(dmabuf->memcg);
- unsigned int nr_pages = PAGE_ALIGN(dmabuf->size) >> PAGE_SHIFT;
This way we can also document in the DMA-buf caller that this conversation must be ok when memcg_account is true.
Regards, Christian.
- if (mem_cgroup_disabled())
return true;- if (!mem_cgroup_reserve_dmabuf(new_memcg, GFP_KERNEL, nr_pages))
return false;- mod_memcg_state(new_memcg, MEMCG_DMABUF, nr_pages);
- if (cmpxchg(&dmabuf->memcg, old_memcg, new_memcg) != old_memcg) {
mod_memcg_state(new_memcg, MEMCG_DMABUF, -nr_pages);mem_cgroup_cancel_dmabuf(new_memcg, nr_pages);return false;- }
- if (old_memcg) {
/* Uncharge old cgroup */mod_memcg_state(old_memcg, MEMCG_DMABUF, -nr_pages);mem_cgroup_cancel_dmabuf(old_memcg, nr_pages);mem_cgroup_put(old_memcg);- }
- return true;
+}
/**
- mem_cgroup_uncharge_dmabuf - uncharge a dma-buf from its memcg
- @dmabuf: buffer to uncharge
On Wed, Oct 28, 2026 at 05:09:37PM +0100, Albert Esteve via OP-TEE wrote:
On Wed, Oct 28, 2026 at 10:08:19AM +0100, Christian König wrote:
On 10/27/26 16:02, Albert Esteve wrote:
dma-heap already supports charging a specific cgroup at allocation time via charge_fd in DMA_HEAP_IOCTL_ALLOC. However, dma-bufs can originate from any exporter (DRM, V4L2, etc.), and the allocator may not know the final owner at allocation time.
Add DMA_BUF_IOCTL_XFER_CHARGE to transfer a dma-buf's memcg charge to a different cgroup after allocation. The target is identified by charge_fd, which may be a cgroupfd or a pidfd.
The transfer charges the new cgroup before uncharging the old one, preserving the original charge on failure. An atomic cmpxchg on dmabuf->memcg prevents races with concurrent transfers.
Not all dma-buf exporters back their buffers with reclaimable system memory. Some represent device registers, VRAM, or carved-out pools where size is not a meaningful memory cost. Add a memcg_account flag to dma_buf_export_info that exporters set when their buffers should participate in memcg accounting. The xfer ioctl returns -EOPNOTSUPP for buffers that do not set the flag. Currently only the system heap sets it, other exporters can opt in as needed.
Usage example: int cg_fd = open("/sys/fs/cgroup/clients/app1", O_RDONLY | O_DIRECTORY);
struct dma_buf_transfer_charge xfer = { .charge_fd = cg_fd, }; ioctl(dmabuf_fd, DMA_BUF_IOCTL_XFER_CHARGE, &xfer); /* charge now on app1's cgroup */ close(cg_fd);
Signed-off-by: Albert Esteve aesteve@redhat.com
drivers/dma-buf/dma-buf.c | 43 +++++++++++++++++++++++++++++++++++++ drivers/dma-buf/heaps/system_heap.c | 1 + include/linux/dma-buf.h | 5 +++++ include/linux/memcontrol.h | 9 ++++++++ include/uapi/linux/dma-buf.h | 26 ++++++++++++++++++++++ mm/memcontrol.c | 41 +++++++++++++++++++++++++++++++++++ 6 files changed, 125 insertions(+)
diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c index f31f49ebd0af..07513ecedc10 100644 --- a/drivers/dma-buf/dma-buf.c +++ b/drivers/dma-buf/dma-buf.c @@ -540,6 +540,45 @@ static long dma_buf_import_sync_file(struct dma_buf *dmabuf, } #endif +/* Transfer a dma-buf's memcg charge; see struct dma_buf_transfer_charge */ +static long dma_buf_xfer_charge(struct dma_buf *dmabuf,
const void __user *user_data)+{
- struct dma_buf_transfer_charge arg;
- struct mem_cgroup *memcg;
- int ret = 0;
- if (copy_from_user(&arg, user_data, sizeof(arg)))
return -EFAULT;- if (!IS_ENABLED(CONFIG_MEMCG))
return 0;- if (arg.flags != 0)
return -EINVAL;- if (!dmabuf->memcg_account)
return -EOPNOTSUPP;- memcg = mem_cgroup_get_from_fd(arg.charge_fd);
- if (IS_ERR(memcg))
return PTR_ERR(memcg);- if (dmabuf->memcg == memcg)
goto skip_transfer;- if (!mem_cgroup_transfer_dmabuf_charge(dmabuf, memcg)) {
ret = -ENOMEM;goto skip_transfer;- }
- return 0;
+skip_transfer:
- mem_cgroup_put(memcg);
- return ret;
+}
static long dma_buf_ioctl(struct file *file, unsigned int cmd, unsigned long arg) { @@ -590,6 +629,9 @@ static long dma_buf_ioctl(struct file *file, return dma_buf_import_sync_file(dmabuf, (const void __user *)arg); #endif
- case DMA_BUF_IOCTL_XFER_CHARGE:
return dma_buf_xfer_charge(dmabuf, (const void __user *)arg);- default: return -ENOTTY; }
@@ -750,6 +792,7 @@ struct dma_buf *dma_buf_export(const struct dma_buf_export_info *exp_info) dmabuf->size = exp_info->size; dmabuf->exp_name = exp_info->exp_name; dmabuf->owner = exp_info->owner;
- dmabuf->memcg_account = exp_info->memcg_account;
The initial charging should probably be done here as well.
This way DMA-buf heaps only needs to pass in the charge_fd and we can re-use the functionality for other DMA-buf exporters as well.
The problem with doing the charge here is that it happens too late. That is the reasoning for this proposal to move all the charging logic into heap allocators before pages are allocated.
What we could do is fill the dmabuf->memcg field when the flag is set (i.e., committing into the dmabuf in the patch semantics). But for that we'd need adding the memcg we want to charge it to into exp_info in order to support charging to cgroups other than the allocator's (for the common case would be simpler as we can just look at current). Maybe that's what you meant?
Actually if I do this we can remove the memcg_account flag and just uses exp_info->memcg as both the memcg to set *and* the flag when it is != NULL. Exporters that set the memcg info are implicitly flagging the dmabuf as chargeable (and transfereable).
The only thing is that mod_memcg_state() still needs nr_pages and we cannot use size as that varies from one exporter to another as you explained before. So that can be left for the exporter to call. WDYT?
spin_lock_init(&dmabuf->name_lock); init_waitqueue_head(&dmabuf->poll); dmabuf->cb_in.poll = dmabuf->cb_out.poll = &dmabuf->poll; diff --git a/drivers/dma-buf/heaps/system_heap.c b/drivers/dma-buf/heaps/system_heap.c index adeea8cd328b..a764167167ba 100644 --- a/drivers/dma-buf/heaps/system_heap.c +++ b/drivers/dma-buf/heaps/system_heap.c @@ -479,6 +479,7 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap, exp_info.size = buffer->len; exp_info.flags = fd_flags; exp_info.priv = buffer;
- exp_info.memcg_account = true; dmabuf = dma_buf_export(&exp_info); if (IS_ERR(dmabuf)) { ret = PTR_ERR(dmabuf);
diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h index 23ec0ff23fba..ac0ec88cab93 100644 --- a/include/linux/dma-buf.h +++ b/include/linux/dma-buf.h @@ -433,6 +433,9 @@ struct dma_buf { /** @memcg: the cgroup to which this buffer is currently attributed */ struct mem_cgroup *memcg;
- /** @memcg_account: account buffer memory to cgroup */
- bool memcg_account;
}; /** @@ -505,6 +508,7 @@ struct dma_buf_attachment {
- @flags: mode flags for the file
- @resv: reservation-object, NULL to allocate default one
- @priv: Attach private data of allocator to this buffer
- @memcg_account: account buffer memory to cgroup
- This structure holds the information required to export the buffer. Used
- with dma_buf_export() only.
@@ -517,6 +521,7 @@ struct dma_buf_export_info { int flags; struct dma_resv *resv; void *priv;
- bool memcg_account;
}; /** diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 6b30baa79fa9..36fddafbef4e 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -679,6 +679,9 @@ bool mem_cgroup_reserve_dmabuf(struct mem_cgroup *memcg, gfp_t gfp, void mem_cgroup_commit_dmabuf(struct dma_buf *dmabuf, struct mem_cgroup *memcg, unsigned int nr_pages); +bool mem_cgroup_transfer_dmabuf_charge(struct dma_buf *dmabuf,
struct mem_cgroup *new_memcg);void __mem_cgroup_uncharge(struct folio *folio); /** @@ -1206,6 +1209,12 @@ static inline void mem_cgroup_cancel_dmabuf(struct mem_cgroup *memcg, { } +static inline bool mem_cgroup_transfer_dmabuf_charge(struct dma_buf *dmabuf,
struct mem_cgroup *new_memcg)+{
- return true;
+}
static inline void mem_cgroup_uncharge(struct folio *folio) { } diff --git a/include/uapi/linux/dma-buf.h b/include/uapi/linux/dma-buf.h index e827c9d20c5d..ea957f4b1d38 100644 --- a/include/uapi/linux/dma-buf.h +++ b/include/uapi/linux/dma-buf.h @@ -168,6 +168,31 @@ struct dma_buf_import_sync_file { __s32 fd; }; +/**
- struct dma_buf_transfer_charge - Transfer a dma-buf's memcg charge
- Any holder of a dma-buf file descriptor can perform a
- DMA_BUF_IOCTL_XFER_CHARGE to attribute the buffer's memory
- cost to a specific cgroup, or move it from one cgroup to
- another. The ioctl charges the new cgroup before uncharging
- the old one atomically. On failure the original charge is
- preserved. If the buffer has no existing charge, a new
- charge is established.
- */
+struct dma_buf_transfer_charge {
- /** @flags: Reserved, must be zero */
- __u32 flags; /* unused for now */
- /**
* @charge_fd: File descriptor identifying the target cgroup.** May be either a pidfd, in which case the charge moves to the* memory cgroup of the referenced process, or a cgroupfd obtained* by opening a cgroup v2 directory, in which case the charge moves* directly to that cgroup's memory controller.*/- __u32 charge_fd;
+};
#define DMA_BUF_BASE 'b' #define DMA_BUF_IOCTL_SYNC _IOW(DMA_BUF_BASE, 0, struct dma_buf_sync) @@ -179,5 +204,6 @@ struct dma_buf_import_sync_file { #define DMA_BUF_SET_NAME_B _IOW(DMA_BUF_BASE, 1, __u64) #define DMA_BUF_IOCTL_EXPORT_SYNC_FILE _IOWR(DMA_BUF_BASE, 2, struct dma_buf_export_sync_file) #define DMA_BUF_IOCTL_IMPORT_SYNC_FILE _IOW(DMA_BUF_BASE, 3, struct dma_buf_import_sync_file) +#define DMA_BUF_IOCTL_XFER_CHARGE _IOW(DMA_BUF_BASE, 4, struct dma_buf_transfer_charge) #endif diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 7b841b905759..3adc84fe6668 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -5786,6 +5786,47 @@ void mem_cgroup_cancel_dmabuf(struct mem_cgroup *memcg, unsigned int nr_pages) refill_stock(memcg, nr_pages); } +/**
- mem_cgroup_transfer_dmabuf_charge - move a dma-buf charge
- @dmabuf: buffer to transfer charge
- @new_memcg: target memcg; caller must hold a reference
- Transfers the charge for @dmabuf to @new_memcg and, on
- success, stores @new_memcg in @dmabuf->memcg.
- Returns %true on success, %false if the charge against @new_memcg
- failed or a concurrent transfer raced on @dmabuf->memcg.
- On failure the caller retains its reference to @new_memcg.
- */
+bool mem_cgroup_transfer_dmabuf_charge(struct dma_buf *dmabuf,
struct mem_cgroup *new_memcg)I would make that more generic instead of dmabuf specific. E.g. something like this:
bool mem_cgroup_transfer_charge(struct struct mem_cgroup **ptr_memcg, struct mem_cgroup *new_memcg, enum memcg_stat_item item, unsigned int nr_pages)
Ah yes, I like this generalization. And the implementation remains mostly the same. Thanks!
+{
- struct mem_cgroup *old_memcg = READ_ONCE(dmabuf->memcg);
- unsigned int nr_pages = PAGE_ALIGN(dmabuf->size) >> PAGE_SHIFT;
This way we can also document in the DMA-buf caller that this conversation must be ok when memcg_account is true.
Regards, Christian.
- if (mem_cgroup_disabled())
return true;- if (!mem_cgroup_reserve_dmabuf(new_memcg, GFP_KERNEL, nr_pages))
return false;- mod_memcg_state(new_memcg, MEMCG_DMABUF, nr_pages);
- if (cmpxchg(&dmabuf->memcg, old_memcg, new_memcg) != old_memcg) {
mod_memcg_state(new_memcg, MEMCG_DMABUF, -nr_pages);mem_cgroup_cancel_dmabuf(new_memcg, nr_pages);return false;- }
- if (old_memcg) {
/* Uncharge old cgroup */mod_memcg_state(old_memcg, MEMCG_DMABUF, -nr_pages);mem_cgroup_cancel_dmabuf(old_memcg, nr_pages);mem_cgroup_put(old_memcg);- }
- return true;
+}
/**
- mem_cgroup_uncharge_dmabuf - uncharge a dma-buf from its memcg
- @dmabuf: buffer to uncharge
On 10/28/26 17:09, Albert Esteve wrote:
On Wed, Oct 28, 2026 at 10:08:19AM +0100, Christian König wrote:
On 10/27/26 16:02, Albert Esteve wrote:
dma-heap already supports charging a specific cgroup at allocation time via charge_fd in DMA_HEAP_IOCTL_ALLOC. However, dma-bufs can originate from any exporter (DRM, V4L2, etc.), and the allocator may not know the final owner at allocation time.
Add DMA_BUF_IOCTL_XFER_CHARGE to transfer a dma-buf's memcg charge to a different cgroup after allocation. The target is identified by charge_fd, which may be a cgroupfd or a pidfd.
The transfer charges the new cgroup before uncharging the old one, preserving the original charge on failure. An atomic cmpxchg on dmabuf->memcg prevents races with concurrent transfers.
Not all dma-buf exporters back their buffers with reclaimable system memory. Some represent device registers, VRAM, or carved-out pools where size is not a meaningful memory cost. Add a memcg_account flag to dma_buf_export_info that exporters set when their buffers should participate in memcg accounting. The xfer ioctl returns -EOPNOTSUPP for buffers that do not set the flag. Currently only the system heap sets it, other exporters can opt in as needed.
Usage example: int cg_fd = open("/sys/fs/cgroup/clients/app1", O_RDONLY | O_DIRECTORY);
struct dma_buf_transfer_charge xfer = { .charge_fd = cg_fd, }; ioctl(dmabuf_fd, DMA_BUF_IOCTL_XFER_CHARGE, &xfer); /* charge now on app1's cgroup */ close(cg_fd);
Signed-off-by: Albert Esteve aesteve@redhat.com
drivers/dma-buf/dma-buf.c | 43 +++++++++++++++++++++++++++++++++++++ drivers/dma-buf/heaps/system_heap.c | 1 + include/linux/dma-buf.h | 5 +++++ include/linux/memcontrol.h | 9 ++++++++ include/uapi/linux/dma-buf.h | 26 ++++++++++++++++++++++ mm/memcontrol.c | 41 +++++++++++++++++++++++++++++++++++ 6 files changed, 125 insertions(+)
diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c index f31f49ebd0af..07513ecedc10 100644 --- a/drivers/dma-buf/dma-buf.c +++ b/drivers/dma-buf/dma-buf.c @@ -540,6 +540,45 @@ static long dma_buf_import_sync_file(struct dma_buf *dmabuf, } #endif +/* Transfer a dma-buf's memcg charge; see struct dma_buf_transfer_charge */ +static long dma_buf_xfer_charge(struct dma_buf *dmabuf,
const void __user *user_data)+{
- struct dma_buf_transfer_charge arg;
- struct mem_cgroup *memcg;
- int ret = 0;
- if (copy_from_user(&arg, user_data, sizeof(arg)))
return -EFAULT;- if (!IS_ENABLED(CONFIG_MEMCG))
return 0;- if (arg.flags != 0)
return -EINVAL;- if (!dmabuf->memcg_account)
return -EOPNOTSUPP;- memcg = mem_cgroup_get_from_fd(arg.charge_fd);
- if (IS_ERR(memcg))
return PTR_ERR(memcg);- if (dmabuf->memcg == memcg)
goto skip_transfer;- if (!mem_cgroup_transfer_dmabuf_charge(dmabuf, memcg)) {
ret = -ENOMEM;goto skip_transfer;- }
- return 0;
+skip_transfer:
- mem_cgroup_put(memcg);
- return ret;
+}
static long dma_buf_ioctl(struct file *file, unsigned int cmd, unsigned long arg) { @@ -590,6 +629,9 @@ static long dma_buf_ioctl(struct file *file, return dma_buf_import_sync_file(dmabuf, (const void __user *)arg); #endif
- case DMA_BUF_IOCTL_XFER_CHARGE:
return dma_buf_xfer_charge(dmabuf, (const void __user *)arg);- default: return -ENOTTY; }
@@ -750,6 +792,7 @@ struct dma_buf *dma_buf_export(const struct dma_buf_export_info *exp_info) dmabuf->size = exp_info->size; dmabuf->exp_name = exp_info->exp_name; dmabuf->owner = exp_info->owner;
- dmabuf->memcg_account = exp_info->memcg_account;
The initial charging should probably be done here as well.
This way DMA-buf heaps only needs to pass in the charge_fd and we can re-use the functionality for other DMA-buf exporters as well.
The problem with doing the charge here is that it happens too late. That is the reasoning for this proposal to move all the charging logic into heap allocators before pages are allocated.
Oh, that is a really good point we need to talk about.
DMA-buf heaps is actually a rather simple use case, but we have way more complex things.
For example in some use cases the backing store of a DMA-buf is only allocated on first use and only then it needs to be comitted to it's memcg or dmem.
I don't mind implementing it for DMA-buf heaps first, but we should have the uAPI and it's handling so solid that it also works for future use cases.
What we could do is fill the dmabuf->memcg field when the flag is set (i.e., committing into the dmabuf in the patch semantics). But for that we'd need adding the memcg we want to charge it to into exp_info in order to support charging to cgroups other than the allocator's (for the common case would be simpler as we can just look at current). Maybe that's what you meant?
At least for the DMA-buf heaps use case that sounds reasonable for now.
Regards, Christian.
spin_lock_init(&dmabuf->name_lock); init_waitqueue_head(&dmabuf->poll); dmabuf->cb_in.poll = dmabuf->cb_out.poll = &dmabuf->poll; diff --git a/drivers/dma-buf/heaps/system_heap.c b/drivers/dma-buf/heaps/system_heap.c index adeea8cd328b..a764167167ba 100644 --- a/drivers/dma-buf/heaps/system_heap.c +++ b/drivers/dma-buf/heaps/system_heap.c @@ -479,6 +479,7 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap, exp_info.size = buffer->len; exp_info.flags = fd_flags; exp_info.priv = buffer;
- exp_info.memcg_account = true; dmabuf = dma_buf_export(&exp_info); if (IS_ERR(dmabuf)) { ret = PTR_ERR(dmabuf);
diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h index 23ec0ff23fba..ac0ec88cab93 100644 --- a/include/linux/dma-buf.h +++ b/include/linux/dma-buf.h @@ -433,6 +433,9 @@ struct dma_buf { /** @memcg: the cgroup to which this buffer is currently attributed */ struct mem_cgroup *memcg;
- /** @memcg_account: account buffer memory to cgroup */
- bool memcg_account;
}; /** @@ -505,6 +508,7 @@ struct dma_buf_attachment {
- @flags: mode flags for the file
- @resv: reservation-object, NULL to allocate default one
- @priv: Attach private data of allocator to this buffer
- @memcg_account: account buffer memory to cgroup
- This structure holds the information required to export the buffer. Used
- with dma_buf_export() only.
@@ -517,6 +521,7 @@ struct dma_buf_export_info { int flags; struct dma_resv *resv; void *priv;
- bool memcg_account;
}; /** diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 6b30baa79fa9..36fddafbef4e 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -679,6 +679,9 @@ bool mem_cgroup_reserve_dmabuf(struct mem_cgroup *memcg, gfp_t gfp, void mem_cgroup_commit_dmabuf(struct dma_buf *dmabuf, struct mem_cgroup *memcg, unsigned int nr_pages); +bool mem_cgroup_transfer_dmabuf_charge(struct dma_buf *dmabuf,
struct mem_cgroup *new_memcg);void __mem_cgroup_uncharge(struct folio *folio); /** @@ -1206,6 +1209,12 @@ static inline void mem_cgroup_cancel_dmabuf(struct mem_cgroup *memcg, { } +static inline bool mem_cgroup_transfer_dmabuf_charge(struct dma_buf *dmabuf,
struct mem_cgroup *new_memcg)+{
- return true;
+}
static inline void mem_cgroup_uncharge(struct folio *folio) { } diff --git a/include/uapi/linux/dma-buf.h b/include/uapi/linux/dma-buf.h index e827c9d20c5d..ea957f4b1d38 100644 --- a/include/uapi/linux/dma-buf.h +++ b/include/uapi/linux/dma-buf.h @@ -168,6 +168,31 @@ struct dma_buf_import_sync_file { __s32 fd; }; +/**
- struct dma_buf_transfer_charge - Transfer a dma-buf's memcg charge
- Any holder of a dma-buf file descriptor can perform a
- DMA_BUF_IOCTL_XFER_CHARGE to attribute the buffer's memory
- cost to a specific cgroup, or move it from one cgroup to
- another. The ioctl charges the new cgroup before uncharging
- the old one atomically. On failure the original charge is
- preserved. If the buffer has no existing charge, a new
- charge is established.
- */
+struct dma_buf_transfer_charge {
- /** @flags: Reserved, must be zero */
- __u32 flags; /* unused for now */
- /**
* @charge_fd: File descriptor identifying the target cgroup.** May be either a pidfd, in which case the charge moves to the* memory cgroup of the referenced process, or a cgroupfd obtained* by opening a cgroup v2 directory, in which case the charge moves* directly to that cgroup's memory controller.*/- __u32 charge_fd;
+};
#define DMA_BUF_BASE 'b' #define DMA_BUF_IOCTL_SYNC _IOW(DMA_BUF_BASE, 0, struct dma_buf_sync) @@ -179,5 +204,6 @@ struct dma_buf_import_sync_file { #define DMA_BUF_SET_NAME_B _IOW(DMA_BUF_BASE, 1, __u64) #define DMA_BUF_IOCTL_EXPORT_SYNC_FILE _IOWR(DMA_BUF_BASE, 2, struct dma_buf_export_sync_file) #define DMA_BUF_IOCTL_IMPORT_SYNC_FILE _IOW(DMA_BUF_BASE, 3, struct dma_buf_import_sync_file) +#define DMA_BUF_IOCTL_XFER_CHARGE _IOW(DMA_BUF_BASE, 4, struct dma_buf_transfer_charge) #endif diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 7b841b905759..3adc84fe6668 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -5786,6 +5786,47 @@ void mem_cgroup_cancel_dmabuf(struct mem_cgroup *memcg, unsigned int nr_pages) refill_stock(memcg, nr_pages); } +/**
- mem_cgroup_transfer_dmabuf_charge - move a dma-buf charge
- @dmabuf: buffer to transfer charge
- @new_memcg: target memcg; caller must hold a reference
- Transfers the charge for @dmabuf to @new_memcg and, on
- success, stores @new_memcg in @dmabuf->memcg.
- Returns %true on success, %false if the charge against @new_memcg
- failed or a concurrent transfer raced on @dmabuf->memcg.
- On failure the caller retains its reference to @new_memcg.
- */
+bool mem_cgroup_transfer_dmabuf_charge(struct dma_buf *dmabuf,
struct mem_cgroup *new_memcg)I would make that more generic instead of dmabuf specific. E.g. something like this:
bool mem_cgroup_transfer_charge(struct struct mem_cgroup **ptr_memcg, struct mem_cgroup *new_memcg, enum memcg_stat_item item, unsigned int nr_pages)
Ah yes, I like this generalization. And the implementation remains mostly the same. Thanks!
+{
- struct mem_cgroup *old_memcg = READ_ONCE(dmabuf->memcg);
- unsigned int nr_pages = PAGE_ALIGN(dmabuf->size) >> PAGE_SHIFT;
This way we can also document in the DMA-buf caller that this conversation must be ok when memcg_account is true.
Regards, Christian.
- if (mem_cgroup_disabled())
return true;- if (!mem_cgroup_reserve_dmabuf(new_memcg, GFP_KERNEL, nr_pages))
return false;- mod_memcg_state(new_memcg, MEMCG_DMABUF, nr_pages);
- if (cmpxchg(&dmabuf->memcg, old_memcg, new_memcg) != old_memcg) {
mod_memcg_state(new_memcg, MEMCG_DMABUF, -nr_pages);mem_cgroup_cancel_dmabuf(new_memcg, nr_pages);return false;- }
- if (old_memcg) {
/* Uncharge old cgroup */mod_memcg_state(old_memcg, MEMCG_DMABUF, -nr_pages);mem_cgroup_cancel_dmabuf(old_memcg, nr_pages);mem_cgroup_put(old_memcg);- }
- return true;
+}
/**
- mem_cgroup_uncharge_dmabuf - uncharge a dma-buf from its memcg
- @dmabuf: buffer to uncharge
On Thu, Oct 29, 2026 at 03:16:12PM +0100, Christian König wrote:
On 10/28/26 17:09, Albert Esteve wrote:
On Wed, Oct 28, 2026 at 10:08:19AM +0100, Christian König wrote:
On 10/27/26 16:02, Albert Esteve wrote:
dma-heap already supports charging a specific cgroup at allocation time via charge_fd in DMA_HEAP_IOCTL_ALLOC. However, dma-bufs can originate from any exporter (DRM, V4L2, etc.), and the allocator may not know the final owner at allocation time.
Add DMA_BUF_IOCTL_XFER_CHARGE to transfer a dma-buf's memcg charge to a different cgroup after allocation. The target is identified by charge_fd, which may be a cgroupfd or a pidfd.
The transfer charges the new cgroup before uncharging the old one, preserving the original charge on failure. An atomic cmpxchg on dmabuf->memcg prevents races with concurrent transfers.
Not all dma-buf exporters back their buffers with reclaimable system memory. Some represent device registers, VRAM, or carved-out pools where size is not a meaningful memory cost. Add a memcg_account flag to dma_buf_export_info that exporters set when their buffers should participate in memcg accounting. The xfer ioctl returns -EOPNOTSUPP for buffers that do not set the flag. Currently only the system heap sets it, other exporters can opt in as needed.
Usage example: int cg_fd = open("/sys/fs/cgroup/clients/app1", O_RDONLY | O_DIRECTORY);
struct dma_buf_transfer_charge xfer = { .charge_fd = cg_fd, }; ioctl(dmabuf_fd, DMA_BUF_IOCTL_XFER_CHARGE, &xfer); /* charge now on app1's cgroup */ close(cg_fd);
Signed-off-by: Albert Esteve aesteve@redhat.com
drivers/dma-buf/dma-buf.c | 43 +++++++++++++++++++++++++++++++++++++ drivers/dma-buf/heaps/system_heap.c | 1 + include/linux/dma-buf.h | 5 +++++ include/linux/memcontrol.h | 9 ++++++++ include/uapi/linux/dma-buf.h | 26 ++++++++++++++++++++++ mm/memcontrol.c | 41 +++++++++++++++++++++++++++++++++++ 6 files changed, 125 insertions(+)
diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c index f31f49ebd0af..07513ecedc10 100644 --- a/drivers/dma-buf/dma-buf.c +++ b/drivers/dma-buf/dma-buf.c @@ -540,6 +540,45 @@ static long dma_buf_import_sync_file(struct dma_buf *dmabuf, } #endif +/* Transfer a dma-buf's memcg charge; see struct dma_buf_transfer_charge */ +static long dma_buf_xfer_charge(struct dma_buf *dmabuf,
const void __user *user_data)+{
- struct dma_buf_transfer_charge arg;
- struct mem_cgroup *memcg;
- int ret = 0;
- if (copy_from_user(&arg, user_data, sizeof(arg)))
return -EFAULT;- if (!IS_ENABLED(CONFIG_MEMCG))
return 0;- if (arg.flags != 0)
return -EINVAL;- if (!dmabuf->memcg_account)
return -EOPNOTSUPP;- memcg = mem_cgroup_get_from_fd(arg.charge_fd);
- if (IS_ERR(memcg))
return PTR_ERR(memcg);- if (dmabuf->memcg == memcg)
goto skip_transfer;- if (!mem_cgroup_transfer_dmabuf_charge(dmabuf, memcg)) {
ret = -ENOMEM;goto skip_transfer;- }
- return 0;
+skip_transfer:
- mem_cgroup_put(memcg);
- return ret;
+}
static long dma_buf_ioctl(struct file *file, unsigned int cmd, unsigned long arg) { @@ -590,6 +629,9 @@ static long dma_buf_ioctl(struct file *file, return dma_buf_import_sync_file(dmabuf, (const void __user *)arg); #endif
- case DMA_BUF_IOCTL_XFER_CHARGE:
return dma_buf_xfer_charge(dmabuf, (const void __user *)arg);- default: return -ENOTTY; }
@@ -750,6 +792,7 @@ struct dma_buf *dma_buf_export(const struct dma_buf_export_info *exp_info) dmabuf->size = exp_info->size; dmabuf->exp_name = exp_info->exp_name; dmabuf->owner = exp_info->owner;
- dmabuf->memcg_account = exp_info->memcg_account;
The initial charging should probably be done here as well.
This way DMA-buf heaps only needs to pass in the charge_fd and we can re-use the functionality for other DMA-buf exporters as well.
The problem with doing the charge here is that it happens too late. That is the reasoning for this proposal to move all the charging logic into heap allocators before pages are allocated.
Oh, that is a really good point we need to talk about.
DMA-buf heaps is actually a rather simple use case, but we have way more complex things.
For example in some use cases the backing store of a DMA-buf is only allocated on first use and only then it needs to be comitted to it's memcg or dmem.
Yes. The idea is focus on heaps for now. But the uAPI is actually agnostic of the time when the charge happens. When we deal with lazy-allocation in the future we can just call the reserve/cancel helpers wherever is more appropiate. As long as the pages are not allocated, exporters do not set dmabuf->memcg, and XFER does not work (returns -EOPNOTSUPP).
I don't mind implementing it for DMA-buf heaps first, but we should have the uAPI and it's handling so solid that it also works for future use cases.
Agreed. I think we should cover all usecases with the current status. Your feedback helped a lot in getting there!
What we could do is fill the dmabuf->memcg field when the flag is set (i.e., committing into the dmabuf in the patch semantics). But for that we'd need adding the memcg we want to charge it to into exp_info in order to support charging to cgroups other than the allocator's (for the common case would be simpler as we can just look at current). Maybe that's what you meant?
At least for the DMA-buf heaps use case that sounds reasonable for now.
Thanks! Next version will have the flag removed and just use dmabuf->memcg as indicator, set within dma_buf_export(). Also, `nr_pages` will be computed within dma_buf_export(), the contract being that exporters setting exp_info.memcg assert that size reflects actual system memory bytes.
Regards, Christian.
spin_lock_init(&dmabuf->name_lock); init_waitqueue_head(&dmabuf->poll); dmabuf->cb_in.poll = dmabuf->cb_out.poll = &dmabuf->poll; diff --git a/drivers/dma-buf/heaps/system_heap.c b/drivers/dma-buf/heaps/system_heap.c index adeea8cd328b..a764167167ba 100644 --- a/drivers/dma-buf/heaps/system_heap.c +++ b/drivers/dma-buf/heaps/system_heap.c @@ -479,6 +479,7 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap, exp_info.size = buffer->len; exp_info.flags = fd_flags; exp_info.priv = buffer;
- exp_info.memcg_account = true; dmabuf = dma_buf_export(&exp_info); if (IS_ERR(dmabuf)) { ret = PTR_ERR(dmabuf);
diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h index 23ec0ff23fba..ac0ec88cab93 100644 --- a/include/linux/dma-buf.h +++ b/include/linux/dma-buf.h @@ -433,6 +433,9 @@ struct dma_buf { /** @memcg: the cgroup to which this buffer is currently attributed */ struct mem_cgroup *memcg;
- /** @memcg_account: account buffer memory to cgroup */
- bool memcg_account;
}; /** @@ -505,6 +508,7 @@ struct dma_buf_attachment {
- @flags: mode flags for the file
- @resv: reservation-object, NULL to allocate default one
- @priv: Attach private data of allocator to this buffer
- @memcg_account: account buffer memory to cgroup
- This structure holds the information required to export the buffer. Used
- with dma_buf_export() only.
@@ -517,6 +521,7 @@ struct dma_buf_export_info { int flags; struct dma_resv *resv; void *priv;
- bool memcg_account;
}; /** diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 6b30baa79fa9..36fddafbef4e 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -679,6 +679,9 @@ bool mem_cgroup_reserve_dmabuf(struct mem_cgroup *memcg, gfp_t gfp, void mem_cgroup_commit_dmabuf(struct dma_buf *dmabuf, struct mem_cgroup *memcg, unsigned int nr_pages); +bool mem_cgroup_transfer_dmabuf_charge(struct dma_buf *dmabuf,
struct mem_cgroup *new_memcg);void __mem_cgroup_uncharge(struct folio *folio); /** @@ -1206,6 +1209,12 @@ static inline void mem_cgroup_cancel_dmabuf(struct mem_cgroup *memcg, { } +static inline bool mem_cgroup_transfer_dmabuf_charge(struct dma_buf *dmabuf,
struct mem_cgroup *new_memcg)+{
- return true;
+}
static inline void mem_cgroup_uncharge(struct folio *folio) { } diff --git a/include/uapi/linux/dma-buf.h b/include/uapi/linux/dma-buf.h index e827c9d20c5d..ea957f4b1d38 100644 --- a/include/uapi/linux/dma-buf.h +++ b/include/uapi/linux/dma-buf.h @@ -168,6 +168,31 @@ struct dma_buf_import_sync_file { __s32 fd; }; +/**
- struct dma_buf_transfer_charge - Transfer a dma-buf's memcg charge
- Any holder of a dma-buf file descriptor can perform a
- DMA_BUF_IOCTL_XFER_CHARGE to attribute the buffer's memory
- cost to a specific cgroup, or move it from one cgroup to
- another. The ioctl charges the new cgroup before uncharging
- the old one atomically. On failure the original charge is
- preserved. If the buffer has no existing charge, a new
- charge is established.
- */
+struct dma_buf_transfer_charge {
- /** @flags: Reserved, must be zero */
- __u32 flags; /* unused for now */
- /**
* @charge_fd: File descriptor identifying the target cgroup.** May be either a pidfd, in which case the charge moves to the* memory cgroup of the referenced process, or a cgroupfd obtained* by opening a cgroup v2 directory, in which case the charge moves* directly to that cgroup's memory controller.*/- __u32 charge_fd;
+};
#define DMA_BUF_BASE 'b' #define DMA_BUF_IOCTL_SYNC _IOW(DMA_BUF_BASE, 0, struct dma_buf_sync) @@ -179,5 +204,6 @@ struct dma_buf_import_sync_file { #define DMA_BUF_SET_NAME_B _IOW(DMA_BUF_BASE, 1, __u64) #define DMA_BUF_IOCTL_EXPORT_SYNC_FILE _IOWR(DMA_BUF_BASE, 2, struct dma_buf_export_sync_file) #define DMA_BUF_IOCTL_IMPORT_SYNC_FILE _IOW(DMA_BUF_BASE, 3, struct dma_buf_import_sync_file) +#define DMA_BUF_IOCTL_XFER_CHARGE _IOW(DMA_BUF_BASE, 4, struct dma_buf_transfer_charge) #endif diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 7b841b905759..3adc84fe6668 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -5786,6 +5786,47 @@ void mem_cgroup_cancel_dmabuf(struct mem_cgroup *memcg, unsigned int nr_pages) refill_stock(memcg, nr_pages); } +/**
- mem_cgroup_transfer_dmabuf_charge - move a dma-buf charge
- @dmabuf: buffer to transfer charge
- @new_memcg: target memcg; caller must hold a reference
- Transfers the charge for @dmabuf to @new_memcg and, on
- success, stores @new_memcg in @dmabuf->memcg.
- Returns %true on success, %false if the charge against @new_memcg
- failed or a concurrent transfer raced on @dmabuf->memcg.
- On failure the caller retains its reference to @new_memcg.
- */
+bool mem_cgroup_transfer_dmabuf_charge(struct dma_buf *dmabuf,
struct mem_cgroup *new_memcg)I would make that more generic instead of dmabuf specific. E.g. something like this:
bool mem_cgroup_transfer_charge(struct struct mem_cgroup **ptr_memcg, struct mem_cgroup *new_memcg, enum memcg_stat_item item, unsigned int nr_pages)
Ah yes, I like this generalization. And the implementation remains mostly the same. Thanks!
+{
- struct mem_cgroup *old_memcg = READ_ONCE(dmabuf->memcg);
- unsigned int nr_pages = PAGE_ALIGN(dmabuf->size) >> PAGE_SHIFT;
This way we can also document in the DMA-buf caller that this conversation must be ok when memcg_account is true.
Regards, Christian.
- if (mem_cgroup_disabled())
return true;- if (!mem_cgroup_reserve_dmabuf(new_memcg, GFP_KERNEL, nr_pages))
return false;- mod_memcg_state(new_memcg, MEMCG_DMABUF, nr_pages);
- if (cmpxchg(&dmabuf->memcg, old_memcg, new_memcg) != old_memcg) {
mod_memcg_state(new_memcg, MEMCG_DMABUF, -nr_pages);mem_cgroup_cancel_dmabuf(new_memcg, nr_pages);return false;- }
- if (old_memcg) {
/* Uncharge old cgroup */mod_memcg_state(old_memcg, MEMCG_DMABUF, -nr_pages);mem_cgroup_cancel_dmabuf(old_memcg, nr_pages);mem_cgroup_put(old_memcg);- }
- return true;
+}
/**
- mem_cgroup_uncharge_dmabuf - uncharge a dma-buf from its memcg
- @dmabuf: buffer to uncharge
On 10/29/26 17:23, Albert Esteve wrote:
On Thu, Oct 29, 2026 at 03:16:12PM +0100, Christian König wrote:
On 10/28/26 17:09, Albert Esteve wrote:
On Wed, Oct 28, 2026 at 10:08:19AM +0100, Christian König wrote:
On 10/27/26 16:02, Albert Esteve wrote:
dma-heap already supports charging a specific cgroup at allocation time via charge_fd in DMA_HEAP_IOCTL_ALLOC. However, dma-bufs can originate from any exporter (DRM, V4L2, etc.), and the allocator may not know the final owner at allocation time.
Add DMA_BUF_IOCTL_XFER_CHARGE to transfer a dma-buf's memcg charge to a different cgroup after allocation. The target is identified by charge_fd, which may be a cgroupfd or a pidfd.
The transfer charges the new cgroup before uncharging the old one, preserving the original charge on failure. An atomic cmpxchg on dmabuf->memcg prevents races with concurrent transfers.
Not all dma-buf exporters back their buffers with reclaimable system memory. Some represent device registers, VRAM, or carved-out pools where size is not a meaningful memory cost. Add a memcg_account flag to dma_buf_export_info that exporters set when their buffers should participate in memcg accounting. The xfer ioctl returns -EOPNOTSUPP for buffers that do not set the flag. Currently only the system heap sets it, other exporters can opt in as needed.
Usage example: int cg_fd = open("/sys/fs/cgroup/clients/app1", O_RDONLY | O_DIRECTORY);
struct dma_buf_transfer_charge xfer = { .charge_fd = cg_fd, }; ioctl(dmabuf_fd, DMA_BUF_IOCTL_XFER_CHARGE, &xfer); /* charge now on app1's cgroup */ close(cg_fd);
Signed-off-by: Albert Esteve aesteve@redhat.com
drivers/dma-buf/dma-buf.c | 43 +++++++++++++++++++++++++++++++++++++ drivers/dma-buf/heaps/system_heap.c | 1 + include/linux/dma-buf.h | 5 +++++ include/linux/memcontrol.h | 9 ++++++++ include/uapi/linux/dma-buf.h | 26 ++++++++++++++++++++++ mm/memcontrol.c | 41 +++++++++++++++++++++++++++++++++++ 6 files changed, 125 insertions(+)
diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c index f31f49ebd0af..07513ecedc10 100644 --- a/drivers/dma-buf/dma-buf.c +++ b/drivers/dma-buf/dma-buf.c @@ -540,6 +540,45 @@ static long dma_buf_import_sync_file(struct dma_buf *dmabuf, } #endif +/* Transfer a dma-buf's memcg charge; see struct dma_buf_transfer_charge */ +static long dma_buf_xfer_charge(struct dma_buf *dmabuf,
const void __user *user_data)+{
- struct dma_buf_transfer_charge arg;
- struct mem_cgroup *memcg;
- int ret = 0;
- if (copy_from_user(&arg, user_data, sizeof(arg)))
return -EFAULT;- if (!IS_ENABLED(CONFIG_MEMCG))
return 0;- if (arg.flags != 0)
return -EINVAL;- if (!dmabuf->memcg_account)
return -EOPNOTSUPP;- memcg = mem_cgroup_get_from_fd(arg.charge_fd);
- if (IS_ERR(memcg))
return PTR_ERR(memcg);- if (dmabuf->memcg == memcg)
goto skip_transfer;- if (!mem_cgroup_transfer_dmabuf_charge(dmabuf, memcg)) {
ret = -ENOMEM;goto skip_transfer;- }
- return 0;
+skip_transfer:
- mem_cgroup_put(memcg);
- return ret;
+}
static long dma_buf_ioctl(struct file *file, unsigned int cmd, unsigned long arg) { @@ -590,6 +629,9 @@ static long dma_buf_ioctl(struct file *file, return dma_buf_import_sync_file(dmabuf, (const void __user *)arg); #endif
- case DMA_BUF_IOCTL_XFER_CHARGE:
return dma_buf_xfer_charge(dmabuf, (const void __user *)arg);- default: return -ENOTTY; }
@@ -750,6 +792,7 @@ struct dma_buf *dma_buf_export(const struct dma_buf_export_info *exp_info) dmabuf->size = exp_info->size; dmabuf->exp_name = exp_info->exp_name; dmabuf->owner = exp_info->owner;
- dmabuf->memcg_account = exp_info->memcg_account;
The initial charging should probably be done here as well.
This way DMA-buf heaps only needs to pass in the charge_fd and we can re-use the functionality for other DMA-buf exporters as well.
The problem with doing the charge here is that it happens too late. That is the reasoning for this proposal to move all the charging logic into heap allocators before pages are allocated.
Oh, that is a really good point we need to talk about.
DMA-buf heaps is actually a rather simple use case, but we have way more complex things.
For example in some use cases the backing store of a DMA-buf is only allocated on first use and only then it needs to be comitted to it's memcg or dmem.
Yes. The idea is focus on heaps for now. But the uAPI is actually agnostic of the time when the charge happens. When we deal with lazy-allocation in the future we can just call the reserve/cancel helpers wherever is more appropiate. As long as the pages are not allocated, exporters do not set dmabuf->memcg, and XFER does not work (returns -EOPNOTSUPP).
Yeah exactly that's the reason why I'm raising this topic. That approach won't work in all cases.
See userspace is not aware that the kernel is delaying allocation or that the kernel is allocating the backing store asynchroniously in a background worker for example.
So returning -EOPNOTSUPP when the backing store is not allocated yet and might be allocated later on is rather questionable uAPI because that is racy by design.
I don't mind implementing it for DMA-buf heaps first, but we should have the uAPI and it's handling so solid that it also works for future use cases.
Agreed. I think we should cover all usecases with the current status. Your feedback helped a lot in getting there!
I mean we can postpone that until we finally need it, but essentially we need to be able to distinct between charging, e.g. making sure the cgroup has enough free limit, and committing, e.g. allowing the resource to be reclaimed.
And then it is possible that when the memcg of a DMA-buf changes that only the charging changes, but there isn't any comitted backing store yet...
What we could do is fill the dmabuf->memcg field when the flag is set (i.e., committing into the dmabuf in the patch semantics). But for that we'd need adding the memcg we want to charge it to into exp_info in order to support charging to cgroups other than the allocator's (for the common case would be simpler as we can just look at current). Maybe that's what you meant?
At least for the DMA-buf heaps use case that sounds reasonable for now.
Thanks! Next version will have the flag removed and just use dmabuf->memcg as indicator, set within dma_buf_export(). Also, `nr_pages` will be computed within dma_buf_export(), the contract being that exporters setting exp_info.memcg assert that size reflects actual system memory bytes.
Thinking more about it I would also put changing dmabuf->memcg under the dmabuf->resv lock instead of doing it with cmpxchg.
We don't need it currently, but the dmabuf->resv lock is used for all state changes of the DMA-buf object and this operation is not performance critical in any way possible.
If there is a problem with locking order we at least get lockdep screaming at us very early and not when we want to extend the functionality in the future.
Regards, Christian.
Regards, Christian.
spin_lock_init(&dmabuf->name_lock); init_waitqueue_head(&dmabuf->poll); dmabuf->cb_in.poll = dmabuf->cb_out.poll = &dmabuf->poll; diff --git a/drivers/dma-buf/heaps/system_heap.c b/drivers/dma-buf/heaps/system_heap.c index adeea8cd328b..a764167167ba 100644 --- a/drivers/dma-buf/heaps/system_heap.c +++ b/drivers/dma-buf/heaps/system_heap.c @@ -479,6 +479,7 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap, exp_info.size = buffer->len; exp_info.flags = fd_flags; exp_info.priv = buffer;
- exp_info.memcg_account = true; dmabuf = dma_buf_export(&exp_info); if (IS_ERR(dmabuf)) { ret = PTR_ERR(dmabuf);
diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h index 23ec0ff23fba..ac0ec88cab93 100644 --- a/include/linux/dma-buf.h +++ b/include/linux/dma-buf.h @@ -433,6 +433,9 @@ struct dma_buf { /** @memcg: the cgroup to which this buffer is currently attributed */ struct mem_cgroup *memcg;
- /** @memcg_account: account buffer memory to cgroup */
- bool memcg_account;
}; /** @@ -505,6 +508,7 @@ struct dma_buf_attachment {
- @flags: mode flags for the file
- @resv: reservation-object, NULL to allocate default one
- @priv: Attach private data of allocator to this buffer
- @memcg_account: account buffer memory to cgroup
- This structure holds the information required to export the buffer. Used
- with dma_buf_export() only.
@@ -517,6 +521,7 @@ struct dma_buf_export_info { int flags; struct dma_resv *resv; void *priv;
- bool memcg_account;
}; /** diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 6b30baa79fa9..36fddafbef4e 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -679,6 +679,9 @@ bool mem_cgroup_reserve_dmabuf(struct mem_cgroup *memcg, gfp_t gfp, void mem_cgroup_commit_dmabuf(struct dma_buf *dmabuf, struct mem_cgroup *memcg, unsigned int nr_pages); +bool mem_cgroup_transfer_dmabuf_charge(struct dma_buf *dmabuf,
struct mem_cgroup *new_memcg);void __mem_cgroup_uncharge(struct folio *folio); /** @@ -1206,6 +1209,12 @@ static inline void mem_cgroup_cancel_dmabuf(struct mem_cgroup *memcg, { } +static inline bool mem_cgroup_transfer_dmabuf_charge(struct dma_buf *dmabuf,
struct mem_cgroup *new_memcg)+{
- return true;
+}
static inline void mem_cgroup_uncharge(struct folio *folio) { } diff --git a/include/uapi/linux/dma-buf.h b/include/uapi/linux/dma-buf.h index e827c9d20c5d..ea957f4b1d38 100644 --- a/include/uapi/linux/dma-buf.h +++ b/include/uapi/linux/dma-buf.h @@ -168,6 +168,31 @@ struct dma_buf_import_sync_file { __s32 fd; }; +/**
- struct dma_buf_transfer_charge - Transfer a dma-buf's memcg charge
- Any holder of a dma-buf file descriptor can perform a
- DMA_BUF_IOCTL_XFER_CHARGE to attribute the buffer's memory
- cost to a specific cgroup, or move it from one cgroup to
- another. The ioctl charges the new cgroup before uncharging
- the old one atomically. On failure the original charge is
- preserved. If the buffer has no existing charge, a new
- charge is established.
- */
+struct dma_buf_transfer_charge {
- /** @flags: Reserved, must be zero */
- __u32 flags; /* unused for now */
- /**
* @charge_fd: File descriptor identifying the target cgroup.** May be either a pidfd, in which case the charge moves to the* memory cgroup of the referenced process, or a cgroupfd obtained* by opening a cgroup v2 directory, in which case the charge moves* directly to that cgroup's memory controller.*/- __u32 charge_fd;
+};
#define DMA_BUF_BASE 'b' #define DMA_BUF_IOCTL_SYNC _IOW(DMA_BUF_BASE, 0, struct dma_buf_sync) @@ -179,5 +204,6 @@ struct dma_buf_import_sync_file { #define DMA_BUF_SET_NAME_B _IOW(DMA_BUF_BASE, 1, __u64) #define DMA_BUF_IOCTL_EXPORT_SYNC_FILE _IOWR(DMA_BUF_BASE, 2, struct dma_buf_export_sync_file) #define DMA_BUF_IOCTL_IMPORT_SYNC_FILE _IOW(DMA_BUF_BASE, 3, struct dma_buf_import_sync_file) +#define DMA_BUF_IOCTL_XFER_CHARGE _IOW(DMA_BUF_BASE, 4, struct dma_buf_transfer_charge) #endif diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 7b841b905759..3adc84fe6668 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -5786,6 +5786,47 @@ void mem_cgroup_cancel_dmabuf(struct mem_cgroup *memcg, unsigned int nr_pages) refill_stock(memcg, nr_pages); } +/**
- mem_cgroup_transfer_dmabuf_charge - move a dma-buf charge
- @dmabuf: buffer to transfer charge
- @new_memcg: target memcg; caller must hold a reference
- Transfers the charge for @dmabuf to @new_memcg and, on
- success, stores @new_memcg in @dmabuf->memcg.
- Returns %true on success, %false if the charge against @new_memcg
- failed or a concurrent transfer raced on @dmabuf->memcg.
- On failure the caller retains its reference to @new_memcg.
- */
+bool mem_cgroup_transfer_dmabuf_charge(struct dma_buf *dmabuf,
struct mem_cgroup *new_memcg)I would make that more generic instead of dmabuf specific. E.g. something like this:
bool mem_cgroup_transfer_charge(struct struct mem_cgroup **ptr_memcg, struct mem_cgroup *new_memcg, enum memcg_stat_item item, unsigned int nr_pages)
Ah yes, I like this generalization. And the implementation remains mostly the same. Thanks!
+{
- struct mem_cgroup *old_memcg = READ_ONCE(dmabuf->memcg);
- unsigned int nr_pages = PAGE_ALIGN(dmabuf->size) >> PAGE_SHIFT;
This way we can also document in the DMA-buf caller that this conversation must be ok when memcg_account is true.
Regards, Christian.
- if (mem_cgroup_disabled())
return true;- if (!mem_cgroup_reserve_dmabuf(new_memcg, GFP_KERNEL, nr_pages))
return false;- mod_memcg_state(new_memcg, MEMCG_DMABUF, nr_pages);
- if (cmpxchg(&dmabuf->memcg, old_memcg, new_memcg) != old_memcg) {
mod_memcg_state(new_memcg, MEMCG_DMABUF, -nr_pages);mem_cgroup_cancel_dmabuf(new_memcg, nr_pages);return false;- }
- if (old_memcg) {
/* Uncharge old cgroup */mod_memcg_state(old_memcg, MEMCG_DMABUF, -nr_pages);mem_cgroup_cancel_dmabuf(old_memcg, nr_pages);mem_cgroup_put(old_memcg);- }
- return true;
+}
/**
- mem_cgroup_uncharge_dmabuf - uncharge a dma-buf from its memcg
- @dmabuf: buffer to uncharge
On Fri, Oct 30, 2026 at 11:14:50AM +0100, Christian König wrote:
On 10/29/26 17:23, Albert Esteve wrote:
On Thu, Oct 29, 2026 at 03:16:12PM +0100, Christian König wrote:
On 10/28/26 17:09, Albert Esteve wrote:
On Wed, Oct 28, 2026 at 10:08:19AM +0100, Christian König wrote:
On 10/27/26 16:02, Albert Esteve wrote:
dma-heap already supports charging a specific cgroup at allocation time via charge_fd in DMA_HEAP_IOCTL_ALLOC. However, dma-bufs can originate from any exporter (DRM, V4L2, etc.), and the allocator may not know the final owner at allocation time.
Add DMA_BUF_IOCTL_XFER_CHARGE to transfer a dma-buf's memcg charge to a different cgroup after allocation. The target is identified by charge_fd, which may be a cgroupfd or a pidfd.
The transfer charges the new cgroup before uncharging the old one, preserving the original charge on failure. An atomic cmpxchg on dmabuf->memcg prevents races with concurrent transfers.
Not all dma-buf exporters back their buffers with reclaimable system memory. Some represent device registers, VRAM, or carved-out pools where size is not a meaningful memory cost. Add a memcg_account flag to dma_buf_export_info that exporters set when their buffers should participate in memcg accounting. The xfer ioctl returns -EOPNOTSUPP for buffers that do not set the flag. Currently only the system heap sets it, other exporters can opt in as needed.
Usage example: int cg_fd = open("/sys/fs/cgroup/clients/app1", O_RDONLY | O_DIRECTORY);
struct dma_buf_transfer_charge xfer = { .charge_fd = cg_fd, }; ioctl(dmabuf_fd, DMA_BUF_IOCTL_XFER_CHARGE, &xfer); /* charge now on app1's cgroup */ close(cg_fd);
Signed-off-by: Albert Esteve aesteve@redhat.com
drivers/dma-buf/dma-buf.c | 43 +++++++++++++++++++++++++++++++++++++ drivers/dma-buf/heaps/system_heap.c | 1 + include/linux/dma-buf.h | 5 +++++ include/linux/memcontrol.h | 9 ++++++++ include/uapi/linux/dma-buf.h | 26 ++++++++++++++++++++++ mm/memcontrol.c | 41 +++++++++++++++++++++++++++++++++++ 6 files changed, 125 insertions(+)
diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c index f31f49ebd0af..07513ecedc10 100644 --- a/drivers/dma-buf/dma-buf.c +++ b/drivers/dma-buf/dma-buf.c @@ -540,6 +540,45 @@ static long dma_buf_import_sync_file(struct dma_buf *dmabuf, } #endif +/* Transfer a dma-buf's memcg charge; see struct dma_buf_transfer_charge */ +static long dma_buf_xfer_charge(struct dma_buf *dmabuf,
const void __user *user_data)+{
- struct dma_buf_transfer_charge arg;
- struct mem_cgroup *memcg;
- int ret = 0;
- if (copy_from_user(&arg, user_data, sizeof(arg)))
return -EFAULT;- if (!IS_ENABLED(CONFIG_MEMCG))
return 0;- if (arg.flags != 0)
return -EINVAL;- if (!dmabuf->memcg_account)
return -EOPNOTSUPP;- memcg = mem_cgroup_get_from_fd(arg.charge_fd);
- if (IS_ERR(memcg))
return PTR_ERR(memcg);- if (dmabuf->memcg == memcg)
goto skip_transfer;- if (!mem_cgroup_transfer_dmabuf_charge(dmabuf, memcg)) {
ret = -ENOMEM;goto skip_transfer;- }
- return 0;
+skip_transfer:
- mem_cgroup_put(memcg);
- return ret;
+}
static long dma_buf_ioctl(struct file *file, unsigned int cmd, unsigned long arg) { @@ -590,6 +629,9 @@ static long dma_buf_ioctl(struct file *file, return dma_buf_import_sync_file(dmabuf, (const void __user *)arg); #endif
- case DMA_BUF_IOCTL_XFER_CHARGE:
return dma_buf_xfer_charge(dmabuf, (const void __user *)arg);- default: return -ENOTTY; }
@@ -750,6 +792,7 @@ struct dma_buf *dma_buf_export(const struct dma_buf_export_info *exp_info) dmabuf->size = exp_info->size; dmabuf->exp_name = exp_info->exp_name; dmabuf->owner = exp_info->owner;
- dmabuf->memcg_account = exp_info->memcg_account;
The initial charging should probably be done here as well.
This way DMA-buf heaps only needs to pass in the charge_fd and we can re-use the functionality for other DMA-buf exporters as well.
The problem with doing the charge here is that it happens too late. That is the reasoning for this proposal to move all the charging logic into heap allocators before pages are allocated.
Oh, that is a really good point we need to talk about.
DMA-buf heaps is actually a rather simple use case, but we have way more complex things.
For example in some use cases the backing store of a DMA-buf is only allocated on first use and only then it needs to be comitted to it's memcg or dmem.
Yes. The idea is focus on heaps for now. But the uAPI is actually agnostic of the time when the charge happens. When we deal with lazy-allocation in the future we can just call the reserve/cancel helpers wherever is more appropiate. As long as the pages are not allocated, exporters do not set dmabuf->memcg, and XFER does not work (returns -EOPNOTSUPP).
Yeah exactly that's the reason why I'm raising this topic. That approach won't work in all cases.
See userspace is not aware that the kernel is delaying allocation or that the kernel is allocating the backing store asynchroniously in a background worker for example.
So returning -EOPNOTSUPP when the backing store is not allocated yet and might be allocated later on is rather questionable uAPI because that is racy by design.
Ah! I see what you meant now.
I don't mind implementing it for DMA-buf heaps first, but we should have the uAPI and it's handling so solid that it also works for future use cases.
Agreed. I think we should cover all usecases with the current status. Your feedback helped a lot in getting there!
I mean we can postpone that until we finally need it, but essentially we need to be able to distinct between charging, e.g. making sure the cgroup has enough free limit, and committing, e.g. allowing the resource to be reclaimed.
And then it is possible that when the memcg of a DMA-buf changes that only the charging changes, but there isn't any comitted backing store yet...
Right. That would need adding a specific flag for committed pages or a similar solution. When transfering an uncommited dmabuf only the charge moves (limit reservation and dmabuf->memcg), the stat is untouched. Once the backing store exists, the exporter commits, it sets the flag and bumps MEMCG_DMABUF stat. For memory where its backing store can go away (e.g., dynamic dma-buf sharing), it can have a mirror function that drops the stat but keeps the charge. That would resolve the -EOPNOTSUPP thingy, as transfer succeeds when dmabuf->memcg is set even for uncommitted pages.
My question would be if all exporters know when memory is committed. For those that only get, e.g., a firmware handle and cannot observe the allocation, I guess the buffer can just remain reserved without even being committed. Or in case of protected memory like OP-TEE they would not even opt-in to begin with.
I do not mind adding that for next version, or leaving it for when lazy exporters actually opt-in. It is not a big change either way. Although, it may look weird while the system heap is the only user and commits right away. But either way I can try to document the flag / explain this discussion in the commit.
What we could do is fill the dmabuf->memcg field when the flag is set (i.e., committing into the dmabuf in the patch semantics). But for that we'd need adding the memcg we want to charge it to into exp_info in order to support charging to cgroups other than the allocator's (for the common case would be simpler as we can just look at current). Maybe that's what you meant?
At least for the DMA-buf heaps use case that sounds reasonable for now.
Thanks! Next version will have the flag removed and just use dmabuf->memcg as indicator, set within dma_buf_export(). Also, `nr_pages` will be computed within dma_buf_export(), the contract being that exporters setting exp_info.memcg assert that size reflects actual system memory bytes.
Thinking more about it I would also put changing dmabuf->memcg under the dmabuf->resv lock instead of doing it with cmpxchg.
We don't need it currently, but the dmabuf->resv lock is used for all state changes of the DMA-buf object and this operation is not performance critical in any way possible.
If there is a problem with locking order we at least get lockdep screaming at us very early and not when we want to extend the functionality in the future.
Sure, will do.
BR, Albert.
Regards, Christian.
Regards, Christian.
spin_lock_init(&dmabuf->name_lock); init_waitqueue_head(&dmabuf->poll); dmabuf->cb_in.poll = dmabuf->cb_out.poll = &dmabuf->poll; diff --git a/drivers/dma-buf/heaps/system_heap.c b/drivers/dma-buf/heaps/system_heap.c index adeea8cd328b..a764167167ba 100644 --- a/drivers/dma-buf/heaps/system_heap.c +++ b/drivers/dma-buf/heaps/system_heap.c @@ -479,6 +479,7 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap, exp_info.size = buffer->len; exp_info.flags = fd_flags; exp_info.priv = buffer;
- exp_info.memcg_account = true; dmabuf = dma_buf_export(&exp_info); if (IS_ERR(dmabuf)) { ret = PTR_ERR(dmabuf);
diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h index 23ec0ff23fba..ac0ec88cab93 100644 --- a/include/linux/dma-buf.h +++ b/include/linux/dma-buf.h @@ -433,6 +433,9 @@ struct dma_buf { /** @memcg: the cgroup to which this buffer is currently attributed */ struct mem_cgroup *memcg;
- /** @memcg_account: account buffer memory to cgroup */
- bool memcg_account;
}; /** @@ -505,6 +508,7 @@ struct dma_buf_attachment {
- @flags: mode flags for the file
- @resv: reservation-object, NULL to allocate default one
- @priv: Attach private data of allocator to this buffer
- @memcg_account: account buffer memory to cgroup
- This structure holds the information required to export the buffer. Used
- with dma_buf_export() only.
@@ -517,6 +521,7 @@ struct dma_buf_export_info { int flags; struct dma_resv *resv; void *priv;
- bool memcg_account;
}; /** diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 6b30baa79fa9..36fddafbef4e 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -679,6 +679,9 @@ bool mem_cgroup_reserve_dmabuf(struct mem_cgroup *memcg, gfp_t gfp, void mem_cgroup_commit_dmabuf(struct dma_buf *dmabuf, struct mem_cgroup *memcg, unsigned int nr_pages); +bool mem_cgroup_transfer_dmabuf_charge(struct dma_buf *dmabuf,
struct mem_cgroup *new_memcg);void __mem_cgroup_uncharge(struct folio *folio); /** @@ -1206,6 +1209,12 @@ static inline void mem_cgroup_cancel_dmabuf(struct mem_cgroup *memcg, { } +static inline bool mem_cgroup_transfer_dmabuf_charge(struct dma_buf *dmabuf,
struct mem_cgroup *new_memcg)+{
- return true;
+}
static inline void mem_cgroup_uncharge(struct folio *folio) { } diff --git a/include/uapi/linux/dma-buf.h b/include/uapi/linux/dma-buf.h index e827c9d20c5d..ea957f4b1d38 100644 --- a/include/uapi/linux/dma-buf.h +++ b/include/uapi/linux/dma-buf.h @@ -168,6 +168,31 @@ struct dma_buf_import_sync_file { __s32 fd; }; +/**
- struct dma_buf_transfer_charge - Transfer a dma-buf's memcg charge
- Any holder of a dma-buf file descriptor can perform a
- DMA_BUF_IOCTL_XFER_CHARGE to attribute the buffer's memory
- cost to a specific cgroup, or move it from one cgroup to
- another. The ioctl charges the new cgroup before uncharging
- the old one atomically. On failure the original charge is
- preserved. If the buffer has no existing charge, a new
- charge is established.
- */
+struct dma_buf_transfer_charge {
- /** @flags: Reserved, must be zero */
- __u32 flags; /* unused for now */
- /**
* @charge_fd: File descriptor identifying the target cgroup.** May be either a pidfd, in which case the charge moves to the* memory cgroup of the referenced process, or a cgroupfd obtained* by opening a cgroup v2 directory, in which case the charge moves* directly to that cgroup's memory controller.*/- __u32 charge_fd;
+};
#define DMA_BUF_BASE 'b' #define DMA_BUF_IOCTL_SYNC _IOW(DMA_BUF_BASE, 0, struct dma_buf_sync) @@ -179,5 +204,6 @@ struct dma_buf_import_sync_file { #define DMA_BUF_SET_NAME_B _IOW(DMA_BUF_BASE, 1, __u64) #define DMA_BUF_IOCTL_EXPORT_SYNC_FILE _IOWR(DMA_BUF_BASE, 2, struct dma_buf_export_sync_file) #define DMA_BUF_IOCTL_IMPORT_SYNC_FILE _IOW(DMA_BUF_BASE, 3, struct dma_buf_import_sync_file) +#define DMA_BUF_IOCTL_XFER_CHARGE _IOW(DMA_BUF_BASE, 4, struct dma_buf_transfer_charge) #endif diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 7b841b905759..3adc84fe6668 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -5786,6 +5786,47 @@ void mem_cgroup_cancel_dmabuf(struct mem_cgroup *memcg, unsigned int nr_pages) refill_stock(memcg, nr_pages); } +/**
- mem_cgroup_transfer_dmabuf_charge - move a dma-buf charge
- @dmabuf: buffer to transfer charge
- @new_memcg: target memcg; caller must hold a reference
- Transfers the charge for @dmabuf to @new_memcg and, on
- success, stores @new_memcg in @dmabuf->memcg.
- Returns %true on success, %false if the charge against @new_memcg
- failed or a concurrent transfer raced on @dmabuf->memcg.
- On failure the caller retains its reference to @new_memcg.
- */
+bool mem_cgroup_transfer_dmabuf_charge(struct dma_buf *dmabuf,
struct mem_cgroup *new_memcg)I would make that more generic instead of dmabuf specific. E.g. something like this:
bool mem_cgroup_transfer_charge(struct struct mem_cgroup **ptr_memcg, struct mem_cgroup *new_memcg, enum memcg_stat_item item, unsigned int nr_pages)
Ah yes, I like this generalization. And the implementation remains mostly the same. Thanks!
+{
- struct mem_cgroup *old_memcg = READ_ONCE(dmabuf->memcg);
- unsigned int nr_pages = PAGE_ALIGN(dmabuf->size) >> PAGE_SHIFT;
This way we can also document in the DMA-buf caller that this conversation must be ok when memcg_account is true.
Regards, Christian.
- if (mem_cgroup_disabled())
return true;- if (!mem_cgroup_reserve_dmabuf(new_memcg, GFP_KERNEL, nr_pages))
return false;- mod_memcg_state(new_memcg, MEMCG_DMABUF, nr_pages);
- if (cmpxchg(&dmabuf->memcg, old_memcg, new_memcg) != old_memcg) {
mod_memcg_state(new_memcg, MEMCG_DMABUF, -nr_pages);mem_cgroup_cancel_dmabuf(new_memcg, nr_pages);return false;- }
- if (old_memcg) {
/* Uncharge old cgroup */mod_memcg_state(old_memcg, MEMCG_DMABUF, -nr_pages);mem_cgroup_cancel_dmabuf(old_memcg, nr_pages);mem_cgroup_put(old_memcg);- }
- return true;
+}
/**
- mem_cgroup_uncharge_dmabuf - uncharge a dma-buf from its memcg
- @dmabuf: buffer to uncharge
On 11/2/26 10:45, Albert Esteve wrote:
On Fri, Oct 30, 2026 at 11:14:50AM +0100, Christian König wrote:
On 10/29/26 17:23, Albert Esteve wrote:
On Thu, Oct 29, 2026 at 03:16:12PM +0100, Christian König wrote:
On 10/28/26 17:09, Albert Esteve wrote:
On Wed, Oct 28, 2026 at 10:08:19AM +0100, Christian König wrote:
On 10/27/26 16:02, Albert Esteve wrote: > dma-heap already supports charging a specific cgroup at allocation > time via charge_fd in DMA_HEAP_IOCTL_ALLOC. However, dma-bufs can > originate from any exporter (DRM, V4L2, etc.), and the allocator may > not know the final owner at allocation time. > > Add DMA_BUF_IOCTL_XFER_CHARGE to transfer a dma-buf's memcg charge > to a different cgroup after allocation. The target is identified by > charge_fd, which may be a cgroupfd or a pidfd. > > The transfer charges the new cgroup before uncharging the old one, > preserving the original charge on failure. An atomic cmpxchg on > dmabuf->memcg prevents races with concurrent transfers. > > Not all dma-buf exporters back their buffers with reclaimable system > memory. Some represent device registers, VRAM, or carved-out pools > where size is not a meaningful memory cost. Add a memcg_account flag > to dma_buf_export_info that exporters set when their buffers should > participate in memcg accounting. The xfer ioctl returns -EOPNOTSUPP > for buffers that do not set the flag. Currently only the system heap > sets it, other exporters can opt in as needed. > > Usage example: > int cg_fd = open("/sys/fs/cgroup/clients/app1", O_RDONLY | O_DIRECTORY); > > struct dma_buf_transfer_charge xfer = { > .charge_fd = cg_fd, > }; > ioctl(dmabuf_fd, DMA_BUF_IOCTL_XFER_CHARGE, &xfer); > /* charge now on app1's cgroup */ > close(cg_fd); > > Signed-off-by: Albert Esteve aesteve@redhat.com > --- > drivers/dma-buf/dma-buf.c | 43 +++++++++++++++++++++++++++++++++++++ > drivers/dma-buf/heaps/system_heap.c | 1 + > include/linux/dma-buf.h | 5 +++++ > include/linux/memcontrol.h | 9 ++++++++ > include/uapi/linux/dma-buf.h | 26 ++++++++++++++++++++++ > mm/memcontrol.c | 41 +++++++++++++++++++++++++++++++++++ > 6 files changed, 125 insertions(+) > > diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c > index f31f49ebd0af..07513ecedc10 100644 > --- a/drivers/dma-buf/dma-buf.c > +++ b/drivers/dma-buf/dma-buf.c > @@ -540,6 +540,45 @@ static long dma_buf_import_sync_file(struct dma_buf *dmabuf, > } > #endif > > +/* Transfer a dma-buf's memcg charge; see struct dma_buf_transfer_charge */ > +static long dma_buf_xfer_charge(struct dma_buf *dmabuf, > + const void __user *user_data) > +{ > + struct dma_buf_transfer_charge arg; > + struct mem_cgroup *memcg; > + int ret = 0; > + > + if (copy_from_user(&arg, user_data, sizeof(arg))) > + return -EFAULT; > + > + if (!IS_ENABLED(CONFIG_MEMCG)) > + return 0; > + > + if (arg.flags != 0) > + return -EINVAL; > + > + if (!dmabuf->memcg_account) > + return -EOPNOTSUPP; > + > + memcg = mem_cgroup_get_from_fd(arg.charge_fd); > + if (IS_ERR(memcg)) > + return PTR_ERR(memcg); > + > + if (dmabuf->memcg == memcg) > + goto skip_transfer; > + > + if (!mem_cgroup_transfer_dmabuf_charge(dmabuf, memcg)) { > + ret = -ENOMEM; > + goto skip_transfer; > + } > + > + return 0; > + > +skip_transfer: > + mem_cgroup_put(memcg); > + return ret; > +} > + > static long dma_buf_ioctl(struct file *file, > unsigned int cmd, unsigned long arg) > { > @@ -590,6 +629,9 @@ static long dma_buf_ioctl(struct file *file, > return dma_buf_import_sync_file(dmabuf, (const void __user *)arg); > #endif > > + case DMA_BUF_IOCTL_XFER_CHARGE: > + return dma_buf_xfer_charge(dmabuf, (const void __user *)arg); > + > default: > return -ENOTTY; > } > @@ -750,6 +792,7 @@ struct dma_buf *dma_buf_export(const struct dma_buf_export_info *exp_info) > dmabuf->size = exp_info->size; > dmabuf->exp_name = exp_info->exp_name; > dmabuf->owner = exp_info->owner; > + dmabuf->memcg_account = exp_info->memcg_account;
The initial charging should probably be done here as well.
This way DMA-buf heaps only needs to pass in the charge_fd and we can re-use the functionality for other DMA-buf exporters as well.
The problem with doing the charge here is that it happens too late. That is the reasoning for this proposal to move all the charging logic into heap allocators before pages are allocated.
Oh, that is a really good point we need to talk about.
DMA-buf heaps is actually a rather simple use case, but we have way more complex things.
For example in some use cases the backing store of a DMA-buf is only allocated on first use and only then it needs to be comitted to it's memcg or dmem.
Yes. The idea is focus on heaps for now. But the uAPI is actually agnostic of the time when the charge happens. When we deal with lazy-allocation in the future we can just call the reserve/cancel helpers wherever is more appropiate. As long as the pages are not allocated, exporters do not set dmabuf->memcg, and XFER does not work (returns -EOPNOTSUPP).
Yeah exactly that's the reason why I'm raising this topic. That approach won't work in all cases.
See userspace is not aware that the kernel is delaying allocation or that the kernel is allocating the backing store asynchroniously in a background worker for example.
So returning -EOPNOTSUPP when the backing store is not allocated yet and might be allocated later on is rather questionable uAPI because that is racy by design.
Ah! I see what you meant now.
I don't mind implementing it for DMA-buf heaps first, but we should have the uAPI and it's handling so solid that it also works for future use cases.
Agreed. I think we should cover all usecases with the current status. Your feedback helped a lot in getting there!
I mean we can postpone that until we finally need it, but essentially we need to be able to distinct between charging, e.g. making sure the cgroup has enough free limit, and committing, e.g. allowing the resource to be reclaimed.
And then it is possible that when the memcg of a DMA-buf changes that only the charging changes, but there isn't any comitted backing store yet...
Right. That would need adding a specific flag for committed pages or a similar solution. When transfering an uncommited dmabuf only the charge moves (limit reservation and dmabuf->memcg), the stat is untouched. Once the backing store exists, the exporter commits, it sets the flag and bumps MEMCG_DMABUF stat. For memory where its backing store can go away (e.g., dynamic dma-buf sharing), it can have a mirror function that drops the stat but keeps the charge. That would resolve the -EOPNOTSUPP thingy, as transfer succeeds when dmabuf->memcg is set even for uncommitted pages.
My question would be if all exporters know when memory is committed.
Yes, the exporter "owns" the backing store and is the only one who can allocate it based on requests from importers (or itself).
For those that only get, e.g., a firmware handle and cannot observe the allocation, I guess the buffer can just remain reserved without even being committed. Or in case of protected memory like OP-TEE they would not even opt-in to begin with.
I do not mind adding that for next version, or leaving it for when lazy exporters actually opt-in. It is not a big change either way. Although, it may look weird while the system heap is the only user and commits right away. But either way I can try to document the flag / explain this discussion in the commit.
I think we should keep it simple and stupid for now, just not make decisions which would block extending it in the future.
What we could do is fill the dmabuf->memcg field when the flag is set (i.e., committing into the dmabuf in the patch semantics). But for that we'd need adding the memcg we want to charge it to into exp_info in order to support charging to cgroups other than the allocator's (for the common case would be simpler as we can just look at current). Maybe that's what you meant?
At least for the DMA-buf heaps use case that sounds reasonable for now.
Thanks! Next version will have the flag removed and just use dmabuf->memcg as indicator, set within dma_buf_export(). Also, `nr_pages` will be computed within dma_buf_export(), the contract being that exporters setting exp_info.memcg assert that size reflects actual system memory bytes.
Thinking more about it I would also put changing dmabuf->memcg under the dmabuf->resv lock instead of doing it with cmpxchg.
We don't need it currently, but the dmabuf->resv lock is used for all state changes of the DMA-buf object and this operation is not performance critical in any way possible.
If there is a problem with locking order we at least get lockdep screaming at us very early and not when we want to extend the functionality in the future.
Sure, will do.
Thanks, Christian.
BR, Albert.
Regards, Christian.
Regards, Christian.
> spin_lock_init(&dmabuf->name_lock); > init_waitqueue_head(&dmabuf->poll); > dmabuf->cb_in.poll = dmabuf->cb_out.poll = &dmabuf->poll; > diff --git a/drivers/dma-buf/heaps/system_heap.c b/drivers/dma-buf/heaps/system_heap.c > index adeea8cd328b..a764167167ba 100644 > --- a/drivers/dma-buf/heaps/system_heap.c > +++ b/drivers/dma-buf/heaps/system_heap.c > @@ -479,6 +479,7 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap, > exp_info.size = buffer->len; > exp_info.flags = fd_flags; > exp_info.priv = buffer; > + exp_info.memcg_account = true; > dmabuf = dma_buf_export(&exp_info); > if (IS_ERR(dmabuf)) { > ret = PTR_ERR(dmabuf); > diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h > index 23ec0ff23fba..ac0ec88cab93 100644 > --- a/include/linux/dma-buf.h > +++ b/include/linux/dma-buf.h > @@ -433,6 +433,9 @@ struct dma_buf { > > /** @memcg: the cgroup to which this buffer is currently attributed */ > struct mem_cgroup *memcg; > + > + /** @memcg_account: account buffer memory to cgroup */ > + bool memcg_account; > }; > > /** > @@ -505,6 +508,7 @@ struct dma_buf_attachment { > * @flags: mode flags for the file > * @resv: reservation-object, NULL to allocate default one > * @priv: Attach private data of allocator to this buffer > + * @memcg_account: account buffer memory to cgroup > * > * This structure holds the information required to export the buffer. Used > * with dma_buf_export() only. > @@ -517,6 +521,7 @@ struct dma_buf_export_info { > int flags; > struct dma_resv *resv; > void *priv; > + bool memcg_account; > }; > > /** > diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h > index 6b30baa79fa9..36fddafbef4e 100644 > --- a/include/linux/memcontrol.h > +++ b/include/linux/memcontrol.h > @@ -679,6 +679,9 @@ bool mem_cgroup_reserve_dmabuf(struct mem_cgroup *memcg, gfp_t gfp, > void mem_cgroup_commit_dmabuf(struct dma_buf *dmabuf, struct mem_cgroup *memcg, > unsigned int nr_pages); > > +bool mem_cgroup_transfer_dmabuf_charge(struct dma_buf *dmabuf, > + struct mem_cgroup *new_memcg); > + > void __mem_cgroup_uncharge(struct folio *folio); > > /** > @@ -1206,6 +1209,12 @@ static inline void mem_cgroup_cancel_dmabuf(struct mem_cgroup *memcg, > { > } > > +static inline bool mem_cgroup_transfer_dmabuf_charge(struct dma_buf *dmabuf, > + struct mem_cgroup *new_memcg) > +{ > + return true; > +} > + > static inline void mem_cgroup_uncharge(struct folio *folio) > { > } > diff --git a/include/uapi/linux/dma-buf.h b/include/uapi/linux/dma-buf.h > index e827c9d20c5d..ea957f4b1d38 100644 > --- a/include/uapi/linux/dma-buf.h > +++ b/include/uapi/linux/dma-buf.h > @@ -168,6 +168,31 @@ struct dma_buf_import_sync_file { > __s32 fd; > }; > > +/** > + * struct dma_buf_transfer_charge - Transfer a dma-buf's memcg charge > + * > + * Any holder of a dma-buf file descriptor can perform a > + * DMA_BUF_IOCTL_XFER_CHARGE to attribute the buffer's memory > + * cost to a specific cgroup, or move it from one cgroup to > + * another. The ioctl charges the new cgroup before uncharging > + * the old one atomically. On failure the original charge is > + * preserved. If the buffer has no existing charge, a new > + * charge is established. > + */ > +struct dma_buf_transfer_charge { > + /** @flags: Reserved, must be zero */ > + __u32 flags; /* unused for now */ > + /** > + * @charge_fd: File descriptor identifying the target cgroup. > + * > + * May be either a pidfd, in which case the charge moves to the > + * memory cgroup of the referenced process, or a cgroupfd obtained > + * by opening a cgroup v2 directory, in which case the charge moves > + * directly to that cgroup's memory controller. > + */ > + __u32 charge_fd; > +}; > + > #define DMA_BUF_BASE 'b' > #define DMA_BUF_IOCTL_SYNC _IOW(DMA_BUF_BASE, 0, struct dma_buf_sync) > > @@ -179,5 +204,6 @@ struct dma_buf_import_sync_file { > #define DMA_BUF_SET_NAME_B _IOW(DMA_BUF_BASE, 1, __u64) > #define DMA_BUF_IOCTL_EXPORT_SYNC_FILE _IOWR(DMA_BUF_BASE, 2, struct dma_buf_export_sync_file) > #define DMA_BUF_IOCTL_IMPORT_SYNC_FILE _IOW(DMA_BUF_BASE, 3, struct dma_buf_import_sync_file) > +#define DMA_BUF_IOCTL_XFER_CHARGE _IOW(DMA_BUF_BASE, 4, struct dma_buf_transfer_charge) > > #endif > diff --git a/mm/memcontrol.c b/mm/memcontrol.c > index 7b841b905759..3adc84fe6668 100644 > --- a/mm/memcontrol.c > +++ b/mm/memcontrol.c > @@ -5786,6 +5786,47 @@ void mem_cgroup_cancel_dmabuf(struct mem_cgroup *memcg, unsigned int nr_pages) > refill_stock(memcg, nr_pages); > } > > +/** > + * mem_cgroup_transfer_dmabuf_charge - move a dma-buf charge > + * @dmabuf: buffer to transfer charge > + * @new_memcg: target memcg; caller must hold a reference > + * > + * Transfers the charge for @dmabuf to @new_memcg and, on > + * success, stores @new_memcg in @dmabuf->memcg. > + * Returns %true on success, %false if the charge against @new_memcg > + * failed or a concurrent transfer raced on @dmabuf->memcg. > + * On failure the caller retains its reference to @new_memcg. > + */ > +bool mem_cgroup_transfer_dmabuf_charge(struct dma_buf *dmabuf, > + struct mem_cgroup *new_memcg)
I would make that more generic instead of dmabuf specific. E.g. something like this:
bool mem_cgroup_transfer_charge(struct struct mem_cgroup **ptr_memcg, struct mem_cgroup *new_memcg, enum memcg_stat_item item, unsigned int nr_pages)
Ah yes, I like this generalization. And the implementation remains mostly the same. Thanks!
> +{ > + struct mem_cgroup *old_memcg = READ_ONCE(dmabuf->memcg);
> + unsigned int nr_pages = PAGE_ALIGN(dmabuf->size) >> PAGE_SHIFT;
This way we can also document in the DMA-buf caller that this conversation must be ok when memcg_account is true.
Regards, Christian.
> + > + if (mem_cgroup_disabled()) > + return true; > + > + if (!mem_cgroup_reserve_dmabuf(new_memcg, GFP_KERNEL, nr_pages)) > + return false; > + > + mod_memcg_state(new_memcg, MEMCG_DMABUF, nr_pages); > + > + if (cmpxchg(&dmabuf->memcg, old_memcg, new_memcg) != old_memcg) { > + mod_memcg_state(new_memcg, MEMCG_DMABUF, -nr_pages); > + mem_cgroup_cancel_dmabuf(new_memcg, nr_pages); > + return false; > + } > + > + if (old_memcg) { > + /* Uncharge old cgroup */ > + mod_memcg_state(old_memcg, MEMCG_DMABUF, -nr_pages); > + mem_cgroup_cancel_dmabuf(old_memcg, nr_pages); > + mem_cgroup_put(old_memcg); > + } > + > + return true; > +} > + > /** > * mem_cgroup_uncharge_dmabuf - uncharge a dma-buf from its memcg > * @dmabuf: buffer to uncharge >
Add tests for the new charge_fd field in struct dma_heap_allocation_data.
When the charge_fd feature is absent (unpatched kernel), the probe in pidfd_alloc_supported() detects this and the tests are skipped gracefully.
Add vmtest.sh similar to other subsystem suites, to orchestrate building the selftests (optionally with a freshly compiled kernel) inside a virtme-ng VM, so the tests can be run without modifying the host system. Add a config fragment with required Kconfig symbols.
Also add test_memcg_dmabuf() to the existing test_memcontrol suite to verify end-to-end cross-cgroup accounting: a parent process opens a pidfd for a child in a separate cgroup, allocates a dma-buf via DMA_HEAP_IOCTL_ALLOC with that pidfd, and asserts that memory.stat dmabuf in the child's cgroup reflects the allocation. If the dmabuf key is missing (unpatched kernel) or /dev/dma_heap/system is absent, the test is skipped.
Assisted-by: LLM Signed-off-by: Albert Esteve aesteve@redhat.com --- tools/testing/selftests/cgroup/Makefile | 2 +- tools/testing/selftests/cgroup/test_memcontrol.c | 290 ++++++++++++++++++++- tools/testing/selftests/dmabuf-heaps/config | 1 + tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c | 227 +++++++++++++++- tools/testing/selftests/dmabuf-heaps/vmtest.sh | 226 ++++++++++++++++ 5 files changed, 742 insertions(+), 4 deletions(-)
diff --git a/tools/testing/selftests/cgroup/Makefile b/tools/testing/selftests/cgroup/Makefile index e01584c2189a..9edfc9f1de5c 100644 --- a/tools/testing/selftests/cgroup/Makefile +++ b/tools/testing/selftests/cgroup/Makefile @@ -1,5 +1,5 @@ # SPDX-License-Identifier: GPL-2.0 -CFLAGS += -Wall -pthread +CFLAGS += -Wall -pthread $(KHDR_INCLUDES)
all: ${HELPER_PROGS}
diff --git a/tools/testing/selftests/cgroup/test_memcontrol.c b/tools/testing/selftests/cgroup/test_memcontrol.c index 0ed82347044e..0b41c3c6f62c 100644 --- a/tools/testing/selftests/cgroup/test_memcontrol.c +++ b/tools/testing/selftests/cgroup/test_memcontrol.c @@ -19,10 +19,20 @@ #include <errno.h> #include <sys/mman.h>
+#include <linux/dma-buf.h> +#include <linux/dma-heap.h> +#include <signal.h> +#include <sys/ioctl.h> + +#include "../pidfd/pidfd.h" #include "kselftest.h" #include "cgroup_util.h"
+#define DMA_HEAP_SYSTEM "/dev/dma_heap/system" +#define ONE_MEG (1024 * 1024) + #define MEMCG_SOCKSTAT_WAIT_RETRIES 30 +#define MEMCG_DMABUF_WAIT_RETRIES 15
static bool has_localevents; static bool has_recursiveprot; @@ -1764,6 +1774,267 @@ static int test_memcg_inotify_delete_dir(const char *root) return ret; }
+static int memcg_dmabuf_child(const char *cgroup, void *arg) +{ + pause(); + return 0; +} + +/* + * This test allocates a dma-buf via DMA_HEAP_IOCTL_ALLOC with a pidfd + * pointing to a child process in a separate cgroup, then checks that + * memory.stat[dmabuf] in the child's cgroup rises by the allocation size + * and returns to zero after the buffer fd is closed. + */ +static int test_memcg_dmabuf(const char *root) +{ + char *parent = NULL, *child_cg = NULL; + int ret = KSFT_FAIL; + int heap_fd = -1, dmabuf_fd = -1, pidfd = -1; + pid_t child_pid; + int child_status; + long dmabuf_stat; + struct dma_heap_allocation_data alloc = { + .len = ONE_MEG, + .fd_flags = O_RDWR | O_CLOEXEC, + }; + + if (access(DMA_HEAP_SYSTEM, R_OK | W_OK)) { + ret = KSFT_SKIP; + goto cleanup; + } + + parent = cg_name(root, "dmabuf_memcg_test"); + if (!parent) + goto cleanup; + + if (cg_create(parent)) + goto cleanup_parent; + + if (cg_write(parent, "cgroup.subtree_control", "+memory")) + goto cleanup_parent; + + child_cg = cg_name(parent, "child"); + if (!child_cg) + goto cleanup_parent; + + if (cg_create(child_cg)) + goto cleanup_parent; + + child_pid = cg_run_nowait(child_cg, memcg_dmabuf_child, NULL); + if (child_pid < 0) + goto cleanup_child; + + if (cg_wait_for_proc_count(child_cg, 1)) + goto cleanup_kill; + + pidfd = sys_pidfd_open(child_pid, 0); + if (pidfd < 0) { + ret = KSFT_SKIP; + goto cleanup_kill; + } + + heap_fd = open(DMA_HEAP_SYSTEM, O_RDWR); + if (heap_fd < 0) { + ret = KSFT_SKIP; + goto cleanup_pidfd; + } + + alloc.charge_fd = (__u32)pidfd; + if (ioctl(heap_fd, DMA_HEAP_IOCTL_ALLOC, &alloc) < 0) + goto cleanup_heap; + dmabuf_fd = (int)alloc.fd; + + dmabuf_stat = cg_read_key_long(child_cg, "memory.stat", "dmabuf "); + if (dmabuf_stat == -1) { + ret = KSFT_SKIP; + goto cleanup_dmabuf; + } + if (dmabuf_stat != ONE_MEG) + dmabuf_stat = cg_read_key_long_poll(child_cg, "memory.stat", + "dmabuf ", ONE_MEG, + MEMCG_DMABUF_WAIT_RETRIES, + DEFAULT_WAIT_INTERVAL_US); + if (dmabuf_stat != ONE_MEG) { + fprintf(stderr, "Expected dmabuf stat %d, got %ld\n", + ONE_MEG, dmabuf_stat); + goto cleanup_dmabuf; + } + + close(dmabuf_fd); + dmabuf_fd = -1; + + dmabuf_stat = cg_read_key_long_poll(child_cg, "memory.stat", + "dmabuf ", 0, + MEMCG_DMABUF_WAIT_RETRIES, + DEFAULT_WAIT_INTERVAL_US); + if (dmabuf_stat != 0) { + fprintf(stderr, "Expected dmabuf stat 0 after close, got %ld\n", + dmabuf_stat); + goto cleanup_heap; + } + + ret = KSFT_PASS; + +cleanup_dmabuf: + if (dmabuf_fd >= 0) + close(dmabuf_fd); +cleanup_heap: + close(heap_fd); +cleanup_pidfd: + close(pidfd); +cleanup_kill: + kill(child_pid, SIGTERM); + waitpid(child_pid, &child_status, 0); +cleanup_child: + cg_destroy(child_cg); + free(child_cg); +cleanup_parent: + cg_destroy(parent); + free(parent); +cleanup: + return ret; +} + +/* + * Allocate a dma-buf in cgroup A, set memory.max on cgroup B below the + * buffer size, then attempt DMA_BUF_IOCTL_XFER_CHARGE to B. The transfer + * must fail with ENOMEM and the charge must remain in A. + */ +static int test_memcg_dmabuf_xfer_at_limit(const char *root) +{ + char *parent = NULL, *cg_a = NULL, *cg_b = NULL; + int ret = KSFT_FAIL; + int heap_fd = -1, dmabuf_fd = -1, cg_b_fd = -1; + pid_t child_pid; + int child_status; + long stat_a; + struct dma_heap_allocation_data alloc = { + .len = ONE_MEG, + .fd_flags = O_RDWR | O_CLOEXEC, + }; + struct dma_buf_transfer_charge xfer = { 0 }; + + if (access(DMA_HEAP_SYSTEM, R_OK | W_OK)) { + ret = KSFT_SKIP; + goto cleanup; + } + + parent = cg_name(root, "dmabuf_xfer_limit_test"); + if (!parent) + goto cleanup; + + if (cg_create(parent)) + goto cleanup_parent; + + if (cg_write(parent, "cgroup.subtree_control", "+memory")) + goto cleanup_parent; + + cg_a = cg_name(parent, "cg_a"); + cg_b = cg_name(parent, "cg_b"); + if (!cg_a || !cg_b) + goto cleanup_cgs; + + if (cg_create(cg_a) || cg_create(cg_b)) + goto cleanup_cgs; + + if (cg_write(cg_b, "memory.max", "524288")) + goto cleanup_cgs; + + child_pid = cg_run_nowait(cg_a, memcg_dmabuf_child, NULL); + if (child_pid < 0) + goto cleanup_cgs; + + if (cg_wait_for_proc_count(cg_a, 1)) + goto cleanup_kill; + + heap_fd = open(DMA_HEAP_SYSTEM, O_RDWR); + if (heap_fd < 0) { + ret = KSFT_SKIP; + goto cleanup_kill; + } + + alloc.charge_fd = (__u32)sys_pidfd_open(child_pid, 0); + if ((int)alloc.charge_fd < 0) { + ret = KSFT_SKIP; + goto cleanup_heap; + } + + if (ioctl(heap_fd, DMA_HEAP_IOCTL_ALLOC, &alloc) < 0) { + close(alloc.charge_fd); + goto cleanup_heap; + } + close(alloc.charge_fd); + dmabuf_fd = (int)alloc.fd; + + stat_a = cg_read_key_long(cg_a, "memory.stat", "dmabuf "); + if (stat_a == -1) { + ret = KSFT_SKIP; + goto cleanup_dmabuf; + } + if (stat_a != ONE_MEG) + stat_a = cg_read_key_long_poll(cg_a, "memory.stat", + "dmabuf ", ONE_MEG, + MEMCG_DMABUF_WAIT_RETRIES, + DEFAULT_WAIT_INTERVAL_US); + if (stat_a != ONE_MEG) { + fprintf(stderr, "Charge not in A after alloc: expected %d, got %ld\n", + ONE_MEG, stat_a); + goto cleanup_dmabuf; + } + + cg_b_fd = open(cg_b, O_RDONLY | O_DIRECTORY); + if (cg_b_fd < 0) + goto cleanup_dmabuf; + + xfer.charge_fd = (__u32)cg_b_fd; + if (ioctl(dmabuf_fd, DMA_BUF_IOCTL_XFER_CHARGE, &xfer) == 0) { + fprintf(stderr, "xfer_charge should have failed at limit\n"); + goto cleanup_cgfd; + } + if (errno != ENOMEM) { + fprintf(stderr, "Expected ENOMEM, got %s\n", strerror(errno)); + goto cleanup_cgfd; + } + + stat_a = cg_read_key_long_poll(cg_a, "memory.stat", + "dmabuf ", ONE_MEG, + MEMCG_DMABUF_WAIT_RETRIES, + DEFAULT_WAIT_INTERVAL_US); + if (stat_a != ONE_MEG) { + fprintf(stderr, "Charge leaked from A: expected %d, got %ld\n", + ONE_MEG, stat_a); + goto cleanup_cgfd; + } + + ret = KSFT_PASS; + +cleanup_cgfd: + close(cg_b_fd); +cleanup_dmabuf: + if (dmabuf_fd >= 0) + close(dmabuf_fd); +cleanup_heap: + close(heap_fd); +cleanup_kill: + kill(child_pid, SIGTERM); + waitpid(child_pid, &child_status, 0); +cleanup_cgs: + if (cg_a) { + cg_destroy(cg_a); + free(cg_a); + } + if (cg_b) { + cg_destroy(cg_b); + free(cg_b); + } +cleanup_parent: + cg_destroy(parent); + free(parent); +cleanup: + return ret; +} + #define T(x) { x, #x } struct memcg_test { int (*fn)(const char *root); @@ -1785,13 +2056,19 @@ struct memcg_test { T(test_memcg_oom_group_score_events), T(test_memcg_inotify_delete_file), T(test_memcg_inotify_delete_dir), + T(test_memcg_dmabuf), + T(test_memcg_dmabuf_xfer_at_limit), }; #undef T
int main(int argc, char **argv) { char root[PATH_MAX]; - int i, proc_status; + int i, proc_status, count = ARRAY_SIZE(tests); + const char *filter = NULL; + + if (argc > 1) + filter = argv[1];
page_size = sysconf(_SC_PAGE_SIZE); if (page_size <= 0) @@ -1822,8 +2099,17 @@ int main(int argc, char **argv) ksft_exit_skip("Failed to query cgroup mount option\n"); has_localevents = proc_status;
- ksft_set_plan(ARRAY_SIZE(tests)); + if (filter) { + count = 0; + for (i = 0; i < ARRAY_SIZE(tests); i++) { + if (!strcmp(tests[i].name, filter)) + count++; + } + } + ksft_set_plan(count); for (i = 0; i < ARRAY_SIZE(tests); i++) { + if (filter && strcmp(tests[i].name, filter)) + continue; switch (tests[i].fn(root)) { case KSFT_PASS: ksft_test_result_pass("%s\n", tests[i].name); diff --git a/tools/testing/selftests/dmabuf-heaps/config b/tools/testing/selftests/dmabuf-heaps/config index be091f1cdfa0..94c8f33b71a2 100644 --- a/tools/testing/selftests/dmabuf-heaps/config +++ b/tools/testing/selftests/dmabuf-heaps/config @@ -1,3 +1,4 @@ +CONFIG_MEMCG=y CONFIG_DMABUF_HEAPS=y CONFIG_DMABUF_HEAPS_SYSTEM=y CONFIG_DRM_VGEM=y diff --git a/tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c b/tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c index 1d49df671919..90ef14d2ecba 100644 --- a/tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c +++ b/tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c @@ -3,6 +3,7 @@ #include <dirent.h> #include <errno.h> #include <fcntl.h> +#include <signal.h> #include <stdio.h> #include <stdlib.h> #include <stdint.h> @@ -10,11 +11,15 @@ #include <unistd.h> #include <sys/ioctl.h> #include <sys/mman.h> +#include <sys/syscall.h> #include <sys/types.h> +#include <sys/wait.h>
#include <linux/dma-buf.h> #include <linux/dma-heap.h> +#include <linux/limits.h> #include <drm/drm.h> +#include "../pidfd/pidfd.h" #include "kselftest.h"
#define DEVPATH "/dev/dma_heap" @@ -320,6 +325,8 @@ static int dmabuf_heap_alloc_newer(int fd, size_t len, unsigned int flags, __u32 fd; __u32 fd_flags; __u64 heap_flags; + __u32 charge_fd; + __u32 __padding; __u64 garbage1; __u64 garbage2; __u64 garbage3; @@ -328,6 +335,8 @@ static int dmabuf_heap_alloc_newer(int fd, size_t len, unsigned int flags, .fd = 0, .fd_flags = O_RDWR | O_CLOEXEC, .heap_flags = flags, + .charge_fd = 0, + .__padding = 0, .garbage1 = 0xffffffff, .garbage2 = 0x88888888, .garbage3 = 0x11111111, @@ -496,6 +505,218 @@ static void test_alloc_no_fd_leak_on_efault(char *heap_name) ksft_test_result(fd_before == fd_after, "fd leak on EFAULT: before=%d after=%d\n", fd_before, fd_after); + +out: + close(heap_fd); +} + +static int dmabuf_heap_alloc_pidfd(int fd, size_t len, unsigned int heap_flags, + unsigned int charge_fd, int *dmabuf_fd) +{ + struct dma_heap_allocation_data data = { + .len = len, + .fd = 0, + .fd_flags = O_RDWR | O_CLOEXEC, + .heap_flags = heap_flags, + .charge_fd = charge_fd, + }; + int ret; + + if (!dmabuf_fd) + return -EINVAL; + + ret = ioctl(fd, DMA_HEAP_IOCTL_ALLOC, &data); + if (ret < 0) + return ret; + *dmabuf_fd = (int)data.fd; + return ret; +} + +/* + * Probe whether the kernel honours charge_fd in DMA_HEAP_IOCTL_ALLOC. + */ +static bool pidfd_alloc_supported(int heap_fd) +{ + int devnull_fd, dmabuf_fd = -1, ret; + + devnull_fd = open("/dev/null", O_RDONLY); + if (devnull_fd < 0) + return false; + + ret = dmabuf_heap_alloc_pidfd(heap_fd, ONE_MEG, 0, devnull_fd, &dmabuf_fd); + if (dmabuf_fd >= 0) { + close(dmabuf_fd); + dmabuf_fd = -1; + } + close(devnull_fd); + return ret < 0; +} + +/* + * Test: allocate charging the calling process's own cgroup via a self pidfd. + */ +static void test_alloc_pidfd_self(char *heap_name) +{ + int heap_fd = -1, pidfd = -1, dmabuf_fd = -1, ret; + + heap_fd = dmabuf_heap_open(heap_name); + + if (!pidfd_alloc_supported(heap_fd)) { + ksft_test_result_skip("charge_fd not supported by this kernel\n"); + goto out; + } + + pidfd = sys_pidfd_open(getpid(), 0); + if (pidfd < 0) { + ksft_test_result_skip("pidfd_open not available\n"); + goto out; + } + + ret = dmabuf_heap_alloc_pidfd(heap_fd, ONE_MEG, 0, pidfd, &dmabuf_fd); + ksft_test_result(!ret, "Allocation with self pidfd %d\n", ret); + if (dmabuf_fd >= 0) + close(dmabuf_fd); + close(pidfd); +out: + close(heap_fd); +} + +/* + * Test: allocate charging a child process's cgroup via a child pidfd. + */ +static void test_alloc_pidfd_child(char *heap_name) +{ + int heap_fd = -1, pidfd = -1, dmabuf_fd = -1; + pid_t child_pid; + int status, ret; + + heap_fd = dmabuf_heap_open(heap_name); + + if (!pidfd_alloc_supported(heap_fd)) { + ksft_test_result_skip("charge_fd not supported by this kernel\n"); + goto out; + } + + child_pid = fork(); + if (child_pid == 0) { + pause(); + _exit(0); + } + if (child_pid < 0) + ksft_exit_fail_msg("fork failed: %s\n", strerror(errno)); + + pidfd = sys_pidfd_open(child_pid, 0); + if (pidfd < 0) { + kill(child_pid, SIGTERM); + waitpid(child_pid, &status, 0); + ksft_test_result_skip("pidfd_open for child failed\n"); + goto out; + } + + ret = dmabuf_heap_alloc_pidfd(heap_fd, ONE_MEG, 0, pidfd, &dmabuf_fd); + ksft_test_result(!ret, "Allocation with child pidfd %d\n", ret); + if (dmabuf_fd >= 0) + close(dmabuf_fd); + close(pidfd); + kill(child_pid, SIGTERM); + waitpid(child_pid, &status, 0); +out: + close(heap_fd); +} + +static int open_self_cgroupfd(void) +{ + char cgroup_path[PATH_MAX]; + char buf[PATH_MAX]; + FILE *f; + + f = fopen("/proc/self/cgroup", "r"); + if (!f) + return -1; + + cgroup_path[0] = '\0'; + while (fgets(buf, sizeof(buf), f)) { + char *p = buf; + + if (strncmp(p, "0::", 3) == 0) { + p += 3; + p[strcspn(p, "\n")] = '\0'; + snprintf(cgroup_path, sizeof(cgroup_path), + "/sys/fs/cgroup%s", p); + break; + } + } + fclose(f); + + if (!cgroup_path[0]) + return -1; + + return open(cgroup_path, O_RDONLY | O_DIRECTORY); +} + +/* + * Test: allocate charging the calling process's own cgroup via a cgroupfd. + */ +static void test_alloc_cgroupfd_self(char *heap_name) +{ + int heap_fd = -1, cgroup_fd = -1, dmabuf_fd = -1, ret; + + heap_fd = dmabuf_heap_open(heap_name); + + if (!pidfd_alloc_supported(heap_fd)) { + ksft_test_result_skip("charge_fd not supported by this kernel\n"); + goto out; + } + + cgroup_fd = open_self_cgroupfd(); + if (cgroup_fd < 0) { + ksft_test_result_skip("cannot open self cgroupfd\n"); + goto out; + } + + ret = dmabuf_heap_alloc_pidfd(heap_fd, ONE_MEG, 0, cgroup_fd, &dmabuf_fd); + ksft_test_result(!ret, "Allocation with cgroupfd %d\n", ret); + if (dmabuf_fd >= 0) + close(dmabuf_fd); + close(cgroup_fd); +out: + close(heap_fd); +} + +/* + * Test: transfer a dma-buf's charge to self cgroup via DMA_BUF_IOCTL_XFER_CHARGE. + */ +static void test_xfer_charge_cgroupfd_self(char *heap_name) +{ + int heap_fd = -1, cgroup_fd = -1, dmabuf_fd = -1, ret; + struct dma_buf_transfer_charge xfer = { 0 }; + + heap_fd = dmabuf_heap_open(heap_name); + + if (!pidfd_alloc_supported(heap_fd)) { + ksft_test_result_skip("charge_fd not supported by this kernel\n"); + goto out; + } + + ret = dmabuf_heap_alloc(heap_fd, ONE_MEG, 0, &dmabuf_fd); + if (ret) { + ksft_test_result_fail("Allocation failed %d\n", ret); + goto out; + } + + cgroup_fd = open_self_cgroupfd(); + if (cgroup_fd < 0) { + ksft_test_result_skip("cannot open self cgroupfd\n"); + close(dmabuf_fd); + goto out; + } + + xfer.charge_fd = cgroup_fd; + ret = ioctl(dmabuf_fd, DMA_BUF_IOCTL_XFER_CHARGE, &xfer); + ksft_test_result(!ret, "XFER_CHARGE to self cgroupfd %d\n", ret); + + close(cgroup_fd); + close(dmabuf_fd); out: close(heap_fd); } @@ -530,7 +751,7 @@ int main(void) return KSFT_SKIP; }
- ksft_set_plan(12 * numer_of_heaps()); + ksft_set_plan(16 * numer_of_heaps());
while ((dir = readdir(d))) { if (!strncmp(dir->d_name, ".", 2)) @@ -546,6 +767,10 @@ int main(void) test_alloc_compat(dir->d_name); test_alloc_errors(dir->d_name); test_alloc_no_fd_leak_on_efault(dir->d_name); + test_alloc_pidfd_self(dir->d_name); + test_alloc_pidfd_child(dir->d_name); + test_alloc_cgroupfd_self(dir->d_name); + test_xfer_charge_cgroupfd_self(dir->d_name); } closedir(d);
diff --git a/tools/testing/selftests/dmabuf-heaps/vmtest.sh b/tools/testing/selftests/dmabuf-heaps/vmtest.sh new file mode 100755 index 000000000000..3574b71ca83e --- /dev/null +++ b/tools/testing/selftests/dmabuf-heaps/vmtest.sh @@ -0,0 +1,226 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +# +# Copyright (c) 2026 Red Hat +# +# Dependencies: +# * virtme-ng +# * qemu (used by virtme-ng) + +readonly SCRIPT_DIR="$(cd -P -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd -P)" +readonly KERNEL_CHECKOUT=$(realpath "${SCRIPT_DIR}"/../../../../) +readonly CGROUP_DIR="${KERNEL_CHECKOUT}/tools/testing/selftests/cgroup" + +source "${SCRIPT_DIR}"/../kselftest/ktap_helpers.sh + +readonly DMABUF_HEAP_TEST="${SCRIPT_DIR}"/dmabuf-heap +readonly MEMCONTROL_TEST="${CGROUP_DIR}"/test_memcontrol +readonly TMP_DIR=$(mktemp -d /tmp/dmabuf-vmtest.XXXXXXXX) + +VERBOSE=false +BUILD=false +BUILD_HOST="" +BUILD_HOST_PODMAN_CONTAINER_NAME="" +SHELL_MODE=false + +usage() { + echo + echo "$0 [OPTIONS]" + echo + echo "Options" + echo " -b: build the kernel from the current source tree and use it for the VM" + echo " -H: hostname for remote build host (used with -b)" + echo " -p: podman container name for remote build host (used with -b)" + echo " Example: -H beefyserver -p vng" + echo " -s: drop into an interactive shell in the VM instead of running tests" + echo " -v: enable verbose vng/qemu output" + echo + + exit 1 +} + +die() { + echo "$*" >&2 + exit "${KSFT_FAIL}" +} + +cleanup() { + rm -rf "${TMP_DIR}" +} + +check_deps() { + for dep in vng make; do + if [[ ! -x $(command -v "${dep}") ]]; then + echo -e "skip: dependency ${dep} not found!\n" + exit "${KSFT_SKIP}" + fi + done + + if ${SHELL_MODE}; then + return + fi + + if [[ ! -x "${DMABUF_HEAP_TEST}" ]]; then + printf "skip: %s not found!" "${DMABUF_HEAP_TEST}" + printf " Please build the kselftest dmabuf-heaps target (or use -b).\n" + exit "${KSFT_SKIP}" + fi + + if [[ ! -x "${MEMCONTROL_TEST}" ]]; then + printf "skip: %s not found!" "${MEMCONTROL_TEST}" + printf " Please build the kselftest cgroup target (or use -b).\n" + exit "${KSFT_SKIP}" + fi +} + +check_vng() { + local tested_versions=("1.36" "1.37") + local version + local ok=0 + + version="$(vng --version)" + for tv in "${tested_versions[@]}"; do + if [[ "${version}" == *"${tv}"* ]]; then + ok=1 + break + fi + done + + if [[ "${ok}" -eq 0 ]]; then + printf "warning: vng version '%s' has not been tested and may " "${version}" >&2 + printf "not function properly.\n\tThe following versions have been tested: " >&2 + echo "${tested_versions[@]}" >&2 + fi +} + +build_selftests() { + make -C "${KERNEL_CHECKOUT}" headers_install \ + INSTALL_HDR_PATH="${TMP_DIR}/usr" -j"$(nproc)" + + local khdr="-isystem ${TMP_DIR}/usr/include" + + if ! make -C "${SCRIPT_DIR}" KHDR_INCLUDES="${khdr}" -j"$(nproc)"; then + die "failed to build dmabuf-heaps selftests" + fi + + if ! make -C "${CGROUP_DIR}" KHDR_INCLUDES="${khdr}" \ + "${MEMCONTROL_TEST}" -j"$(nproc)"; then + die "failed to build cgroup/test_memcontrol selftest" + fi +} + +handle_build() { + if ! ${BUILD}; then + return + fi + + if [[ ! -d "${KERNEL_CHECKOUT}" ]]; then + echo "-b requires vmtest.sh called from the kernel source tree" >&2 + exit 1 + fi + + pushd "${KERNEL_CHECKOUT}" &>/dev/null + + if ! vng --kconfig --config "${SCRIPT_DIR}/config"; then + die "failed to generate .config for kernel source tree (${KERNEL_CHECKOUT})" + fi + + local vng_args=("-v" "--config" "${SCRIPT_DIR}/config" "--build") + + if [[ -n "${BUILD_HOST}" ]]; then + vng_args+=("--build-host" "${BUILD_HOST}") + fi + + if [[ -n "${BUILD_HOST_PODMAN_CONTAINER_NAME}" ]]; then + vng_args+=("--build-host-exec-prefix" \ + "podman exec -ti ${BUILD_HOST_PODMAN_CONTAINER_NAME}") + fi + + if ! vng "${vng_args[@]}"; then + die "failed to build kernel from source tree (${KERNEL_CHECKOUT})" + fi + + build_selftests + + popd &>/dev/null +} + +make_runner() { + # virtme-ng shares the host filesystem, so TMP_DIR is accessible + # inside the VM at the same absolute path. + cat > "${TMP_DIR}/run_tests.sh" <<-EOF + #!/bin/sh + set -u + + # Enable memory controller for child cgroups + echo +memory > /sys/fs/cgroup/cgroup.subtree_control 2>/dev/null + + PASS=0; FAIL=0; SKIP=0; N=0 + + run() { + name="$1"; shift + N=$((N+1)) + "$@"; rc=$? + if [ $rc -eq 0 ]; then echo "ok $N $name"; PASS=$((PASS+1)) + elif [ $rc -eq 4 ]; then echo "ok $N $name # SKIP"; SKIP=$((SKIP+1)) + else echo "not ok $N $name"; FAIL=$((FAIL+1)) + fi + } + + run "dmabuf-heap charge_fd ioctl" ${DMABUF_HEAP_TEST} + run "memcontrol dma-buf memcg" ${MEMCONTROL_TEST} test_memcg_dmabuf + run "memcontrol dma-buf xfer limit" \ + ${MEMCONTROL_TEST} test_memcg_dmabuf_xfer_at_limit + echo "# PASS=$PASS SKIP=$SKIP FAIL=$FAIL" + [ $FAIL -eq 0 ] + EOF + chmod +x "${TMP_DIR}/run_tests.sh" +} + +run_vm() { + local verbose_opt="" + local kernel_opt="" + + ${VERBOSE} && verbose_opt="--verbose" + + # If we are running from within the kernel source tree, use the kernel + # source tree as the kernel to boot, otherwise use the running kernel. + if [[ "$(realpath "$(pwd)")" == "${KERNEL_CHECKOUT}"* ]]; then + kernel_opt="${KERNEL_CHECKOUT}" + fi + + if ${SHELL_MODE}; then + echo "Starting interactive shell in VM. Exit to stop VM." + vng --run ${kernel_opt} ${verbose_opt} --user root --memory 512M \ + --append "dma_heap.mem_accounting=1" \ + -- bash --noprofile --norc + exit "${KSFT_PASS}" + fi + + vng --run ${kernel_opt} ${verbose_opt} --user root --memory 512M \ + --append "dma_heap.mem_accounting=1" \ + --exec "${TMP_DIR}/run_tests.sh" +} + +while getopts :hvsbH:p: o +do + case $o in + v) VERBOSE=true;; + b) BUILD=true;; + s) SHELL_MODE=true;; + H) BUILD_HOST=$OPTARG;; + p) BUILD_HOST_PODMAN_CONTAINER_NAME=$OPTARG;; + h|*) usage;; + esac +done +shift $((OPTIND-1)) + +trap cleanup EXIT + +check_deps +check_vng +handle_build +make_runner + +echo "Booting VM and running tests..." +run_vm
linaro-mm-sig@lists.linaro.org
-
Albert Esteve -
Christian König