On Wed, Apr 23, 2025 at 9:03 AM Cosmin Ratiu cratiu@nvidia.com wrote:
Drivers that are told to allocate RX buffers from pools of DMA memory should have enough memory in the pool to satisfy projected allocation requests (a function of ring size, MTU & other parameters). If there's not enough memory, RX ring refill might fail later at inconvenient times (e.g. during NAPI poll).
My understanding is that if the RX ring refill fails, the driver will post the buffers it was able to allocate data for, and will not post other buffers. So it will run with a degraded performance but nothing overly bad should happen. This should be the same behavior if the machine is under memory pressure.
In general I don't know about this change. If the user wants to use very small dmabufs, they should be able to, without going through hoops reducing the number of rx ring slots the driver has (if it supports configuring that).
I think maybe printing an error or warning that the dmabuf is too small for the pool_size may be fine. But outright failing this configuration? I don't think so.
This commit adds a check at dmabuf pool init time that compares the amount of memory in the underlying chunk pool (configured by the user space application providing dmabuf memory) with the desired pool size (previously set by the driver) and fails with an error message if chunk memory isn't enough.
Fixes: 0f9214046893 ("memory-provider: dmabuf devmem memory provider") Signed-off-by: Cosmin Ratiu cratiu@nvidia.com
net/core/devmem.c | 11 +++++++++++ 1 file changed, 11 insertions(+)
diff --git a/net/core/devmem.c b/net/core/devmem.c index 6e27a47d0493..651cd55ebb28 100644 --- a/net/core/devmem.c +++ b/net/core/devmem.c @@ -299,6 +299,7 @@ net_devmem_bind_dmabuf(struct net_device *dev, unsigned int dmabuf_fd, int mp_dmabuf_devmem_init(struct page_pool *pool) { struct net_devmem_dmabuf_binding *binding = pool->mp_priv;
size_t size; if (!binding) return -EINVAL;@@ -312,6 +313,16 @@ int mp_dmabuf_devmem_init(struct page_pool *pool) if (pool->p.order != 0) return -E2BIG;
/* Validate that the underlying dmabuf has enough memory to satisfy* requested pool size.*/size = gen_pool_size(binding->chunk_pool) >> PAGE_SHIFT;if (size < pool->p.pool_size) {
pool_size seems to be the number of ptr_ring slots in the page_pool, not some upper or lower bound on the amount of memory the page_pool can provide. So this check seems useless? The page_pool can still not provide this amount of memory with dmabuf (if the netmems aren't being recycled fast enough) or with normal memory (under memory pressure).
pr_warn("%s: Insufficient dmabuf memory (%zu pages) to satisfy pool_size (%u pages)\n",__func__, size, pool->p.pool_size);return -ENOMEM;
In general I think maybe printing an error/warn to dmesg to warn the user that this is a weird configuration may be fine, but return -ENOMEM? I don't think this should be an unsupported configuration and I don't think checking against p.pool_size guarantees anything.
}net_devmem_dmabuf_binding_get(binding); return 0;}
2.45.0