On Thu, Jul 2, 2026 at 4:10 AM David Laight david.laight.linux@gmail.com wrote:
On Thu, 2 Jul 2026 00:56:40 -0400 David Hu xuehaohu@google.com wrote:
On Tue, Jun 30, 2026 at 8:42 AM Jason Gunthorpe jgg@ziepe.ca wrote:
On Tue, Jun 23, 2026 at 11:53:50PM +0100, David Laight wrote:
If we restrict incoming dmabuf transfers to fit within VFS-centric limits (2GB), we impose unnecessary overhead on the RDMA stack, forcing it to manage a significantly higher number of memory registrations. By cleanly splitting these massive contiguous device buffers into page-aligned SGL entries, we directly improve the efficiency of P2P transfers and memory registration.
But a divide by '4G - PAGE_SIZE' is also non-trivial and (I think affects a lot of io) when the quotient is always 1. Splitting into 2G chunks is a lot cheaper.
Doesn't matter this isn't fast path stuff. It is better to use fewer SGL entries, IHMO.
Since this change doesn't seem to have a negative impact on standard file I/O or break existing VFS constraints, I'm curious why we shouldn't support splitting these >4GB P2P transfers? Am I missing something?
I was only wondering whether it was needed... It does bring up the question of why the >4GB transfers even need splitting. But that is another question.
SGL can only store an unsigned int size, so any large physical range has to be split down.
rdma now a days has code to process the sgl and restore back the > 4G sizes since mode RDMA HW can accept that.
commit 486055f5e09df959ad4e3aa4ee75b5c91ddeec2e Author: Michael Margolin mrgolin@amazon.com Date: Mon Feb 17 14:16:23 2025 +0000
RDMA/core: Fix best page size finding when it can cross SG entriesSo whatever this produces needs to be compatible with that to undo it.
Thank you everyone. It looks like most open issues are sorted out. I'll wait for maintainers to weigh in before sending out v3 (which will remove the type cast for min() per David L.'s feedback, and revert to ALIGN_DOWN(UINT_MAX, PAGE_SIZE) per Jason's feedback).
Does this code get used a lot for 'normal' transfers? I'm away from my normal systems and can't check. But if pretty much all of the fragments are small (< 4G) then it is probably worth adding a check for 'size < limit' before anything else and optimising that case.
Hi David,
Thank you for raising this. This file (`dma-buf-mapping.c`) was recently added [1] to exclusively export MMIO device memory. Therefore, it is bypassed completely for `normal` transfers (IIUC, e.g., video buffers for V4L2 or DRM).
Regards, David
[1] https://lore.kernel.org/all/20251120-dmabuf-vfio-v9-6-d7f71607f371@nvidia.co...