On Tue, Jun 23, 2026 at 11:53:50PM +0100, David Laight wrote:
If we restrict incoming dmabuf transfers to fit within VFS-centric limits (2GB), we impose unnecessary overhead on the RDMA stack, forcing it to manage a significantly higher number of memory registrations. By cleanly splitting these massive contiguous device buffers into page-aligned SGL entries, we directly improve the efficiency of P2P transfers and memory registration.
But a divide by '4G - PAGE_SIZE' is also non-trivial and (I think affects a lot of io) when the quotient is always 1. Splitting into 2G chunks is a lot cheaper.
Doesn't matter this isn't fast path stuff. It is better to use fewer SGL entries, IHMO.
Since this change doesn't seem to have a negative impact on standard file I/O or break existing VFS constraints, I'm curious why we shouldn't support splitting these >4GB P2P transfers? Am I missing something?
I was only wondering whether it was needed... It does bring up the question of why the >4GB transfers even need splitting. But that is another question.
SGL can only store an unsigned int size, so any large physical range has to be split down.
rdma now a days has code to process the sgl and restore back the > 4G sizes since mode RDMA HW can accept that.
commit 486055f5e09df959ad4e3aa4ee75b5c91ddeec2e Author: Michael Margolin mrgolin@amazon.com Date: Mon Feb 17 14:16:23 2025 +0000
RDMA/core: Fix best page size finding when it can cross SG entries
So whatever this produces needs to be compatible with that to undo it.
Jason