On Thu, 23 Jul 2026 16:58:14 -0700 Bobby Eshleman wrote:
> > > + if (info->attrs[NETDEV_A_DMABUF_RX_BUF_SIZE]) {
> > > + u32 rx_buf_size = nla_get_u32(info->attrs[NETDEV_A_DMABUF_RX_BUF_SIZE]);
> > > +
> > > + if (!rx_buf_size || !is_power_of_2(rx_buf_size) ||
> > > + rx_buf_size < PAGE_SIZE) {
> >
> > we should add a check: min: page-size in the Netlink policy?
>
> I played around with this adding:
>
> Documentation/netlink/specs/netdev.yaml:
> definitions:
> + -
> + type: const
> + name: page-size
> + value: 4096 # dummy value, to pass ynl_gen_c.py checks
> + header: asm/page.h
> + scope: kernel
>
> Generating:
>
> +static const struct netlink_range_validation
> netdev_a_dmabuf_rx_page_size_range = {
> + .min = PAGE_SIZE,
> + .max = U32_MAX,
> +};
> +
>
> ... but the dummy 4096 is kind of annoying. ynl_gen_c.py can't know the
> value of PAGE_SIZE but needs some value for its arithmetic checks (e.g.,
> confirm min < max is true).
>
> Should we stick with using a dummy value, or should we add a patch
> teaching ynl_gen_c.py to allow value-less consts (skip the arithmetic
> checks)?
Let's stick to a dummy one for now, but maybe something obviously dummy
like 0 ?
BTW did you add both min and max checks? Cause the only risk with using
a dummy value would be that the policy will be rendered inline, and
inline policy is u16 so 64k wouldn't fit. But your sample above has a
max of u32_max which forces the out-of-line policy, which is what we
want.
In case MMIO size is bigger than 4G and peer2peer DMA goes
through host bridge, we trigger a code path that assigns the
total linked IOVA (which is greater than 4G) to mapped_len.
Previously, `mapped_len` was declared as 32-bit `unsigned int`.
When accumulating `size_t` lengths, this leads to a silent wrap-around.
This truncation causes truncated lengths to be passed to functions
like `fill_sg_entry()`.
Fix this by changing `mapped_len` to `size_t` (64-bit). While
at it, fix similar potential overflow issues in `calc_sg_nents`
by using `check_add_overflow()` for `nents` and using
`unsigned int` for the loop iterator in `fill_sg_entry` to match.
Fixes: 3aa31a8bb11e ("dma-buf: provide phys_vec to scatter-gather mapping routine")
Cc: stable(a)vger.kernel.org
Cc: iommu(a)lists.linux.dev
Reviewed-by: Pranjal Shrivastava <praan(a)google.com>
Reviewed-by: Kevin Tian <kevin.tian(a)intel.com>
Reviewed-by: Leon Romanovsky <leon(a)kernel.org>
Signed-off-by: David Hu <xuehaohu(a)google.com>
---
Changes in v7:
- Added a missing blank line after local variable declaration in
`calc_sg_nents()` (Leon).
- Collected Reviewed-by from Leon Romanovsky.
Changes in v6:
- Used `check_add_overflow()` in `calc_sg_nents()` for safer
accumulation (Leon).
- Dropped explicit `!nents` check and added a comment noting that
`sg_alloc_table` handles `nents == 0` (Leon).
- Collected Reviewed-by from Kevin Tian.
Changes in v5:
- Removed WARN_ON_ONCE from calc_sg_nents() to avoid log noise (Jason).
- Added explicit check for `!nents` in dma_buf_phys_vec_to_sgt() to
cleanly return -EINVAL on overflow (Jason).
Changes in v4:
- Added WARN_ON_ONCE() to the nents overflow check to prevent silent
failures (Claude Bot).
Changes in v3:
- Removed leftover sentence fragment from the commit message.
- Kept `nents = 0` initialization (previously stated as removed in the
v2 changelog) as it is strictly required for the `+=` accumulation
loop in `calc_sg_nents()`.
Changes in v2:
- Fixed 'IVOA' -> 'IOVA' typo and expanded commit message (Claude Bot).
- Added Reverse Xmas tree formatting (Pranjal).
- Folded in extra bounds checking for calc_sg_nents() (Pranjal).
- Folded in type consistency fix for fill_sg_entry() (Pranjal).
- Collected Reviewed-by from Pranjal Shrivastava.
drivers/dma-buf/dma-buf-mapping.c | 16 ++++++++++++----
1 file changed, 12 insertions(+), 4 deletions(-)
diff --git a/drivers/dma-buf/dma-buf-mapping.c b/drivers/dma-buf/dma-buf-mapping.c
index 794acff2546a..80f6ab2f4809 100644
--- a/drivers/dma-buf/dma-buf-mapping.c
+++ b/drivers/dma-buf/dma-buf-mapping.c
@@ -5,12 +5,13 @@
*/
#include <linux/dma-buf-mapping.h>
#include <linux/dma-resv.h>
+#include <linux/overflow.h>
static struct scatterlist *fill_sg_entry(struct scatterlist *sgl, size_t length,
dma_addr_t addr)
{
unsigned int len, nents;
- int i;
+ unsigned int i;
nents = DIV_ROUND_UP(length, UINT_MAX);
for (i = 0; i < nents; i++) {
@@ -40,8 +41,12 @@ static unsigned int calc_sg_nents(struct dma_iova_state *state,
size_t i;
if (!state || !dma_use_iova(state)) {
- for (i = 0; i < nr_ranges; i++)
- nents += DIV_ROUND_UP(phys_vec[i].len, UINT_MAX);
+ for (i = 0; i < nr_ranges; i++) {
+ unsigned int added = DIV_ROUND_UP(phys_vec[i].len, UINT_MAX);
+
+ if (check_add_overflow(nents, added, &nents))
+ return 0;
+ }
} else {
/*
* In IOVA case, there is only one SG entry which spans
@@ -95,9 +100,10 @@ struct sg_table *dma_buf_phys_vec_to_sgt(struct dma_buf_attachment *attach,
size_t nr_ranges, size_t size,
enum dma_data_direction dir)
{
- unsigned int nents, mapped_len = 0;
struct dma_buf_dma *dma;
struct scatterlist *sgl;
+ size_t mapped_len = 0;
+ unsigned int nents;
dma_addr_t addr;
size_t i;
int ret;
@@ -133,6 +139,8 @@ struct sg_table *dma_buf_phys_vec_to_sgt(struct dma_buf_attachment *attach,
}
nents = calc_sg_nents(dma->state, phys_vec, nr_ranges, size);
+
+ /* sg_alloc_table will cleanly fail and return -EINVAL if nents == 0 */
ret = sg_alloc_table(&dma->sgt, nents, GFP_KERNEL | __GFP_ZERO);
if (ret)
goto err_free_state;
--
2.54.0.1064.gd145956f57-goog
On Wed, Jul 22, 2026 at 11:39:32PM +0000, dhu(a)x6u.co wrote:
> From: David Hu <xuehaohu(a)google.com>
>
> Currently, `fill_sg_entry()` splits the scatterlist using `UINT_MAX`.
> This creates a non-page-aligned DMA length (`0xFFFFFFFF`) for the
> first entry, resulting in non-page-aligned DMA addresses for all
> subsequent entries.
>
> While the underlying IOMMU mapping may be contiguous, hardware
> DMA engines often require explicit address alignment (e.g., page,
> cacheline, or storage sector boundaries). Passing unaligned
> addresses and lengths can cause explicit failures in DMA descriptor
> creation or silent data corruption if lower unaligned bits are
> truncated.
>
> In addition, a non-page-aligned sgl length will trigger an edge case
> in `ib_umem_find_best_pgsz()`. In case of a discontinuity in later
> buffers, we will have a `va` with lowest bit set to 1. That will lead
> to `ib_umem_find_best_pgsz()` always return 0, and break the promise
> to find best page size for the mapping on the NIC side.
>
> Fix this by splitting the scatterlist by the largest possible page
> aligned chunk within `UINT_MAX` (`ALIGN_DOWN(UINT_MAX, PAGE_SIZE)`).
> This ensures all scatterlist DMA addresses and lengths remain page
> aligned, while minimizing the total number of sgl entries.
>
> Page-aligned entries allow the system to cleanly chunk payloads into
> PCIe MaxPayloadSize (MPS) (e.g., 128 bytes, 256 bytes, 512 bytes).
> As a result, this may help reduce TLP fragmentation in P2P transfers
> and alleviate potential congestion within a logical PCIe switch
> partition, especially when Relaxed Ordering is not possible due to
> hardware constraints.
>
> Reported-by: sashiko-bot <sashiko-bot(a)kernel.org>
> Closes: https://lore.kernel.org/all/20260609165431.778061F00893@smtp.kernel.org/
> Fixes: 3aa31a8bb11e ("dma-buf: provide phys_vec to scatter-gather mapping routine")
> Cc: stable(a)vger.kernel.org
> Signed-off-by: David Hu <xuehaohu(a)google.com>
> ---
> Changes in v3:
> - Removed the type cast for `min` (David Laight)
> - Reverted max ent size to be `ALIGN_DOWN(UINT_MAX, PAGE_SIZE)` and
> updated commit message to reflect that (Jason Gunthorpe)
> - Updated commit message to reflect that this also fixes an edge case
> in `ib_umem_find_best_pgsz()`
>
> Changes in v2:
> - Updated commit title and message to reflect the switch to 2G chunks
> - Switch to using 2G as the max sg entry size as it naturally aligns
> with most hardware boundaries, while allowing compiler optimizations
> with bit shifts (David Laight)
> - Optimized away division calculation for `nent`, and multiplication
> calculation for sgl address, by dropping the `for` loop in favor of a
> `while (length)` loop (David Laight)
> - Dropped `min_t` in favor of `min()` to maintain a strict type
> checking safety net (David Laight)
>
> drivers/dma-buf/dma-buf-mapping.c | 19 +++++++++++--------
> 1 file changed, 11 insertions(+), 8 deletions(-)
Could you please avoid sending patches as replies? It severely disrupts
the reading flow when using mutt's threaded view.
Regarding the patch,
Reviewed-by: Leon Romanovsky <leonro(a)nvidia.com>
Thanks
On Wed, 2026-07-22 at 18:38 +0000, Timur Tabi wrote:
> On Wed, 2026-07-22 at 16:49 +0200, Philipp Stanner wrote:
> > The Rust dma_fence abstractions need the ECANCELED error code.
> >
> > Add ECANCELED error code.
> >
> > Signed-off-by: Philipp Stanner <phasta(a)kernel.org>
> > Tested-by: Daniel Almeida <daniel.almeida(a)collabora.com>
>
> No need for this, all remaining codes are being added via
>
> https://lore.kernel.org/rust-for-linux/20260629183022.2709524-1-ttabi@nvidi…
>
> Please pick up that patch instead.
Ok, pulled it in my local branch for now, thx for the heads-up.
I see that your patch is from June. If it lands in drm-rust-next
soonish I'd simply rebase.
Regards
P.
Hi Robert,
> Subject: [PATCH v1] dma-buf/udmabuf: Disable the size limit by default
>
> As udmabuf increasingly enjoys popularity - being used in projects like
> libcamera, Gstreamer, Mesa, KWin and Weston - users more frequently
> encounter cases where the current default size limit of 64MB is too low.
> Examples include allocating video buffers at a 8K resolution - and even 4K
> is affected when using non-subsampled video formats and high bit depths.
>
> In its current form the size limit for individual buffers does not seem to
> provide any additional level of protection - such as limiting the amount of
> memory a process can pin - as the later can just allocate multiple buffers.
> If additional guardrails are desired, they would likely require some kind
> accounting not limited to individual buffers.
>
> Therefor let's disable the size limit by default. Use the special value
> of zero to do so, which prevously could be used to effectively disable the
> interface. Using other means, such as file permissions, appears to be a
> much better fit for that purpose.
>
> Signed-off-by: Robert Mader <robert.mader(a)collabora.com>
>
> ---
>
> Please let me know if changing the meaning of the parameter value of zero
> is considered a breaking change / not acceptable. In that case INT_MAX
> might be a better option.
Yeah, I think using INT_MAX might be better.
Thanks,
Vivek
>
> See
> https://lore.kernel.org/dri-devel/20260711144814.8205-1-
> robert.mader(a)collabora.com/
> for a previous attempt to make the value configurable via kconfig - and
> in particular
> https://lore.kernel.org/dri-devel/6764ca6f-b4d8-4baa-9d27-
> 2ca867ac2d41(a)amd.com/
> for the suggestion and discussion to remove the default limit.
> ---
> drivers/dma-buf/udmabuf.c | 6 +++---
> 1 file changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/dma-buf/udmabuf.c b/drivers/dma-buf/udmabuf.c
> index bced421c0d65..3509b690d8e2 100644
> --- a/drivers/dma-buf/udmabuf.c
> +++ b/drivers/dma-buf/udmabuf.c
> @@ -20,9 +20,9 @@ static int list_limit = 1024;
> module_param(list_limit, int, 0644);
> MODULE_PARM_DESC(list_limit, "udmabuf_create_list->count limit.
> Default is 1024.");
>
> -static int size_limit_mb = 64;
> +static int size_limit_mb = 0;
> module_param(size_limit_mb, int, 0644);
> -MODULE_PARM_DESC(size_limit_mb, "Max size of a dmabuf, in
> megabytes. Default is 64.");
> +MODULE_PARM_DESC(size_limit_mb, "Max size of a dmabuf, in
> megabytes. Setting 0 disables the limit. Default is 0.");
>
> struct udmabuf {
> pgoff_t pagecount;
> @@ -373,7 +373,7 @@ static long udmabuf_create(struct miscdevice
> *device,
>
> subpgcnt = list[i].size >> PAGE_SHIFT;
> pgcnt += subpgcnt;
> - if (pgcnt > pglimit)
> + if (pglimit && pglimit < pgcnt)
> goto err_noinit;
>
> max_nr_folios = max_t(unsigned long, subpgcnt,
> max_nr_folios);
> --
> 2.55.0
Incredible, professional, and fast! Accurate and polite. Understanding and genuinely caring about horrible situations. Thanks, Dune Nectar Web Expert. Dune Nectar Web Expert is a great crypto recovery company to deal with. They have gone above and beyond to help, and they have worked side by side with my local law enforcement. +1, 5,1,6,4,6,7,6,7,8,3
Every devmem dmabuf binding hands the page_pool PAGE_SIZE niovs today.
On NICs that consume one descriptor per netmem, this caps a single RX
descriptor at PAGE_SIZE and burns CPU on buffer churn.
In this series, we add a bind-time netlink attribute,
NETDEV_A_DMABUF_RX_BUF_SIZE, that lets userspace request a larger niov
size (power of two >= PAGE_SIZE). Drivers must opt in via
queue_mgmt_ops.QCFG_RX_PAGE_SIZE.
Measurements:
Setup: kperf devmem RX/TX cuda, 4 flows, 64 MB messages, 60s, dctcp,
num-rx-queues=4, dmabuf-rx/tx-size-mb=2048, 10 runs per niov size,
mlx5.
niov RX dev Gbps RX flow avg Gbps app sys %
----- ---------------- ----------------- ----------------
4K 300.63 +/- 53.21 75.16 +/- 13.30 54.15 +/- 10.23
16K 321.35 +/- 28.20 80.34 +/- 7.05 41.05 +/- 8.87
32K 347.63 +/- 2.20 86.91 +/- 0.55 44.54 +/- 3.51
64K 332.11 +/- 14.26 83.03 +/- 3.56 35.47 +/- 3.11
RX app sys % drops ~19% from 4K to 64K.
kperf support (not yet merged):
https://github.com/facebookexperimental/kperf/commit/8837577f920876bce6986e…
Signed-off-by: Bobby Eshleman <bobbyeshleman(a)meta.com>
---
Changes in v5:
- removed unnecessary change from ssize_t to size_t (Mina)
- removed '--------' lines in the commit message (Paolo)
- removed commit msg about CONFIG_HUGETLB since that change was already
merged
- Link to v4: https://lore.kernel.org/r/20260701-tcpdm-large-niovs-v4-0-ca4654f37570@meta…
Changes in v4:
- ncdevmem: fix the possible overflow in ncdevmem (Sashiko)
- drop the udmabuf patch because the fix is now already in net-next
- silenced two pylint complaints in devmem_lib.py
- Link to v3: https://lore.kernel.org/r/20260612-tcpdm-large-niovs-v3-0-a3b693e76fcb@meta…
Changes in v3:
- fix a bunch of non-reverse christmas tree declarations (Stan)
- remove extra uint32 cast for getpagesize() (Stan)
- remove overzealous strtoul checking (Stan)
- remove value checks that the kernel already performs on rx_buf_size
(Stan)
- Link to v2: https://lore.kernel.org/r/20260611-tcpdm-large-niovs-v2-0-ee2bf15e7523@meta…
Changes in v2:
- Use NL_SET_ERR_MSG_FMT for sg alignment failure details (Stan)
- Keep -E2BIG (not a direct ask, but seemed preferred, Stan)
- Update udmabuf commit message and comments explaining why
"one sg ent per folio" is useful (Christian)
- Set/restore nr_hugepages in py harness (Stan)
- Link to v1: https://lore.kernel.org/r/20260603-tcpdm-large-niovs-v1-0-f37a4ac6726c@meta…
---
Bobby Eshleman (3):
net: devmem: allow rx-buf-size > PAGE_SIZE per dmabuf binding
selftests/net: ncdevmem: add -b option to set rx-buf-size on bind
selftests/net: devmem.py: add check_rx_large_niov
Documentation/netlink/specs/netdev.yaml | 8 +++
include/uapi/linux/netdev.h | 1 +
net/core/devmem.c | 51 +++++++++++--------
net/core/devmem.h | 13 +++--
net/core/netdev-genl-gen.c | 5 +-
net/core/netdev-genl.c | 19 ++++++-
tools/include/uapi/linux/netdev.h | 1 +
tools/testing/selftests/drivers/net/hw/devmem.py | 12 ++++-
.../testing/selftests/drivers/net/hw/devmem_lib.py | 59 +++++++++++++++++++++-
tools/testing/selftests/drivers/net/hw/ncdevmem.c | 36 +++++++++++--
.../testing/selftests/drivers/net/hw/nk_devmem.py | 11 +++-
11 files changed, 178 insertions(+), 38 deletions(-)
---
base-commit: 474cff6868129755cf889edf40d7f491729fc588
change-id: 20260602-tcpdm-large-niovs-56523a3a1077
Best regards,
--
Bobby Eshleman <bobbyeshleman(a)meta.com>
7.1-stable review patch. If anyone has any objections, please let me know.
------------------
From: Tvrtko Ursulin <tvrtko.ursulin(a)igalia.com>
[ Upstream commit e94b9f01543cc6a83538c2c2cc645a424d3015ca ]
Trace_dma_fence_signaled, trace_dma_fence_wait_end and
trace_dma_fence_destroy can all currently dereference a null fence->ops
pointer after it has been reset on fence signalling.
Lets use the safe string getters for most tracepoints to avoid this class
of a problem, while for the signal tracepoint we move it to before ops are
cleared to avoid losing the driver and timeline name information. Apart
from moving it we also need to add a new tracepoint class to bypass the
safe name getters since the signaled bit is already set.
For dma_fence_init we also need to use the new tracepoint class since the
rcu read lock is not held there, and we can do the same for the enable
signaling since there we are certain the fence cannot be signaled while
we are holding the lock and have even validated the fence->ops.
Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin(a)igalia.com>
Fixes: 541c8f2468b9 ("dma-buf: detach fence ops on signal v3")
Cc: Christian König <christian.koenig(a)amd.com>
Cc: Philipp Stanner <phasta(a)kernel.org>
Cc: Boris Brezillon <boris.brezillon(a)collabora.com>
Cc: linux-media(a)vger.kernel.org
Cc: linaro-mm-sig(a)lists.linaro.org
Reviewed-by: Christian König <christian.koenig(a)amd.com>
Signed-off-by: Tvrtko Ursulin <tursulin(a)ursulin.net>
Link: https://lore.kernel.org/r/20260415083207.40513-2-tvrtko.ursulin@igalia.com
Signed-off-by: Sasha Levin <sashal(a)kernel.org>
---
drivers/dma-buf/dma-fence.c | 3 ++-
include/trace/events/dma_fence.h | 40 +++++++++++++++++++++++++++-----
2 files changed, 36 insertions(+), 7 deletions(-)
diff --git a/drivers/dma-buf/dma-fence.c b/drivers/dma-buf/dma-fence.c
index a2aa82f4eedd49..b3bfa6943a8e13 100644
--- a/drivers/dma-buf/dma-fence.c
+++ b/drivers/dma-buf/dma-fence.c
@@ -363,6 +363,8 @@ void dma_fence_signal_timestamp_locked(struct dma_fence *fence,
&fence->flags)))
return;
+ trace_dma_fence_signaled(fence);
+
/*
* When neither a release nor a wait operation is specified set the ops
* pointer to NULL to allow the fence structure to become independent
@@ -377,7 +379,6 @@ void dma_fence_signal_timestamp_locked(struct dma_fence *fence,
fence->timestamp = timestamp;
set_bit(DMA_FENCE_FLAG_TIMESTAMP_BIT, &fence->flags);
- trace_dma_fence_signaled(fence);
list_for_each_entry_safe(cur, tmp, &cb_list, node) {
INIT_LIST_HEAD(&cur->node);
diff --git a/include/trace/events/dma_fence.h b/include/trace/events/dma_fence.h
index 3abba45c0601a4..5b10a9e06fb4ec 100644
--- a/include/trace/events/dma_fence.h
+++ b/include/trace/events/dma_fence.h
@@ -9,12 +9,40 @@
struct dma_fence;
+DECLARE_EVENT_CLASS(dma_fence,
+
+ TP_PROTO(struct dma_fence *fence),
+
+ TP_ARGS(fence),
+
+ TP_STRUCT__entry(
+ __string(driver, dma_fence_driver_name(fence))
+ __string(timeline, dma_fence_timeline_name(fence))
+ __field(unsigned int, context)
+ __field(unsigned int, seqno)
+ ),
+
+ TP_fast_assign(
+ __assign_str(driver);
+ __assign_str(timeline);
+ __entry->context = fence->context;
+ __entry->seqno = fence->seqno;
+ ),
+
+ TP_printk("driver=%s timeline=%s context=%u seqno=%u",
+ __get_str(driver), __get_str(timeline), __entry->context,
+ __entry->seqno)
+);
+
/*
* Safe only for call sites which are guaranteed to not race with fence
- * signaling,holding the fence->lock and having checked for not signaled, or the
- * signaling path itself.
+ * signaling, holding the fence->lock and having checked for not signaled, or
+ * the signaling path itself.
+ *
+ * TODO: Remove the need for this event class when drivers switch to independent
+ * fences.
*/
-DECLARE_EVENT_CLASS(dma_fence,
+DECLARE_EVENT_CLASS(dma_fence_ops,
TP_PROTO(struct dma_fence *fence),
@@ -46,7 +74,7 @@ DEFINE_EVENT(dma_fence, dma_fence_emit,
TP_ARGS(fence)
);
-DEFINE_EVENT(dma_fence, dma_fence_init,
+DEFINE_EVENT(dma_fence_ops, dma_fence_init,
TP_PROTO(struct dma_fence *fence),
@@ -60,14 +88,14 @@ DEFINE_EVENT(dma_fence, dma_fence_destroy,
TP_ARGS(fence)
);
-DEFINE_EVENT(dma_fence, dma_fence_enable_signal,
+DEFINE_EVENT(dma_fence_ops, dma_fence_enable_signal,
TP_PROTO(struct dma_fence *fence),
TP_ARGS(fence)
);
-DEFINE_EVENT(dma_fence, dma_fence_signaled,
+DEFINE_EVENT(dma_fence_ops, dma_fence_signaled,
TP_PROTO(struct dma_fence *fence),
--
2.53.0
On Fri, 2026-07-17 at 14:14 -0300, Daniel Almeida wrote:
> >
> > > > + try_pin_init!(Self {
> > > > + // SAFETY: `dma_fence_context_alloc()` merely works on a global atomic.
> > > > + // Parameter `1` is the number of contexts we want to allocate.
> > > > + nr: unsafe { bindings::dma_fence_context_alloc(1) },
> > > > + seqno: AtomicU64::new(0),
> > >
> > > Do we really need to force a 0 here? i.e.: can’t we take the initial seqno
> > > as an argument?
> >
> > We could. What would that be useful for?
>
>
> On Mali, the hardware syncobj starts at 0. If you see a 0, is this the default
> state, or should you signal seqno 0?
>
> This problem goes away if we can have seqnos starting at a custom value, like
> 1. It seems like the C machinery also special-cases 0 in a few other places too.
ACK.
>
> >
> > Hm, no, we don't.
> >
> > For the most part that's irrelevant, since all critical components then
> > only get set in new_fence(). Correct typization is enforced through T.
> >
> > The notable exception is the fence_ctx reference itself.
> >
> > What should we do about it?
> >
> > We could keep the fctx field as a MaybeUninit and set it later. Or we
> > check through the fctx identifier number whether it's the correct one
> > in new_fence(), but then new_fence() could fail with some error, and
> > it's probably better to have it be completely fail-free.
>
> Agree about the fail-free part.
>
> The problem I see here is that new_fence() will use "seqno" and "nr" from
> whatever context called new_fence(), but DriverFenceAllocation has some other
> (possibly unrelated) context as its DriverFenceData::fctx.
>
> The lifetimes are apparently broken too, because 'a is the lifetime of the
> context where new_fence_allocation was called, meaning that the context that
> actually called new_fence() can drop, even though it provided the state for
> dma_fence_init().
>
> I guess this can be solved by moving new_fence() to impl DriverFenceAllocation?
> That already has a context, and most importantly, the right context.
Yes that is / was broken. Having the wrong fctx would cause wrong
container_of() calls and therefore explode.
Fixed it locally following your suggestion of creating fences on the
allocation object.
> >
[…]
> > > > +
> > > > +/// The receiving counterpart of a [`DriverFence`], designed to register callbacks
> > > > +/// on, check the signalled state etc. A [`Fence`] cannot be signalled.
> > > > +/// A [`Fence`] is always refcounted.
> > >
> > > I would explain this a tad better.
> >
> > What exactly? The refcounting? The dualism between DriverFence and
> > Fence? :)
>
> For example, you say “a Fence cannot be signaled”. A person seeing this
> code for the first time might ask why. Specially if they start by reading the
> docs for Fence first.
>
> I think explaining a bit more about the DriverFence/Fence/refcounting as you
> said is already enough to settle it.
OK, I flesh that out a bit.
> > >
[…]
> > > > + let ret = unsafe { bindings::dma_fence_is_signaled(fence) };
> > > > +
> > > > + // To guarantee that an API caller can 100% rely on the signalling being
> > > > + // completed (i.e., all fence callbacks ran), we have to take the lock.
> > > > + //
> > > > + // The reason is that the C dma_fence backend currently does not carefully
> > > > + // synchronize the `dma_fence_is_signaled()` function with the proper
> > > > + // spinlock. This can lead to the function returning `true` while fence
> > > > + // callbacks are still being executed. This can be mitigated by guarding
> > > > + // the entire function with the spinlock.
> > > > + //
> > > > + // See commit c8a5d5ea3ba6a.
> > > > +
> > > > + // SAFETY: `fence` is valid because `self` is valid. `flag_ptr` is
> > > > + // merely a pointer to an integer, which lives as long as this function.
> > > > + unsafe { bindings::dma_fence_lock_irqsave(fence, flag_ptr) };
> > >
> > > Shouldn’t this be before the “is_signaled” ffi call? Or is this
> > > only about ensuring all callbacks have run? i.e.: is “ret” valid even
> > > though it was computed before taking the lock?
> >
> > OK, this is where it gets ugly.
> >
> > So during the last weeks I've been struggling to get the C backend into
> > better shape. One issue from my POV is that the C dma_fence spinlock
> > does not protect the fence state; there is insistence that the lock
> > shall only protect the callback list.
> >
> > The function dma_fence_is_signaled() has an unlocked fast path check:
> >
> > https://elixir.bootlin.com/linux/v7.2-rc3/source/include/linux/dma-fence.h#…
> >
> > whereas setting of that bit is done under lock-protection:
> >
> > https://elixir.bootlin.com/linux/v7.2-rc3/source/drivers/dma-buf/dma-fence.…
> >
> >
> > This can lead to funny races like in the commit mentioned in the
> > comment block above (c8a5d5ea3ba6a).
> >
> > And it also leads to weird hacks like this:
> >
> > https://elixir.bootlin.com/linux/v7.2-rc3/source/drivers/gpu/drm/amd/amdgpu…
> >
> >
> > Now, in principle I agree with you that a pattern like this:
> >
> > dma_fence_lock_irqsave(…);
> > let signaled = dma_fence_is_signaled_locked(…);
> > dma_fence_unlock_irqrestore(…);
> >
> > would be better.
> >
> > However, lengthy discussions with Christian seem to settle at the point
> > where Christian sees the very strict requirement of never calling fence
> > callbacks under lock protection, and where he views
> > dma_fence_is_signaled_locked() as a broken function that should be
> > removed.
> >
> > He's currently working on removing all bits where fence callbacks are
> > invoked under lock protection:
> >
> > https://lore.kernel.org/dri-devel/20260624122917.2483-1-christian.koenig@am…
> >
> > There's been a ton of discussions and proposals about that in recent
> > weeks
> >
> > https://lore.kernel.org/dri-devel/20260608142436.265820-2-phasta@kernel.org/
> > https://lore.kernel.org/dri-devel/20260612104251.2264707-2-phasta@kernel.or…
> >
> >
> > So tl;dr: The weird code you're commenting on above ensures that
> >
> > a) the fence->ops->is_signaled() callback is not called under lock
> > protection and
> > b) taking and releasing the lock guarantees that all callbacks are
> > really finished, i.e. they have run.
> >
> >
> > (I continue to believe that setting the bit under lock protection and
> > reading it without lock is fundamentally broken and needs to be fixed,
> > but fixes are being rejected because of claimed performance regressions
> > years ago when this was tried, because checking the bit is some sort of
> > fast path check for.. parties that spin on dma_fence_is_signaled() ??)
>
> I see, there is a lot more context on this then. Can you merely add a comment
> saying it’s ok to call dma_fence_is_signaled() without the locks? Otherwise
> people might try to “fix” this down the line...
I tried to make it clear with the comment above:
// To guarantee that an API caller can 100% rely on the signalling being
// completed (i.e., all fence callbacks ran), we have to take the lock.
//
// The reason is that the C dma_fence backend currently does not carefully
// synchronize the `dma_fence_is_signaled()` function with the proper
// spinlock. This can lead to the function returning `true` while fence
// callbacks are still being executed. This can be mitigated by guarding
// the entire function with the spinlock.
//
// See commit c8a5d5ea3ba6a.
I can try to make it more explicit.
Actually, I suppose with our design in Rust we would not actually need
this lock-unlock, because we ensure that it does not matter whether a
callback already ran.
But since we don't know who will be working on this in 5 years, adding
this or that exotic callback, or maybe registering callbacks on his own
fence, I put this sequence there for robustness.
So if the function returns true, you really know that all callbacks are
gone forever.
>
> >
> > >
> > > >
> >
> > […]
> > >
> > >
> > > > + /// The API user's data. This must either not need drop, or must delay its
> > > > + /// drop by a grace period. It is essential that the data only performs
> > > > + /// operations legal in atomic context in its [`Drop`] implementation.
> > > > + #[pin]
> > > > + data: T::FenceDataType,
> > > > +}
> > > > +
> > > >
> >
> > […]
> >
> > > > +
> > > > + // DriverFenceData is repr(C) and a Fence is its first member.
> > >
> > > > + let fence_data_ptr = fence_ptr as *mut DriverFenceData<'a, T>;
> > >
> > > Without a “CAST:” keyword, I think this will trigger the linter?
> > >
> >
> > Didn't see a complaint from clippy nor compiler.
>
> I recommend the CAST thing anyways. It’s being adopted in other parts of the kernel
> crate.
I can write a CAST comment, no problem.
Thx
P.