On 1/21/26 07:37, Nitesh Shetty wrote:
On 23/11/25 10:51PM, Pavel Begunkov wrote:
Add blk-mq infrastructure to handle dmabuf tokens. There are two main objects. The first is struct blk_mq_dma_token, which is an extension of struct dma_token and passed in an iterator. The second is struct blk_mq_dma_map, which keeps the actual mapping and unlike the token, can be ejected (e.g. by move_notify) and recreated.
The token keeps an rcu protected pointer to the mapping, so when it resolves a token into a mapping to pass it to a request, it'll do an rcu protected lookup and get a percpu reference to the mapping.
If there is no current mapping attached to a token, it'll need to be created by calling the driver (e.g. nvme) via a new callback. It requires waiting, thefore can't be done for nowait requests and couldn't happen deeper in the stack, e.g. during nvme request submission.
The structure split is needed because move_notify can request to invalidate the dma mapping at any moment, and we need a way to concurrently remove it and wait for the inflight requests using the previous mapping to complete.
Signed-off-by: Pavel Begunkov asml.silence@gmail.com
block/Makefile | 1 + block/bdev.c | 14 ++ block/blk-mq-dma-token.c | 236 +++++++++++++++++++++++++++++++ block/blk-mq.c | 20 +++ block/fops.c | 1 + include/linux/blk-mq-dma-token.h | 60 ++++++++ include/linux/blk-mq.h | 21 +++ include/linux/blkdev.h | 3 + 8 files changed, 356 insertions(+) create mode 100644 block/blk-mq-dma-token.c create mode 100644 include/linux/blk-mq-dma-token.h
diff --git a/block/Makefile b/block/Makefile index c65f4da93702..0190e5aa9f00 100644
...
diff --git a/block/blk-mq.c b/block/blk-mq.c index f2650c97a75e..1ff3a7e3191b 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -29,6 +29,7 @@ #include <linux/blk-crypto.h> #include <linux/part_stat.h> #include <linux/sched/isolation.h> +#include <linux/blk-mq-dma-token.h>
#include <trace/events/block.h>
@@ -439,6 +440,7 @@ static struct request *blk_mq_rq_ctx_init(struct blk_mq_alloc_data *data, rq->nr_integrity_segments = 0; rq->end_io = NULL; rq->end_io_data = NULL; + rq->dma_map = NULL;
blk_crypto_rq_set_defaults(rq); INIT_LIST_HEAD(&rq->queuelist); @@ -794,6 +796,7 @@ static void __blk_mq_free_request(struct request *rq) blk_pm_mark_last_busy(rq); rq->mq_hctx = NULL;
+ blk_rq_drop_dma_map(rq);
blk_rq_drop_dma_map(rq), needs to be added in blk_mq_end_request_batch as well[1], otherwise I am seeing we leave with increased reference count in dma-buf exporter side.
Thanks, Nitesh
[1] --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -1214,6 +1214,7 @@ void blk_mq_end_request_batch(struct io_comp_batch *iob)
blk_crypto_free_request(rq); blk_pm_mark_last_busy(rq); + blk_rq_drop_dma_map(rq);
Ah yes, thanks Nitesh