On Mon, 15 Apr 2024, Bart Van Assche wrote:
The following behavior is inconsistent:
- For request-based dm queues the default value of rq_affinity is 1.
- For bio-based dm queues the default value of rq_affinity is 0.
The default value for request-based dm queues is 1 because of the following code in blk_mq_init_allocated_queue():
q->queue_flags |= QUEUE_FLAG_MQ_DEFAULT;From <linux/blkdev.h>:
#define QUEUE_FLAG_MQ_DEFAULT ((1UL << QUEUE_FLAG_IO_STAT) | \ (1UL << QUEUE_FLAG_SAME_COMP) | \ (1UL << QUEUE_FLAG_NOWAIT))The default value of rq_affinity for bio-based dm queues is 0 because the dm alloc_dev() function does not set any of the QUEUE_FLAG_SAME_* flags. I think the different default values are the result of an oversight when blk-mq support was added in the device mapper code. Hence this patch that changes the default value of rq_affinity from 0 to 1 for bio-based dm queues.
This patch reduces the boot time from 12.23 to 12.20 seconds on my test
Are you sure that this is not jitter?
I am wondering how should QUEUE_FLAG_SAME_COMP work for bio-based devices.
I grepped the kernel for QUEUE_FLAG_SAME_COMP and it is tested in block/blk-mq.c in blk_mq_complete_need_ipi (this code path is taken only for request-based devices) and in block/blk-sysfs.c in queue_rq_affinity_show (this just displays the value in sysfs). There are no other places where QUEUE_FLAG_SAME_COMP is tested, so I don't see what effect is it supposed to have.
Mikulas
setup, a Pixel 2023 development board. The storage controller on that test setup supports a single completion interrupt and hence benefits from redirecting I/O completions to a CPU core that is closer to the submitter.
Cc: Mikulas Patocka mpatocka@redhat.com Cc: Eric Biggers ebiggers@kernel.org Cc: Jaegeuk Kim jaegeuk@kernel.org Cc: Daniel Lee chullee@google.com Cc: stable@vger.kernel.org Fixes: bfebd1cdb497 ("dm: add full blk-mq support to request-based DM") Signed-off-by: Bart Van Assche bvanassche@acm.org
drivers/md/dm.c | 1 + 1 file changed, 1 insertion(+)
diff --git a/drivers/md/dm.c b/drivers/md/dm.c index 56aa2a8b9d71..9af216c11cf7 100644 --- a/drivers/md/dm.c +++ b/drivers/md/dm.c @@ -2106,6 +2106,7 @@ static struct mapped_device *alloc_dev(int minor) if (IS_ERR(md->disk)) goto bad; md->queue = md->disk->queue;
- blk_queue_flag_set(QUEUE_FLAG_SAME_COMP, md->queue);
init_waitqueue_head(&md->wait); INIT_WORK(&md->work, dm_wq_work);