On Mon, Jun 30, 2025 at 02:25:23PM -0600, Keith Busch wrote:
I think the PRP handling is broken. At the very least, handling the last element is wrong if it appears at the end of the list, so I think we need something like this:
Yeah.
But even that, the PRP setup doesn't match the teardown. We're calling dma_map_page() on each PRP even if consecutive PRP's came from the same dma mapping segment. So even if it had been coalesced, but if the device doesn't support SGLs, then it would use the prp unmap path.
Yes, that's broken, and I remember fixing it before. A little digging shows that my fixes disappeared between the oct 30 version of Leon's dma-split branch and the latest one somewhere. Below is what should restore it, but at least when forcing my Intel IOMMU down this path it still has issues with VPTEs already set. So maybe Bob should not try it quite yet. I'll try to get to it, but my availability today and tomorrow is a bit limited.
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c index 38be1505dbd9..02bb5cf5db1a 100644 --- a/drivers/nvme/host/pci.c +++ b/drivers/nvme/host/pci.c @@ -678,40 +678,55 @@ static void nvme_free_prps(struct request *req) enum dma_data_direction dir = rq_dma_dir(req); int length = iod->total_len; dma_addr_t dma_addr; - int i, desc; + int prp_len, i, desc; __le64 *prp_list; + dma_addr_t dma_start; u32 dma_len;
dma_addr = le64_to_cpu(iod->cmd.common.dptr.prp1); - dma_len = min_t(u32, length, - NVME_CTRL_PAGE_SIZE - (dma_addr & (NVME_CTRL_PAGE_SIZE - 1))); - length -= dma_len; + prp_len = NVME_CTRL_PAGE_SIZE - (dma_addr & (NVME_CTRL_PAGE_SIZE - 1)); + prp_len = min(length, prp_len); + length -= prp_len; if (!length) { - dma_unmap_page(dma_dev, dma_addr, dma_len, dir); + dma_unmap_page(dma_dev, dma_addr, prp_len, dir); return; }
+ dma_start = dma_addr; + dma_len = prp_len; + dma_addr = le64_to_cpu(iod->cmd.common.dptr.prp2); + if (length <= NVME_CTRL_PAGE_SIZE) { - dma_unmap_page(dma_dev, dma_addr, dma_len, dir); - dma_addr = le64_to_cpu(iod->cmd.common.dptr.prp2); - dma_unmap_page(dma_dev, dma_addr, length, dir); - return; + if (dma_addr != dma_start + dma_len) { + dma_unmap_page(dma_dev, dma_start, dma_len, dir); + dma_start = dma_addr; + dma_len = 0; + } + dma_len += length; + goto done; }
i = 0; desc = 0; prp_list = iod->descriptors[desc]; do { - dma_unmap_page(dma_dev, dma_addr, dma_len, dir); if (i == NVME_CTRL_PAGE_SIZE >> 3) { prp_list = iod->descriptors[++desc]; i = 0; }
dma_addr = le64_to_cpu(prp_list[i++]); - dma_len = min(length, NVME_CTRL_PAGE_SIZE); - length -= dma_len; + if (dma_addr != dma_start + dma_len) { + dma_unmap_page(dma_dev, dma_start, dma_len, dir); + dma_start = dma_addr; + dma_len = 0; + } + prp_len = min(length, NVME_CTRL_PAGE_SIZE); + dma_len += prp_len; + length -= prp_len; } while (length); +done: + dma_unmap_page(dma_dev, dma_start, dma_len, dir); }
static void nvme_free_sgls(struct request *req)