iommu/amd: fix amd iotlb flush range in unmap

This was fixed in mainline in 6b080c4e815ceba3c08ffa980c858595c07e786a, but we do not backport the full patch, only the part that changes IOTLB flush range calculation(the bug fix part). The other part of the patch is about API change and refactoring of the code to use the new API, which are not required for the bug fix. Besides, the API change in AMD IOMMU driver from map/unmap to map_pages/unmap_pages also has a dependency on the IOMMU core driver's changes that is not included in 5.15.

This bug fix patch only applies to 5.15.y. For 5.10 LTS and earlier LTS versions, they don't have selective IOTLB flush support, so this patch does not apply. For 6.1 and later, they already have the fix as part of the API change.

AMD IOMMU driver supports power of 2 KB page size, it can be 4K, 8K, 16K, etc. So when VFIO driver ask AMD IOMMU driver to unmap a IOVA with a page_size 4K, it actually can unmap a page_size of 8K, depending on the page used during mapping. However, the iotlb gather function use the page_size as the range of unmap range, instead of the real unmapped page size r.

This miscalculation of iotlb flush range will make the unflushed IOTLB entry stale. It triggered hard-to-debug silent data corruption issue as DMA engine who used the stale IOTLB entry will DMA into unmapped memory region.

Note, the upstream commit aims at changing API from map/unmap_page() to map/unmap_pages() and changed the gather range calculation along with it. It accidentally fixed this bug in the mainline since 6.1 without realizing there was a bug in the old range calculation.

Cc: Nadav Amit namit@vmware.com Cc: Joerg Roedel joro@8bytes.org Cc: Will Deacon will@kernel.org Cc: Robin Murphy robin.murphy@arm.com Cc: Lu Baolu baolu.lu@linux.intel.com Cc: iommu@lists.linux-foundation.org Fixes: fc65d0acaf23 ("iommu/amd: Selective flush on unmap") Signed-off-by: Zhichuang Sun zhichuang@google.com --- drivers/iommu/amd/iommu.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c index 714c78bf69db..d3a11be8d1dd 100644 --- a/drivers/iommu/amd/iommu.c +++ b/drivers/iommu/amd/iommu.c @@ -2121,7 +2121,8 @@ static size_t amd_iommu_unmap(struct iommu_domain *dom, unsigned long iova,

r = (ops->unmap) ? ops->unmap(ops, iova, page_size, gather) : 0;

- amd_iommu_iotlb_gather_add_page(dom, gather, iova, page_size); + if (r) + amd_iommu_iotlb_gather_add_page(dom, gather, iova, r);

return r; }