During the review of iommufd pasid series, Kevin and Jason suggested attaching PASID to the blocked domain hence replacing the usage of remove_dev_pasid() op [1]. This makes sense as it makes the PASID path aligned with the RID path which attaches the RID to the blocked_domain when it is to be blocked. To do it, it requires passing the old domain to the iommu driver. This has been done in [2].
This series makes the Intel iommu driver and ARM SMMUv3 driver support attaching PASID to the blocked domain. While the AMD iommu driver does not have the blocked domain yet, so still uses the remove_dev_pasid() op.
[1] https://lore.kernel.org/linux-iommu/20240816130202.GB2032816@nvidia.com/ [2] https://lore.kernel.org/linux-iommu/20240912130427.10119-1-yi.l.liu@intel.co...
Regards, Yi Liu
Jason Gunthorpe (1): iommu/arm-smmu-v3: Make smmuv3 blocked domain support PASID
Yi Liu (2): iommu/vt-d: Make blocked domain support PASID iommu: Add a wrapper for remove_dev_pasid
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 12 ++++----- drivers/iommu/intel/iommu.c | 17 ++++++++---- drivers/iommu/iommu.c | 30 ++++++++++++++++----- 3 files changed, 42 insertions(+), 17 deletions(-)
From: Jason Gunthorpe jgg@nvidia.com
The blocked domain is used to park RID to be blocking DMA state. This can be extended to PASID as well. By this, the remove_dev_pasid() op of ARM SMMUv3 can be dropped.
Signed-off-by: Jason Gunthorpe jgg@nvidia.com Signed-off-by: Yi Liu yi.l.liu@intel.com --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index f7a73b854151..ca768f04f378 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2933,13 +2933,12 @@ int arm_smmu_set_pasid(struct arm_smmu_master *master, return ret; }
-static void arm_smmu_remove_dev_pasid(struct device *dev, ioasid_t pasid, - struct iommu_domain *domain) +static int arm_smmu_blocking_set_dev_pasid(struct iommu_domain *new_domain, + struct device *dev, ioasid_t pasid, + struct iommu_domain *old_domain) { + struct arm_smmu_domain *smmu_domain = to_smmu_domain(old_domain); struct arm_smmu_master *master = dev_iommu_priv_get(dev); - struct arm_smmu_domain *smmu_domain; - - smmu_domain = to_smmu_domain(domain);
mutex_lock(&arm_smmu_asid_lock); arm_smmu_clear_cd(master, pasid); @@ -2960,6 +2959,7 @@ static void arm_smmu_remove_dev_pasid(struct device *dev, ioasid_t pasid, sid_domain->type == IOMMU_DOMAIN_BLOCKED) sid_domain->ops->attach_dev(sid_domain, dev); } + return 0; }
static void arm_smmu_attach_dev_ste(struct iommu_domain *domain, @@ -3041,6 +3041,7 @@ static int arm_smmu_attach_dev_blocked(struct iommu_domain *domain,
static const struct iommu_domain_ops arm_smmu_blocked_ops = { .attach_dev = arm_smmu_attach_dev_blocked, + .set_dev_pasid = arm_smmu_blocking_set_dev_pasid, };
static struct iommu_domain arm_smmu_blocked_domain = { @@ -3483,7 +3484,6 @@ static struct iommu_ops arm_smmu_ops = { .device_group = arm_smmu_device_group, .of_xlate = arm_smmu_of_xlate, .get_resv_regions = arm_smmu_get_resv_regions, - .remove_dev_pasid = arm_smmu_remove_dev_pasid, .dev_enable_feat = arm_smmu_dev_enable_feature, .dev_disable_feat = arm_smmu_dev_disable_feature, .page_response = arm_smmu_page_response,
From: Liu, Yi L yi.l.liu@intel.com Sent: Thursday, September 12, 2024 9:07 PM
From: Jason Gunthorpe jgg@nvidia.com
The blocked domain is used to park RID to be blocking DMA state. This can be extended to PASID as well. By this, the remove_dev_pasid() op of ARM SMMUv3 can be dropped.
Signed-off-by: Jason Gunthorpe jgg@nvidia.com Signed-off-by: Yi Liu yi.l.liu@intel.com
Reviewed-by: Kevin Tian kevin.tian@intel.com
The blocked domain can be extended to park PASID of a device to be the DMA blocking state. By this the remove_dev_pasid() op is dropped.
Signed-off-by: Yi Liu yi.l.liu@intel.com --- drivers/iommu/intel/iommu.c | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-)
diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index 749ee7741ec4..ed34ee45a25c 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -3426,10 +3426,15 @@ static int blocking_domain_attach_dev(struct iommu_domain *domain, return 0; }
+static int blocking_domain_set_dev_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid, + struct iommu_domain *old); + static struct iommu_domain blocking_domain = { .type = IOMMU_DOMAIN_BLOCKED, .ops = &(const struct iommu_domain_ops) { .attach_dev = blocking_domain_attach_dev, + .set_dev_pasid = blocking_domain_set_dev_pasid, } };
@@ -4280,8 +4285,9 @@ static void domain_remove_dev_pasid(struct iommu_domain *domain, kfree(dev_pasid); }
-static void intel_iommu_remove_dev_pasid(struct device *dev, ioasid_t pasid, - struct iommu_domain *domain) +static int blocking_domain_set_dev_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid, + struct iommu_domain *old) { struct device_domain_info *info = dev_iommu_priv_get(dev); struct intel_iommu *iommu = info->iommu; @@ -4289,8 +4295,10 @@ static void intel_iommu_remove_dev_pasid(struct device *dev, ioasid_t pasid, intel_pasid_tear_down_entry(iommu, dev, pasid, INTEL_PASID_TEARDOWN_DRAIN_PRQ); if (domain->type == IOMMU_DOMAIN_IDENTITY) - return; - domain_remove_dev_pasid(domain, dev, pasid); + goto out; + domain_remove_dev_pasid(old, dev, pasid); +out: + return 0; }
static struct dev_pasid_info * @@ -4651,7 +4659,6 @@ const struct iommu_ops intel_iommu_ops = { .dev_disable_feat = intel_iommu_dev_disable_feat, .is_attach_deferred = intel_iommu_is_attach_deferred, .def_domain_type = device_def_domain_type, - .remove_dev_pasid = intel_iommu_remove_dev_pasid, .pgsize_bitmap = SZ_4K, #ifdef CONFIG_INTEL_IOMMU_SVM .page_response = intel_svm_page_response,
From: Yi Liu yi.l.liu@intel.com Sent: Thursday, September 12, 2024 9:07 PM
The blocked domain can be extended to park PASID of a device to be the DMA blocking state. By this the remove_dev_pasid() op is dropped.
Signed-off-by: Yi Liu yi.l.liu@intel.com
Reviewed-by: Kevin Tian kevin.tian@intel.com
The iommu drivers are on the way to drop the remove_dev_pasid op by extending the blocked_domain to support PASID. However, this cannot be done in one shot. So far, the Intel iommu and the ARM SMMUv3 driver have supported it, while the AMD iommu driver has not yet. During this transition, the IOMMU core needs to support both ways to destroy the attachment of device/PASID and domain.
Signed-off-by: Yi Liu yi.l.liu@intel.com --- drivers/iommu/iommu.c | 30 ++++++++++++++++++++++++------ 1 file changed, 24 insertions(+), 6 deletions(-)
diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index f3f81c04b8fb..b6b44b184004 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -3324,6 +3324,28 @@ bool iommu_group_dma_owner_claimed(struct iommu_group *group) } EXPORT_SYMBOL_GPL(iommu_group_dma_owner_claimed);
+/* + * This is only needed in the transition of dropping remove_dev_pasid op + * by adding set_dev_pasid op for the blocked domains. + */ +static void iommu_remove_dev_pasid(struct device *dev, ioasid_t pasid, + struct iommu_domain *domain) +{ + const struct iommu_ops *ops = dev_iommu_ops(dev); + struct iommu_domain *blocked_domain = ops->blocked_domain; + int ret = 1; + + if (blocked_domain->ops->set_dev_pasid) { + ret = blocked_domain->ops->set_dev_pasid(blocked_domain, + dev, pasid, domain); + } else if (ops->remove_dev_pasid) { + ops->remove_dev_pasid(dev, pasid, domain); + ret = 0; + } + + WARN_ON(ret); +} + static int __iommu_set_group_pasid(struct iommu_domain *domain, struct iommu_group *group, ioasid_t pasid) { @@ -3342,11 +3364,9 @@ static int __iommu_set_group_pasid(struct iommu_domain *domain, err_revert: last_gdev = device; for_each_group_device(group, device) { - const struct iommu_ops *ops = dev_iommu_ops(device->dev); - if (device == last_gdev) break; - ops->remove_dev_pasid(device->dev, pasid, domain); + iommu_remove_dev_pasid(device->dev, pasid, domain); } return ret; } @@ -3356,11 +3376,9 @@ static void __iommu_remove_group_pasid(struct iommu_group *group, struct iommu_domain *domain) { struct group_device *device; - const struct iommu_ops *ops;
for_each_group_device(group, device) { - ops = dev_iommu_ops(device->dev); - ops->remove_dev_pasid(device->dev, pasid, domain); + iommu_remove_dev_pasid(device->dev, pasid, domain); } }
On 9/12/24 9:06 PM, Yi Liu wrote:
The iommu drivers are on the way to drop the remove_dev_pasid op by extending the blocked_domain to support PASID. However, this cannot be done in one shot. So far, the Intel iommu and the ARM SMMUv3 driver have supported it, while the AMD iommu driver has not yet. During this transition, the IOMMU core needs to support both ways to destroy the attachment of device/PASID and domain.
Signed-off-by: Yi Liuyi.l.liu@intel.com
drivers/iommu/iommu.c | 30 ++++++++++++++++++++++++------ 1 file changed, 24 insertions(+), 6 deletions(-)
This patch should be moved before the change in drivers. After all drivers convert to use set blocking domain to pasid, the remove_dev_pasid could be removed as the last step.
Thanks, baolu
On 2024/9/13 10:24, Baolu Lu wrote:
On 9/12/24 9:06 PM, Yi Liu wrote:
The iommu drivers are on the way to drop the remove_dev_pasid op by extending the blocked_domain to support PASID. However, this cannot be done in one shot. So far, the Intel iommu and the ARM SMMUv3 driver have supported it, while the AMD iommu driver has not yet. During this transition, the IOMMU core needs to support both ways to destroy the attachment of device/PASID and domain.
Signed-off-by: Yi Liuyi.l.liu@intel.com
drivers/iommu/iommu.c | 30 ++++++++++++++++++++++++------ 1 file changed, 24 insertions(+), 6 deletions(-)
This patch should be moved before the change in drivers. After all drivers convert to use set blocking domain to pasid, the remove_dev_pasid could be removed as the last step.
you are right! let me fix it.
From: Liu, Yi L yi.l.liu@intel.com Sent: Thursday, September 12, 2024 9:07 PM
The iommu drivers are on the way to drop the remove_dev_pasid op by extending the blocked_domain to support PASID. However, this cannot be done in one shot. So far, the Intel iommu and the ARM SMMUv3 driver have supported it, while the AMD iommu driver has not yet. During this transition, the IOMMU core needs to support both ways to destroy the attachment of device/PASID and domain.
Signed-off-by: Yi Liu yi.l.liu@intel.com
drivers/iommu/iommu.c | 30 ++++++++++++++++++++++++------ 1 file changed, 24 insertions(+), 6 deletions(-)
diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index f3f81c04b8fb..b6b44b184004 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -3324,6 +3324,28 @@ bool iommu_group_dma_owner_claimed(struct iommu_group *group) } EXPORT_SYMBOL_GPL(iommu_group_dma_owner_claimed);
+/*
- This is only needed in the transition of dropping remove_dev_pasid op
- by adding set_dev_pasid op for the blocked domains.
- */
let's be specific that it's gated by AMD's support and temporary.
with the order adjusted as Baolu suggested:
Reviewed-by: Kevin Tian kevin.tian@intel.com
linux-kselftest-mirror@lists.linaro.org
-
Baolu Lu -
Tian, Kevin -
Yi Liu