Hi Jason,
On Wed, Sep 03, 2025 at 02:46:40PM -0300, Jason Gunthorpe wrote:
From: Alejandro Jimenez alejandro.j.jimenez@oracle.com
Replace the io_pgtable versions with pt_iommu versions. The v2 page table uses the x86 implementation that will be eventually shared with VT-D.
This supports the same special features as the original code:
- increase_top for the v1 format to allow scaling from 3 to 6 levels
- non-present flushing
- Dirty tracking for v1 only
- __sme_set() to adjust the PTEs for CC
- Optimization for flushing with virtualization to minimize the range
- amd_iommu_pgsize_bitmap override of the native page sizes
- page tables allocate from the device's NUMA node
Rework the domain ops so that v1/v2 get their own ops. Make dedicated allocation functions for v1 and v2. Hook up invalidation for a top change to struct pt_iommu_flush_ops. Delete some of the iopgtable related code that becomes unused in this patch. The next patch will delete the rest of it.
This fixes a race bug in AMD's increase_address_space() implementation. It stores the top level and top pointer in different memory, which prevents other threads from reading a coherent version:
increase_address_space() alloc_pte() level = pgtable->mode - 1; pgtable->root = pte; pgtable->mode += 1; pte = &pgtable->root[PM_LEVEL_INDEX(level, address)];
The iommupt version is careful to put mode and root under a single READ_ONCE and then is careful to only READ_ONCE a single time per walk.
Signed-off-by: Alejandro Jimenez alejandro.j.jimenez@oracle.com Tested-by: Alejandro Jimenez alejandro.j.jimenez@oracle.com Signed-off-by: Jason Gunthorpe jgg@nvidia.com
drivers/iommu/amd/Kconfig | 5 +- drivers/iommu/amd/amd_iommu.h | 1 - drivers/iommu/amd/amd_iommu_types.h | 12 +- drivers/iommu/amd/io_pgtable.c | 1 - drivers/iommu/amd/iommu.c | 538 ++++++++++++++-------------- 5 files changed, 282 insertions(+), 275 deletions(-)
../..
+static struct iommu_domain *amd_iommu_domain_alloc_paging_v1(struct device *dev,
u32 flags)+{
- struct pt_iommu_amdv1_cfg cfg = {}; struct protection_domain *domain; int ret;
- if (amd_iommu_hatdis)
return ERR_PTR(-EOPNOTSUPP);- domain = protection_domain_alloc(); if (!domain) return ERR_PTR(-ENOMEM);
- domain->pd_mode = pgtable;
- ret = pdom_setup_pgtable(domain, dev);
- domain->pd_mode = PD_MODE_V1;
- domain->iommu.hw_flush_ops = &amd_hw_flush_ops_v1;
- domain->iommu.nid = dev_to_node(dev);
- if (flags & IOMMU_HWPT_ALLOC_DIRTY_TRACKING)
domain->domain.dirty_ops = &amdv1_dirty_ops;- /*
* Someday FORCE_COHERENCE should be set by* amd_iommu_enforce_cache_coherency() like VT-D does.*/- cfg.common.features = BIT(PT_FEAT_DYNAMIC_TOP) |
BIT(PT_FEAT_AMDV1_ENCRYPT_TABLES) |BIT(PT_FEAT_AMDV1_FORCE_COHERENCE);- /*
* AMD's IOMMU can flush as many pages as necessary in a single flush.* Unless we run in a virtual machine, which can be inferred according* to whether "non-present cache" is on, it is probably best to prefer* (potentially) too extensive TLB flushing (i.e., more misses) over* multiple TLB flushes (i.e., more flushes). For virtual machines the* hypervisor needs to synchronize the host IOMMU PTEs with those of* the guest, and the trade-off is different: unnecessary TLB flushes* should be avoided.*/- if (amd_iommu_np_cache)
cfg.common.features |= BIT(PT_FEAT_FLUSH_RANGE_NO_GAPS);- else
cfg.common.features |= BIT(PT_FEAT_FLUSH_RANGE);- cfg.common.hw_max_vasz_lg2 =
max(64, (amd_iommu_hpt_level - 1) * 9 + 21);
s/max/min with 6 level page table max will be 66 and i am seeing boot failure with this. Please refer below fail log.
[ 5.458439] BUG: kernel NULL pointer dereference, address: 0000000000000030 [ 5.459338] #PF: supervisor read access in kernel mode [ 5.459338] #PF: error_code(0x0000) - not-present page [ 5.459338] PGD 0 [ 5.459338] Oops: Oops: 0000 [#1] SMP NOPTI [ 5.459338] CPU: 0 UID: 0 PID: 1 Comm: swapper/0 Not tainted 6.17.0-rc3+ #83 PREEMPT(voluntary) [ 5.459338] Hardware name: AMD Corporation RUBY/RUBY, BIOS RRR100DB 08/09/2024 [ 5.459338] RIP: 0010:sysfs_add_link_to_group+0x16/0x60 [ 5.459338] Code: 00 00 00 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 0f 1f 44 00 00 55 48 89 e5 41 55 49 89 cd 41 54 49 89 d4 31 d2 53 <48> 8b 7f 30 e8 b1 8d ff ff 48 85 c0 74 32 48 89 c3 4c 89 e6 48 89 [ 5.459338] RSP: 0018:ff7421b640067c00 EFLAGS: 00010246 [ 5.459338] RAX: ff1b1885cb669a20 RBX: ff1b1885c028e0c8 RCX: ff1b1885cb6289b0 [ 5.459338] RDX: 0000000000000000 RSI: ffffffffacb14932 RDI: 0000000000000000 [ 5.459338] RBP: ff7421b640067c18 R08: 0000000000000000 R09: 0000000000000000 [ 5.459338] R10: 0000000000000000 R11: 0000000000000000 R12: ff1b1885c028e0c8 [ 5.459338] R13: ff1b1885cb6289b0 R14: ff7421b640067d20 R15: ff1b1885c0a20800 [ 5.459338] FS: 0000000000000000(0000) GS:ff1b18951a09a000(0000) knlGS:0000000000000000 [ 5.459338] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5.459338] CR2: 0000000000000030 CR3: 0000000f65640001 CR4: 0000000000f71ef0 [ 5.459338] PKRU: 55555554 [ 5.459338] Call Trace: [ 5.459338] <TASK> [ 5.459338] iommu_device_link+0x36/0xb0 [ 5.459338] __iommu_probe_device+0x123/0x550 [ 5.459338] ? __pfx_probe_iommu_group+0x10/0x10 [ 5.459338] probe_iommu_group+0x29/0x50 [ 5.459338] bus_for_each_dev+0x89/0xf0 [ 5.459338] iommu_device_register+0xca/0x240 [ 5.459338] iommu_go_to_state+0x7d2/0x2340 [ 5.459338] ? __pfx_pci_iommu_init+0x10/0x10 [ 5.459338] amd_iommu_init+0x14/0x40 [ 5.459338] ? __pfx_pci_iommu_init+0x10/0x10 [ 5.459338] pci_iommu_init+0x13/0x70 [ 5.459338] ? __pfx_pci_iommu_init+0x10/0x10 [ 5.459338] do_one_initcall+0x5a/0x340 [ 5.459338] kernel_init_freeable+0x34e/0x530 [ 5.459338] ? __pfx_kernel_init+0x10/0x10 [ 5.459338] kernel_init+0x1b/0x200 [ 5.459338] ? __pfx_kernel_init+0x10/0x10 [ 5.459338] ret_from_fork+0x1a1/0x1d0 [ 5.459338] ? __pfx_kernel_init+0x10/0x10 [ 5.459338] ret_from_fork_asm+0x1a/0x30 [ 5.459338] RIP: 1f0f:0x0 [ 5.459338] Code: Unable to access opcode bytes at 0xffffffffffffffd6. [ 5.459338] RSP: 0000:0000000000000000 EFLAGS: 841f0f2e66 ORIG_RAX: 1f0f2e6600000000 [ 5.459338] RAX: 0000000000000000 RBX: 1f0f2e6600000000 RCX: 2e66000000000084 [ 5.459338] RDX: 0000000000841f0f RSI: 000000841f0f2e66 RDI: 00841f0f2e660000 [ 5.459338] RBP: 00841f0f2e660000 R08: 00841f0f2e660000 R09: 000000841f0f2e66 [ 5.459338] R10: 0000000000841f0f R11: 2e66000000000084 R12: 000000841f0f2e66 [ 5.459338] R13: 0000000000841f0f R14: 2e66000000000084 R15: 1f0f2e6600000000 [ 5.459338] </TASK> [ 5.459338] Modules linked in: [ 5.459338] CR2: 0000000000000030 [ 5.459338] ---[ end trace 0000000000000000 ]--- [ 5.459338] RIP: 0010:sysfs_add_link_to_group+0x16/0x60 [ 5.459338] Code: 00 00 00 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 0f 1f 44 00 00 55 48 89 e5 41 55 49 89 cd 41 54 49 89 d4 31 d2 53 <48> 8b 7f 30 e8 b1 8d ff ff 48 85 c0 74 32 48 89 c3 4c 89 e6 48 89 [ 5.459338] RSP: 0018:ff7421b640067c00 EFLAGS: 00010246 [ 5.459338] RAX: ff1b1885cb669a20 RBX: ff1b1885c028e0c8 RCX: ff1b1885cb6289b0 [ 5.459338] RDX: 0000000000000000 RSI: ffffffffacb14932 RDI: 0000000000000000 [ 5.459338] RBP: ff7421b640067c18 R08: 0000000000000000 R09: 0000000000000000 [ 5.459338] R10: 0000000000000000 R11: 0000000000000000 R12: ff1b1885c028e0c8 [ 5.459338] R13: ff1b1885cb6289b0 R14: ff7421b640067d20 R15: ff1b1885c0a20800 [ 5.459338] FS: 0000000000000000(0000) GS:ff1b18951a09a000(0000) knlGS:0000000000000000 [ 5.459338] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 5.459338] CR2: 0000000000000030 CR3: 0000000f65640001 CR4: 0000000000f71ef0 [ 5.459338] PKRU: 55555554 [ 5.459338] note: swapper/0[1] exited with irqs disabled [ 5.874958] Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000009 [ 5.875940] ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000009 ]---
- cfg.common.hw_max_oasz_lg2 = 52;
- cfg.starting_level = 2;
- domain->domain.ops = &amdv1_ops;
- ret = pt_iommu_amdv1_init(&domain->amdv1, &cfg, GFP_KERNEL); if (ret) {
pdom_id_free(domain->id);kfree(domain);
amd_iommu_domain_free(&domain->domain);
- domain->domain.geometry.aperture_start = 0;
- domain->domain.geometry.aperture_end = dma_max_address(pgtable);
- domain->domain.geometry.force_aperture = true;
- domain->domain.pgsize_bitmap = domain->iop.pgtbl.cfg.pgsize_bitmap;
- /*
* Narrow the supported page sizes to those selected by the kernel* command line.*/- domain->domain.pgsize_bitmap &= amd_iommu_pgsize_bitmap;
- return &domain->domain;
+} 2.43.0
Thanks, Ankit