With joint effort from the upstream KVM community, we come up with the 4th version of mediated vPMU for x86. We have made the following changes on top of the previous RFC v3.
v3 -> v4 - Rebase whole patchset on 6.14-rc3 base. - Address Peter's comments on Perf part. - Address Sean's comments on KVM part. * Change key word "passthrough" to "mediated" in all patches * Change static enabling to user space dynamic enabling via KVM_CAP_PMU_CAPABILITY. * Only support GLOBAL_CTRL save/restore with VMCS exec_ctrl, drop the MSR save/retore list support for GLOBAL_CTRL, thus the support of mediated vPMU is constrained to SapphireRapids and later CPUs on Intel side. * Merge some small changes into a single patch. - Address Sandipan's comment on invalid pmu pointer. - Add back "eventsel_hw" and "fixed_ctr_ctrl_hw" to avoid to directly manipulate pmc->eventsel and pmu->fixed_ctr_ctrl.
Testing (Intel side): - Perf-based legacy vPMU (force emulation on/off) * Kselftests pmu_counters_test, pmu_event_filter_test and vmx_pmu_caps_test pass. * KUT PMU tests pmu, pmu_lbr, pmu_pebs pass. * Basic perf counting/sampling tests in 3 scenarios, guest-only, host-only and host-guest coexistence all pass.
- Mediated vPMU (force emulation on/off) * Kselftests pmu_counters_test, pmu_event_filter_test and vmx_pmu_caps_test pass. * KUT PMU tests pmu, pmu_lbr, pmu_pebs pass. * Basic perf counting/sampling tests in 3 scenarios, guest-only, host-only and host-guest coexistence all pass.
- Failures. All above tests passed on Intel Granite Rapids as well except a failure on KUT/pmu_pebs. * GP counter 0 (0xfffffffffffe): PEBS record (written seq 0) is verified (including size, counters and cfg). * The pebs_data_cfg (0xb500000000) doesn't match with the effective MSR_PEBS_DATA_CFG (0x0). * This failure has nothing to do with this mediated vPMU patch set. The failure is caused by Granite Rapids supported timed PEBS which needs extra support on Qemu and KUT/pmu_pebs. These extra support would be sent in separate patches later.
Testing (AMD side): - Kselftests pmu_counters_test, pmu_event_filter_test and vmx_pmu_caps_test all pass
- legacy guest with KUT/pmu: * qmeu option: -cpu host, -perfctr-core * when set force_emulation_prefix=1, passes * when set force_emulation_prefix=0, passes - perfmon-v1 guest with KUT/pmu: * qmeu option: -cpu host, -perfmon-v2 * when set force_emulation_prefix=1, passes * when set force_emulation_prefix=0, passes - perfmon-v2 guest with KUT/pmu: * qmeu option: -cpu host * when set force_emulation_prefix=1, passes * when set force_emulation_prefix=0, passes
- perf_fuzzer (perfmon-v2): * fails with soft lockup in guest in current version. * culprit could be between 6.13 ~ 6.14-rc3 within KVM * Series tested on 6.12 and 6.13 without issue.
Note: a QEMU series is needed to run mediated vPMU v4: - https://lore.kernel.org/all/20250324123712.34096-1-dapeng1.mi@linux.intel.co...
History: - RFC v3: https://lore.kernel.org/all/20240801045907.4010984-1-mizhang@google.com/ - RFC v2: https://lore.kernel.org/all/20240506053020.3911940-1-mizhang@google.com/ - RFC v1: https://lore.kernel.org/all/20240126085444.324918-1-xiong.y.zhang@linux.inte...
Dapeng Mi (18): KVM: x86/pmu: Introduce enable_mediated_pmu global parameter KVM: x86/pmu: Check PMU cpuid configuration from user space KVM: x86: Rename vmx_vmentry/vmexit_ctrl() helpers KVM: x86/pmu: Add perf_capabilities field in struct kvm_host_values{} KVM: x86/pmu: Move PMU_CAP_{FW_WRITES,LBR_FMT} into msr-index.h header KVM: VMX: Add macros to wrap around {secondary,tertiary}_exec_controls_changebit() KVM: x86/pmu: Check if mediated vPMU can intercept rdpmc KVM: x86/pmu/vmx: Save/load guest IA32_PERF_GLOBAL_CTRL with vm_exit/entry_ctrl KVM: x86/pmu: Optimize intel/amd_pmu_refresh() helpers KVM: x86/pmu: Setup PMU MSRs' interception mode KVM: x86/pmu: Handle PMU MSRs interception and event filtering KVM: x86/pmu: Switch host/guest PMU context at vm-exit/vm-entry KVM: x86/pmu: Handle emulated instruction for mediated vPMU KVM: nVMX: Add macros to simplify nested MSR interception setting KVM: selftests: Add mediated vPMU supported for pmu tests KVM: Selftests: Support mediated vPMU for vmx_pmu_caps_test KVM: Selftests: Fix pmu_counters_test error for mediated vPMU KVM: x86/pmu: Expose enable_mediated_pmu parameter to user space
Kan Liang (8): perf: Support get/put mediated PMU interfaces perf: Skip pmu_ctx based on event_type perf: Clean up perf ctx time perf: Add a EVENT_GUEST flag perf: Add generic exclude_guest support perf: Add switch_guest_ctx() interface perf/x86: Support switch_guest_ctx interface perf/x86/intel: Support PERF_PMU_CAP_MEDIATED_VPMU
Mingwei Zhang (5): perf/x86: Forbid PMI handler when guest own PMU perf/x86/core: Plumb mediated PMU capability from x86_pmu to x86_pmu_cap KVM: x86/pmu: Exclude PMU MSRs in vmx_get_passthrough_msr_slot() KVM: x86/pmu: introduce eventsel_hw to prepare for pmu event filtering KVM: nVMX: Add nested virtualization support for mediated PMU
Sandipan Das (4): perf/x86/core: Do not set bit width for unavailable counters KVM: x86/pmu: Add AMD PMU registers to direct access list KVM: x86/pmu/svm: Set GuestOnly bit and clear HostOnly bit when guest write to event selectors perf/x86/amd: Support PERF_PMU_CAP_MEDIATED_VPMU for AMD host
Xiong Zhang (3): x86/irq: Factor out common code for installing kvm irq handler perf: core/x86: Register a new vector for KVM GUEST PMI KVM: x86/pmu: Register KVM_GUEST_PMI_VECTOR handler
arch/x86/events/amd/core.c | 2 + arch/x86/events/core.c | 40 +- arch/x86/events/intel/core.c | 5 + arch/x86/include/asm/hardirq.h | 1 + arch/x86/include/asm/idtentry.h | 1 + arch/x86/include/asm/irq.h | 2 +- arch/x86/include/asm/irq_vectors.h | 5 +- arch/x86/include/asm/kvm-x86-pmu-ops.h | 2 + arch/x86/include/asm/kvm_host.h | 10 + arch/x86/include/asm/msr-index.h | 18 +- arch/x86/include/asm/perf_event.h | 1 + arch/x86/include/asm/vmx.h | 1 + arch/x86/kernel/idt.c | 1 + arch/x86/kernel/irq.c | 39 +- arch/x86/kvm/cpuid.c | 15 + arch/x86/kvm/pmu.c | 254 ++++++++- arch/x86/kvm/pmu.h | 45 ++ arch/x86/kvm/svm/pmu.c | 148 ++++- arch/x86/kvm/svm/svm.c | 26 + arch/x86/kvm/svm/svm.h | 2 +- arch/x86/kvm/vmx/capabilities.h | 11 +- arch/x86/kvm/vmx/nested.c | 68 ++- arch/x86/kvm/vmx/pmu_intel.c | 224 ++++++-- arch/x86/kvm/vmx/vmx.c | 89 +-- arch/x86/kvm/vmx/vmx.h | 11 +- arch/x86/kvm/x86.c | 63 ++- arch/x86/kvm/x86.h | 2 + include/linux/perf_event.h | 47 +- kernel/events/core.c | 519 ++++++++++++++---- .../beauty/arch/x86/include/asm/irq_vectors.h | 5 +- .../selftests/kvm/include/kvm_test_harness.h | 13 + .../testing/selftests/kvm/include/kvm_util.h | 3 + .../selftests/kvm/include/x86/processor.h | 8 + tools/testing/selftests/kvm/lib/kvm_util.c | 23 + .../selftests/kvm/x86/pmu_counters_test.c | 24 +- .../selftests/kvm/x86/pmu_event_filter_test.c | 8 +- .../selftests/kvm/x86/vmx_pmu_caps_test.c | 2 +- 37 files changed, 1480 insertions(+), 258 deletions(-)
base-commit: 0ad2507d5d93f39619fc42372c347d6006b64319