This series adds the base support to preserve a VFIO device file across
a Live Update. "Base support" means that this allows userspace to
safetly preserve a VFIO device file with LIVEUPDATE_SESSION_PRESERVE_FD
and retrieve a preserved VFIO device file with
LIVEUPDATE_SESSION_RETRIEVE_FD, but the device itself is not preserved
in a fully running state across Live Update.
This series unblocks 2 parallel but related streams of work:
- iommufd preservation across Live Update. This work spans iommufd,
the IOMMU subsystem, and IOMMU drivers [1]
- Preservation of VFIO device state across Live Update (config space,
BAR addresses, power state, SR-IOV state, etc.). This work spans both
VFIO and the core PCI subsystem.
While we need all of the above to fully preserve a VFIO device across a
Live Update without disrupting the workload on the device, this series
aims to be functional and safe enough to merge as the first incremental
step toward that goal.
Areas for Discussion
--------------------
BDF Stability across Live Update
The PCI support for tracking preserved devices across a Live Update to
prevent auto-probing relies on PCI segment numbers and BDFs remaining
stable. For now I have disallowed VFs, as the BDFs assigned to VFs can
vary depending on how the kernel chooses to allocate bus numbers. For
non-VFs I am wondering if there is any more needed to ensure BDF
stability across Live Update.
While we would like to support many different systems and
configurations in due time (including preserving VFs), I'd like to
keep this first serses constrained to simple use-cases.
FLB Locking
I don't see a way to properly synchronize pci_flb_finish() with
pci_liveupdate_incoming_is_preserved() since the incoming FLB mutex is
dropped by liveupdate_flb_get_incoming() when it returns the pointer
to the object, and taking pci_flb_incoming_lock in pci_flb_finish()
could result in a deadlock due to reversing the lock ordering.
FLB Retrieving
The first patch of this series includes a fix to prevent an FLB from
being retrieved again it is finished. I am wondering if this is the
right approach or if subsystems are expected to stop calling
liveupdate_flb_get_incoming() after an FLB is finished.
Testing
-------
The patches at the end of this series provide comprehensive selftests
for the new code added by this series. The selftests have been validated
in both a VM environment using a virtio-net PCIe device, and in a
baremetal environment on an Intel EMR server with an Intel DSA device.
Here is an example of how to run the new selftests:
vfio_pci_liveupdate_uapi_test:
$ tools/testing/selftests/vfio/scripts/setup.sh 0000:00:04.0
$ tools/testing/selftests/vfio/vfio_pci_liveupdate_uapi_test 0000:00:04.0
$ tools/testing/selftests/vfio/scripts/cleanup.sh
vfio_pci_liveupdate_kexec_test:
$ tools/testing/selftests/vfio/scripts/setup.sh 0000:00:04.0
$ tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test --stage 1 0000:00:04.0
$ kexec [...] # NOTE: distro-dependent
$ tools/testing/selftests/vfio/scripts/setup.sh 0000:00:04.0
$ tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test --stage 2 0000:00:04.0
$ tools/testing/selftests/vfio/scripts/cleanup.sh
Dependencies
------------
This series was constructed on top of several in-flight series and on
top of mm-nonmm-unstable [2].
+-- This series
|
+-- [PATCH v2 00/18] vfio: selftests: Support for multi-device tests
| https://lore.kernel.org/kvm/20251112192232.442761-1-dmatlack@google.com/
|
+-- [PATCH v3 0/4] vfio: selftests: update DMA mapping tests to use queried IOVA ranges
| https://lore.kernel.org/kvm/20251111-iova-ranges-v3-0-7960244642c5@fb.com/
|
+-- [PATCH v8 0/2] Live Update: File-Lifecycle-Bound (FLB) State
| https://lore.kernel.org/linux-mm/20251125225006.3722394-1-pasha.tatashin@so…
|
+-- [PATCH v8 00/18] Live Update Orchestrator
| https://lore.kernel.org/linux-mm/20251125165850.3389713-1-pasha.tatashin@so…
|
To simplify checking out the code, this series can be found on GitHub:
https://github.com/dmatlack/linux/tree/liveupdate/vfio/cdev/v1
Changelog
---------
v1:
- Rebase series on top of LUOv8 and VFIO selftests improvements
- Drop commits to preserve config space fields across Live Update.
These changes require changes to the PCI layer. For exmaple,
preserving rbars could lead to an inconsistent device state until
device BARs addresses are preserved across Live Update.
- Drop commits to preserve Bus Master Enable on the device. There's no
reason to preserve this until iommufd preservation is fully working.
Furthermore, preserving Bus Master Enable could lead to memory
corruption when the device if the device is bound to the default
identity-map domain after Live Update.
- Drop commits to preserve saved PCI state. This work is not needed
until we are ready to preserve the device's config space, and
requires more thought to make the PCI state data layout ABI-friendly.
- Add support to skip auto-probing devices that are preserved by VFIO
to avoid them getting bound to a different driver by the next kernel.
- Restrict device preservation further (no VFs, no intel-graphics).
- Various refactoring and small edits to improve readability and
eliminate code duplication.
rfc: https://lore.kernel.org/kvm/20251018000713.677779-1-vipinsh@google.com/
Cc: Saeed Mahameed <saeedm(a)nvidia.com>
Cc: Adithya Jayachandran <ajayachandra(a)nvidia.com>
Cc: Jason Gunthorpe <jgg(a)nvidia.com>
Cc: Parav Pandit <parav(a)nvidia.com>
Cc: Leon Romanovsky <leonro(a)nvidia.com>
Cc: William Tu <witu(a)nvidia.com>
Cc: Jacob Pan <jacob.pan(a)linux.microsoft.com>
Cc: Lukas Wunner <lukas(a)wunner.de>
Cc: Pasha Tatashin <pasha.tatashin(a)soleen.com>
Cc: Mike Rapoport <rppt(a)kernel.org>
Cc: Pratyush Yadav <pratyush(a)kernel.org>
Cc: Samiullah Khawaja <skhawaja(a)google.com>
Cc: Chris Li <chrisl(a)kernel.org>
Cc: Josh Hilke <jrhilke(a)google.com>
Cc: David Rientjes <rientjes(a)google.com>
[1] https://lore.kernel.org/linux-iommu/20250928190624.3735830-1-skhawaja@googl…
[2] https://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm.git/log/?h=mm-nonmm…
David Matlack (12):
liveupdate: luo_flb: Prevent retrieve() after finish()
PCI: Add API to track PCI devices preserved across Live Update
PCI: Require driver_override for incoming Live Update preserved
devices
vfio/pci: Notify PCI subsystem about devices preserved across Live
Update
vfio: Enforce preserved devices are retrieved via
LIVEUPDATE_SESSION_RETRIEVE_FD
vfio/pci: Store Live Update state in struct vfio_pci_core_device
vfio: selftests: Add Makefile support for TEST_GEN_PROGS_EXTENDED
vfio: selftests: Add vfio_pci_liveupdate_uapi_test
vfio: selftests: Expose iommu_modes to tests
vfio: selftests: Expose low-level helper routines for setting up
struct vfio_pci_device
vfio: selftests: Verify that opening VFIO device fails during Live
Update
vfio: selftests: Add continuous DMA to vfio_pci_liveupdate_kexec_test
Vipin Sharma (9):
vfio/pci: Register a file handler with Live Update Orchestrator
vfio/pci: Preserve vfio-pci device files across Live Update
vfio/pci: Retrieve preserved device files after Live Update
vfio/pci: Skip reset of preserved device after Live Update
selftests/liveupdate: Move luo_test_utils.* into a reusable library
selftests/liveupdate: Add helpers to preserve/retrieve FDs
vfio: selftests: Build liveupdate library in VFIO selftests
vfio: selftests: Initialize vfio_pci_device using a VFIO cdev FD
vfio: selftests: Add vfio_pci_liveupdate_kexec_test
MAINTAINERS | 1 +
drivers/pci/Makefile | 1 +
drivers/pci/liveupdate.c | 248 ++++++++++++++++
drivers/pci/pci-driver.c | 12 +-
drivers/vfio/device_cdev.c | 25 +-
drivers/vfio/group.c | 9 +
drivers/vfio/pci/Makefile | 1 +
drivers/vfio/pci/vfio_pci.c | 11 +-
drivers/vfio/pci/vfio_pci_core.c | 23 +-
drivers/vfio/pci/vfio_pci_liveupdate.c | 278 ++++++++++++++++++
drivers/vfio/pci/vfio_pci_priv.h | 16 +
drivers/vfio/vfio.h | 13 -
drivers/vfio/vfio_main.c | 22 +-
include/linux/kho/abi/pci.h | 53 ++++
include/linux/kho/abi/vfio_pci.h | 45 +++
include/linux/liveupdate.h | 3 +
include/linux/pci.h | 38 +++
include/linux/vfio.h | 51 ++++
include/linux/vfio_pci_core.h | 7 +
kernel/liveupdate/luo_flb.c | 4 +
tools/testing/selftests/liveupdate/.gitignore | 1 +
tools/testing/selftests/liveupdate/Makefile | 14 +-
.../include/libliveupdate.h} | 11 +-
.../selftests/liveupdate/lib/libliveupdate.mk | 20 ++
.../{luo_test_utils.c => lib/liveupdate.c} | 43 ++-
.../selftests/liveupdate/luo_kexec_simple.c | 2 +-
.../selftests/liveupdate/luo_multi_session.c | 2 +-
tools/testing/selftests/vfio/Makefile | 23 +-
.../vfio/lib/include/libvfio/iommu.h | 2 +
.../lib/include/libvfio/vfio_pci_device.h | 8 +
tools/testing/selftests/vfio/lib/iommu.c | 4 +-
.../selftests/vfio/lib/vfio_pci_device.c | 60 +++-
.../vfio/vfio_pci_liveupdate_kexec_test.c | 255 ++++++++++++++++
.../vfio/vfio_pci_liveupdate_uapi_test.c | 93 ++++++
34 files changed, 1313 insertions(+), 86 deletions(-)
create mode 100644 drivers/pci/liveupdate.c
create mode 100644 drivers/vfio/pci/vfio_pci_liveupdate.c
create mode 100644 include/linux/kho/abi/pci.h
create mode 100644 include/linux/kho/abi/vfio_pci.h
rename tools/testing/selftests/liveupdate/{luo_test_utils.h => lib/include/libliveupdate.h} (80%)
create mode 100644 tools/testing/selftests/liveupdate/lib/libliveupdate.mk
rename tools/testing/selftests/liveupdate/{luo_test_utils.c => lib/liveupdate.c} (89%)
create mode 100644 tools/testing/selftests/vfio/vfio_pci_liveupdate_kexec_test.c
create mode 100644 tools/testing/selftests/vfio/vfio_pci_liveupdate_uapi_test.c
--
2.52.0.487.g5c8c507ade-goog
The top level kselftest Makefile supports an option FORCE_TARGETS which
causes any failures during the build to be propagated to the exit status
of the top level make, useful during build testing. Currently the recursion
done by the arm64 selftests ignores this option, meaning arm64 failures are
not reported via this mechanism. Add the logic to implement FORCE_TARGETS
so that it works for arm64.
Signed-off-by: Mark Brown <broonie(a)kernel.org>
---
tools/testing/selftests/arm64/Makefile | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/tools/testing/selftests/arm64/Makefile b/tools/testing/selftests/arm64/Makefile
index c4c72ee2ef55..e456f3b62fa1 100644
--- a/tools/testing/selftests/arm64/Makefile
+++ b/tools/testing/selftests/arm64/Makefile
@@ -30,13 +30,15 @@ all:
@for DIR in $(ARM64_SUBTARGETS); do \
BUILD_TARGET=$(OUTPUT)/$$DIR; \
mkdir -p $$BUILD_TARGET; \
- make OUTPUT=$$BUILD_TARGET -C $$DIR $@; \
+ make OUTPUT=$$BUILD_TARGET -C $$DIR $@ \
+ $(if $(FORCE_TARGETS),|| exit); \
done
install: all
@for DIR in $(ARM64_SUBTARGETS); do \
BUILD_TARGET=$(OUTPUT)/$$DIR; \
- make OUTPUT=$$BUILD_TARGET -C $$DIR $@; \
+ make OUTPUT=$$BUILD_TARGET -C $$DIR $@ \
+ $(if $(FORCE_TARGETS),|| exit); \
done
run_tests: all
---
base-commit: 8f0b4cce4481fb22653697cced8d0d04027cb1e8
change-id: 20251219-kselftest-arm64-force-targets-2ba8cebd1748
Best regards,
--
Mark Brown <broonie(a)kernel.org>
For consistency with __vdso_clock_gettime64() there should also be a
64-bit variant of clock_getres(). This will allow the extension of
CONFIG_COMPAT_32BIT_TIME to the vDSO and finally the removal of 32-bit
time types from the kernel and UAPI.
Signed-off-by: Thomas Weißschuh <thomas.weissschuh(a)linutronix.de>
---
Thomas Weißschuh (9):
vdso: Add prototype for __vdso_clock_getres_time64()
selftests: vDSO: vdso_config: Add configurations for clock_getres_time64()
selftests: vDSO: vdso_test_abi: Use UAPI system call numbers
selftests: vDSO: vdso_test_abi: Add test for clock_getres_time64()
x86/vdso: Provide clock_getres_time64() for x86-32
ARM: VDSO: also patch out __vdso_clock_getres() if unavailable
ARM: VDSO: provide clock_getres_time64()
arm64: vdso32: Provide clock_getres_time64()
MIPS: vdso: Provide getres_time64() for 32-bit ABIs
arch/arm/kernel/vdso.c | 2 +
arch/arm/vdso/vdso.lds.S | 1 +
arch/arm/vdso/vgettimeofday.c | 6 +++
arch/arm64/kernel/vdso32/vdso.lds.S | 1 +
arch/arm64/kernel/vdso32/vgettimeofday.c | 6 +++
arch/mips/vdso/vdso.lds.S | 1 +
arch/mips/vdso/vgettimeofday.c | 6 +++
arch/x86/entry/vdso/vclock_gettime.c | 8 ++++
arch/x86/entry/vdso/vdso32/vdso32.lds.S | 1 +
include/vdso/gettime.h | 1 +
tools/testing/selftests/vDSO/vdso_config.h | 4 +-
tools/testing/selftests/vDSO/vdso_test_abi.c | 55 +++++++++++++++++++++++++++-
12 files changed, 89 insertions(+), 3 deletions(-)
---
base-commit: 15a11f3ffb629cbbf6efd272239c04a9eb3180e2
change-id: 20251120-vdso-compat-time32-f4684ff250ba
Best regards,
--
Thomas Weißschuh <thomas.weissschuh(a)linutronix.de>
A few improvements/fixes for the mm kselftests:
- Patch 1-2 extend support for more build configurations: out-of-tree
$KDIR, cross-compilation, etc.
- Patch 3-4 fix issues in the pagemap_ioctl tests, most importantly that
it does not report failures: ./run_kselftests.sh would report OK
even if some pagemap_ioctl tests fail. That's probably why the issue
in patch 3 went unnoticed.
---
Cc: Andrew Morton <akpm(a)linux-foundation.org>
Cc: David Hildenbrand <david(a)kernel.org>
Cc: Lorenzo Stoakes <lorenzo.stoakes(a)oracle.com>
Cc: Mark Brown <broonie(a)kernel.org>
Cc: Ryan Roberts <ryan.roberts(a)arm.com>
Cc: Shuah Khan <shuah(a)kernel.org>
---
Kevin Brodsky (4):
selftests/mm: remove flaky header check
selftests/mm: pass down full CC and CFLAGS to check_config.sh
selftests/mm: fix faulting-in code in pagemap_ioctl test
selftests/mm: fix exit code in pagemap_ioctl
tools/testing/selftests/mm/Makefile | 6 +-----
tools/testing/selftests/mm/check_config.sh | 3 +--
tools/testing/selftests/mm/pagemap_ioctl.c | 12 ++++++------
3 files changed, 8 insertions(+), 13 deletions(-)
base-commit: 8f0b4cce4481fb22653697cced8d0d04027cb1e8
--
2.51.2
While debugging issues related to aarch64 only systems I ran into
speedbumps due to the lack of detail in the results reported when the
guest register read and reset value preservation tests were run, they
generated an immediately fatal assert without indicating which register
was being tested. Update these tests to report a result per register,
making it much easier to see what the problem being reported is.
A similar, though less severe, issue exists with the validation of the
individual bitfields in registers due to the use of immediately fatal
asserts. Update those asserts to be standard kselftest reports.
Finally we have a fix for spurious errors on some NV systems.
Signed-off-by: Mark Brown <broonie(a)kernel.org>
---
Changes in v3:
- Rebase onto v6.19-rc1.
- Link to v2: https://patch.msgid.link/20251114-kvm-arm64-set-id-regs-aarch64-v2-0-672f21…
Changes in v2:
- Add a fix for spurious failures with 64 bit only guests.
- Link to v1: https://patch.msgid.link/20251030-kvm-arm64-set-id-regs-aarch64-v1-0-96fe0d…
---
Mark Brown (4):
KVM: selftests: arm64: Report set_id_reg reads of test registers as tests
KVM: selftests: arm64: Report register reset tests individually
KVM: selftests: arm64: Make set_id_regs bitfield validatity checks non-fatal
KVM: selftests: arm64: Skip all 32 bit IDs when set_id_regs is aarch64 only
tools/testing/selftests/kvm/arm64/set_id_regs.c | 150 ++++++++++++++++++------
1 file changed, 111 insertions(+), 39 deletions(-)
---
base-commit: 8f0b4cce4481fb22653697cced8d0d04027cb1e8
change-id: 20251028-kvm-arm64-set-id-regs-aarch64-ebb77969401c
Best regards,
--
Mark Brown <broonie(a)kernel.org>
This unintended LRU eviction issue was observed while developing the
selftest for
"[PATCH bpf-next v10 0/8] bpf: Introduce BPF_F_CPU and BPF_F_ALL_CPUS flags for percpu maps" [1].
When updating an existing element in lru_hash or lru_percpu_hash maps,
the current implementation calls prealloc_lru_pop() to get a new node
before checking if the key already exists. If the map is full, this
triggers LRU eviction and removes an existing element, even though the
update operation only needs to modify the value in-place.
In the selftest of the aforementioned patch, this was to be worked around by
reserving an extra entry to
void triggering eviction in __htab_lru_percpu_map_update_elem(). However, the
underlying issue remains problematic because:
1. Users may unexpectedly lose entries when updating existing keys in a
full map.
2. The eviction overhead is unnecessary for existing key updates.
This patchset fixes the issue by first checking if the key exists before
allocating a new node. If the key is found, update the value using the extra
LRU node without triggering any eviction. Only proceed with node allocation
if the key does not exist.
Links:
[1] https://lore.kernel.org/bpf/20251117162033.6296-1-leon.hwang@linux.dev/
Changes:
v1 -> v2:
* Tidy hash handling in LRU code.
* Factor out bpf_lru_node_reset_state helper.
* Factor out bpf_lru_move_next_inactive_rotation helper.
* Update element using preallocated extra elements in order to avoid
breaking the update atomicity (per Alexei).
* Check values on other CPUs in tests (per bot).
* v1: https://lore.kernel.org/bpf/20251202153032.10118-1-leon.hwang@linux.dev/
Leon Hwang (5):
bpf: lru: Tidy hash handling in LRU code
bpf: lru: Factor out bpf_lru_node_reset_state helper
bpf: lru: Factor out bpf_lru_move_next_inactive_rotation helper
bpf: lru: Fix unintended eviction when updating lru hash maps
selftests/bpf: Add tests to verify no unintended eviction when
updating lru_[percpu_,]hash maps
kernel/bpf/bpf_lru_list.c | 228 ++++++++++++++----
kernel/bpf/bpf_lru_list.h | 10 +-
kernel/bpf/hashtab.c | 90 ++++++-
.../selftests/bpf/prog_tests/htab_update.c | 129 ++++++++++
4 files changed, 402 insertions(+), 55 deletions(-)
--
2.52.0
From: Abhishek Dubey <adubey(a)linux.ibm.com>
This patch series enables support for two BPF JIT features
on powerpc64. The first three patches target support for
tail calls with subprogram combinations. The third patch
includes an optimization in which NVRs are accommodated in
the stack save area of the trampoline frame. Implementation
details are provided in the commit messages.
The last three patches add support for BPF exceptions. An
architecture-specific stack walker is implemented to assist
with stack walk during exceptions. BPF selftest results and
implementation details are presented in the corresponding
commits.
Abhishek Dubey (6):
powerpc64/bpf: Support tailcalls with subprogs
powerpc64/bpf: Tailcall handling with trampolines
powerpc/bpf: use BPF_PPC_STACK_SAVE to spill trampoline NVRs
powerpc64/bpf: Add arch_bpf_stack_walk() for BPF JIT
powerpc64/bpf: Support exceptions
powerpc64/bpf: Additional NVR handling for bpf_throw
arch/powerpc/net/bpf_jit.h | 16 ++-
arch/powerpc/net/bpf_jit_comp.c | 90 +++++++++----
arch/powerpc/net/bpf_jit_comp64.c | 214 ++++++++++++++++++++++++------
3 files changed, 256 insertions(+), 64 deletions(-)
--
2.48.1
From: Andrea Righi <andrea.righi(a)canonical.com>
[ Upstream commit e7e4785fa30f9b5d1b60ed2d8e221891325dfc5f ]
It looks like test_vxlan_under_vrf.sh is always failing to verify the
connectivity test during the ping between the two simulated VMs.
This is due to the fact that veth-hv in each VM should have a distinct
MAC address.
Fix by setting a unique MAC address on each simulated VM interface.
Without this fix:
$ sudo ./tools/testing/selftests/net/test_vxlan_under_vrf.sh
Checking HV connectivity [ OK ]
Check VM connectivity through VXLAN (underlay in the default VRF) [FAIL]
With this fix applied:
$ sudo ./tools/testing/selftests/net/test_vxlan_under_vrf.sh
Checking HV connectivity [ OK ]
Check VM connectivity through VXLAN (underlay in the default VRF) [ OK ]
Check VM connectivity through VXLAN (underlay in a VRF) [FAIL]
NOTE: the connectivity test with the underlay VRF is still failing; it
seems that ARP requests are blocked at the simulated hypervisor level,
probably due to some missing ARP forwarding rules. This requires more
investigation (in the meantime we may consider to set that test as
expected failure - XFAIL).
Signed-off-by: Andrea Righi <andrea.righi(a)canonical.com>
Signed-off-by: David S. Miller <davem(a)davemloft.net>
Signed-off-by: Cao Jiaqiang <caojiaqiang(a)huawei.com>
---
tools/testing/selftests/net/test_vxlan_under_vrf.sh | 2 ++
1 file changed, 2 insertions(+)
diff --git a/tools/testing/selftests/net/test_vxlan_under_vrf.sh b/tools/testing/selftests/net/test_vxlan_under_vrf.sh
index 6fadc8e2f116..1fd1250ebc66 100755
--- a/tools/testing/selftests/net/test_vxlan_under_vrf.sh
+++ b/tools/testing/selftests/net/test_vxlan_under_vrf.sh
@@ -101,6 +101,8 @@ setup-vm() {
ip -netns hv-$id link set veth-tap master br0
ip -netns hv-$id link set veth-tap up
+ ip link set veth-hv address 02:1d:8d:dd:0c:6$id
+
ip link set veth-hv netns vm-$id
ip -netns vm-$id addr add 10.0.0.$id/24 dev veth-hv
ip -netns vm-$id link set veth-hv up
--
2.22.0