On Fri, Jun 20, 2025 at 11:19:58PM +0000, David Matlack wrote:
This series introduces VFIO selftests, located in tools/testing/selftests/vfio/.
Sorry for coming late to the party. Only recently got some cycles to go through this. This seems very similar to what we are trying to do with iommutests [3].
VFIO selftests aim to enable kernel developers to write and run tests that take the form of userspace programs that interact with VFIO and IOMMUFD uAPIs. VFIO selftests can be used to write functional tests for new features, regression tests for bugs, and performance tests for optimizations.
Have you considered implementing something outside the kernel sources? Something similar to fstests [1] or blktests [2]?
I ask because I have always seen it as a suit that tests (regression and performance) the Linux kernel as well as the hardware. By hardware I mean IOMMU devices, peripherals as well as their QEMU implementations. Since the scope is quite big, seemed (to me) like a good idea to keep it out of the kernel sources.
Can you speak to the pros/cons of having it in selftests? (sorry if this was already answered)
These tests are designed to interact with real PCI devices, i.e. they do not rely on mocking out or faking any behavior in the kernel. This allows the tests to exercise not only VFIO but also IOMMUFD, the IOMMU driver, interrupt remapping, IRQ handling, etc.
And depending on how you execute them, you might also be exercising the QEMU emulation paths. You could even test different HW firmwares if you wanted to :)
For more background on the motivation and design of this series, please see the RFC:
https://lore.kernel.org/kvm/20250523233018.1702151-1-dmatlack@google.com/
This series can also be found on GitHub:
...
Instructions
Running VFIO selftests requires at a PCI device bound to vfio-pci for the tests to use. The address of this device is passed to the test as a segment:bus:device.function string, which must match the path to the device in /sys/bus/pci/devices/ (e.g. 0000:00:04.0).
Would you be able to autodetect the devices that are vfio-testable? I saw this question in the thread, but did not see the answer (sorry if I missed it).
Once you have chosen a device, there is a helper script provided to unbind the device from its current driver, bind it to vfio-pci, export the environment variable $VFIO_SELFTESTS_BDF, and launch a shell:
If I'm reading the series correctly there is a fair amount of helper code needed: device (un)binding, checking capabilities, setting up DMAable memory.... Have you considered a library like libvfn [4] (is there any other?) to take care of all this?
One of the good things about having all this in a library is that it can be used in other contexts besides testing.
$ tools/testing/selftests/vfio/run.sh -d 0000:00:04.0 -s
The -d option tells the script which device to use and the -s option tells the script to launch a shell.
Additionally, the VFIO selftest vfio_dma_mapping_test has test cases that rely on HugeTLB pages being available, otherwise they are skipped. To enable those tests make sure at least 1 2MB and 1 1GB HugeTLB pages are available.
$ echo 1 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages $ echo 1 > /sys/kernel/mm/hugepages/hugepages-1048576kB/nr_hugepages
Should this be automatic? You can just modify nr_hugepages everytime you execute a test that requires it.
To run all VFIO selftests using make:
$ make -C tools/testing/selftests/vfio run_tests
This ties back to having the test suit outside the Linux Kernel sources. I might not always want/have a Linux Kernel selftests. Like if I would want to test the Intel/AMD IOMMU implementation in QEMU.
To run individual tests:
$ tools/testing/selftests/vfio/vfio_dma_mapping_test $ tools/testing/selftests/vfio/vfio_dma_mapping_test -v iommufd_anonymous_hugetlb_2mb $ tools/testing/selftests/vfio/vfio_dma_mapping_test -r vfio_dma_mapping_test.iommufd_anonymous_hugetlb_2mb.dma_map_unmap
The environment variable $VFIO_SELFTESTS_BDF can be overridden for a specific test by passing in the BDF on the command line as the last positional argument.
$ tools/testing/selftests/vfio/vfio_dma_mapping_test 0000:00:04.0 $ tools/testing/selftests/vfio/vfio_dma_mapping_test -v iommufd_anonymous_hugetlb_2mb 0000:00:04.0 $ tools/testing/selftests/vfio/vfio_dma_mapping_test -r vfio_dma_mapping_test.iommufd_anonymous_hugetlb_2mb.dma_map_unmap 0000:00:04.0
When you are done, free the HugeTLB pages and exit the shell started by run.sh. Exiting the shell will cause the device to be unbound from vfio-pci and bound back to its original driver.
$ echo 0 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages $ echo 0 > /sys/kernel/mm/hugepages/hugepages-1048576kB/nr_hugepages $ exit
As before: Can this be done automatically?
It's also possible to use run.sh to run just a single test hermetically, rather than dropping into a shell:
$ tools/testing/selftests/vfio/run.sh -d 0000:00:04.0 -- tools/testing/selftests/vfio/vfio_dma_mapping_test -v iommufd_anonymous
Tests
There are 5 tests in this series, mostly to demonstrate as a proof-of-concept:
- tools/testing/selftests/vfio/vfio_pci_device_test.c
- tools/testing/selftests/vfio/vfio_pci_driver_test.c
- tools/testing/selftests/vfio/vfio_iommufd_setup_test.c
- tools/testing/selftests/vfio/vfio_dma_mapping_test.c
- tools/testing/selftests/kvm/vfio_pci_device_irq_test.c
Future Areas of Development
Library:
- Driver support for devices that can be used on AMD, ARM, and other platforms (e.g. mlx5).
- Driver support for a device available in QEMU VMs (e.g. pcie-ats-testdev [1])
- Support for tests that use multiple devices.
- Support for IOMMU groups with multiple devices.
- Support for multiple devices sharing the same container/iommufd.
- Sharing TEST_ASSERT() macros and other common code between KVM and VFIO selftests.
Same as before: How about a lib?
Tests:
- DMA mapping performance tests for BARs/HugeTLB/etc.
- Porting tests from https://github.com/awilliam/tests/commits/for-clg/ to selftests.
- Live Update selftests.
- Porting Sean's KVM selftest for posted interrupts to use the VFIO selftests library [2]
...
base-commit: e271ed52b344ac02d4581286961d0c40acc54c03 prerequisite-patch-id: c1decca4653262d3d2451e6fd4422ebff9c0b589 -- 2.50.0.rc2.701.gf1e915cc24-goog
Best
[1] https://github.com/kdave/xfstests [2] https://github.com/linux-blktests/blktests [3] https://github.com/SamsungDS/iommutests [4] https://github.com/SamsungDS/libvfn