This patch series makes two changes to how KUnit test suites are stored
and executed:
- The .kunit_test_suites section is now used for tests in modules (in
lieu of a module_init funciton), as well as for built-in tests. The
module loader will now trigger test execution. This frees up the
module_init function for other uses.
- Instead of storing an array of arrays of suites, have the
kunit_test_suite() and kunit_test_suites() macros append to one global
(or per-module) list of test suites. This removes a needless layer of
indirection.
The upshot of this is that it should now be possible to use the
kunit_test_suite() and kunit_test_suites() macros to register test
suites even from within modules which otherwise had module_init
functions. This was proving to be quite a common issue, resulting in
several modules calling into KUnit's private suite execution functions
to run their tests (often introducing incompatibilities with the KUnit
tooling).
This series also fixes the thunderbolt, nitro_enclaves, and
sdhci-of-aspeed tests to use kunit_test_suite() now that it works.
Huge thanks to Jeremy Kerr, who designed and implemented the module
loader changes, and to Daniel Latypov for pushing the simplification of
the nested arrays in .kunit_test_suites.
I've tested this series both with builtin tests, and with modules on
x86_64, but there's always the possibility that there's something subtle
and nasty on another architecture, so please test!
Cheers,
-- David
Daniel Latypov (1):
kunit: flatten kunit_suite*** to kunit_suite** in .kunit_test_suites
David Gow (3):
thunderbolt: test: Use kunit_test_suite() macro
nitro_enclaves: test: Use kunit_test_suite() macro
mmc: sdhci-of-aspeed: test: Use kunit_test_suite() macro
Jeremy Kerr (1):
kunit: unify module and builtin suite definitions
drivers/mmc/host/Kconfig | 5 +-
drivers/mmc/host/sdhci-of-aspeed-test.c | 8 +-
drivers/mmc/host/sdhci-of-aspeed.c | 27 ----
drivers/thunderbolt/Kconfig | 5 +-
drivers/thunderbolt/domain.c | 3 -
drivers/thunderbolt/tb.h | 8 -
drivers/thunderbolt/test.c | 12 +-
drivers/virt/nitro_enclaves/Kconfig | 5 +-
drivers/virt/nitro_enclaves/ne_misc_dev.c | 27 ----
.../virt/nitro_enclaves/ne_misc_dev_test.c | 5 +-
include/kunit/test.h | 60 ++------
include/linux/module.h | 5 +
kernel/module/main.c | 6 +
lib/kunit/executor.c | 117 ++++-----------
lib/kunit/executor_test.c | 139 +++++-------------
lib/kunit/test.c | 54 ++++++-
16 files changed, 152 insertions(+), 334 deletions(-)
--
2.36.1.476.g0c4daa206d-goog
KUnit unifies the test structure and provides helper tools that simplify
the development of tests. The basic use case allows running tests as regular
processes, which makes it easier to run unit tests on a development machine
and to integrate the tests into a CI system.
That said, the conversion of selftests for DRM to KUnit tests is beneficial
as it unifies the testing API by using the KUnit API.
KUnit is beneficial for developers as it eases the process to run unit tests.
It is possible to run the tests by using the kunit-tool on userspace with the
following command:
./tools/testing/kunit/kunit.py run --kunitconfig=drivers/gpu/drm/tests --arch=x86_64
For CI system, it is possible to execute during the build. But, we also think
about IGT: we are developing a patch to introduce KUnit to IGT.
These patches were developed during a KUnit hackathon [0] last October. Now,
we believe that both the IGT side and the Kernel side are in good shape for
submission.
If you are willing to check the output, here is the Pastebin with the output
and execution times [1].
[0] https://groups.google.com/g/kunit-dev/c/YqFR1q2uZvk/m/IbvItSfHBAAJ
[1] https://pastebin.com/FJjLPKsC
- Arthur Grillo, Isabella Basso, and Maíra Canal
Arthur Grillo (2):
drm: selftest: refactor drm_cmdline_parser
drm: selftest: convert drm_mm selftest to KUnit
Maíra Canal (8):
drm: selftest: convert drm_damage_helper selftest to KUnit
drm: selftest: convert drm_cmdline_parser selftest to KUnit
drm: selftest: convert drm_rect selftest to KUnit
drm: selftest: convert drm_format selftest to KUnit
drm: selftest: convert drm_plane_helper selftest to KUnit
drm: selftest: convert drm_dp_mst_helper selftest to KUnit
drm: selftest: convert drm_framebuffer selftest to KUnit
drm: selftest: convert drm_buddy selftest to KUnit
drivers/gpu/drm/Kconfig | 20 +-
drivers/gpu/drm/Makefile | 2 +-
drivers/gpu/drm/selftests/Makefile | 8 -
.../gpu/drm/selftests/drm_buddy_selftests.h | 15 -
.../gpu/drm/selftests/drm_cmdline_selftests.h | 68 -
drivers/gpu/drm/selftests/drm_mm_selftests.h | 28 -
.../gpu/drm/selftests/drm_modeset_selftests.h | 40 -
drivers/gpu/drm/selftests/drm_selftest.c | 109 --
drivers/gpu/drm/selftests/drm_selftest.h | 41 -
drivers/gpu/drm/selftests/test-drm_buddy.c | 994 --------------
.../drm/selftests/test-drm_cmdline_parser.c | 1141 -----------------
.../drm/selftests/test-drm_damage_helper.c | 667 ----------
drivers/gpu/drm/selftests/test-drm_format.c | 280 ----
.../drm/selftests/test-drm_modeset_common.c | 32 -
.../drm/selftests/test-drm_modeset_common.h | 52 -
drivers/gpu/drm/tests/.kunitconfig | 3 +
drivers/gpu/drm/tests/Kconfig | 130 ++
drivers/gpu/drm/tests/Makefile | 10 +
drivers/gpu/drm/tests/test-drm_buddy.c | 748 +++++++++++
.../gpu/drm/tests/test-drm_cmdline_parser.c | 799 ++++++++++++
.../gpu/drm/tests/test-drm_damage_helper.c | 633 +++++++++
.../test-drm_dp_mst_helper.c | 82 +-
drivers/gpu/drm/tests/test-drm_format.c | 284 ++++
.../test-drm_framebuffer.c | 25 +-
.../drm/{selftests => tests}/test-drm_mm.c | 1135 +++++++---------
.../test-drm_plane_helper.c | 101 +-
.../drm/{selftests => tests}/test-drm_rect.c | 124 +-
27 files changed, 3240 insertions(+), 4331 deletions(-)
delete mode 100644 drivers/gpu/drm/selftests/Makefile
delete mode 100644 drivers/gpu/drm/selftests/drm_buddy_selftests.h
delete mode 100644 drivers/gpu/drm/selftests/drm_cmdline_selftests.h
delete mode 100644 drivers/gpu/drm/selftests/drm_mm_selftests.h
delete mode 100644 drivers/gpu/drm/selftests/drm_modeset_selftests.h
delete mode 100644 drivers/gpu/drm/selftests/drm_selftest.c
delete mode 100644 drivers/gpu/drm/selftests/drm_selftest.h
delete mode 100644 drivers/gpu/drm/selftests/test-drm_buddy.c
delete mode 100644 drivers/gpu/drm/selftests/test-drm_cmdline_parser.c
delete mode 100644 drivers/gpu/drm/selftests/test-drm_damage_helper.c
delete mode 100644 drivers/gpu/drm/selftests/test-drm_format.c
delete mode 100644 drivers/gpu/drm/selftests/test-drm_modeset_common.c
delete mode 100644 drivers/gpu/drm/selftests/test-drm_modeset_common.h
create mode 100644 drivers/gpu/drm/tests/.kunitconfig
create mode 100644 drivers/gpu/drm/tests/Kconfig
create mode 100644 drivers/gpu/drm/tests/Makefile
create mode 100644 drivers/gpu/drm/tests/test-drm_buddy.c
create mode 100644 drivers/gpu/drm/tests/test-drm_cmdline_parser.c
create mode 100644 drivers/gpu/drm/tests/test-drm_damage_helper.c
rename drivers/gpu/drm/{selftests => tests}/test-drm_dp_mst_helper.c (73%)
create mode 100644 drivers/gpu/drm/tests/test-drm_format.c
rename drivers/gpu/drm/{selftests => tests}/test-drm_framebuffer.c (96%)
rename drivers/gpu/drm/{selftests => tests}/test-drm_mm.c (58%)
rename drivers/gpu/drm/{selftests => tests}/test-drm_plane_helper.c (62%)
rename drivers/gpu/drm/{selftests => tests}/test-drm_rect.c (53%)
--
2.36.1
Commit 8ff978b8b222 ("ipv4/raw: support binding to nonlocal addresses")
introduced support for binding to nonlocal addresses, as well as some
basic test coverage for some of the cases.
Commit b4a028c4d031 ("ipv4: ping: fix bind address validity check")
fixes a regression which incorrectly removed some checks for bind
address validation. In addition, it introduces regression tests for
those specific checks. However, those regression tests are defective, in
that they perform the tests using an incorrect combination of bind
flags. As a result, those tests fail when they should succeed.
This commit introduces additional regression tests for nonlocal binding
and fixes the defective regression tests.
PLEASE NOTE THAT THIS PATCH SHOULD NOT BE APPLIED AS-IS. The ICMP
broadcast and multicast regression tests succeed, but they do so while
returning the wrong error status. In particular, it isn't the bind that
fails, but the socket creation. This is /not/ correct, and it must be
investigated to have proper regression testing. Other instances where
this happens are: 1) if the broadcast/multicast addresses are replace
with an allowed (e.g. local) address (bind should work, but socket is
never created in the first place); 2) the commented out tests (nonlocal
bind should work but ditto.) Additionally, please note that when the
test cases are manually (i.e. without the network namespace setup from
fcnal-test.sh) ran, the expected/correct outcome is observed. The reason
I'm submitting this patch for comments, is that I'm failing to
understand where the issue lies. (Disclamer: might be something
stupid/trivial that I'm plainly missing due to tunnel vision.)
Signed-off-by: Riccardo Paolo Bestetti <pbl(a)bestov.io>
---
tools/testing/selftests/net/fcnal-test.sh | 36 +++++++++++++++++------
1 file changed, 27 insertions(+), 9 deletions(-)
diff --git a/tools/testing/selftests/net/fcnal-test.sh b/tools/testing/selftests/net/fcnal-test.sh
index 75223b63e3c8..778288539879 100755
--- a/tools/testing/selftests/net/fcnal-test.sh
+++ b/tools/testing/selftests/net/fcnal-test.sh
@@ -1800,24 +1800,33 @@ ipv4_addr_bind_novrf()
done
#
- # raw socket with nonlocal bind
+ # tests for nonlocal bind
#
a=${NL_IP}
log_start
- run_cmd nettest -s -R -P icmp -f -l ${a} -I ${NSA_DEV} -b
- log_test_addr ${a} $? 0 "Raw socket bind to nonlocal address after device bind"
+ run_cmd nettest -s -R -f -l ${a} -b
+ log_test_addr ${a} $? 0 "Raw socket bind to nonlocal address"
+
+ log_start
+ run_cmd nettest -s -f -l ${a} -b
+ log_test_addr ${a} $? 0 "TCP socket bind to nonlocal address"
+
+ # currently fails with ACCES
+ #log_start
+ #run_cmd nettest -s -D -P icmp -f -l ${a} -b
+ #log_test_addr ${a} $? 0 "ICMP socket bind to nonlocal address"
#
# check that ICMP sockets cannot bind to broadcast and multicast addresses
#
a=${BCAST_IP}
log_start
- run_cmd nettest -s -R -P icmp -l ${a} -b
+ run_cmd nettest -s -D -P icmp -l ${a} -b
log_test_addr ${a} $? 1 "ICMP socket bind to broadcast address"
a=${MCAST_IP}
log_start
- run_cmd nettest -s -R -P icmp -f -l ${a} -b
+ run_cmd nettest -s -D -P icmp -l ${a} -b
log_test_addr ${a} $? 1 "ICMP socket bind to multicast address"
#
@@ -1870,24 +1879,33 @@ ipv4_addr_bind_vrf()
log_test_addr ${a} $? 1 "Raw socket bind to out of scope address after VRF bind"
#
- # raw socket with nonlocal bind
+ # tests for nonlocal bind
#
a=${NL_IP}
log_start
- run_cmd nettest -s -R -P icmp -f -l ${a} -I ${VRF} -b
+ run_cmd nettest -s -R -f -l ${a} -I ${VRF} -b
log_test_addr ${a} $? 0 "Raw socket bind to nonlocal address after VRF bind"
+ log_start
+ run_cmd nettest -s -f -l ${a} -I ${VRF} -b
+ log_test_addr ${a} $? 0 "TCP socket bind to nonlocal address after VRF bind"
+
+ # currently fails with ACCES
+ #log_start
+ #run_cmd nettest -s -D -P icmp -f -l ${a} -I ${VRF} -b
+ #log_test_addr ${a} $? 0 "ICMP socket bind to nonlocal address after VRF bind"
+
#
# check that ICMP sockets cannot bind to broadcast and multicast addresses
#
a=${BCAST_IP}
log_start
- run_cmd nettest -s -R -P icmp -l ${a} -I ${VRF} -b
+ run_cmd nettest -s -D -P icmp -l ${a} -I ${VRF} -b
log_test_addr ${a} $? 1 "ICMP socket bind to broadcast address after VRF bind"
a=${MCAST_IP}
log_start
- run_cmd nettest -s -R -P icmp -f -l ${a} -I ${VRF} -b
+ run_cmd nettest -s -D -P icmp -l ${a} -I ${VRF} -b
log_test_addr ${a} $? 1 "ICMP socket bind to multicast address after VRF bind"
#
--
2.36.1
A couple of test updates are included:
* With this option [1,2], the kernel's altstack check becomes stringent.
The x86 sigaltstack test is ignorant about this. Adjust the test now.
This check was established [3] to ensure every AMX task's altstack is
sufficient (regardless of that option) [4].
* The AMX test wrongly fails on non-AMX machines. Fix the code to skip the
test instead.
The series is available in this repository:
git://github.com/intel/amx-linux.git selftest
[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arc…
[2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Doc…
[3] 3aac3ebea08f ("x86/signal: Implement sigaltstack size validation")
[4] 4b7ca609a33d ("x86/signal: Use fpu::__state_user_size for sigalt stack validation")
Chang S. Bae (2):
selftests/x86/signal: Adjust the test to the kernel's altstack check
selftests/x86/amx: Fix the test to avoid failure when AMX is
unavailable
tools/testing/selftests/x86/amx.c | 42 +++++++++++++++++------
tools/testing/selftests/x86/sigaltstack.c | 12 ++++++-
2 files changed, 42 insertions(+), 12 deletions(-)
base-commit: f443e374ae131c168a065ea1748feac6b2e76613
--
2.17.1
Hi there,
The first patch moves the current livepatch tests to selftests, allowing it
be better suited to contain more complex tests, like using userspace C code
to use the livepatched kernel code. As a bonus it allows to use
"gen_tar" to export the livepatch selftests, rebuild the modules by
running make in selftests/livepatch directory and simplifies the process
of creating and debugging new selftests.
It keeps the ability to execute the tests by running the shell scripts,
like "test-livepatch.sh", but beware that the kernel modules
might not be up-to-date.
The second patch includes a new test to exercise the functionality to livepatch
a heavy hammered function. The test uses getpid in this case.
I tested the changes by running the tests within the kernel source tree and running
from the gen_tar extracted directory.
Marcos Paulo de Souza (2):
livepatch: Move tests from lib/livepatch to selftests/livepatch
selftests: livepatch: Test livepatching a heavily called syscall
arch/s390/configs/debug_defconfig | 1 -
arch/s390/configs/defconfig | 1 -
lib/Kconfig.debug | 22 -------
lib/Makefile | 2 -
lib/livepatch/Makefile | 14 -----
tools/testing/selftests/livepatch/Makefile | 35 ++++++++++-
tools/testing/selftests/livepatch/README | 5 +-
tools/testing/selftests/livepatch/config | 1 -
.../testing/selftests/livepatch/functions.sh | 34 ++++-------
.../selftests/livepatch/test-callbacks.sh | 50 ++++++++--------
.../selftests/livepatch/test-ftrace.sh | 6 +-
.../selftests/livepatch/test-livepatch.sh | 10 ++--
.../selftests/livepatch/test-shadow-vars.sh | 2 +-
.../testing/selftests/livepatch/test-state.sh | 18 +++---
.../selftests/livepatch/test-syscall.sh | 46 ++++++++++++++
.../test_binaries/test_klp-call_getpid.c | 48 +++++++++++++++
.../selftests/livepatch/test_modules/Makefile | 25 ++++++++
.../test_modules}/test_klp_atomic_replace.c | 0
.../test_modules}/test_klp_callbacks_busy.c | 0
.../test_modules}/test_klp_callbacks_demo.c | 0
.../test_modules}/test_klp_callbacks_demo2.c | 0
.../test_modules}/test_klp_callbacks_mod.c | 0
.../test_modules}/test_klp_livepatch.c | 0
.../test_modules}/test_klp_shadow_vars.c | 0
.../livepatch/test_modules}/test_klp_state.c | 0
.../livepatch/test_modules}/test_klp_state2.c | 0
.../livepatch/test_modules}/test_klp_state3.c | 0
.../livepatch/test_modules/test_klp_syscall.c | 60 +++++++++++++++++++
28 files changed, 269 insertions(+), 111 deletions(-)
delete mode 100644 lib/livepatch/Makefile
create mode 100755 tools/testing/selftests/livepatch/test-syscall.sh
create mode 100644 tools/testing/selftests/livepatch/test_binaries/test_klp-call_getpid.c
create mode 100644 tools/testing/selftests/livepatch/test_modules/Makefile
rename {lib/livepatch => tools/testing/selftests/livepatch/test_modules}/test_klp_atomic_replace.c (100%)
rename {lib/livepatch => tools/testing/selftests/livepatch/test_modules}/test_klp_callbacks_busy.c (100%)
rename {lib/livepatch => tools/testing/selftests/livepatch/test_modules}/test_klp_callbacks_demo.c (100%)
rename {lib/livepatch => tools/testing/selftests/livepatch/test_modules}/test_klp_callbacks_demo2.c (100%)
rename {lib/livepatch => tools/testing/selftests/livepatch/test_modules}/test_klp_callbacks_mod.c (100%)
rename {lib/livepatch => tools/testing/selftests/livepatch/test_modules}/test_klp_livepatch.c (100%)
rename {lib/livepatch => tools/testing/selftests/livepatch/test_modules}/test_klp_shadow_vars.c (100%)
rename {lib/livepatch => tools/testing/selftests/livepatch/test_modules}/test_klp_state.c (100%)
rename {lib/livepatch => tools/testing/selftests/livepatch/test_modules}/test_klp_state2.c (100%)
rename {lib/livepatch => tools/testing/selftests/livepatch/test_modules}/test_klp_state3.c (100%)
create mode 100644 tools/testing/selftests/livepatch/test_modules/test_klp_syscall.c
--
2.35.3