- Linux-kselftest-mirror - lists.linaro.org

[PATCHSET 0/3] Add REQ_F_CQE_SKIP support to io_uring zerocopy

by Oliver Crumrine

This patchset allows for io_uring zerocopy to support REQ_F_CQE_SKIP, skipping the normal completion notification, but not the zerocopy buffer release notification. This patchset also includes a test to test these changes, and a patch to mini_liburing to enable io_uring_peek_cqe, which is needed for the test. Oliver Crumrine (3): io_uring: Add REQ_F_CQE_SKIP support for io_uring zerocopy io_uring: Add io_uring_peek_cqe to mini_liburing io_uring: Support IOSQE_CQE_SKIP_SUCCESS in io_uring zerocopy test io_uring/net.c | 6 +-- tools/include/io_uring/mini_liburing.h | 18 +++++++++ .../selftests/net/io_uring_zerocopy_tx.c | 37 +++++++++++++++++-- .../selftests/net/io_uring_zerocopy_tx.sh | 7 +++- 4 files changed, 59 insertions(+), 10 deletions(-) -- 2.44.0

1 year, 7 months

1
0
0 0

[PATCH 0/2] Fix selftests/mm build without requiring "make headers"

by John Hubbard

Hi, As mentioned in each patch, this implements the solution that we discussed in December 2023, in [1]. This turned out to be very clean and easy. It should also be quite easy to maintain. This should also make Peter Zijlstra happy, because it directly addresses the root cause of his "NAK NAK NAK" reply [2]. :) I haven't done much build testing, because selftests are not so easy to build with a cross-compiler. So it's just tested on x86 64-bit so far. [1] https://lore.kernel.org/all/783a4178-1dec-4e30-989a-5174b8176b09@redhat.com/ [2] https://lore.kernel.org/lkml/20231103121652.GA6217@noisy.programming.kicks-… Cc: David Hildenbrand <david(a)redhat.com> Cc: Mark Brown <broonie(a)kernel.org> Cc: Muhammad Usama Anjum <usama.anjum(a)collabora.com> Cc: Suren Baghdasaryan <surenb(a)google.com> Cc: Peter Zijlstra <peterz(a)infradead.org> John Hubbard (2): selftests: break the dependency upon local header files selftests/mm: fix additional build errors for selftests tools/include/uapi/linux/memfd.h | 39 +++ tools/include/uapi/linux/userfaultfd.h | 386 +++++++++++++++++++++++++ tools/testing/selftests/lib.mk | 9 + tools/testing/selftests/mm/Makefile | 2 +- 4 files changed, 435 insertions(+), 1 deletion(-) create mode 100644 tools/include/uapi/linux/memfd.h create mode 100644 tools/include/uapi/linux/userfaultfd.h base-commit: 98560e9019851bf55b8a4073978a623a3bcf98c0 -- 2.44.0

1 year, 7 months

3
6
0 0

[PATCH 0/2] selftests: add ksft_exit_fail_perror()

by Muhammad Usama Anjum

In this series, ksft_exit_fail_perror() is being added which is helper function on top of ksft_exit_fail_msg(). It prints errno and its string form always. After writing and porting several kselftests, I've found out that most of times ksft_exit_fail_msg() isn't useful if errno value isn't printed. The ksft_exit_fail_perror() provides a convenient way to always print errno when its used. Muhammad Usama Anjum (2): selftests: add ksft_exit_fail_perror() selftests: exec: Use new ksft_exit_fail_perror() helper tools/testing/selftests/exec/recursion-depth.c | 10 +++++----- tools/testing/selftests/kselftest.h | 14 ++++++++++++++ 2 files changed, 19 insertions(+), 5 deletions(-) -- 2.39.2

1 year, 7 months

2
3
0 0

[PATCH] kselftest: Add missing signature to the comments

by Muhammad Usama Anjum

The comment on top of the file is used by many developers to glance over all the available functions. Add the recently added ksft_perror() to it. Signed-off-by: Muhammad Usama Anjum <usama.anjum(a)collabora.com> --- tools/testing/selftests/kselftest.h | 1 + 1 file changed, 1 insertion(+) diff --git a/tools/testing/selftests/kselftest.h b/tools/testing/selftests/kselftest.h index 7d650a06ca359..159bf8e314fa3 100644 --- a/tools/testing/selftests/kselftest.h +++ b/tools/testing/selftests/kselftest.h @@ -16,6 +16,7 @@ * For each test, report any progress, debugging, etc with: * * ksft_print_msg(fmt, ...); + * ksft_perror(msg); * * and finally report the pass/fail/skip/xfail state of the test with one of: * -- 2.39.2

1 year, 7 months

2
1
0 0

[BUG] seltests/iommu: runaway ./iommufd consuming 99% CPU after a failed assert()

by Mirsad Todorovac

Hi, (This is verified on the second test box.) In the most recent 6.8.0 release of torvalds tree kernel with selftest configs on, process ./iommufd appears to consume 99% of a CPU core for quote a while in an endless loop: root 59502 8816 0 Mar11 pts/2 00:00:00 make OUTPUT=/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/iommu -C iommu run_tests O=/home/marvin/linux/kernel/linux_torvalds root 59503 59502 0 Mar11 pts/2 00:00:00 /bin/sh -c BASE_DIR="/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests"; . /home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/kselftest/runner.sh; if [ "X" != "X" ]; then per_test_logging=1; fi; run_many /home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/iommu/iommufd /home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/iommu/iommufd_fail_nth root 59516 59503 0 Mar11 pts/2 00:00:00 /bin/sh -c BASE_DIR="/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests"; . /home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/kselftest/runner.sh; if [ "X" != "X" ]; then per_test_logging=1; fi; run_many /home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/iommu/iommufd /home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/iommu/iommufd_fail_nth root 59517 59516 0 Mar11 pts/2 00:00:00 /bin/sh -c BASE_DIR="/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests"; . /home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/kselftest/runner.sh; if [ "X" != "X" ]; then per_test_logging=1; fi; run_many /home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/iommu/iommufd /home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/iommu/iommufd_fail_nth root 59518 59517 0 Mar11 pts/2 00:00:00 /bin/sh -c BASE_DIR="/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests"; . /home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/kselftest/runner.sh; if [ "X" != "X" ]; then per_test_logging=1; fi; run_many /home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/iommu/iommufd /home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/iommu/iommufd_fail_nth root 59522 59518 0 Mar11 pts/2 00:00:00 /bin/sh -c BASE_DIR="/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests"; . /home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/kselftest/runner.sh; if [ "X" != "X" ]; then per_test_logging=1; fi; run_many /home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/iommu/iommufd /home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/iommu/iommufd_fail_nth root 59523 59522 0 Mar11 pts/2 00:00:00 perl /home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/kselftest/prefix.pl root 59635 2367 99 Mar11 pts/2 11:28:03 ./iommufd root@stargazer:/home/marvin# strace -p 59635 ioctl(5, _IOC(_IOC_NONE, 0x3b, 0xa0, 0), 0x7ffdd9eebc00) = 0 ioctl(5, _IOC(_IOC_NONE, 0x3b, 0xa0, 0), 0x7ffdd9eebc00) = 0 ioctl(5, _IOC(_IOC_NONE, 0x3b, 0xa0, 0), 0x7ffdd9eebc00) = 0 ioctl(5, _IOC(_IOC_NONE, 0x3b, 0xa0, 0), 0x7ffdd9eebc00) = 0 ioctl(5, _IOC(_IOC_NONE, 0x3b, 0xa0, 0), 0x7ffdd9eebc00) = 0 ioctl(5, _IOC(_IOC_NONE, 0x3b, 0xa0, 0), 0x7ffdd9eebc00) = 0 ioctl(5, _IOC(_IOC_NONE, 0x3b, 0xa0, 0), 0x7ffdd9eebc00) = 0 ioctl(5, _IOC(_IOC_NONE, 0x3b, 0xa0, 0), 0x7ffdd9eebc00) = 0 ioctl(5, _IOC(_IOC_NONE, 0x3b, 0xa0, 0), 0x7ffdd9eebc00) = 0 ioctl(5, _IOC(_IOC_NONE, 0x3b, 0xa0, 0), 0x7ffdd9eebc00) = 0 ioctl(5, _IOC(_IOC_NONE, 0x3b, 0xa0, 0), 0x7ffdd9eebc00) = 0 ioctl(5, _IOC(_IOC_NONE, 0x3b, 0xa0, 0), 0x7ffdd9eebc00) = 0 ioctl(5, _IOC(_IOC_NONE, 0x3b, 0xa0, 0), 0x7ffdd9eebc00) = 0 ioctl(5, _IOC(_IOC_NONE, 0x3b, 0xa0, 0), 0x7ffdd9eebc00) = 0 ioctl(5, _IOC(_IOC_NONE, 0x3b, 0xa0, 0), 0x7ffdd9eebc00) = 0 ioctl(5, _IOC(_IOC_NONE, 0x3b, 0xa0, 0), 0x7ffdd9eebc00) = 0 ioctl(5, _IOC(_IOC_NONE, 0x3b, 0xa0, 0), 0x7ffdd9eebc00) = 0 . . . Please find attached config. It is the vanilla kernel, the build suite marked it "dirty" because of the modifications to the selftests (adding debug option, mostly). The kseltest output is: make[3]: Entering directory '/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/iommu' TAP version 13 1..2 # timeout set to 45 # selftests: iommu: iommufd # TAP version 13 # 1..180 # # Starting 180 tests from 18 test cases. # # RUN iommufd.simple_close ... # # OK iommufd.simple_close # ok 1 iommufd.simple_close # # RUN iommufd.cmd_fail ... # # OK iommufd.cmd_fail # ok 2 iommufd.cmd_fail # # RUN iommufd.cmd_length ... # # OK iommufd.cmd_length # ok 3 iommufd.cmd_length # # RUN iommufd.cmd_ex_fail ... # # OK iommufd.cmd_ex_fail # ok 4 iommufd.cmd_ex_fail # # RUN iommufd.global_options ... # # OK iommufd.global_options # ok 5 iommufd.global_options # # RUN iommufd.simple_ioctls ... # # OK iommufd.simple_ioctls # ok 6 iommufd.simple_ioctls # # RUN iommufd.unmap_cmd ... # # OK iommufd.unmap_cmd # ok 7 iommufd.unmap_cmd # # RUN iommufd.map_cmd ... # # OK iommufd.map_cmd # ok 8 iommufd.map_cmd # # RUN iommufd.info_cmd ... # # OK iommufd.info_cmd # ok 9 iommufd.info_cmd # # RUN iommufd.set_iommu_cmd ... # # OK iommufd.set_iommu_cmd # ok 10 iommufd.set_iommu_cmd # # RUN iommufd.vfio_ioas ... # # OK iommufd.vfio_ioas # ok 11 iommufd.vfio_ioas # # RUN iommufd_ioas.no_domain.ioas_auto_destroy ... # # OK iommufd_ioas.no_domain.ioas_auto_destroy # ok 12 iommufd_ioas.no_domain.ioas_auto_destroy # # RUN iommufd_ioas.no_domain.ioas_destroy ... # # OK iommufd_ioas.no_domain.ioas_destroy # ok 13 iommufd_ioas.no_domain.ioas_destroy # # RUN iommufd_ioas.no_domain.alloc_hwpt_nested ... # # OK iommufd_ioas.no_domain.alloc_hwpt_nested # ok 14 iommufd_ioas.no_domain.alloc_hwpt_nested # # RUN iommufd_ioas.no_domain.hwpt_attach ... # # iommufd.c:541:hwpt_attach:Expected 2 (2) == errno (22) # # hwpt_attach: Test failed at step #6 # # FAIL iommufd_ioas.no_domain.hwpt_attach # not ok 15 iommufd_ioas.no_domain.hwpt_attach # # RUN iommufd_ioas.no_domain.ioas_area_destroy ... # # OK iommufd_ioas.no_domain.ioas_area_destroy # ok 16 iommufd_ioas.no_domain.ioas_area_destroy # # RUN iommufd_ioas.no_domain.ioas_area_auto_destroy ... # # OK iommufd_ioas.no_domain.ioas_area_auto_destroy # ok 17 iommufd_ioas.no_domain.ioas_area_auto_destroy # # RUN iommufd_ioas.no_domain.get_hw_info ... # # OK iommufd_ioas.no_domain.get_hw_info # ok 18 iommufd_ioas.no_domain.get_hw_info # # RUN iommufd_ioas.no_domain.area ... # # OK iommufd_ioas.no_domain.area # ok 19 iommufd_ioas.no_domain.area # # RUN iommufd_ioas.no_domain.unmap_fully_contained_areas ... # # OK iommufd_ioas.no_domain.unmap_fully_contained_areas # ok 20 iommufd_ioas.no_domain.unmap_fully_contained_areas # # RUN iommufd_ioas.no_domain.area_auto_iova ... # # OK iommufd_ioas.no_domain.area_auto_iova # ok 21 iommufd_ioas.no_domain.area_auto_iova # # RUN iommufd_ioas.no_domain.area_allowed ... # # OK iommufd_ioas.no_domain.area_allowed # ok 22 iommufd_ioas.no_domain.area_allowed # # RUN iommufd_ioas.no_domain.copy_area ... # # OK iommufd_ioas.no_domain.copy_area # ok 23 iommufd_ioas.no_domain.copy_area # # RUN iommufd_ioas.no_domain.iova_ranges ... # # OK iommufd_ioas.no_domain.iova_ranges # ok 24 iommufd_ioas.no_domain.iova_ranges # # RUN iommufd_ioas.no_domain.access_domain_destory ... # # iommufd.c:916:access_domain_destory:Expected MAP_FAILED (18446744073709551615) != buf (18446744073709551615) # # access_domain_destory: Test terminated by timeout # # FAIL iommufd_ioas.no_domain.access_domain_destory # not ok 25 iommufd_ioas.no_domain.access_domain_destory # # RUN iommufd_ioas.no_domain.access_pin ... # # iommufd.c:991:access_pin:Expected 0 (0) == _test_cmd_mock_domain(self->fd, self->ioas_id, &mock_stdev_id, &mock_hwpt_id, ((void *)0)) (-1) The failing assert seems to be here: 987 /* Add/remove a domain with a user */ 988 ASSERT_EQ(0, ioctl(self->fd, 989 _IOMMU_TEST_CMD(IOMMU_TEST_OP_ACCESS_PAGES), 990 &access_cmd)); → 991 test_cmd_mock_domain(self->ioas_id, &mock_stdev_id, 992 &mock_hwpt_id, NULL); 993 check_map_cmd.id = mock_hwpt_id; 994 ASSERT_EQ(0, ioctl(self->fd, 995 _IOMMU_TEST_CMD(IOMMU_TEST_OP_MD_CHECK_MAP), 996 &check_map_cmd)); For those of you who still do not have a clue what went wrong (like myself), I am trying to generate a reproducer. attempt to tap gdb on the running ./iommufd gave this: root@defiant:/home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/iommu# gdb ./iommufd --pid 63963 GNU gdb (Ubuntu 12.1-0ubuntu1~22.04.1) 12.1 Copyright (C) 2022 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: <https://www.gnu.org/software/gdb/bugs/>. Find the GDB manual and other documentation resources online at: <http://www.gnu.org/software/gdb/documentation/>. For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from ./iommufd... Attaching to program: /home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/iommu/iommufd, process 63963 Reading symbols from /usr/libexec/coreutils/libstdbuf.so... (No debugging symbols found in /usr/libexec/coreutils/libstdbuf.so) Reading symbols from /lib/x86_64-linux-gnu/libc.so.6... Reading symbols from /usr/lib/debug/.build-id/c2/89da5071a3399de893d2af81d6a30c62646e1e.debug... Reading symbols from /lib64/ld-linux-x86-64.so.2... Reading symbols from /usr/lib/debug/.build-id/15/921ea631d9f36502d20459c43e5c85b7d6ab76.debug... [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". __GI___ioctl (fd=5, request=request@entry=15264) at ../sysdeps/unix/sysv/linux/ioctl.c:36 36 ../sysdeps/unix/sysv/linux/ioctl.c: No such file or directory. (gdb) bt #0 __GI___ioctl (fd=5, request=request@entry=15264) at ../sysdeps/unix/sysv/linux/ioctl.c:36 #1 0x000057ed23d1f1ae in _test_ioctl_set_temp_memory_limit (limit=65536, fd=<optimized out>) at /home/marvin/linux/kernel/linux_torvalds/tools/testing/selftests/iommu/iommufd_utils.h:585 #2 iommufd_ioas_teardown (_metadata=_metadata@entry=0x57ed23d42860 <_iommufd_ioas_access_domain_destory_object>, self=self@entry=0x7ffdcdce5ef0, variant=<optimized out>) at iommufd.c:229 #3 0x000057ed23d23b7f in wrapper_iommufd_ioas_access_domain_destory (_metadata=0x57ed23d42860 <_iommufd_ioas_access_domain_destory_object>, variant=0x57ed23d43860 <_iommufd_ioas_mock_domain_object>) at iommufd.c:902 #4 0x000057ed23d1bfe9 in __run_test (f=f@entry=0x57ed23d438a0 <_iommufd_ioas_fixture_object>, variant=variant@entry=0x57ed23d43860 <_iommufd_ioas_mock_domain_object>, t=t@entry=0x57ed23d42860 <_iommufd_ioas_access_domain_destory_object>) at ../kselftest_harness.h:1134 #5 0x000057ed23d12146 in test_harness_run (argv=0x7ffdcdce61a8, argc=1) at ../kselftest_harness.h:1199 #6 main (argc=1, argv=0x7ffdcdce61a8) at iommufd.c:2349 (gdb) list iommufd.c:2349 2344 &unmap_cmd)); 2345 } 2346 } 2347 } 2348 2349 TEST_HARNESS_MAIN (gdb) Hope this helps someone. Best regards, Mirsad Todorovac

1 year, 7 months

4
13
0 0

[PATCH] selftests/harness: Prevent infinite loop due to Assert in FIXTURE_TEARDOWN

by Shengyu Li

This patch addresses an issue in the selftests/harness where an assertion within FIXTURE_TEARDOWN could trigger an infinite loop. The problem arises because the teardown procedure is meant to execute once, but the presence of failing assertions (ASSERT_EQ(0, 1)) leads to repeated attempts to execute teardown due to the long jump mechanism used by the harness for handling assertions. To resolve this, the patch ensures that the teardown process runs only once, regardless of assertion outcomes, preventing the infinite loop and allowing tests to fail. Signed-off-by: Shengyu Li <shengyu.li.evgeny(a)gmail.com> --- tools/testing/selftests/kselftest_harness.h | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/kselftest_harness.h b/tools/testing/selftests/kselftest_harness.h index 4fd735e48ee7..230d62884885 100644 --- a/tools/testing/selftests/kselftest_harness.h +++ b/tools/testing/selftests/kselftest_harness.h @@ -383,6 +383,7 @@ FIXTURE_DATA(fixture_name) self; \ pid_t child = 1; \ int status = 0; \ + bool jmp = false; \ memset(&self, 0, sizeof(FIXTURE_DATA(fixture_name))); \ if (setjmp(_metadata->env) == 0) { \ /* Use the same _metadata. */ \ @@ -399,8 +400,10 @@ _metadata->exit_code = KSFT_FAIL; \ } \ } \ + else \ + jmp = true; \ if (child == 0) { \ - if (_metadata->setup_completed && !_metadata->teardown_parent) \ + if (_metadata->setup_completed && !_metadata->teardown_parent && !jmp) \ fixture_name##_teardown(_metadata, &self, variant->data); \ _exit(0); \ } \ -- 2.25.1

1 year, 7 months

2
4
0 0

[Bug Report] selftests: Clang: kselftests build errors in KernelCI

by Muhammad Usama Anjum

Hi, We have caught bugs in kselftest suites on linux-next and on stable-RCs etc when using clang. There are two types of bugs (logs with clang-17 are attached.): As usually people use GCC, there are GCC-specific flags added to the Makefiles that clang doesn't recognize. For example: * clang: error: argument unused during compilation: '-pie' [-Werror,-Wunused-command-line-argument] * clang: error: unknown argument '-static-libasan'; did you mean '-static-libsan'? * clang: error: cannot specify -o when generating multiple output files Clang has best static analysis tools. It is reporting static errors. For example: * test_execve.c:121:13: warning: variable 'have_outer_privilege' is used uninitialized whenever 'if' condition is false [-Wsometimes-uninitialized] * test_execve.c:121:9: note: remove the 'if' if its condition is always true * test_memcontrol.c:727:6: warning: variable 'fd' is used uninitialized whenever 'if' condition is true [-Wsometimes-uninitialized] We have found these issues through our new KernelCI system when enabling kselftest and clang there. The new system dashboard is a WIP, so It is not the web dashboard you are used-to with in KernelCI. We can show you ways of pulling the data if you are interest into. Unless the above is some sort of false-positive or misconfiguration, it would be great to support clang for kselftests. What we can do from our side is that clang kselftests builds should be enabled on KernelCI to find and fix the errors. What is your stance about this? Thanks, Usama

1 year, 7 months

3
2
0 0

[PATCH v2] selftests/ftrace: Limit length in subsystem-enable tests

by Yuanhe Shu

While sched* events being traced and sched* events continuously happen, "[xx] event tracing - enable/disable with subsystem level files" would not stop as on some slower systems it seems to take forever. Select the first 100 lines of output would be enough to judge whether there are more than 3 types of sched events. Fixes: 815b18ea66d6 ("ftracetest: Add basic event tracing test cases") Cc: stable(a)vger.kernel.org Signed-off-by: Yuanhe Shu <xiangzao(a)linux.alibaba.com> --- .../selftests/ftrace/test.d/event/subsystem-enable.tc | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/tools/testing/selftests/ftrace/test.d/event/subsystem-enable.tc b/tools/testing/selftests/ftrace/test.d/event/subsystem-enable.tc index b1ede6249866..b7c8f29c09a9 100644 --- a/tools/testing/selftests/ftrace/test.d/event/subsystem-enable.tc +++ b/tools/testing/selftests/ftrace/test.d/event/subsystem-enable.tc @@ -18,7 +18,7 @@ echo 'sched:*' > set_event yield -count=`cat trace | grep -v ^# | awk '{ print $5 }' | sort -u | wc -l` +count=`head -n 100 trace | grep -v ^# | awk '{ print $5 }' | sort -u | wc -l` if [ $count -lt 3 ]; then fail "at least fork, exec and exit events should be recorded" fi @@ -29,7 +29,7 @@ echo 1 > events/sched/enable yield -count=`cat trace | grep -v ^# | awk '{ print $5 }' | sort -u | wc -l` +count=`head -n 100 trace | grep -v ^# | awk '{ print $5 }' | sort -u | wc -l` if [ $count -lt 3 ]; then fail "at least fork, exec and exit events should be recorded" fi @@ -40,7 +40,7 @@ echo 0 > events/sched/enable yield -count=`cat trace | grep -v ^# | awk '{ print $5 }' | sort -u | wc -l` +count=`head -n 100 trace | grep -v ^# | awk '{ print $5 }' | sort -u | wc -l` if [ $count -ne 0 ]; then fail "any of scheduler events should not be recorded" fi -- 2.39.3

1 year, 7 months

4
3
0 0

[PATCH net-next 0/7] selftests: net: groundwork for YNL-based tests

by Jakub Kicinski

Currently the options for writing networking tests are C, bash or some mix of the two. YAML/Netlink gives us the ability to easily interface with Netlink in higher level laguages. In particular, there is a Python library already available in tree, under tools/net. Add the scaffolding which allows writing tests using this library. The "scaffolding" is needed because the library lives under tools/net and uses YAML files from under Documentation/. So we need a small amount of glue code to find those things and add them to TEST_FILES. This series adds both a basic SW sanity test and driver test which can be run against netdevsim or a real device. When I develop core code I usually test with netdevsim, then a real device, and then a backport to Meta's kernel. Because of the lack of integration, until now I had to throw away the (YNL-based) test script and netdevsim code. Running tests in tree directly: $ ./tools/testing/selftests/net/nl_netdev.py KTAP version 1 1..2 ok 1 nl_netdev.empty_check ok 2 nl_netdev.lo_check # Totals: pass:2 fail:0 xfail:0 xpass:0 skip:0 error:0 in tree via make: $ make -C tools/testing/selftests/ TARGETS=net \ TEST_PROGS=nl_netdev.py TEST_GEN_PROGS="" run_tests [ ... ] and installed externally, all seem to work: $ make -C tools/testing/selftests/ TARGETS=net \ install INSTALL_PATH=/tmp/ksft-net $ /tmp/ksft-net/run_kselftest.sh -t net:nl_netdev.py [ ... ] For driver tests I followed the lead of net/forwarding and get the device name from env and/or a config file. Jakub Kicinski (7): netlink: specs: define ethtool header flags tools: ynl: copy netlink error to NlError selftests: net: add scaffolding for Netlink tests in Python selftests: nl_netdev: add a trivial Netlink netdev test netdevsim: report stats by default, like a real device selftests: drivers: add scaffolding for Netlink tests in Python testing: net-drv: add a driver test for stats reporting Documentation/netlink/specs/ethtool.yaml | 5 + drivers/net/netdevsim/ethtool.c | 11 ++ drivers/net/netdevsim/netdev.c | 45 +++++++ tools/net/ynl/lib/ynl.py | 3 +- tools/testing/selftests/Makefile | 8 ++ tools/testing/selftests/drivers/net/Makefile | 7 ++ .../testing/selftests/drivers/net/README.rst | 30 +++++ .../selftests/drivers/net/lib/py/__init__.py | 17 +++ .../selftests/drivers/net/lib/py/env.py | 41 ++++++ tools/testing/selftests/drivers/net/stats.py | 85 +++++++++++++ tools/testing/selftests/net/Makefile | 1 + tools/testing/selftests/net/lib/Makefile | 8 ++ .../testing/selftests/net/lib/py/__init__.py | 7 ++ tools/testing/selftests/net/lib/py/consts.py | 9 ++ tools/testing/selftests/net/lib/py/ksft.py | 96 ++++++++++++++ tools/testing/selftests/net/lib/py/nsim.py | 118 ++++++++++++++++++ tools/testing/selftests/net/lib/py/utils.py | 47 +++++++ tools/testing/selftests/net/lib/py/ynl.py | 49 ++++++++ tools/testing/selftests/net/nl_netdev.py | 24 ++++ 19 files changed, 610 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/drivers/net/Makefile create mode 100644 tools/testing/selftests/drivers/net/README.rst create mode 100644 tools/testing/selftests/drivers/net/lib/py/__init__.py create mode 100644 tools/testing/selftests/drivers/net/lib/py/env.py create mode 100755 tools/testing/selftests/drivers/net/stats.py create mode 100644 tools/testing/selftests/net/lib/Makefile create mode 100644 tools/testing/selftests/net/lib/py/__init__.py create mode 100644 tools/testing/selftests/net/lib/py/consts.py create mode 100644 tools/testing/selftests/net/lib/py/ksft.py create mode 100644 tools/testing/selftests/net/lib/py/nsim.py create mode 100644 tools/testing/selftests/net/lib/py/utils.py create mode 100644 tools/testing/selftests/net/lib/py/ynl.py create mode 100755 tools/testing/selftests/net/nl_netdev.py -- 2.44.0

1 year, 7 months

3
25
0 0

[KTAP V2 PATCH v4] ktap_v2: add test metadata

by Rae Moar

Add specification for test metadata to the KTAP v2 spec. KTAP v1 only specifies the output format of very basic test information: test result and test name. Any additional test information either gets added to general diagnostic data or is not included in the output at all. The purpose of KTAP metadata is to create a framework to include and easily identify additional important test information in KTAP. KTAP metadata could include any test information that is pertinent for user interaction before or after the running of the test. For example, the test file path or the test speed. Since this includes a large variety of information, this specification will recognize notable types of KTAP metadata to ensure consistent format across test frameworks. See the full list of types in the specification. Example of KTAP Metadata: KTAP version 2 #:ktap_test: main #:ktap_arch: uml 1..1 KTAP version 2 #:ktap_test: suite_1 #:ktap_subsystem: example #:ktap_test_file: lib/test.c 1..2 ok 1 test_1 #:ktap_test: test_2 #:ktap_speed: very_slow # test_2 has begun #:custom_is_flaky: true ok 2 test_2 # suite_1 has passed ok 1 suite_1 The changes to the KTAP specification outline the format, location, and different types of metadata. Reviewed-by: Kees Cook <keescook(a)chromium.org> Reviewed-by: David Gow <davidgow(a)google.com> Signed-off-by: Rae Moar <rmoar(a)google.com> --- Note this version is in reponse to comments made off the list asking for more explanation on inheritance and edge cases. Changes since v3: - Add two metadata ktap_config and ktap_id - Add section on metadata inheritance - Add edge case examples Documentation/dev-tools/ktap.rst | 248 ++++++++++++++++++++++++++++++- 1 file changed, 244 insertions(+), 4 deletions(-) diff --git a/Documentation/dev-tools/ktap.rst b/Documentation/dev-tools/ktap.rst index ff77f4aaa6ef..55bc43cd5aea 100644 --- a/Documentation/dev-tools/ktap.rst +++ b/Documentation/dev-tools/ktap.rst @@ -17,19 +17,22 @@ KTAP test results describe a series of tests (which may be nested: i.e., test can have subtests), each of which can contain both diagnostic data -- e.g., log lines -- and a final result. The test structure and results are machine-readable, whereas the diagnostic data is unstructured and is there to -aid human debugging. +aid human debugging. Since version 2, tests can also contain metadata which +consists of important supplemental test information and can be +machine-readable. + +KTAP output is built from five different types of lines: -KTAP output is built from four different types of lines: - Version lines - Plan lines - Test case result lines - Diagnostic lines +- Metadata lines In general, valid KTAP output should also form valid TAP output, but some information, in particular nested test results, may be lost. Also note that there is a stagnant draft specification for TAP14, KTAP diverges from this in -a couple of places (notably the "Subtest" header), which are described where -relevant later in this document. +a couple of places, which are described where relevant later in this document. Version lines ------------- @@ -166,6 +169,237 @@ even if they do not start with a "#": this is to capture any other useful kernel output which may help debug the test. It is nevertheless recommended that tests always prefix any diagnostic output they have with a "#" character. +KTAP metadata lines +------------------- + +KTAP metadata lines are used to include and easily identify important +supplemental test information in KTAP. These lines may appear similar to +diagnostic lines. The format of metadata lines is below: + +.. code-block:: none + + #:<prefix>_<metadata type>: <metadata value> + +The <prefix> indicates where to find the specification for the type of +metadata, such as the name of a test framework or "ktap" to indicate this +specification. The list of currently approved prefixes and where to find the +documentation of the metadata types is below. Note any metadata type that does +not use a prefix from the list below must use the prefix "custom". + +Current List of Approved Prefixes: + +- ``ktap``: See Types of KTAP Metadata below for the list of metadata types. + +The format of <metadata type> and <value> varies based on the type. See the +individual specification. For "custom" types the <metadata type> can be any +string excluding ":", spaces, or newline characters and the <value> can be any +string. + +**Location:** + +The first KTAP metadata line for a test must be "#:ktap_test: <test name>", +which acts as a header to associate metadata with the correct test. Metadata +for the main KTAP level uses the test name "main". A test's metadata ends +with a "ktap_test" line for a different test. + +For test cases, the location of the metadata is between the prior test result +line and the current test result line. For test suites, the location of the +metadata is between the suite's version line and test plan line. For the main +level, the location of the metadata is between the main version line and main +test plan line. See the example below. + +Note that a test case's metadata is inline with the test's result line. Whereas +a suite's metadata is inline with the suite's version line and thus will be +more indented than the suite's result line. Additionally, metadata for the main +level is inline with the main version line. + +KTAP metadata for a test does not need to be contiguous. For example, a kernel +warning or other diagnostic output could interrupt metadata lines. However, it +is recommended to keep a test's metadata lines in the correct location and +together when possible, as this improves readability. + +**Example of KTAP metadata:** + +:: + + KTAP version 2 + #:ktap_test: main + #:ktap_arch: uml + 1..1 + KTAP version 2 + #:ktap_test: suite_1 + #:ktap_subsystem: example + #:ktap_test_file: lib/test.c + 1..2 + # WARNING: test_1 skipped + ok 1 test_1 # SKIP + #:ktap_test: test_2 + #:ktap_speed: very_slow + # test_2 has begun + #:custom_is_flaky: true + ok 2 test_2 + # suite_1 passed + ok 1 suite_1 + +In this example, the tests are running on UML. The test suite "suite_1" is part +of the subsystem "example" and belongs to the file "lib/test.c". It has +two subtests, "test_1" and "test_2". The subtest "test_2" has a speed of +"very_slow" and has been marked with a custom KTAP metadata type called +"custom_is_flaky" with the value of "true". + +**Inheritance of KTAP metadata** + +Tests can inherit KTAP metadata. A child test inherits all the parent test's +KTAP metadata except for directly opposing metadata. For example, if a suite +has a property of "#:ktap_speed: slow", all child test cases are also marked as +slow. However, if one of the test cases has metadata of "#:ktap_speed: +very_slow" then that test case would be marked as very_slow instead and not +slow. + +Note if a test case inherits metadata it does not need to appear as a line in +the KTAP. Using the example above, not every test case would have the line +"#:ktap_speed: slow" in their metadata. + +**Edge Case Examples of KTAP metadata** + +Here are a few edge case examples of KTAP metadata. The first example shows +metadata in the wrong location. + +:: + + KTAP version 2 + 1..1 + KTAP version 2 + #:ktap_test: suite_1 + 1..3 + ok 1 test_1 + #:ktap_test: test_2 + #:ktap_speed: very_slow + ok 2 test_2 + #:ktap_duration: 1.342s + #:ktap_test: test_3 + #:ktap_speed: slow + ok 3 test_3 + ok 1 suite_1 + +In this example, the metadata "#:ktap_duration: 1.342s" is in the wrong +location. It was meant to belong to test_2 but was printed late. The location +of this metadata is not recommended. However, it is allowed because the line is +still below "#:ktap_test: test_2" and above any other ktap_test lines. + +This second example shows metadata in the correct location but without the +proper header. + +:: + + KTAP version 2 + 1..1 + KTAP version 2 + #:ktap_test: suite_1 + 1..2 + not ok 1 test_1 + #:ktap_speed: very_slow + ok 2 test_2 + ok 1 suite_1 + +In this example, the metadata "#:ktap_speed: very_slow" is meant to belong to +test_2. It is in the correct location but does not fall below a ktap_test line +for test_2. Instead this metadata might be mistaken for belonging to suite_1 +because it does fall under the ktap_test line for suite_1. This lack of header +is not allowed. + +**Types of KTAP Metadata:** + +This is the current list of KTAP metadata types recognized in this +specification. Note that all of these metadata types are optional (except for +ktap_test as the KTAP metadata header). + +- ``ktap_test``: Name of test (used as header of KTAP metadata). This should + match the test name printed in the test result line: "ok 1 [test_name]". + +- ``ktap_module``: Name of the module containing the test + +- ``ktap_subsystem``: Name of the subsystem being tested + +- ``ktap_start_time``: Time tests started in ISO8601 format + + - Example: "#:ktap_start_time: 2024-01-09T13:09:01.990000+00:00" + +- ``ktap_duration``: Time taken (in seconds) to execute the test + + - Example: "#:ktap_duration: 10.154s" + +- ``ktap_speed``: Category of how fast test runs: "normal", "slow", or + "very_slow" + +- ``ktap_test_file``: Path to source file containing the test. This metadata + line can be repeated if the test is spread across multiple files. + + - Example: "#:ktap_test_file: lib/test.c" + +- ``ktap_generated_file``: Description of and path to file generated during + test execution. This could be a core dump, generated filesystem image, some + form of visual output (for graphics drivers), etc. This metadata line can be + repeated to attach multiple files to the test. Note use ktap_log_file or + ktap_error_file instead of this type if more applicable. + + - Example: "#:ktap_generated_file: Core dump: /var/lib/systemd/coredump/hello.core" + +- ``ktap_log_file``: Path to file containing kernel log test output + + - Example: "#:ktap_log_file: /sys/kernel/debugfs/kunit/example/results" + +- ``ktap_error_file``: Path to file containing context for test failure or + error. This could include the difference between optimal test output and + actual test output. + + - Example: "#:ktap_error_file: fs/results/example.out.bad" + +- ``ktap_results_url``: Link to webpage describing this test run and its + results + + - Example: "#:ktap_results_url: https://kcidb.kernelci.org/hello" + +- ``ktap_arch``: Architecture used during test run + + - Example: "#:ktap_arch: x86_64" + +- ``ktap_compiler``: Compiler used during test run + + - Example: "#:ktap_compiler: gcc (GCC) 10.1.1 20200507 (Red Hat 10.1.1-1)" + +- ``ktap_respository_url``: Link to git repository of the checked out code. + + - Example: "#:ktap_respository_url: https://github.com/torvalds/linux.git" + +- ``ktap_git_branch``: Name of git branch of checked out code + + - Example: "#:ktap_git_branch: kselftest/kunit" + +- ``ktap_kernel_version``: Version of Linux Kernel being used during test run + + - Example: "#:ktap_kernel_version: 6.7-rc1" + +- ``ktap_config``: Config name and value. This does not necessarly need to be + restricted to Kconfig. + + - Example: "#:ktap_config: CONFIG_SYSFS=y" + +- ``ktap_id``: Description of ID and ID value. This is an open-ended metadata + used for IDs, such as checkout id or test run id. + + - Example: "#:ktap_id: Test run id: 14e782" + +- ``ktap_commit_hash``: The full git commit hash of the checked out base code. + + - Example: "#:ktap_commit_hash: 064725faf8ec2e6e36d51e22d3b86d2707f0f47f" + +**Other Metadata Types:** + +There can also be KTAP metadata that is not included in the recognized list +above. This metadata must be prefixed with the test framework, ie. "kselftest", +or with the prefix "custom". For example, "# custom_batch: 20". + Unknown lines ------------- @@ -206,6 +440,7 @@ An example of a test with two nested subtests: KTAP version 2 1..1 KTAP version 2 + #:ktap_test: example 1..2 ok 1 test_1 not ok 2 test_2 @@ -219,6 +454,7 @@ An example format with multiple levels of nested testing: KTAP version 2 1..2 KTAP version 2 + #:ktap_test: example_test_1 1..2 KTAP version 2 1..2 @@ -254,6 +490,7 @@ Example KTAP output KTAP version 2 1..1 KTAP version 2 + #:ktap_test: main_test 1..3 KTAP version 2 1..1 @@ -261,11 +498,14 @@ Example KTAP output ok 1 test_1 ok 1 example_test_1 KTAP version 2 + #:ktap_test: example_test_2 + #:ktap_speed: slow 1..2 ok 1 test_1 # SKIP test_1 skipped ok 2 test_2 ok 2 example_test_2 KTAP version 2 + #:ktap_test: example_test_3 1..3 ok 1 test_1 # test_2: FAIL base-commit: 906f02e42adfbd5ae70d328ee71656ecb602aaf5 -- 2.44.0.478.gd926399ef9-goog

1 year, 7 months

1
0
0 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

Linux-kselftest-mirror