Although "TAP" word is being used already in documentation, but it hasn't
been defined in informative way for developers that how to write TAP
conformant tests and what are the benefits. Write a short brief about it.
Signed-off-by: Muhammad Usama Anjum <usama.anjum(a)collabora.com>
---
Changes since v1:
- Update documentation by modifying and removing sentences
---
Documentation/dev-tools/kselftest.rst | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/Documentation/dev-tools/kselftest.rst b/Documentation/dev-tools/kselftest.rst
index dcf634e411bd9..f3766e326d1e3 100644
--- a/Documentation/dev-tools/kselftest.rst
+++ b/Documentation/dev-tools/kselftest.rst
@@ -228,6 +228,13 @@ In general, the rules for selftests are
* Don't cause the top-level "make run_tests" to fail if your feature is
unconfigured.
+ * The output of tests must conform to the TAP standard to ensure high
+ testing quality and to capture failures/errors with specific details.
+ The kselftest.h and kselftest_harness.h headers provide wrappers for
+ outputting test results. These wrappers should be used for pass,
+ fail, exit, and skip messages. CI systems can easily parse TAP output
+ messages to detect test results.
+
Contributing new tests (details)
================================
--
2.39.2
Changes since v2:
a) After some disussion with David Hildenbrand, simplified the "fix
missing __NR_mseal" patch down to just two lines of diff, after all.
b) Improved the "kvm, mdwe fixes to avoid requiring "make headers""
patch by taking a snapshot of the prctl.h, instead of manually creating
defines. Thanks also to David Hildenbrand for that.
c) Fixed up the final patch to use asm-generic/unistd.h.
d) Added acks.
e) Rebased onto 6.10-rc4+ (today's top of tree).
Changes since v1:
Jeff Xu, I apologize for this churn: I was forced to drop your
Reviewed-by and Tested-by tags from 2 of the 3 mseal patches, because
the __NR_mseal fix is completely different now.
a) Reworked the mseal fix to use the kernel's in-tree unistd*.h files,
instead of hacking in a __NR_mseal definition directly. (Thanks to David
Hildenbrand for pointing out that this needed to be done.)
b) Fixed the subject line of the kvm and mdwe patch.
c) Reordered the patches so as to group the mseal changes together.
d) ADDED an additional patch, 6/6, to remove various __NR_xx items and
checks from the mm selftests.
Cover letter, updated for v2:
Eventually, once the build succeeds on a sufficiently old distro, the
idea is to delete $(KHDR_INCLUDES) from the selftests/mm build, and then
after that, from selftests/lib.mk and all of the other selftest builds.
For now, this series merely achieves a clean build of selftests/mm on a
not-so-old distro: Ubuntu 23.04. In other words, after this series is
applied, it is possible to delete $(KHDR_INCLUDES) from
selftests/mm/Makefile and the build will still succeed.
1. Add tools/uapi/asm/unistd_[32|x32|64].h files, which include
definitions of __NR_mseal, and include them (indirectly) from the files
that use __NR_mseal. The new files are copied from ./usr/include/asm,
which is how we have agreed to do this sort of thing, see [1].
2. Add fs.h, similarly created: it was copied directly from a snapshot
of ./usr/include/linux/fs.h after running "make headers".
3. Add a few selected prctl.h values that the ksm and mdwe tests require.
4. Factor out some common code from mseal_test.c and seal_elf.c, into a
new mseal_helpers.h file.
5. Remove local __NR_* definitions and checks.
[1] commit e076eaca5906 ("selftests: break the dependency upon local
header files")
John Hubbard (6):
selftests/mm: mseal, self_elf: fix missing __NR_mseal
selftests/mm: mseal, self_elf: factor out test macros and other
duplicated items
selftests/mm: mseal, self_elf: rename TEST_END_CHECK to
REPORT_TEST_PASS
selftests/mm: fix vm_util.c build failures: add snapshot of fs.h
selftests/mm: kvm, mdwe fixes to avoid requiring "make headers"
selftests/mm: remove local __NR_* definitions
tools/include/uapi/linux/fs.h | 392 ++++++++++++++++++
tools/include/uapi/linux/prctl.h | 331 +++++++++++++++
tools/testing/selftests/mm/hugepage-mremap.c | 2 +-
.../selftests/mm/ksm_functional_tests.c | 8 +-
tools/testing/selftests/mm/memfd_secret.c | 14 +-
tools/testing/selftests/mm/mkdirty.c | 8 +-
tools/testing/selftests/mm/mlock2.h | 1 +
tools/testing/selftests/mm/mseal_helpers.h | 41 ++
tools/testing/selftests/mm/mseal_test.c | 143 +++----
tools/testing/selftests/mm/pagemap_ioctl.c | 2 +-
tools/testing/selftests/mm/protection_keys.c | 2 +-
tools/testing/selftests/mm/seal_elf.c | 37 +-
tools/testing/selftests/mm/uffd-common.c | 4 -
tools/testing/selftests/mm/uffd-stress.c | 16 +-
tools/testing/selftests/mm/uffd-unit-tests.c | 14 +-
15 files changed, 824 insertions(+), 191 deletions(-)
create mode 100644 tools/include/uapi/linux/fs.h
create mode 100644 tools/include/uapi/linux/prctl.h
create mode 100644 tools/testing/selftests/mm/mseal_helpers.h
base-commit: 14d7c92f8df9c0964ae6f8b813c1b3ac38120825
--
2.45.2
Hi Linus,
Please pull this kselftest fixes update for Linux 6.10-rc5.
This kselftest fixes update consists of 4 fixes to the following
build warnings:
- filesystems: warn_unused_result warnings
- seccomp: format-zero-length warnings
- fchmodat2: clang build warnings due to-static-libasan
- openat2: clang build warnings due to static-libasan, LOCAL_HDRS
diff for pull request is attached.
thanks,
-- Shuah
----------------------------------------------------------------
The following changes since commit 4bf15b1c657d22d1d70173e43264e4606dfe75ff:
selftests/futex: don't pass a const char* to asprintf(3) (2024-05-31 14:37:10 -0600)
are available in the Git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest tags/linux_kselftest-fixes-6.10-rc5
for you to fetch changes up to ed3994ac847e0d6605f248e7f6776b1d4f445f4b:
selftests/fchmodat2: fix clang build failure due to -static-libasan (2024-06-11 15:05:05 -0600)
----------------------------------------------------------------
linux_kselftest-fixes-6.10-rc5
This kselftest fixes update consists of 4 fixes to the following
build warnings:
- filesystems: warn_unused_result warnings
- seccomp: format-zero-length warnings
- fchmodat2: clang build warnings due to-static-libasan
- openat2: clang build warnings due to static-libasan, LOCAL_HDRS
----------------------------------------------------------------
Amer Al Shanawany (2):
selftests: filesystems: fix warn_unused_result build warnings
selftests: seccomp: fix format-zero-length warnings
John Hubbard (2):
selftests/openat2: fix clang build failures: -static-libasan, LOCAL_HDRS
selftests/fchmodat2: fix clang build failure due to -static-libasan
tools/testing/selftests/fchmodat2/Makefile | 11 ++++++++++-
.../selftests/filesystems/statmount/statmount_test.c | 12 ++++++++++--
tools/testing/selftests/openat2/Makefile | 14 ++++++++++++--
tools/testing/selftests/seccomp/seccomp_benchmark.c | 6 +++---
4 files changed, 35 insertions(+), 8 deletions(-)
----------------------------------------------------------------
This patchset enables both detecting as well as dumping compilable
prototypes for kfuncs.
The first commit instructs pahole to DECL_TAG kfuncs when available.
This requires v1.27 or newer. v1.27 is nearing release at time of
writing. Following this, users will be able to look at BTF inside
vmlinux (or modules) and check if the kfunc they want is available.
The final commit teaches bpftool how to dump kfunc prototypes. This
is done for developer convenience.
The rest of the commits are fixups to enable selftests to use the
newly dumped kfunc prototypes. With these, selftests will regularly
exercise the newly added codepaths.
=== Changelog ===
From v3:
* Teach selftests to use dumped prototypes
From v2:
* Update Makefile.btf with pahole flag
* More error checking
* Output formatting changes
* Drop already-merged commit
From v1:
* Add __weak annotation
* Use btf_dump for kfunc prototypes
* Update kernel bpf_rdonly_cast() signature
Daniel Xu (12):
kbuild: bpf: Tell pahole to DECL_TAG kfuncs
bpf: selftests: Fix bpf_iter_task_vma_new() prototype
bpf: selftests: Fix fentry test kfunc prototypes
bpf: selftests: Fix bpf_cpumask_first_zero() kfunc prototype
bpf: selftests: Fix bpf_map_sum_elem_count() kfunc prototype
bpf: selftests: Fix bpf_session_cookie() kfunc prototype
bpf: selftests: Namespace struct_opt callbacks in bpf_dctcp
bpf: verifier: Relax caller requirements for kfunc projection type
args
bpf: treewide: Align kfunc signatures to prog point-of-view
bpf: selftests: nf: Opt out of using generated kfunc prototypes
bpf: selftests: xfrm: Opt out of using generated kfunc prototypes
bpftool: Support dumping kfunc prototypes from BTF
fs/verity/measure.c | 3 +-
include/linux/bpf.h | 8 +--
kernel/bpf/crypto.c | 24 +++++---
kernel/bpf/helpers.c | 39 +++++++++----
kernel/bpf/verifier.c | 12 +++-
kernel/trace/bpf_trace.c | 9 ++-
net/core/filter.c | 32 +++++++----
scripts/Makefile.btf | 2 +-
tools/bpf/bpftool/btf.c | 55 +++++++++++++++++++
.../testing/selftests/bpf/bpf_experimental.h | 2 +-
tools/testing/selftests/bpf/bpf_kfuncs.h | 2 +-
tools/testing/selftests/bpf/progs/bpf_dctcp.c | 36 ++++++------
.../selftests/bpf/progs/get_func_ip_test.c | 14 ++---
.../selftests/bpf/progs/ip_check_defrag.c | 10 ++--
.../bpf/progs/kprobe_multi_session_cookie.c | 2 +-
.../selftests/bpf/progs/map_percpu_stats.c | 2 +-
.../selftests/bpf/progs/nested_trust_common.h | 2 +-
.../testing/selftests/bpf/progs/test_bpf_nf.c | 1 +
.../selftests/bpf/progs/test_bpf_nf_fail.c | 1 +
.../bpf/progs/verifier_netfilter_ctx.c | 6 +-
.../selftests/bpf/progs/xdp_synproxy_kern.c | 1 +
tools/testing/selftests/bpf/progs/xfrm_info.c | 1 +
22 files changed, 183 insertions(+), 81 deletions(-)
--
2.44.0
Currently, if a user wants to run pmtu.sh and cover all the provided test
cases, they need to install the Open vSwitch userspace utilities. This
dependency is difficult for users as well as CI environments, because the
userspace build and setup may require lots of support and devel packages
to be installed, system setup to be correct, and things like permissions
and selinux policies to be properly configured.
The kernel selftest suite includes an ovs-dpctl.py utility which can
interact with the openvswitch module directly. This lets developers and
CI environments run without needing too many extra dependencies - just
the pyroute2 python package.
This series enhances the ovs-dpctl utility to provide support for set()
and tunnel() flow specifiers, better ipv6 handling support, and the
ability to add tunnel vports, and LWT interfaces. Finally, it modifies
the pmtu.sh script to call the ovs-dpctl.py utility rather than the
typical OVS userspace utilities.
Aaron Conole (7):
selftests: openvswitch: Support explicit tunnel port creation.
selftests: openvswitch: Refactor actions parsing.
selftests: openvswitch: Add set() and set_masked() support.
selftests: openvswitch: Add support for tunnel() key.
selftests: openvswitch: Support implicit ipv6 arguments.
selftests: net: Use the provided dpctl rather than the vswitchd for
tests.
selftests: net: add config for openvswitch
tools/testing/selftests/net/config | 5 +
.../selftests/net/openvswitch/ovs-dpctl.py | 372 +++++++++++++++---
tools/testing/selftests/net/pmtu.sh | 145 +++++--
3 files changed, 453 insertions(+), 69 deletions(-)
--
2.45.1