- Linux-kselftest-mirror - lists.linaro.org

next-20200326 kselftest results

by ci_notify＠linaro.org

Summary ------------------------------------------------------------------------ kernel: 5.6.0-rc7 git repo: https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git git branch: master git commit: 89295c59c1f063b533d071ca49d0fa0c0783ca6f git describe: next-20200326 Test details: https://qa-reports.linaro.org/lkft/linux-next-oe/build/next-20200326 Regressions (compared to build next-20200325) ------------------------------------------------------------------------ No regressions Fixes (compared to build next-20200325) ------------------------------------------------------------------------ No fixes In total: ------------------------------------------------------------------------ Ran 0 total tests in the following environments and test suites. pass 0 fail 0 xfail 0 skip 0 Environments -------------- - dragonboard-410c - arm64 - hi6220-hikey - arm64 - i386 - juno-r2 - arm64 - juno-r2-compat - juno-r2-kasan - x15 - arm - x86_64 - x86-kasan Test Suites ----------- Failures ------------------------------------------------------------------------ juno-r2: juno-r2-kasan: i386: juno-r2-compat: x86: x86-kasan: hi6220-hikey: dragonboard-410c: x15: Skips ------------------------------------------------------------------------ No skips -- Linaro LKFT https://lkft.linaro.org

5 years, 3 months

1
0
0 0

[PATCHv2] selftests/powerpc: Turn off timeout setting for benchmarks, dscr, signal, tm

by Po-Hsu Lin

Some specific tests in powerpc can take longer than the default 45 seconds that added in commit 852c8cbf34d3 ("selftests/kselftest/runner.sh: Add 45 second timeout per test") to run, the following test result was collected across 2 Power8 nodes and 1 Power9 node in our pool: powerpc/benchmarks/futex_bench - 52s powerpc/dscr/dscr_sysfs_test - 116s powerpc/signal/signal_fuzzer - 88s powerpc/tm/tm_unavailable_test - 168s powerpc/tm/tm-poison - 240s Thus they will fail with TIMEOUT error. Disable the timeout setting for these sub-tests to allow them finish properly. https://bugs.launchpad.net/bugs/1864642 Fixes: 852c8cbf34d3 ("selftests/kselftest/runner.sh: Add 45 second timeout per test") Signed-off-by: Po-Hsu Lin <po-hsu.lin(a)canonical.com> --- tools/testing/selftests/powerpc/benchmarks/Makefile | 2 ++ tools/testing/selftests/powerpc/benchmarks/settings | 1 + tools/testing/selftests/powerpc/dscr/Makefile | 2 ++ tools/testing/selftests/powerpc/dscr/settings | 1 + tools/testing/selftests/powerpc/signal/Makefile | 2 ++ tools/testing/selftests/powerpc/signal/settings | 1 + tools/testing/selftests/powerpc/tm/Makefile | 2 ++ tools/testing/selftests/powerpc/tm/settings | 1 + 8 files changed, 12 insertions(+) create mode 100644 tools/testing/selftests/powerpc/benchmarks/settings create mode 100644 tools/testing/selftests/powerpc/dscr/settings create mode 100644 tools/testing/selftests/powerpc/signal/settings create mode 100644 tools/testing/selftests/powerpc/tm/settings diff --git a/tools/testing/selftests/powerpc/benchmarks/Makefile b/tools/testing/selftests/powerpc/benchmarks/Makefile index d40300a..a32a6ab 100644 --- a/tools/testing/selftests/powerpc/benchmarks/Makefile +++ b/tools/testing/selftests/powerpc/benchmarks/Makefile @@ -2,6 +2,8 @@ TEST_GEN_PROGS := gettimeofday context_switch fork mmap_bench futex_bench null_syscall TEST_GEN_FILES := exec_target +TEST_FILES := settings + CFLAGS += -O2 top_srcdir = ../../../../.. diff --git a/tools/testing/selftests/powerpc/benchmarks/settings b/tools/testing/selftests/powerpc/benchmarks/settings new file mode 100644 index 0000000..e7b9417 --- /dev/null +++ b/tools/testing/selftests/powerpc/benchmarks/settings @@ -0,0 +1 @@ +timeout=0 diff --git a/tools/testing/selftests/powerpc/dscr/Makefile b/tools/testing/selftests/powerpc/dscr/Makefile index 5df4763..cfa6eed 100644 --- a/tools/testing/selftests/powerpc/dscr/Makefile +++ b/tools/testing/selftests/powerpc/dscr/Makefile @@ -3,6 +3,8 @@ TEST_GEN_PROGS := dscr_default_test dscr_explicit_test dscr_user_test \ dscr_inherit_test dscr_inherit_exec_test dscr_sysfs_test \ dscr_sysfs_thread_test +TEST_FILES := settings + top_srcdir = ../../../../.. include ../../lib.mk diff --git a/tools/testing/selftests/powerpc/dscr/settings b/tools/testing/selftests/powerpc/dscr/settings new file mode 100644 index 0000000..e7b9417 --- /dev/null +++ b/tools/testing/selftests/powerpc/dscr/settings @@ -0,0 +1 @@ +timeout=0 diff --git a/tools/testing/selftests/powerpc/signal/Makefile b/tools/testing/selftests/powerpc/signal/Makefile index 113838f..153fafc 100644 --- a/tools/testing/selftests/powerpc/signal/Makefile +++ b/tools/testing/selftests/powerpc/signal/Makefile @@ -5,6 +5,8 @@ CFLAGS += -maltivec $(OUTPUT)/signal_tm: CFLAGS += -mhtm $(OUTPUT)/sigfuz: CFLAGS += -pthread -m64 +TEST_FILES := settings + top_srcdir = ../../../../.. include ../../lib.mk diff --git a/tools/testing/selftests/powerpc/signal/settings b/tools/testing/selftests/powerpc/signal/settings new file mode 100644 index 0000000..e7b9417 --- /dev/null +++ b/tools/testing/selftests/powerpc/signal/settings @@ -0,0 +1 @@ +timeout=0 diff --git a/tools/testing/selftests/powerpc/tm/Makefile b/tools/testing/selftests/powerpc/tm/Makefile index b15a1a3..7b99d09 100644 --- a/tools/testing/selftests/powerpc/tm/Makefile +++ b/tools/testing/selftests/powerpc/tm/Makefile @@ -7,6 +7,8 @@ TEST_GEN_PROGS := tm-resched-dscr tm-syscall tm-signal-msr-resv tm-signal-stack $(SIGNAL_CONTEXT_CHK_TESTS) tm-sigreturn tm-signal-sigreturn-nt \ tm-signal-context-force-tm tm-poison +TEST_FILES := settings + top_srcdir = ../../../../.. include ../../lib.mk diff --git a/tools/testing/selftests/powerpc/tm/settings b/tools/testing/selftests/powerpc/tm/settings new file mode 100644 index 0000000..e7b9417 --- /dev/null +++ b/tools/testing/selftests/powerpc/tm/settings @@ -0,0 +1 @@ +timeout=0 -- 2.7.4

5 years, 3 months

2
1
0 0

next-20200325 kselftest results

by ci_notify＠linaro.org

Summary ------------------------------------------------------------------------ kernel: 5.6.0-rc7 git repo: https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git git branch: master git commit: bfd7a248502373da8b1c8eb0e811fdb19cc3f8b6 git describe: next-20200325 Test details: https://qa-reports.linaro.org/lkft/linux-next-oe/build/next-20200325 Regressions (compared to build next-20200323) ------------------------------------------------------------------------ No regressions Fixes (compared to build next-20200323) ------------------------------------------------------------------------ No fixes In total: ------------------------------------------------------------------------ Ran 0 total tests in the following environments and test suites. pass 0 fail 0 xfail 0 skip 0 Environments -------------- - dragonboard-410c - arm64 - hi6220-hikey - arm64 - i386 - juno-r2 - arm64 - x15 - arm - x86_64 Test Suites ----------- Failures ------------------------------------------------------------------------ dragonboard-410c: x15: hi6220-hikey: i386: juno-r2: x86: Skips ------------------------------------------------------------------------ No skips -- Linaro LKFT https://lkft.linaro.org

5 years, 3 months

1
0
0 0

[PATCH v2 0/6] Fix errors when try to build kvm selftests on specified output

by Xiaoyao Li

I attempted to build KVM selftests on a specified dir, unfortunately neither "make O=/path/to/mydir TARGETS=kvm" in tools/testing/selftests, nor "make OUTPUT=/path/to/mydir" in tools/testing/selftests/kvm work. This series aims to fix them. Patch 1 fixes the issue that output directory is not exist. Patch 2 and 3 are the preparation for kvm to get the right path of installed linux headers. Patch 4 and 6 prepare the INSTALL_HDR_PATH to tell sub TARGET where the linux headers are installed. Patch 5 fixes the issue that with OUTPUT specified, it still make the linux tree dirty. I only test the sub TARGET of kvm. In theory, it won't break other TARGET of selftests. Changes in v2: - fix the no directory issue in lib.mk - make kvm fixes seperate patch - Add the patch to fix linux src tree not clean issue v1: https://lore.kernel.org/kvm/20200315093425.33600-1-xiaoyao.li@intel.com/ Xiaoyao Li (6): selftests: Create directory when OUTPUT specified selftests: kvm: Include lib.mk earlier selftests: kvm: Use the default linux header path only when INSTALL_HDR_PATH not defined selftests: Create variable INSTALL_HDR_PATH if need to install linux headers to $(OUTPUT)/usr selftests: Generate build output of linux headers to $(OUTPUT)/linux-header-build selftests: export INSTALL_HDR_PATH if using "O" to specify output dir tools/testing/selftests/Makefile | 6 +++++- tools/testing/selftests/kvm/Makefile | 9 +++++---- tools/testing/selftests/lib.mk | 19 ++++++++++++++++++- 3 files changed, 28 insertions(+), 6 deletions(-) -- 2.20.1

5 years, 3 months

2
8
0 0

[PATCH] selftests: enforce local header dependency in lib.mk

by Shuah Khan

Add local header dependency in lib.mk. This enforces the dependency blindly even when a test doesn't include the file, with the benefit of a simpler common logic without requiring individual tests to have special rule for it. Signed-off-by: Shuah Khan <skhan(a)linuxfoundation.org> --- tools/testing/selftests/lib.mk | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/lib.mk b/tools/testing/selftests/lib.mk index 3ed0134a764d..b0556c752443 100644 --- a/tools/testing/selftests/lib.mk +++ b/tools/testing/selftests/lib.mk @@ -137,7 +137,8 @@ endif # Selftest makefiles can override those targets by setting # OVERRIDE_TARGETS = 1. ifeq ($(OVERRIDE_TARGETS),) -$(OUTPUT)/%:%.c +LOCAL_HDRS := $(selfdir)/kselftest_harness.h $(selfdir)/kselftest.h +$(OUTPUT)/%:%.c $(LOCAL_HDRS) $(LINK.c) $^ $(LDLIBS) -o $@ $(OUTPUT)/%.o:%.S -- 2.20.1

5 years, 3 months

2
1
0 0

[PATCH v4] selftests: Fix seccomp to support relocatable build (O=objdir)

by Shuah Khan

Fix seccomp relocatable builds. This is a simple fix to use the right lib.mk variable TEST_GEN_PROGS. Local header dependency is addressed in a change to lib.mk as a framework change that enforces the dependency without requiring changes to individual tests. The following use-cases work with this change: In seccomp directory: make all and make clean >From top level from main Makefile: make kselftest-install O=objdir ARCH=arm64 HOSTCC=gcc \ CROSS_COMPILE=aarch64-linux-gnu- TARGETS=seccomp Signed-off-by: Shuah Khan <skhan(a)linuxfoundation.org> --- Changes since v3: Simplified logic based on comments from Kees and Michael tools/testing/selftests/seccomp/Makefile | 17 +++-------------- 1 file changed, 3 insertions(+), 14 deletions(-) diff --git a/tools/testing/selftests/seccomp/Makefile b/tools/testing/selftests/seccomp/Makefile index 1760b3e39730..0ebfe8b0e147 100644 --- a/tools/testing/selftests/seccomp/Makefile +++ b/tools/testing/selftests/seccomp/Makefile @@ -1,17 +1,6 @@ # SPDX-License-Identifier: GPL-2.0 -all: - -include ../lib.mk - -.PHONY: all clean - -BINARIES := seccomp_bpf seccomp_benchmark CFLAGS += -Wl,-no-as-needed -Wall +LDFLAGS += -lpthread -seccomp_bpf: seccomp_bpf.c ../kselftest_harness.h - $(CC) $(CFLAGS) $(LDFLAGS) $< -lpthread -o $@ - -TEST_PROGS += $(BINARIES) -EXTRA_CLEAN := $(BINARIES) - -all: $(BINARIES) +TEST_GEN_PROGS := seccomp_bpf seccomp_benchmark +include ../lib.mk -- 2.20.1

5 years, 3 months

2
1
0 0

[PATCH v7 kunit-next 0/4] kunit: add debugfs representation to show results

by Alan Maguire

When kunit tests are run on native (i.e. non-UML) environments, the results of test execution are often intermixed with dmesg output. This patch series attempts to solve this by providing a debugfs representation of the results of the last test run, available as /sys/kernel/debug/kunit/<testsuite>/results Changes since v6: - fixed regexp parsing in kunit_parser.py to ensure test results are read successfully with 4-space indentation (Brendan, patch 3) Changes since v5: - replaced undefined behaviour use of snprintf(buf, ..., buf) in kunit_log() with a function to append string to existing log (Frank, patch 1) - added clarification on log size limitations to documentation (Frank, patch 4) Changes since v4: - added suite-level log expectations to kunit log test (Brendan, patch 2) - added log expectations (of it being NULL) for case where CONFIG_KUNIT_DEBUGFS=n to kunit log test (patch 2) - added patch 3 which replaces subtest tab indentation with 4 space indentation as per TAP 14 spec (Frank, patch 3) Changes since v3: - added CONFIG_KUNIT_DEBUGFS to support conditional compilation of debugfs representation, including string logging (Frank, patch 1) - removed unneeded NULL check for test_case in kunit_suite_for_each_test_case() (Frank, patch 1) - added kunit log test to verify logging multiple strings works (Frank, patch 2) - rephrased description of results file (Frank, patch 3) Changes since v2: - updated kunit_status2str() to kunit_status_to_string() and made it static inline in include/kunit/test.h (Brendan) - added log string to struct kunit_suite and kunit_case, with log pointer in struct kunit pointing at the case log. This allows us to collect kunit_[err|info|warning]() messages at the same time as we printk() them. This solves for the most part the sharing of log messages between test execution and debugfs since we just print the suite log (which contains the test suite preamble) and the individual test logs. The only exception is the suite-level status, which we cannot store in the suite log as it would mean we'd print the suite and its status prior to the suite's results. (Brendan, patch 1) - dropped debugfs-based kunit run patch for now so as not to cause problems with tests currently under development (Brendan) - fixed doc issues with code block (Brendan, patch 3) Changes since v1: - trimmed unneeded include files in lib/kunit/debugfs.c (Greg) - renamed global debugfs functions to be prefixed with kunit_ (Greg) - removed error checking for debugfs operations (Greg) Alan Maguire (4): kunit: add debugfs /sys/kernel/debug/kunit/<suite>/results display kunit: add log test kunit: subtests should be indented 4 spaces according to TAP kunit: update documentation to describe debugfs representation Documentation/dev-tools/kunit/usage.rst | 14 +++ include/kunit/test.h | 59 +++++++++++-- lib/kunit/Kconfig | 8 ++ lib/kunit/Makefile | 4 + lib/kunit/assert.c | 79 ++++++++--------- lib/kunit/debugfs.c | 116 +++++++++++++++++++++++++ lib/kunit/debugfs.h | 30 +++++++ lib/kunit/kunit-test.c | 45 +++++++++- lib/kunit/test.c | 147 +++++++++++++++++++++++++------- tools/testing/kunit/kunit_parser.py | 10 +-- 10 files changed, 426 insertions(+), 86 deletions(-) create mode 100644 lib/kunit/debugfs.c create mode 100644 lib/kunit/debugfs.h -- 1.8.3.1

5 years, 3 months

3
9
0 0

[PATCH net] selftests/net/forwarding: define libs as TEST_PROGS_EXTENDED

by Hangbin Liu

The lib files should not be defined as TEST_PROGS, or we will run them in run_kselftest.sh. Also remove ethtool_lib.sh exec permission. Fixes: 81573b18f26d ("selftests/net/forwarding: add Makefile to install tests") Signed-off-by: Hangbin Liu <liuhangbin(a)gmail.com> --- .../testing/selftests/net/forwarding/Makefile | 31 ++++++++++--------- .../selftests/net/forwarding/ethtool_lib.sh | 0 2 files changed, 16 insertions(+), 15 deletions(-) mode change 100755 => 100644 tools/testing/selftests/net/forwarding/ethtool_lib.sh diff --git a/tools/testing/selftests/net/forwarding/Makefile b/tools/testing/selftests/net/forwarding/Makefile index 44616103508b..250fbb2d1625 100644 --- a/tools/testing/selftests/net/forwarding/Makefile +++ b/tools/testing/selftests/net/forwarding/Makefile @@ -5,11 +5,7 @@ TEST_PROGS = bridge_igmp.sh \ bridge_sticky_fdb.sh \ bridge_vlan_aware.sh \ bridge_vlan_unaware.sh \ - devlink_lib.sh \ - ethtool_lib.sh \ ethtool.sh \ - fib_offload_lib.sh \ - forwarding.config.sample \ gre_inner_v4_multipath.sh \ gre_inner_v6_multipath.sh \ gre_multipath.sh \ @@ -21,8 +17,6 @@ TEST_PROGS = bridge_igmp.sh \ ipip_hier_gre_key.sh \ ipip_hier_gre_keys.sh \ ipip_hier_gre.sh \ - ipip_lib.sh \ - lib.sh \ loopback.sh \ mirror_gre_bound.sh \ mirror_gre_bridge_1d.sh \ @@ -32,15 +26,11 @@ TEST_PROGS = bridge_igmp.sh \ mirror_gre_changes.sh \ mirror_gre_flower.sh \ mirror_gre_lag_lacp.sh \ - mirror_gre_lib.sh \ mirror_gre_neigh.sh \ mirror_gre_nh.sh \ mirror_gre.sh \ - mirror_gre_topo_lib.sh \ mirror_gre_vlan_bridge_1q.sh \ mirror_gre_vlan.sh \ - mirror_lib.sh \ - mirror_topo_lib.sh \ mirror_vlan.sh \ router_bridge.sh \ router_bridge_vlan.sh \ @@ -50,17 +40,12 @@ TEST_PROGS = bridge_igmp.sh \ router_multipath.sh \ router.sh \ router_vid_1.sh \ - sch_ets_core.sh \ sch_ets.sh \ - sch_ets_tests.sh \ - sch_tbf_core.sh \ - sch_tbf_etsprio.sh \ sch_tbf_ets.sh \ sch_tbf_prio.sh \ sch_tbf_root.sh \ tc_actions.sh \ tc_chains.sh \ - tc_common.sh \ tc_flower_router.sh \ tc_flower.sh \ tc_shblocks.sh \ @@ -72,4 +57,20 @@ TEST_PROGS = bridge_igmp.sh \ vxlan_bridge_1q.sh \ vxlan_symmetric.sh +TEST_PROGS_EXTENDED := devlink_lib.sh \ + ethtool_lib.sh \ + fib_offload_lib.sh \ + forwarding.config.sample \ + ipip_lib.sh \ + lib.sh \ + mirror_gre_lib.sh \ + mirror_gre_topo_lib.sh \ + mirror_lib.sh \ + mirror_topo_lib.sh \ + sch_ets_core.sh \ + sch_ets_tests.sh \ + sch_tbf_core.sh \ + sch_tbf_etsprio.sh \ + tc_common.sh + include ../../lib.mk diff --git a/tools/testing/selftests/net/forwarding/ethtool_lib.sh b/tools/testing/selftests/net/forwarding/ethtool_lib.sh old mode 100755 new mode 100644 -- 2.19.2

5 years, 3 months

3
3
0 0

[RFC PATCH kunit-next] kunit: kunit_tool: Separate out config/build/exec/parse

by David Gow

Add new subcommands to kunit.py to allow stages of the existing 'run' subcommand to be run independently: - 'config': Verifies that .config is a subset of .kunitconfig - 'build': Compiles a UML kernel for KUnit - 'exec': Runs the kernel, and outputs the test results. - 'parse': Parses test results from a file or stdin Most of these are not hugely useful by themselves yet, and there's room for plenty of bikeshedding on the names, but this hopefully can form a foundation for future improvements. Signed-off-by: David Gow <davidgow(a)google.com> --- [Whoops: typo-ed Brendan's email. Sorry about that!] As was briefly disccussed in [1], this change is part of a "separation of concerns" in kunit_tool. This should make it easier to integrate kunit_tool into other setups. Of particular intrest is probably 'kunit.py parse', which should allow KUnit results to be parsed from other sources, such as after loading a module, or from a non-UML kernel, or from debugfs when that's supported[2]. [1]: https://lkml.org/lkml/2020/2/5/552 [2]: https://patchwork.kernel.org/patch/11419901/ tools/testing/kunit/kunit.py | 236 ++++++++++++++++++++----- tools/testing/kunit/kunit_tool_test.py | 55 ++++++ 2 files changed, 242 insertions(+), 49 deletions(-) diff --git a/tools/testing/kunit/kunit.py b/tools/testing/kunit/kunit.py index 180ad1e1b04f..92a634594cf6 100755 --- a/tools/testing/kunit/kunit.py +++ b/tools/testing/kunit/kunit.py @@ -20,8 +20,12 @@ import kunit_config import kunit_kernel import kunit_parser -KunitResult = namedtuple('KunitResult', ['status','result']) +KunitResult = namedtuple('KunitResult', ['status','result','elapsed_time']) +KunitConfigRequest = namedtuple('KunitConfigRequest', ['build_dir', 'defconfig']) +KunitBuildRequest = namedtuple('KunitBuildRequest', ['jobs', 'build_dir']) +KunitExecRequest = namedtuple('KunitExecRequest', ['timeout', 'build_dir']) +KunitParseRequest = namedtuple('KunitParseRequest', ['raw_output', 'input_data']) KunitRequest = namedtuple('KunitRequest', ['raw_output','timeout', 'jobs', 'build_dir', 'defconfig']) KernelDirectoryPath = sys.argv[0].split('tools/testing/kunit/')[0] @@ -44,14 +48,25 @@ def get_kernel_root_path(): sys.exit(1) return parts[0] -def run_tests(linux: kunit_kernel.LinuxSourceTree, - request: KunitRequest) -> KunitResult: +def config_tests(linux: kunit_kernel.LinuxSourceTree, + request: KunitConfigRequest) -> KunitResult: + kunit_parser.print_with_timestamp('Configuring KUnit Kernel ...') + config_start = time.time() + if request.defconfig: + create_default_kunitconfig() success = linux.build_reconfig(request.build_dir) config_end = time.time() if not success: - return KunitResult(KunitStatus.CONFIG_FAILURE, 'could not configure kernel') + return KunitResult(KunitStatus.CONFIG_FAILURE, + 'could not configure kernel', + config_end - config_start) + return KunitResult(KunitStatus.SUCCESS, + 'configured kernel successfully', + config_end - config_start) +def build_tests(linux: kunit_kernel.LinuxSourceTree, + request: KunitBuildRequest) -> KunitResult: kunit_parser.print_with_timestamp('Building KUnit Kernel ...') build_start = time.time() @@ -59,73 +74,156 @@ def run_tests(linux: kunit_kernel.LinuxSourceTree, build_end = time.time() if not success: return KunitResult(KunitStatus.BUILD_FAILURE, 'could not build kernel') + if not success: + return KunitResult(KunitStatus.BUILD_FAILURE, + 'could not build kernel', + build_end - build_start) + return KunitResult(KunitStatus.SUCCESS, + 'built kernel successfully', + build_end - build_start) +def exec_tests(linux: kunit_kernel.LinuxSourceTree, + request: KunitExecRequest) -> KunitResult: kunit_parser.print_with_timestamp('Starting KUnit Kernel ...') test_start = time.time() + result = linux.run_kernel(timeout=request.timeout, + build_dir=request.build_dir) + test_end = time.time() + + return KunitResult(KunitStatus.SUCCESS, + result, + test_end - test_start) + +def parse_tests(request: KunitParseRequest) -> KunitResult: + parse_start = time.time() + test_result = kunit_parser.TestResult(kunit_parser.TestStatus.SUCCESS, [], 'Tests not Parsed.') if request.raw_output: - kunit_parser.raw_output( - linux.run_kernel(timeout=request.timeout, - build_dir=request.build_dir)) + kunit_parser.raw_output(request.input_data) else: - kunit_output = linux.run_kernel(timeout=request.timeout, - build_dir=request.build_dir) - test_result = kunit_parser.parse_run_tests(kunit_output) - test_end = time.time() + test_result = kunit_parser.parse_run_tests(request.input_data) + parse_end = time.time() + + if test_result.status != kunit_parser.TestStatus.SUCCESS: + return KunitResult(KunitStatus.TEST_FAILURE, test_result, + parse_end - parse_start) + + return KunitResult(KunitStatus.SUCCESS, test_result, + parse_end - parse_start) + + +def run_tests(linux: kunit_kernel.LinuxSourceTree, + request: KunitRequest) -> KunitResult: + run_start = time.time() + + config_request = KunitConfigRequest(request.build_dir, request.defconfig) + config_result = config_tests(linux, config_request) + if config_result.status != KunitStatus.SUCCESS: + return config_result + + build_request = KunitBuildRequest(request.jobs, request.build_dir) + build_result = build_tests(linux, build_request) + if build_result.status != KunitStatus.SUCCESS: + return build_result + + exec_request = KunitExecRequest(request.timeout, request.build_dir) + exec_result = exec_tests(linux, exec_request) + if exec_result.status != KunitStatus.SUCCESS: + return exec_result + + parse_request = KunitParseRequest(request.raw_output, exec_result.result) + parse_result = parse_tests(parse_request) + + run_end = time.time() kunit_parser.print_with_timestamp(( 'Elapsed time: %.3fs total, %.3fs configuring, %.3fs ' + 'building, %.3fs running\n') % ( - test_end - config_start, - config_end - config_start, - build_end - build_start, - test_end - test_start)) + run_end - run_start, + config_result.elapsed_time, + build_result.elapsed_time, + exec_result.elapsed_time)) + return parse_result + +def add_common_opts(parser): + parser.add_argument('--build_dir', + help='As in the make command, it specifies the build ' + 'directory.', + type=str, default='', metavar='build_dir') + +def add_config_opts(parser): + parser.add_argument('--defconfig', + help='Uses a default .kunitconfig.', + action='store_true') + +def add_build_opts(parser): + parser.add_argument('--jobs', + help='As in the make command, "Specifies the number of ' + 'jobs (commands) to run simultaneously."', + type=int, default=8, metavar='jobs') + +def add_exec_opts(parser): + parser.add_argument('--timeout', + help='maximum number of seconds to allow for all tests ' + 'to run. This does not include time taken to build the ' + 'tests.', + type=int, + default=300, + metavar='timeout') + +def add_parse_opts(parser): + parser.add_argument('--raw_output', help='don\'t format output from kernel', + action='store_true') - if test_result.status != kunit_parser.TestStatus.SUCCESS: - return KunitResult(KunitStatus.TEST_FAILURE, test_result) - else: - return KunitResult(KunitStatus.SUCCESS, test_result) def main(argv, linux=None): parser = argparse.ArgumentParser( description='Helps writing and running KUnit tests.') subparser = parser.add_subparsers(dest='subcommand') + # The 'run' command will config, build, exec, and parse in one go. run_parser = subparser.add_parser('run', help='Runs KUnit tests.') - run_parser.add_argument('--raw_output', help='don\'t format output from kernel', - action='store_true') + add_common_opts(run_parser) + add_config_opts(run_parser) + add_build_opts(run_parser) + add_exec_opts(run_parser) + add_parse_opts(run_parser) - run_parser.add_argument('--timeout', - help='maximum number of seconds to allow for all tests ' - 'to run. This does not include time taken to build the ' - 'tests.', - type=int, - default=300, - metavar='timeout') - - run_parser.add_argument('--jobs', - help='As in the make command, "Specifies the number of ' - 'jobs (commands) to run simultaneously."', - type=int, default=8, metavar='jobs') - - run_parser.add_argument('--build_dir', - help='As in the make command, it specifies the build ' - 'directory.', - type=str, default='', metavar='build_dir') - - run_parser.add_argument('--defconfig', - help='Uses a default .kunitconfig.', - action='store_true') + config_parser = subparser.add_parser('config', + help='Ensures that .config contains all of ' + 'the options in .kunitconfig') + add_common_opts(config_parser) + add_config_opts(config_parser) - cli_args = parser.parse_args(argv) + build_parser = subparser.add_parser('build', help='Builds a kernel with KUnit tests') + add_common_opts(build_parser) + add_build_opts(build_parser) - if cli_args.subcommand == 'run': - if get_kernel_root_path(): - os.chdir(get_kernel_root_path()) + exec_parser = subparser.add_parser('exec', help='Run a kernel with KUnit tests') + add_common_opts(exec_parser) + add_exec_opts(exec_parser) + add_parse_opts(exec_parser) + + # The 'parse' option is special, as it doesn't need the kernel source + # (therefore there is no need for a build_dir, hence no add_common_opts) + # and the '--file' argument is not relevant to 'run', so isn't in + # add_parse_opts() + parse_parser = subparser.add_parser('parse', + help='Parses KUnit results from a file, ' + 'and parses formatted results.') + add_parse_opts(parse_parser) + parse_parser.add_argument('file', + help='Specifies the file to read results from.', + type=str, default='-', metavar='input_file') + + cli_args = parser.parse_args(argv) + # Set up the build_dir and source tree for commands which use it + # (everything but 'parse') + if cli_args.subcommand != 'parse': if cli_args.build_dir: if not os.path.exists(cli_args.build_dir): os.mkdir(cli_args.build_dir) @@ -133,12 +231,11 @@ def main(argv, linux=None): cli_args.build_dir, kunit_kernel.kunitconfig_path) - if cli_args.defconfig: - create_default_kunitconfig() - if not linux: linux = kunit_kernel.LinuxSourceTree() + + if cli_args.subcommand == 'run': request = KunitRequest(cli_args.raw_output, cli_args.timeout, cli_args.jobs, @@ -147,6 +244,47 @@ def main(argv, linux=None): result = run_tests(linux, request) if result.status != KunitStatus.SUCCESS: sys.exit(1) + elif cli_args.subcommand == 'config': + request = KunitConfigRequest(cli_args.build_dir, + cli_args.defconfig) + result = config_tests(linux, request) + kunit_parser.print_with_timestamp(( + 'Elapsed time: %.3fs\n') % ( + result.elapsed_time)) + if result.status != KunitStatus.SUCCESS: + sys.exit(1) + elif cli_args.subcommand == 'build': + request = KunitBuildRequest(cli_args.jobs, + cli_args.build_dir) + result = build_tests(linux, request) + kunit_parser.print_with_timestamp(( + 'Elapsed time: %.3fs\n') % ( + result.elapsed_time)) + if result.status != KunitStatus.SUCCESS: + sys.exit(1) + elif cli_args.subcommand == 'exec': + exec_request = KunitExecRequest(cli_args.timeout, + cli_args.build_dir) + exec_result = exec_tests(linux, exec_request) + parse_request = KunitParseRequest(cli_args.raw_output, + exec_result.result) + result = parse_tests(parse_request) + kunit_parser.print_with_timestamp(( + 'Elapsed time: %.3fs\n') % ( + exec_result.elapsed_time)) + if result.status != KunitStatus.SUCCESS: + sys.exit(1) + elif cli_args.subcommand == 'parse': + if cli_args.file == '-': + kunit_output = sys.stdin + else: + with open(cli_args.file, 'r') as f: + kunit_output = f.read().splitlines() + request = KunitParseRequest(cli_args.raw_output, + kunit_output) + result = parse_tests(request) + if result.status != KunitStatus.SUCCESS: + sys.exit(1) else: parser.print_help() diff --git a/tools/testing/kunit/kunit_tool_test.py b/tools/testing/kunit/kunit_tool_test.py index cba97756ac4a..0d04425ead04 100755 --- a/tools/testing/kunit/kunit_tool_test.py +++ b/tools/testing/kunit/kunit_tool_test.py @@ -170,6 +170,24 @@ class KUnitMainTest(unittest.TestCase): self.print_patch.stop() pass + def test_config_passes_args_pass(self): + kunit.main(['config'], self.linux_source_mock) + assert self.linux_source_mock.build_reconfig.call_count == 1 + assert self.linux_source_mock.run_kernel.call_count == 0 + + def test_build_passes_args_pass(self): + kunit.main(['build'], self.linux_source_mock) + assert self.linux_source_mock.build_reconfig.call_count == 0 + self.linux_source_mock.build_um_kernel.assert_called_once_with(8, '') + assert self.linux_source_mock.run_kernel.call_count == 0 + + def test_exec_passes_args_pass(self): + kunit.main(['exec'], self.linux_source_mock) + assert self.linux_source_mock.build_reconfig.call_count == 0 + assert self.linux_source_mock.run_kernel.call_count == 1 + self.linux_source_mock.run_kernel.assert_called_once_with(build_dir='', timeout=300) + self.print_mock.assert_any_call(StrContains('Testing complete.')) + def test_run_passes_args_pass(self): kunit.main(['run'], self.linux_source_mock) assert self.linux_source_mock.build_reconfig.call_count == 1 @@ -177,6 +195,13 @@ class KUnitMainTest(unittest.TestCase): self.linux_source_mock.run_kernel.assert_called_once_with(build_dir='', timeout=300) self.print_mock.assert_any_call(StrContains('Testing complete.')) + def test_exec_passes_args_fail(self): + self.linux_source_mock.run_kernel = mock.Mock(return_value=[]) + with self.assertRaises(SystemExit) as e: + kunit.main(['exec'], self.linux_source_mock) + assert type(e.exception) == SystemExit + assert e.exception.code == 1 + def test_run_passes_args_fail(self): self.linux_source_mock.run_kernel = mock.Mock(return_value=[]) with self.assertRaises(SystemExit) as e: @@ -187,6 +212,14 @@ class KUnitMainTest(unittest.TestCase): assert self.linux_source_mock.run_kernel.call_count == 1 self.print_mock.assert_any_call(StrContains(' 0 tests run')) + def test_exec_raw_output(self): + self.linux_source_mock.run_kernel = mock.Mock(return_value=[]) + kunit.main(['exec', '--raw_output'], self.linux_source_mock) + assert self.linux_source_mock.run_kernel.call_count == 1 + for kall in self.print_mock.call_args_list: + assert kall != mock.call(StrContains('Testing complete.')) + assert kall != mock.call(StrContains(' 0 tests run')) + def test_run_raw_output(self): self.linux_source_mock.run_kernel = mock.Mock(return_value=[]) kunit.main(['run', '--raw_output'], self.linux_source_mock) @@ -196,6 +229,12 @@ class KUnitMainTest(unittest.TestCase): assert kall != mock.call(StrContains('Testing complete.')) assert kall != mock.call(StrContains(' 0 tests run')) + def test_exec_timeout(self): + timeout = 3453 + kunit.main(['exec', '--timeout', str(timeout)], self.linux_source_mock) + self.linux_source_mock.run_kernel.assert_called_once_with(build_dir='', timeout=timeout) + self.print_mock.assert_any_call(StrContains('Testing complete.')) + def test_run_timeout(self): timeout = 3453 kunit.main(['run', '--timeout', str(timeout)], self.linux_source_mock) @@ -210,5 +249,21 @@ class KUnitMainTest(unittest.TestCase): self.linux_source_mock.run_kernel.assert_called_once_with(build_dir=build_dir, timeout=300) self.print_mock.assert_any_call(StrContains('Testing complete.')) + def test_config_builddir(self): + build_dir = '.kunit' + kunit.main(['config', '--build_dir', build_dir], self.linux_source_mock) + assert self.linux_source_mock.build_reconfig.call_count == 1 + + def test_build_builddir(self): + build_dir = '.kunit' + kunit.main(['build', '--build_dir', build_dir], self.linux_source_mock) + self.linux_source_mock.build_um_kernel.assert_called_once_with(8, build_dir) + + def test_exec_builddir(self): + build_dir = '.kunit' + kunit.main(['exec', '--build_dir', build_dir], self.linux_source_mock) + self.linux_source_mock.run_kernel.assert_called_once_with(build_dir=build_dir, timeout=300) + self.print_mock.assert_any_call(StrContains('Testing complete.')) + if __name__ == '__main__': unittest.main() -- 2.25.1.481.gfbce0eb801-goog

5 years, 3 months

2
1
0 0

[PATCH kselftest/test] kunit: Always print actual pointer values in asserts

by David Gow

KUnit assertions and expectations will print the values being tested. If these are pointers (e.g., KUNIT_EXPECT_PTR_EQ(test, a, b)), these pointers are currently printed with the %pK format specifier, which -- to prevent information leaks which may compromise, e.g., ASLR -- are often either hashed or replaced with ____ptrval____ or similar, making debugging tests difficult. By replacing %pK with %px as Documentation/core-api/printk-formats.rst suggests, we disable this security feature for KUnit assertions and expectations, allowing the actual pointer values to be printed. Given that KUnit is not intended for use in production kernels, and the pointers are only printed on failing tests, this seems like a worthwhile tradeoff. Signed-off-by: David Gow <davidgow(a)google.com> --- This seems like the best way of solving this problem to me, but if anyone has a better solution I'd love to hear it. Note also that this does trigger two checkpatch.pl warnings, which warn that the change will potentially cause the kernel memory layout to be exposed. Since that's the whole point of the change, they probably sohuld stay there. lib/kunit/assert.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/lib/kunit/assert.c b/lib/kunit/assert.c index 86013d4cf891..a87960409bd4 100644 --- a/lib/kunit/assert.c +++ b/lib/kunit/assert.c @@ -110,10 +110,10 @@ void kunit_binary_ptr_assert_format(const struct kunit_assert *assert, binary_assert->left_text, binary_assert->operation, binary_assert->right_text); - string_stream_add(stream, "\t\t%s == %pK\n", + string_stream_add(stream, "\t\t%s == %px\n", binary_assert->left_text, binary_assert->left_value); - string_stream_add(stream, "\t\t%s == %pK", + string_stream_add(stream, "\t\t%s == %px", binary_assert->right_text, binary_assert->right_value); kunit_assert_print_msg(assert, stream); -- 2.24.0.432.g9d3f5f5b63-goog

5 years, 3 months

3
4
0 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

Linux-kselftest-mirror