- Linux-kselftest-mirror - lists.linaro.org

[PATCH v2 0/6] Fix errors when try to build kvm selftests on specified output

by Xiaoyao Li

I attempted to build KVM selftests on a specified dir, unfortunately neither "make O=/path/to/mydir TARGETS=kvm" in tools/testing/selftests, nor "make OUTPUT=/path/to/mydir" in tools/testing/selftests/kvm work. This series aims to fix them. Patch 1 fixes the issue that output directory is not exist. Patch 2 and 3 are the preparation for kvm to get the right path of installed linux headers. Patch 4 and 6 prepare the INSTALL_HDR_PATH to tell sub TARGET where the linux headers are installed. Patch 5 fixes the issue that with OUTPUT specified, it still make the linux tree dirty. I only test the sub TARGET of kvm. In theory, it won't break other TARGET of selftests. Changes in v2: - fix the no directory issue in lib.mk - make kvm fixes seperate patch - Add the patch to fix linux src tree not clean issue v1: https://lore.kernel.org/kvm/20200315093425.33600-1-xiaoyao.li@intel.com/ Xiaoyao Li (6): selftests: Create directory when OUTPUT specified selftests: kvm: Include lib.mk earlier selftests: kvm: Use the default linux header path only when INSTALL_HDR_PATH not defined selftests: Create variable INSTALL_HDR_PATH if need to install linux headers to $(OUTPUT)/usr selftests: Generate build output of linux headers to $(OUTPUT)/linux-header-build selftests: export INSTALL_HDR_PATH if using "O" to specify output dir tools/testing/selftests/Makefile | 6 +++++- tools/testing/selftests/kvm/Makefile | 9 +++++---- tools/testing/selftests/lib.mk | 19 ++++++++++++++++++- 3 files changed, 28 insertions(+), 6 deletions(-) -- 2.20.1

5 years, 1 month

2
8
0 0

[PATCH] selftests: enforce local header dependency in lib.mk

by Shuah Khan

Add local header dependency in lib.mk. This enforces the dependency blindly even when a test doesn't include the file, with the benefit of a simpler common logic without requiring individual tests to have special rule for it. Signed-off-by: Shuah Khan <skhan(a)linuxfoundation.org> --- tools/testing/selftests/lib.mk | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/lib.mk b/tools/testing/selftests/lib.mk index 3ed0134a764d..b0556c752443 100644 --- a/tools/testing/selftests/lib.mk +++ b/tools/testing/selftests/lib.mk @@ -137,7 +137,8 @@ endif # Selftest makefiles can override those targets by setting # OVERRIDE_TARGETS = 1. ifeq ($(OVERRIDE_TARGETS),) -$(OUTPUT)/%:%.c +LOCAL_HDRS := $(selfdir)/kselftest_harness.h $(selfdir)/kselftest.h +$(OUTPUT)/%:%.c $(LOCAL_HDRS) $(LINK.c) $^ $(LDLIBS) -o $@ $(OUTPUT)/%.o:%.S -- 2.20.1

5 years, 1 month

2
1
0 0

[PATCH v4] selftests: Fix seccomp to support relocatable build (O=objdir)

by Shuah Khan

Fix seccomp relocatable builds. This is a simple fix to use the right lib.mk variable TEST_GEN_PROGS. Local header dependency is addressed in a change to lib.mk as a framework change that enforces the dependency without requiring changes to individual tests. The following use-cases work with this change: In seccomp directory: make all and make clean >From top level from main Makefile: make kselftest-install O=objdir ARCH=arm64 HOSTCC=gcc \ CROSS_COMPILE=aarch64-linux-gnu- TARGETS=seccomp Signed-off-by: Shuah Khan <skhan(a)linuxfoundation.org> --- Changes since v3: Simplified logic based on comments from Kees and Michael tools/testing/selftests/seccomp/Makefile | 17 +++-------------- 1 file changed, 3 insertions(+), 14 deletions(-) diff --git a/tools/testing/selftests/seccomp/Makefile b/tools/testing/selftests/seccomp/Makefile index 1760b3e39730..0ebfe8b0e147 100644 --- a/tools/testing/selftests/seccomp/Makefile +++ b/tools/testing/selftests/seccomp/Makefile @@ -1,17 +1,6 @@ # SPDX-License-Identifier: GPL-2.0 -all: - -include ../lib.mk - -.PHONY: all clean - -BINARIES := seccomp_bpf seccomp_benchmark CFLAGS += -Wl,-no-as-needed -Wall +LDFLAGS += -lpthread -seccomp_bpf: seccomp_bpf.c ../kselftest_harness.h - $(CC) $(CFLAGS) $(LDFLAGS) $< -lpthread -o $@ - -TEST_PROGS += $(BINARIES) -EXTRA_CLEAN := $(BINARIES) - -all: $(BINARIES) +TEST_GEN_PROGS := seccomp_bpf seccomp_benchmark +include ../lib.mk -- 2.20.1

5 years, 1 month

2
1
0 0

[PATCH v7 kunit-next 0/4] kunit: add debugfs representation to show results

by Alan Maguire

When kunit tests are run on native (i.e. non-UML) environments, the results of test execution are often intermixed with dmesg output. This patch series attempts to solve this by providing a debugfs representation of the results of the last test run, available as /sys/kernel/debug/kunit/<testsuite>/results Changes since v6: - fixed regexp parsing in kunit_parser.py to ensure test results are read successfully with 4-space indentation (Brendan, patch 3) Changes since v5: - replaced undefined behaviour use of snprintf(buf, ..., buf) in kunit_log() with a function to append string to existing log (Frank, patch 1) - added clarification on log size limitations to documentation (Frank, patch 4) Changes since v4: - added suite-level log expectations to kunit log test (Brendan, patch 2) - added log expectations (of it being NULL) for case where CONFIG_KUNIT_DEBUGFS=n to kunit log test (patch 2) - added patch 3 which replaces subtest tab indentation with 4 space indentation as per TAP 14 spec (Frank, patch 3) Changes since v3: - added CONFIG_KUNIT_DEBUGFS to support conditional compilation of debugfs representation, including string logging (Frank, patch 1) - removed unneeded NULL check for test_case in kunit_suite_for_each_test_case() (Frank, patch 1) - added kunit log test to verify logging multiple strings works (Frank, patch 2) - rephrased description of results file (Frank, patch 3) Changes since v2: - updated kunit_status2str() to kunit_status_to_string() and made it static inline in include/kunit/test.h (Brendan) - added log string to struct kunit_suite and kunit_case, with log pointer in struct kunit pointing at the case log. This allows us to collect kunit_[err|info|warning]() messages at the same time as we printk() them. This solves for the most part the sharing of log messages between test execution and debugfs since we just print the suite log (which contains the test suite preamble) and the individual test logs. The only exception is the suite-level status, which we cannot store in the suite log as it would mean we'd print the suite and its status prior to the suite's results. (Brendan, patch 1) - dropped debugfs-based kunit run patch for now so as not to cause problems with tests currently under development (Brendan) - fixed doc issues with code block (Brendan, patch 3) Changes since v1: - trimmed unneeded include files in lib/kunit/debugfs.c (Greg) - renamed global debugfs functions to be prefixed with kunit_ (Greg) - removed error checking for debugfs operations (Greg) Alan Maguire (4): kunit: add debugfs /sys/kernel/debug/kunit/<suite>/results display kunit: add log test kunit: subtests should be indented 4 spaces according to TAP kunit: update documentation to describe debugfs representation Documentation/dev-tools/kunit/usage.rst | 14 +++ include/kunit/test.h | 59 +++++++++++-- lib/kunit/Kconfig | 8 ++ lib/kunit/Makefile | 4 + lib/kunit/assert.c | 79 ++++++++--------- lib/kunit/debugfs.c | 116 +++++++++++++++++++++++++ lib/kunit/debugfs.h | 30 +++++++ lib/kunit/kunit-test.c | 45 +++++++++- lib/kunit/test.c | 147 +++++++++++++++++++++++++------- tools/testing/kunit/kunit_parser.py | 10 +-- 10 files changed, 426 insertions(+), 86 deletions(-) create mode 100644 lib/kunit/debugfs.c create mode 100644 lib/kunit/debugfs.h -- 1.8.3.1

5 years, 1 month

3
9
0 0

[PATCH net] selftests/net/forwarding: define libs as TEST_PROGS_EXTENDED

by Hangbin Liu

The lib files should not be defined as TEST_PROGS, or we will run them in run_kselftest.sh. Also remove ethtool_lib.sh exec permission. Fixes: 81573b18f26d ("selftests/net/forwarding: add Makefile to install tests") Signed-off-by: Hangbin Liu <liuhangbin(a)gmail.com> --- .../testing/selftests/net/forwarding/Makefile | 31 ++++++++++--------- .../selftests/net/forwarding/ethtool_lib.sh | 0 2 files changed, 16 insertions(+), 15 deletions(-) mode change 100755 => 100644 tools/testing/selftests/net/forwarding/ethtool_lib.sh diff --git a/tools/testing/selftests/net/forwarding/Makefile b/tools/testing/selftests/net/forwarding/Makefile index 44616103508b..250fbb2d1625 100644 --- a/tools/testing/selftests/net/forwarding/Makefile +++ b/tools/testing/selftests/net/forwarding/Makefile @@ -5,11 +5,7 @@ TEST_PROGS = bridge_igmp.sh \ bridge_sticky_fdb.sh \ bridge_vlan_aware.sh \ bridge_vlan_unaware.sh \ - devlink_lib.sh \ - ethtool_lib.sh \ ethtool.sh \ - fib_offload_lib.sh \ - forwarding.config.sample \ gre_inner_v4_multipath.sh \ gre_inner_v6_multipath.sh \ gre_multipath.sh \ @@ -21,8 +17,6 @@ TEST_PROGS = bridge_igmp.sh \ ipip_hier_gre_key.sh \ ipip_hier_gre_keys.sh \ ipip_hier_gre.sh \ - ipip_lib.sh \ - lib.sh \ loopback.sh \ mirror_gre_bound.sh \ mirror_gre_bridge_1d.sh \ @@ -32,15 +26,11 @@ TEST_PROGS = bridge_igmp.sh \ mirror_gre_changes.sh \ mirror_gre_flower.sh \ mirror_gre_lag_lacp.sh \ - mirror_gre_lib.sh \ mirror_gre_neigh.sh \ mirror_gre_nh.sh \ mirror_gre.sh \ - mirror_gre_topo_lib.sh \ mirror_gre_vlan_bridge_1q.sh \ mirror_gre_vlan.sh \ - mirror_lib.sh \ - mirror_topo_lib.sh \ mirror_vlan.sh \ router_bridge.sh \ router_bridge_vlan.sh \ @@ -50,17 +40,12 @@ TEST_PROGS = bridge_igmp.sh \ router_multipath.sh \ router.sh \ router_vid_1.sh \ - sch_ets_core.sh \ sch_ets.sh \ - sch_ets_tests.sh \ - sch_tbf_core.sh \ - sch_tbf_etsprio.sh \ sch_tbf_ets.sh \ sch_tbf_prio.sh \ sch_tbf_root.sh \ tc_actions.sh \ tc_chains.sh \ - tc_common.sh \ tc_flower_router.sh \ tc_flower.sh \ tc_shblocks.sh \ @@ -72,4 +57,20 @@ TEST_PROGS = bridge_igmp.sh \ vxlan_bridge_1q.sh \ vxlan_symmetric.sh +TEST_PROGS_EXTENDED := devlink_lib.sh \ + ethtool_lib.sh \ + fib_offload_lib.sh \ + forwarding.config.sample \ + ipip_lib.sh \ + lib.sh \ + mirror_gre_lib.sh \ + mirror_gre_topo_lib.sh \ + mirror_lib.sh \ + mirror_topo_lib.sh \ + sch_ets_core.sh \ + sch_ets_tests.sh \ + sch_tbf_core.sh \ + sch_tbf_etsprio.sh \ + tc_common.sh + include ../../lib.mk diff --git a/tools/testing/selftests/net/forwarding/ethtool_lib.sh b/tools/testing/selftests/net/forwarding/ethtool_lib.sh old mode 100755 new mode 100644 -- 2.19.2

5 years, 1 month

3
3
0 0

[RFC PATCH kunit-next] kunit: kunit_tool: Separate out config/build/exec/parse

by David Gow

Add new subcommands to kunit.py to allow stages of the existing 'run' subcommand to be run independently: - 'config': Verifies that .config is a subset of .kunitconfig - 'build': Compiles a UML kernel for KUnit - 'exec': Runs the kernel, and outputs the test results. - 'parse': Parses test results from a file or stdin Most of these are not hugely useful by themselves yet, and there's room for plenty of bikeshedding on the names, but this hopefully can form a foundation for future improvements. Signed-off-by: David Gow <davidgow(a)google.com> --- [Whoops: typo-ed Brendan's email. Sorry about that!] As was briefly disccussed in [1], this change is part of a "separation of concerns" in kunit_tool. This should make it easier to integrate kunit_tool into other setups. Of particular intrest is probably 'kunit.py parse', which should allow KUnit results to be parsed from other sources, such as after loading a module, or from a non-UML kernel, or from debugfs when that's supported[2]. [1]: https://lkml.org/lkml/2020/2/5/552 [2]: https://patchwork.kernel.org/patch/11419901/ tools/testing/kunit/kunit.py | 236 ++++++++++++++++++++----- tools/testing/kunit/kunit_tool_test.py | 55 ++++++ 2 files changed, 242 insertions(+), 49 deletions(-) diff --git a/tools/testing/kunit/kunit.py b/tools/testing/kunit/kunit.py index 180ad1e1b04f..92a634594cf6 100755 --- a/tools/testing/kunit/kunit.py +++ b/tools/testing/kunit/kunit.py @@ -20,8 +20,12 @@ import kunit_config import kunit_kernel import kunit_parser -KunitResult = namedtuple('KunitResult', ['status','result']) +KunitResult = namedtuple('KunitResult', ['status','result','elapsed_time']) +KunitConfigRequest = namedtuple('KunitConfigRequest', ['build_dir', 'defconfig']) +KunitBuildRequest = namedtuple('KunitBuildRequest', ['jobs', 'build_dir']) +KunitExecRequest = namedtuple('KunitExecRequest', ['timeout', 'build_dir']) +KunitParseRequest = namedtuple('KunitParseRequest', ['raw_output', 'input_data']) KunitRequest = namedtuple('KunitRequest', ['raw_output','timeout', 'jobs', 'build_dir', 'defconfig']) KernelDirectoryPath = sys.argv[0].split('tools/testing/kunit/')[0] @@ -44,14 +48,25 @@ def get_kernel_root_path(): sys.exit(1) return parts[0] -def run_tests(linux: kunit_kernel.LinuxSourceTree, - request: KunitRequest) -> KunitResult: +def config_tests(linux: kunit_kernel.LinuxSourceTree, + request: KunitConfigRequest) -> KunitResult: + kunit_parser.print_with_timestamp('Configuring KUnit Kernel ...') + config_start = time.time() + if request.defconfig: + create_default_kunitconfig() success = linux.build_reconfig(request.build_dir) config_end = time.time() if not success: - return KunitResult(KunitStatus.CONFIG_FAILURE, 'could not configure kernel') + return KunitResult(KunitStatus.CONFIG_FAILURE, + 'could not configure kernel', + config_end - config_start) + return KunitResult(KunitStatus.SUCCESS, + 'configured kernel successfully', + config_end - config_start) +def build_tests(linux: kunit_kernel.LinuxSourceTree, + request: KunitBuildRequest) -> KunitResult: kunit_parser.print_with_timestamp('Building KUnit Kernel ...') build_start = time.time() @@ -59,73 +74,156 @@ def run_tests(linux: kunit_kernel.LinuxSourceTree, build_end = time.time() if not success: return KunitResult(KunitStatus.BUILD_FAILURE, 'could not build kernel') + if not success: + return KunitResult(KunitStatus.BUILD_FAILURE, + 'could not build kernel', + build_end - build_start) + return KunitResult(KunitStatus.SUCCESS, + 'built kernel successfully', + build_end - build_start) +def exec_tests(linux: kunit_kernel.LinuxSourceTree, + request: KunitExecRequest) -> KunitResult: kunit_parser.print_with_timestamp('Starting KUnit Kernel ...') test_start = time.time() + result = linux.run_kernel(timeout=request.timeout, + build_dir=request.build_dir) + test_end = time.time() + + return KunitResult(KunitStatus.SUCCESS, + result, + test_end - test_start) + +def parse_tests(request: KunitParseRequest) -> KunitResult: + parse_start = time.time() + test_result = kunit_parser.TestResult(kunit_parser.TestStatus.SUCCESS, [], 'Tests not Parsed.') if request.raw_output: - kunit_parser.raw_output( - linux.run_kernel(timeout=request.timeout, - build_dir=request.build_dir)) + kunit_parser.raw_output(request.input_data) else: - kunit_output = linux.run_kernel(timeout=request.timeout, - build_dir=request.build_dir) - test_result = kunit_parser.parse_run_tests(kunit_output) - test_end = time.time() + test_result = kunit_parser.parse_run_tests(request.input_data) + parse_end = time.time() + + if test_result.status != kunit_parser.TestStatus.SUCCESS: + return KunitResult(KunitStatus.TEST_FAILURE, test_result, + parse_end - parse_start) + + return KunitResult(KunitStatus.SUCCESS, test_result, + parse_end - parse_start) + + +def run_tests(linux: kunit_kernel.LinuxSourceTree, + request: KunitRequest) -> KunitResult: + run_start = time.time() + + config_request = KunitConfigRequest(request.build_dir, request.defconfig) + config_result = config_tests(linux, config_request) + if config_result.status != KunitStatus.SUCCESS: + return config_result + + build_request = KunitBuildRequest(request.jobs, request.build_dir) + build_result = build_tests(linux, build_request) + if build_result.status != KunitStatus.SUCCESS: + return build_result + + exec_request = KunitExecRequest(request.timeout, request.build_dir) + exec_result = exec_tests(linux, exec_request) + if exec_result.status != KunitStatus.SUCCESS: + return exec_result + + parse_request = KunitParseRequest(request.raw_output, exec_result.result) + parse_result = parse_tests(parse_request) + + run_end = time.time() kunit_parser.print_with_timestamp(( 'Elapsed time: %.3fs total, %.3fs configuring, %.3fs ' + 'building, %.3fs running\n') % ( - test_end - config_start, - config_end - config_start, - build_end - build_start, - test_end - test_start)) + run_end - run_start, + config_result.elapsed_time, + build_result.elapsed_time, + exec_result.elapsed_time)) + return parse_result + +def add_common_opts(parser): + parser.add_argument('--build_dir', + help='As in the make command, it specifies the build ' + 'directory.', + type=str, default='', metavar='build_dir') + +def add_config_opts(parser): + parser.add_argument('--defconfig', + help='Uses a default .kunitconfig.', + action='store_true') + +def add_build_opts(parser): + parser.add_argument('--jobs', + help='As in the make command, "Specifies the number of ' + 'jobs (commands) to run simultaneously."', + type=int, default=8, metavar='jobs') + +def add_exec_opts(parser): + parser.add_argument('--timeout', + help='maximum number of seconds to allow for all tests ' + 'to run. This does not include time taken to build the ' + 'tests.', + type=int, + default=300, + metavar='timeout') + +def add_parse_opts(parser): + parser.add_argument('--raw_output', help='don\'t format output from kernel', + action='store_true') - if test_result.status != kunit_parser.TestStatus.SUCCESS: - return KunitResult(KunitStatus.TEST_FAILURE, test_result) - else: - return KunitResult(KunitStatus.SUCCESS, test_result) def main(argv, linux=None): parser = argparse.ArgumentParser( description='Helps writing and running KUnit tests.') subparser = parser.add_subparsers(dest='subcommand') + # The 'run' command will config, build, exec, and parse in one go. run_parser = subparser.add_parser('run', help='Runs KUnit tests.') - run_parser.add_argument('--raw_output', help='don\'t format output from kernel', - action='store_true') + add_common_opts(run_parser) + add_config_opts(run_parser) + add_build_opts(run_parser) + add_exec_opts(run_parser) + add_parse_opts(run_parser) - run_parser.add_argument('--timeout', - help='maximum number of seconds to allow for all tests ' - 'to run. This does not include time taken to build the ' - 'tests.', - type=int, - default=300, - metavar='timeout') - - run_parser.add_argument('--jobs', - help='As in the make command, "Specifies the number of ' - 'jobs (commands) to run simultaneously."', - type=int, default=8, metavar='jobs') - - run_parser.add_argument('--build_dir', - help='As in the make command, it specifies the build ' - 'directory.', - type=str, default='', metavar='build_dir') - - run_parser.add_argument('--defconfig', - help='Uses a default .kunitconfig.', - action='store_true') + config_parser = subparser.add_parser('config', + help='Ensures that .config contains all of ' + 'the options in .kunitconfig') + add_common_opts(config_parser) + add_config_opts(config_parser) - cli_args = parser.parse_args(argv) + build_parser = subparser.add_parser('build', help='Builds a kernel with KUnit tests') + add_common_opts(build_parser) + add_build_opts(build_parser) - if cli_args.subcommand == 'run': - if get_kernel_root_path(): - os.chdir(get_kernel_root_path()) + exec_parser = subparser.add_parser('exec', help='Run a kernel with KUnit tests') + add_common_opts(exec_parser) + add_exec_opts(exec_parser) + add_parse_opts(exec_parser) + + # The 'parse' option is special, as it doesn't need the kernel source + # (therefore there is no need for a build_dir, hence no add_common_opts) + # and the '--file' argument is not relevant to 'run', so isn't in + # add_parse_opts() + parse_parser = subparser.add_parser('parse', + help='Parses KUnit results from a file, ' + 'and parses formatted results.') + add_parse_opts(parse_parser) + parse_parser.add_argument('file', + help='Specifies the file to read results from.', + type=str, default='-', metavar='input_file') + + cli_args = parser.parse_args(argv) + # Set up the build_dir and source tree for commands which use it + # (everything but 'parse') + if cli_args.subcommand != 'parse': if cli_args.build_dir: if not os.path.exists(cli_args.build_dir): os.mkdir(cli_args.build_dir) @@ -133,12 +231,11 @@ def main(argv, linux=None): cli_args.build_dir, kunit_kernel.kunitconfig_path) - if cli_args.defconfig: - create_default_kunitconfig() - if not linux: linux = kunit_kernel.LinuxSourceTree() + + if cli_args.subcommand == 'run': request = KunitRequest(cli_args.raw_output, cli_args.timeout, cli_args.jobs, @@ -147,6 +244,47 @@ def main(argv, linux=None): result = run_tests(linux, request) if result.status != KunitStatus.SUCCESS: sys.exit(1) + elif cli_args.subcommand == 'config': + request = KunitConfigRequest(cli_args.build_dir, + cli_args.defconfig) + result = config_tests(linux, request) + kunit_parser.print_with_timestamp(( + 'Elapsed time: %.3fs\n') % ( + result.elapsed_time)) + if result.status != KunitStatus.SUCCESS: + sys.exit(1) + elif cli_args.subcommand == 'build': + request = KunitBuildRequest(cli_args.jobs, + cli_args.build_dir) + result = build_tests(linux, request) + kunit_parser.print_with_timestamp(( + 'Elapsed time: %.3fs\n') % ( + result.elapsed_time)) + if result.status != KunitStatus.SUCCESS: + sys.exit(1) + elif cli_args.subcommand == 'exec': + exec_request = KunitExecRequest(cli_args.timeout, + cli_args.build_dir) + exec_result = exec_tests(linux, exec_request) + parse_request = KunitParseRequest(cli_args.raw_output, + exec_result.result) + result = parse_tests(parse_request) + kunit_parser.print_with_timestamp(( + 'Elapsed time: %.3fs\n') % ( + exec_result.elapsed_time)) + if result.status != KunitStatus.SUCCESS: + sys.exit(1) + elif cli_args.subcommand == 'parse': + if cli_args.file == '-': + kunit_output = sys.stdin + else: + with open(cli_args.file, 'r') as f: + kunit_output = f.read().splitlines() + request = KunitParseRequest(cli_args.raw_output, + kunit_output) + result = parse_tests(request) + if result.status != KunitStatus.SUCCESS: + sys.exit(1) else: parser.print_help() diff --git a/tools/testing/kunit/kunit_tool_test.py b/tools/testing/kunit/kunit_tool_test.py index cba97756ac4a..0d04425ead04 100755 --- a/tools/testing/kunit/kunit_tool_test.py +++ b/tools/testing/kunit/kunit_tool_test.py @@ -170,6 +170,24 @@ class KUnitMainTest(unittest.TestCase): self.print_patch.stop() pass + def test_config_passes_args_pass(self): + kunit.main(['config'], self.linux_source_mock) + assert self.linux_source_mock.build_reconfig.call_count == 1 + assert self.linux_source_mock.run_kernel.call_count == 0 + + def test_build_passes_args_pass(self): + kunit.main(['build'], self.linux_source_mock) + assert self.linux_source_mock.build_reconfig.call_count == 0 + self.linux_source_mock.build_um_kernel.assert_called_once_with(8, '') + assert self.linux_source_mock.run_kernel.call_count == 0 + + def test_exec_passes_args_pass(self): + kunit.main(['exec'], self.linux_source_mock) + assert self.linux_source_mock.build_reconfig.call_count == 0 + assert self.linux_source_mock.run_kernel.call_count == 1 + self.linux_source_mock.run_kernel.assert_called_once_with(build_dir='', timeout=300) + self.print_mock.assert_any_call(StrContains('Testing complete.')) + def test_run_passes_args_pass(self): kunit.main(['run'], self.linux_source_mock) assert self.linux_source_mock.build_reconfig.call_count == 1 @@ -177,6 +195,13 @@ class KUnitMainTest(unittest.TestCase): self.linux_source_mock.run_kernel.assert_called_once_with(build_dir='', timeout=300) self.print_mock.assert_any_call(StrContains('Testing complete.')) + def test_exec_passes_args_fail(self): + self.linux_source_mock.run_kernel = mock.Mock(return_value=[]) + with self.assertRaises(SystemExit) as e: + kunit.main(['exec'], self.linux_source_mock) + assert type(e.exception) == SystemExit + assert e.exception.code == 1 + def test_run_passes_args_fail(self): self.linux_source_mock.run_kernel = mock.Mock(return_value=[]) with self.assertRaises(SystemExit) as e: @@ -187,6 +212,14 @@ class KUnitMainTest(unittest.TestCase): assert self.linux_source_mock.run_kernel.call_count == 1 self.print_mock.assert_any_call(StrContains(' 0 tests run')) + def test_exec_raw_output(self): + self.linux_source_mock.run_kernel = mock.Mock(return_value=[]) + kunit.main(['exec', '--raw_output'], self.linux_source_mock) + assert self.linux_source_mock.run_kernel.call_count == 1 + for kall in self.print_mock.call_args_list: + assert kall != mock.call(StrContains('Testing complete.')) + assert kall != mock.call(StrContains(' 0 tests run')) + def test_run_raw_output(self): self.linux_source_mock.run_kernel = mock.Mock(return_value=[]) kunit.main(['run', '--raw_output'], self.linux_source_mock) @@ -196,6 +229,12 @@ class KUnitMainTest(unittest.TestCase): assert kall != mock.call(StrContains('Testing complete.')) assert kall != mock.call(StrContains(' 0 tests run')) + def test_exec_timeout(self): + timeout = 3453 + kunit.main(['exec', '--timeout', str(timeout)], self.linux_source_mock) + self.linux_source_mock.run_kernel.assert_called_once_with(build_dir='', timeout=timeout) + self.print_mock.assert_any_call(StrContains('Testing complete.')) + def test_run_timeout(self): timeout = 3453 kunit.main(['run', '--timeout', str(timeout)], self.linux_source_mock) @@ -210,5 +249,21 @@ class KUnitMainTest(unittest.TestCase): self.linux_source_mock.run_kernel.assert_called_once_with(build_dir=build_dir, timeout=300) self.print_mock.assert_any_call(StrContains('Testing complete.')) + def test_config_builddir(self): + build_dir = '.kunit' + kunit.main(['config', '--build_dir', build_dir], self.linux_source_mock) + assert self.linux_source_mock.build_reconfig.call_count == 1 + + def test_build_builddir(self): + build_dir = '.kunit' + kunit.main(['build', '--build_dir', build_dir], self.linux_source_mock) + self.linux_source_mock.build_um_kernel.assert_called_once_with(8, build_dir) + + def test_exec_builddir(self): + build_dir = '.kunit' + kunit.main(['exec', '--build_dir', build_dir], self.linux_source_mock) + self.linux_source_mock.run_kernel.assert_called_once_with(build_dir=build_dir, timeout=300) + self.print_mock.assert_any_call(StrContains('Testing complete.')) + if __name__ == '__main__': unittest.main() -- 2.25.1.481.gfbce0eb801-goog

5 years, 1 month

2
1
0 0

[PATCH kselftest/test] kunit: Always print actual pointer values in asserts

by David Gow

KUnit assertions and expectations will print the values being tested. If these are pointers (e.g., KUNIT_EXPECT_PTR_EQ(test, a, b)), these pointers are currently printed with the %pK format specifier, which -- to prevent information leaks which may compromise, e.g., ASLR -- are often either hashed or replaced with ____ptrval____ or similar, making debugging tests difficult. By replacing %pK with %px as Documentation/core-api/printk-formats.rst suggests, we disable this security feature for KUnit assertions and expectations, allowing the actual pointer values to be printed. Given that KUnit is not intended for use in production kernels, and the pointers are only printed on failing tests, this seems like a worthwhile tradeoff. Signed-off-by: David Gow <davidgow(a)google.com> --- This seems like the best way of solving this problem to me, but if anyone has a better solution I'd love to hear it. Note also that this does trigger two checkpatch.pl warnings, which warn that the change will potentially cause the kernel memory layout to be exposed. Since that's the whole point of the change, they probably sohuld stay there. lib/kunit/assert.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/lib/kunit/assert.c b/lib/kunit/assert.c index 86013d4cf891..a87960409bd4 100644 --- a/lib/kunit/assert.c +++ b/lib/kunit/assert.c @@ -110,10 +110,10 @@ void kunit_binary_ptr_assert_format(const struct kunit_assert *assert, binary_assert->left_text, binary_assert->operation, binary_assert->right_text); - string_stream_add(stream, "\t\t%s == %pK\n", + string_stream_add(stream, "\t\t%s == %px\n", binary_assert->left_text, binary_assert->left_value); - string_stream_add(stream, "\t\t%s == %pK", + string_stream_add(stream, "\t\t%s == %px", binary_assert->right_text, binary_assert->right_value); kunit_assert_print_msg(assert, stream); -- 2.24.0.432.g9d3f5f5b63-goog

5 years, 1 month

3
4
0 0

[PATCH kunit-next] kunit: kunit_tool: Allow .kunitconfig to disable config items

by David Gow

Rework kunit_tool in order to allow .kunitconfig files to better enforce that disabled items in .kunitconfig are disabled in the generated .config. Previously, kunit_tool simply enforced that any line present in .kunitconfig was also present in .config, but this could cause problems if a config option was disabled in .kunitconfig, but not listed in .config due to (for example) having disabled dependencies. To fix this, re-work the parser to track config names and values, and require values to match unless they are explicitly disabled with the "CONFIG_x is not set" comment (or by setting its value to 'n'). Those "disabled" values will pass validation if omitted from the .config, but not if they have a different value. Signed-off-by: David Gow <davidgow(a)google.com> --- tools/testing/kunit/kunit_config.py | 41 ++++++++++++++++++++------ tools/testing/kunit/kunit_tool_test.py | 22 +++++++------- 2 files changed, 43 insertions(+), 20 deletions(-) diff --git a/tools/testing/kunit/kunit_config.py b/tools/testing/kunit/kunit_config.py index ebf3942b23f5..e75063d603b5 100644 --- a/tools/testing/kunit/kunit_config.py +++ b/tools/testing/kunit/kunit_config.py @@ -9,16 +9,18 @@ import collections import re -CONFIG_IS_NOT_SET_PATTERN = r'^# CONFIG_\w+ is not set$' -CONFIG_PATTERN = r'^CONFIG_\w+=\S+$' - -KconfigEntryBase = collections.namedtuple('KconfigEntry', ['raw_entry']) +CONFIG_IS_NOT_SET_PATTERN = r'^# CONFIG_(\w+) is not set$' +CONFIG_PATTERN = r'^CONFIG_(\w+)=(\S+)$' +KconfigEntryBase = collections.namedtuple('KconfigEntry', ['name', 'value']) class KconfigEntry(KconfigEntryBase): def __str__(self) -> str: - return self.raw_entry + if self.value == 'n': + return r'# CONFIG_%s is not set' % (self.name) + else: + return r'CONFIG_%s=%s' % (self.name, self.value) class KconfigParseError(Exception): @@ -38,7 +40,17 @@ class Kconfig(object): self._entries.append(entry) def is_subset_of(self, other: 'Kconfig') -> bool: - return self.entries().issubset(other.entries()) + for a in self.entries(): + found = False + for b in other.entries(): + if a.name != b.name: + continue + if a.value != b.value: + return False + found = True + if a.value != 'n' and found == False: + return False + return True def write_to_file(self, path: str) -> None: with open(path, 'w') as f: @@ -54,9 +66,20 @@ class Kconfig(object): line = line.strip() if not line: continue - elif config_matcher.match(line) or is_not_set_matcher.match(line): - self._entries.append(KconfigEntry(line)) - elif line[0] == '#': + + match = config_matcher.match(line) + if match: + entry = KconfigEntry(match.group(1), match.group(2)) + self.add_entry(entry) + continue + + empty_match = is_not_set_matcher.match(line) + if empty_match: + entry = KconfigEntry(empty_match.group(1), 'n') + self.add_entry(entry) + continue + + if line[0] == '#': continue else: raise KconfigParseError('Failed to parse: ' + line) diff --git a/tools/testing/kunit/kunit_tool_test.py b/tools/testing/kunit/kunit_tool_test.py index ce47e87b633a..984588d6ba95 100755 --- a/tools/testing/kunit/kunit_tool_test.py +++ b/tools/testing/kunit/kunit_tool_test.py @@ -37,7 +37,7 @@ class KconfigTest(unittest.TestCase): self.assertTrue(kconfig0.is_subset_of(kconfig0)) kconfig1 = kunit_config.Kconfig() - kconfig1.add_entry(kunit_config.KconfigEntry('CONFIG_TEST=y')) + kconfig1.add_entry(kunit_config.KconfigEntry('TEST', 'y')) self.assertTrue(kconfig1.is_subset_of(kconfig1)) self.assertTrue(kconfig0.is_subset_of(kconfig1)) self.assertFalse(kconfig1.is_subset_of(kconfig0)) @@ -51,15 +51,15 @@ class KconfigTest(unittest.TestCase): expected_kconfig = kunit_config.Kconfig() expected_kconfig.add_entry( - kunit_config.KconfigEntry('CONFIG_UML=y')) + kunit_config.KconfigEntry('UML', 'y')) expected_kconfig.add_entry( - kunit_config.KconfigEntry('CONFIG_MMU=y')) + kunit_config.KconfigEntry('MMU', 'y')) expected_kconfig.add_entry( - kunit_config.KconfigEntry('CONFIG_TEST=y')) + kunit_config.KconfigEntry('TEST', 'y')) expected_kconfig.add_entry( - kunit_config.KconfigEntry('CONFIG_EXAMPLE_TEST=y')) + kunit_config.KconfigEntry('EXAMPLE_TEST', 'y')) expected_kconfig.add_entry( - kunit_config.KconfigEntry('# CONFIG_MK8 is not set')) + kunit_config.KconfigEntry('MK8', 'n')) self.assertEqual(kconfig.entries(), expected_kconfig.entries()) @@ -68,15 +68,15 @@ class KconfigTest(unittest.TestCase): expected_kconfig = kunit_config.Kconfig() expected_kconfig.add_entry( - kunit_config.KconfigEntry('CONFIG_UML=y')) + kunit_config.KconfigEntry('UML', 'y')) expected_kconfig.add_entry( - kunit_config.KconfigEntry('CONFIG_MMU=y')) + kunit_config.KconfigEntry('MMU', 'y')) expected_kconfig.add_entry( - kunit_config.KconfigEntry('CONFIG_TEST=y')) + kunit_config.KconfigEntry('TEST', 'y')) expected_kconfig.add_entry( - kunit_config.KconfigEntry('CONFIG_EXAMPLE_TEST=y')) + kunit_config.KconfigEntry('EXAMPLE_TEST', 'y')) expected_kconfig.add_entry( - kunit_config.KconfigEntry('# CONFIG_MK8 is not set')) + kunit_config.KconfigEntry('MK8', 'n')) expected_kconfig.write_to_file(kconfig_path) -- 2.25.1.696.g5e7596f4ac-goog

5 years, 1 month

2
1
0 0

[PATCH 0/3] kselftest: Extend vDSO tests

by Vincenzo Frascino

This series extends the kselftests for the vDSO library making sure: that they compile correctly on non x86 platforms, that they can be cross compiled and introducing new tests that verify the correctness of the library. The so extended vDSO kselftests have been verified on all the platforms supported by the unified vDSO library [1]. The only new patch that this series introduces is the first one, patch 2 and patch 3 have already been reviewed in past as part of other series [2] [3]. [1] https://lore.kernel.org/lkml/20190621095252.32307-1-vincenzo.frascino@arm.c… [2] https://lore.kernel.org/lkml/20190621095252.32307-26-vincenzo.frascino@arm.… [3] https://lore.kernel.org/lkml/20190523112116.19233-4-vincenzo.frascino@arm.c… Cc: Shuah Khan <shuah(a)kernel.org> Cc: Thomas Gleixner <tglx(a)linutronix.de> Cc: Andy Lutomirski <luto(a)kernel.org> Signed-off-by: Vincenzo Frascino <vincenzo.frascino(a)arm.com> Vincenzo Frascino (3): kselftest: Enable vDSO test on non x86 platforms kselftest: Extend vDSO selftest kselftest: Extend vDSO selftest to clock_getres tools/testing/selftests/Makefile | 1 + tools/testing/selftests/vDSO/Makefile | 6 +- .../selftests/vDSO/vdso_clock_getres.c | 124 +++++++++ tools/testing/selftests/vDSO/vdso_config.h | 90 +++++++ tools/testing/selftests/vDSO/vdso_full_test.c | 244 ++++++++++++++++++ 5 files changed, 463 insertions(+), 2 deletions(-) create mode 100644 tools/testing/selftests/vDSO/vdso_clock_getres.c create mode 100644 tools/testing/selftests/vDSO/vdso_config.h create mode 100644 tools/testing/selftests/vDSO/vdso_full_test.c -- 2.25.2

5 years, 1 month

2
5
0 0

next-20200324 kselftest results

by ci_notify＠linaro.org

Summary ------------------------------------------------------------------------ kernel: 5.6.0-rc7 git repo: https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git git branch: master git commit: f15e8108f6251f3480d572bca5729c1b849edfb3 git describe: next-20200324 Test details: https://qa-reports.linaro.org/lkft/linux-next-oe/build/next-20200324 Regressions (compared to build next-20200323) ------------------------------------------------------------------------ No regressions Fixes (compared to build next-20200323) ------------------------------------------------------------------------ No fixes In total: ------------------------------------------------------------------------ Ran 0 total tests in the following environments and test suites. pass 0 fail 0 xfail 0 skip 0 Environments -------------- - dragonboard-410c - arm64 - hi6220-hikey - arm64 - i386 - juno-r2 - arm64 - x15 - arm - x86_64 Test Suites ----------- Failures ------------------------------------------------------------------------ dragonboard-410c: i386: x15: x86: juno-r2: hi6220-hikey: Skips ------------------------------------------------------------------------ No skips -- Linaro LKFT https://lkft.linaro.org

5 years, 1 month

1
0
0 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

Linux-kselftest-mirror