- Linux-kselftest-mirror - lists.linaro.org

[PATCH v2 0/2] Selftest for cpuidle latency measurement

by Pratik Rajesh Sampat

v1: https://lkml.org/lkml/2020/7/7/1036 Changelog v1 --> v2 1. Based on Shuah Khan's comment, changed exit code to ksft_skip to indicate the test is being skipped 2. Change the busy workload for baseline measurement from "yes > /dev/null" to "cat /dev/random to /dev/null", based on observed CPU utilization for "yes" consuming ~60% CPU while the latter consumes 100% of CPUs, giving more accurate baseline numbers --- The patch series introduces a mechanism to measure wakeup latency for IPI and timer based interrupts The motivation behind this series is to find significant deviations behind advertised latency and resisdency values To achieve this, we introduce a kernel module and expose its control knobs through the debugfs interface that the selftests can engage with. The kernel module provides the following interfaces within /sys/kernel/debug/latency_test/ for, 1. IPI test: ipi_cpu_dest # Destination CPU for the IPI ipi_cpu_src # Origin of the IPI ipi_latency_ns # Measured latency time in ns 2. Timeout test: timeout_cpu_src # CPU on which the timer to be queued timeout_expected_ns # Timer duration timeout_diff_ns # Difference of actual duration vs expected timer To include the module, check option and include as module kernel hacking -> Cpuidle latency selftests The selftest inserts the module, disables all the idle states and enables them one by one testing the following: 1. Keeping source CPU constant, iterates through all the CPUS measuring IPI latency for baseline (CPU is busy with "cat /dev/random > /dev/null" workload) and the when the CPU is allowed to be at rest 2. Iterating through all the CPUs, sending expected timer durations to be equivalent to the residency of the the deepest idle state enabled and extracting the difference in time between the time of wakeup and the expected timer duration Usage ----- Can be used in conjuction to the rest of the selftests. Default Output location in: tools/testing/cpuidle/cpuidle.log To run this test specifically: $ make -C tools/testing/selftests TARGETS="cpuidle" run_tests There are a few optinal arguments too that the script can take [-h <help>] [-m <location of the module>] [-o <location of the output>] Sample output snippet --------------------- --IPI Latency Test--- --Baseline IPI Latency measurement: CPU Busy-- SRC_CPU DEST_CPU IPI_Latency(ns) ... 0 8 1996 0 9 2125 0 10 1264 0 11 1788 0 12 2045 Baseline Average IPI latency(ns): 1843 ---Enabling state: 5--- SRC_CPU DEST_CPU IPI_Latency(ns) 0 8 621719 0 9 624752 0 10 622218 0 11 623968 0 12 621303 Expected IPI latency(ns): 100000 Observed Average IPI latency(ns): 622792 --Timeout Latency Test-- --Baseline Timeout Latency measurement: CPU Busy-- Wakeup_src Baseline_delay(ns) ... 8 2249 9 2226 10 2211 11 2183 12 2263 Baseline Average timeout diff(ns): 2226 ---Enabling state: 5--- 8 10749 9 10911 10 10912 11 12100 12 73276 Expected timeout(ns): 10000200 Observed Average timeout diff(ns): 23589 Pratik Rajesh Sampat (2): cpuidle: Trace IPI based and timer based wakeup latency from idle states selftest/cpuidle: Add support for cpuidle latency measurement drivers/cpuidle/Makefile | 1 + drivers/cpuidle/test-cpuidle_latency.c | 150 ++++++++++++ lib/Kconfig.debug | 10 + tools/testing/selftests/Makefile | 1 + tools/testing/selftests/cpuidle/Makefile | 6 + tools/testing/selftests/cpuidle/cpuidle.sh | 257 +++++++++++++++++++++ tools/testing/selftests/cpuidle/settings | 1 + 7 files changed, 426 insertions(+) create mode 100644 drivers/cpuidle/test-cpuidle_latency.c create mode 100644 tools/testing/selftests/cpuidle/Makefile create mode 100755 tools/testing/selftests/cpuidle/cpuidle.sh create mode 100644 tools/testing/selftests/cpuidle/settings -- 2.25.4

4 years, 6 months

3
5
0 0

[PATCH v2 0/5] mm/migrate: avoid device private invalidations

by Ralph Campbell

The goal for this series is to avoid device private memory TLB invalidations when migrating a range of addresses from system memory to device private memory and some of those pages have already been migrated. The approach taken is to introduce a new mmu notifier invalidation event type and use that in the device driver to skip invalidation callbacks from migrate_vma_setup(). The device driver is also then expected to handle device MMU invalidations as part of the migrate_vma_setup(), migrate_vma_pages(), migrate_vma_finalize() process. Note that this is opt-in. A device driver can simply invalidate its MMU in the mmu notifier callback and not handle MMU invalidations in the migration sequence. This series is based on Jason Gunthorpe's HMM tree (linux-5.8.0-rc4). Also, this replaces the need for the following two patches I sent: ("mm: fix migrate_vma_setup() src_owner and normal pages") https://lore.kernel.org/linux-mm/20200622222008.9971-1-rcampbell@nvidia.com ("nouveau: fix mixed normal and device private page migration") https://lore.kernel.org/lkml/20200622233854.10889-3-rcampbell@nvidia.com Changes in v2: Rebase to Jason Gunthorpe's HMM tree. Added reviewed-by from Bharata B Rao. Rename the mmu_notifier_range::data field to migrate_pgmap_owner as suggested by Jason Gunthorpe. Ralph Campbell (5): nouveau: fix storing invalid ptes mm/migrate: add a direction parameter to migrate_vma mm/notifier: add migration invalidation type nouveau/svm: use the new migration invalidation mm/hmm/test: use the new migration invalidation arch/powerpc/kvm/book3s_hv_uvmem.c | 2 ++ drivers/gpu/drm/nouveau/nouveau_dmem.c | 13 ++++++-- drivers/gpu/drm/nouveau/nouveau_svm.c | 10 +++++- drivers/gpu/drm/nouveau/nouveau_svm.h | 1 + .../drm/nouveau/nvkm/subdev/mmu/vmmgp100.c | 13 +++++--- include/linux/migrate.h | 12 +++++-- include/linux/mmu_notifier.h | 7 ++++ lib/test_hmm.c | 33 +++++++++++-------- mm/migrate.c | 13 ++++++-- 9 files changed, 77 insertions(+), 27 deletions(-) -- 2.20.1

4 years, 6 months

3
15
0 0

[RFC v2] kunit: tool: Allows to install and load modules in root filesystem

by Vitor Massaru Iha

This makes possible to install and load KUnit tests as modules in a root filesystem and boot UML with this filesystem. The filesystem was created using debootstrap: sudo debootstrap buster .uml_rootfs And change the owner of the root filesystem files for your user: sudo chown -R $USER:$USER .uml_rootfs For the kunit-tool to correctly capture the test results, uml_utilities must be installed on the host to halt uml. Signed-off-by: Vitor Massaru Iha <vitor(a)massaru.org> Reviewed-by: Brendan Higgins <brendanhiggins(a)google.com> --- v2: * splitted patch in 3: - Allows to install and load modules in root filesystem; - Provides an userspace memory context when tests are compiled as module; - Convert test_user_copy to KUnit test; * added documentation; * removed the use of global variable; * capitalized make_cmd to look like a constant; * adjusted unit tests to the new parameter (--uml_rootfs_dir); --- Documentation/dev-tools/kunit/start.rst | 15 ++++ tools/testing/kunit/kunit.py | 40 +++++++-- tools/testing/kunit/kunit_kernel.py | 106 ++++++++++++++++++++---- tools/testing/kunit/kunit_tool_test.py | 16 ++-- 4 files changed, 146 insertions(+), 31 deletions(-) diff --git a/Documentation/dev-tools/kunit/start.rst b/Documentation/dev-tools/kunit/start.rst index bb112cf70624..af2b6b9f870f 100644 --- a/Documentation/dev-tools/kunit/start.rst +++ b/Documentation/dev-tools/kunit/start.rst @@ -9,6 +9,21 @@ Installing dependencies KUnit has the same dependencies as the Linux kernel. As long as you can build the kernel, you can run KUnit. +Unless you need to run the tests with a root filesystem, in this case you will +need to install debootsrap and uml_tools (uml-tools in Debian flavor distro). + +To install the root filesystem: + +.. code-block:: bash + + sudo debootstrap buster .uml_rootfs + +And change the owner of the root filesystem for your user: + +.. code-block:: bash + + sudo chown -R $USER:$USER .uml_rootfs + Running tests with the KUnit Wrapper ==================================== Included with KUnit is a simple Python wrapper which runs tests under User Mode diff --git a/tools/testing/kunit/kunit.py b/tools/testing/kunit/kunit.py index 2ece17e9eab5..8abe3a61f798 100755 --- a/tools/testing/kunit/kunit.py +++ b/tools/testing/kunit/kunit.py @@ -23,16 +23,16 @@ import kunit_parser KunitResult = namedtuple('KunitResult', ['status','result','elapsed_time']) KunitConfigRequest = namedtuple('KunitConfigRequest', - ['build_dir', 'make_options']) + ['build_dir', 'uml_rootfs_path', 'make_options']) KunitBuildRequest = namedtuple('KunitBuildRequest', - ['jobs', 'build_dir', 'alltests', + ['jobs', 'build_dir', 'uml_rootfs_path', 'alltests', 'make_options']) KunitExecRequest = namedtuple('KunitExecRequest', - ['timeout', 'build_dir', 'alltests']) + ['timeout', 'build_dir', 'uml_rootfs_path', 'alltests']) KunitParseRequest = namedtuple('KunitParseRequest', ['raw_output', 'input_data']) KunitRequest = namedtuple('KunitRequest', ['raw_output','timeout', 'jobs', - 'build_dir', 'alltests', + 'build_dir', 'uml_rootfs_path', 'alltests', 'make_options']) KernelDirectoryPath = sys.argv[0].split('tools/testing/kunit/')[0] @@ -47,7 +47,6 @@ def create_default_kunitconfig(): if not os.path.exists(kunit_kernel.kunitconfig_path): shutil.copyfile('arch/um/configs/kunit_defconfig', kunit_kernel.kunitconfig_path) - def get_kernel_root_path(): parts = sys.argv[0] if not __file__ else __file__ parts = os.path.realpath(parts).split('tools/testing/kunit') @@ -61,7 +60,8 @@ def config_tests(linux: kunit_kernel.LinuxSourceTree, config_start = time.time() create_default_kunitconfig() - success = linux.build_reconfig(request.build_dir, request.make_options) + + success = linux.build_reconfig(request.build_dir, request.uml_rootfs_path, request.make_options) config_end = time.time() if not success: return KunitResult(KunitStatus.CONFIG_FAILURE, @@ -79,10 +79,9 @@ def build_tests(linux: kunit_kernel.LinuxSourceTree, success = linux.build_um_kernel(request.alltests, request.jobs, request.build_dir, + request.uml_rootfs_path, request.make_options) build_end = time.time() - if not success: - return KunitResult(KunitStatus.BUILD_FAILURE, 'could not build kernel') if not success: return KunitResult(KunitStatus.BUILD_FAILURE, 'could not build kernel', @@ -97,7 +96,8 @@ def exec_tests(linux: kunit_kernel.LinuxSourceTree, test_start = time.time() result = linux.run_kernel( timeout=None if request.alltests else request.timeout, - build_dir=request.build_dir) + build_dir=request.build_dir, + uml_rootfs_path=request.uml_rootfs_path) test_end = time.time() @@ -130,12 +130,14 @@ def run_tests(linux: kunit_kernel.LinuxSourceTree, run_start = time.time() config_request = KunitConfigRequest(request.build_dir, + request.uml_rootfs_path, request.make_options) config_result = config_tests(linux, config_request) if config_result.status != KunitStatus.SUCCESS: return config_result build_request = KunitBuildRequest(request.jobs, request.build_dir, + request.uml_rootfs_path, request.alltests, request.make_options) build_result = build_tests(linux, build_request) @@ -143,6 +145,7 @@ def run_tests(linux: kunit_kernel.LinuxSourceTree, return build_result exec_request = KunitExecRequest(request.timeout, request.build_dir, + request.uml_rootfs_path, request.alltests) exec_result = exec_tests(linux, exec_request) if exec_result.status != KunitStatus.SUCCESS: @@ -168,6 +171,10 @@ def add_common_opts(parser): help='As in the make command, it specifies the build ' 'directory.', type=str, default='.kunit', metavar='build_dir') + parser.add_argument('--uml_rootfs_dir', + help='To load modules, a root filesystem is ' + 'required to install and load these modules.', + type=str, default=None, metavar='uml_rootfs_dir') parser.add_argument('--make_options', help='X=Y make option, can be repeated.', action='append') @@ -196,6 +203,7 @@ def add_parse_opts(parser): def main(argv, linux=None): + uml_rootfs_path = None parser = argparse.ArgumentParser( description='Helps writing and running KUnit tests.') subparser = parser.add_subparsers(dest='subcommand') @@ -235,6 +243,16 @@ def main(argv, linux=None): cli_args = parser.parse_args(argv) + if cli_args.uml_rootfs_dir \ + and os.path.exists(cli_args.uml_rootfs_dir) \ + and os.path.abspath(cli_args.uml_rootfs_dir) != \ + os.path.abspath(os.getcwd()) \ + and os.path.abspath(cli_args.uml_rootfs_dir) != '/': + uml_rootfs_path = os.path.abspath(cli_args.uml_rootfs_dir) + elif cli_args.uml_rootfs_dir != None: + print("Invalid uml_rootfs_dir: {}".format(cli_args.uml_rootfs_dir)) + sys.exit(1) + if cli_args.subcommand == 'run': if not os.path.exists(cli_args.build_dir): os.mkdir(cli_args.build_dir) @@ -246,6 +264,7 @@ def main(argv, linux=None): cli_args.timeout, cli_args.jobs, cli_args.build_dir, + uml_rootfs_path, cli_args.alltests, cli_args.make_options) result = run_tests(linux, request) @@ -260,6 +279,7 @@ def main(argv, linux=None): linux = kunit_kernel.LinuxSourceTree() request = KunitConfigRequest(cli_args.build_dir, + uml_rootfs_path, cli_args.make_options) result = config_tests(linux, request) kunit_parser.print_with_timestamp(( @@ -277,6 +297,7 @@ def main(argv, linux=None): request = KunitBuildRequest(cli_args.jobs, cli_args.build_dir, + uml_rootfs_path, cli_args.alltests, cli_args.make_options) result = build_tests(linux, request) @@ -295,6 +316,7 @@ def main(argv, linux=None): exec_request = KunitExecRequest(cli_args.timeout, cli_args.build_dir, + uml_rootfs_path, cli_args.alltests) exec_result = exec_tests(linux, exec_request) parse_request = KunitParseRequest(cli_args.raw_output, diff --git a/tools/testing/kunit/kunit_kernel.py b/tools/testing/kunit/kunit_kernel.py index 63dbda2d029f..aeffd024816c 100644 --- a/tools/testing/kunit/kunit_kernel.py +++ b/tools/testing/kunit/kunit_kernel.py @@ -11,6 +11,7 @@ import logging import subprocess import os import signal +import time from contextlib import ExitStack @@ -19,8 +20,58 @@ import kunit_parser KCONFIG_PATH = '.config' kunitconfig_path = '.kunitconfig' +X86_64_DEFCONFIG_PATH = 'arch/um/configs/x86_64_defconfig' BROKEN_ALLCONFIG_PATH = 'tools/testing/kunit/configs/broken_on_uml.config' +MAKE_CMD = { + 'make': { + 'command': ['make', 'ARCH=um'], + 'msg_error': 'Could not call execute make: ', + }, + 'make_modules': { + 'command': ['make', 'modules', 'ARCH=um'], + 'msg_error': 'Could not call execute make modules: ', + }, + 'make_modules_install': { + 'command': ['make', 'modules_install', 'ARCH=um'], + 'msg_error': 'Could not call execute make modules_install: ', + } +} + +def halt_uml(): + try: + subprocess.call(['uml_mconsole', 'kunitid', 'halt']) + except OSError as e: + raise ConfigError('Could not call uml_mconsole ' + e) + except subprocess.CalledProcessError as e: + raise ConfigError(e.output) + +def enable_uml_modules_on_boot(output_command, uml_rootfs_path): + uml_modules_path = None + found_kernel_version = False + modules = [] + for i in output_command.decode('utf-8').split(): + if found_kernel_version: + kernel_version = i + uml_modules_path = os.path.join(uml_rootfs_path, + 'lib/modules/', kernel_version, 'kernel/lib/') + break + if 'DEPMOD' in i: + found_kernel_version = True + + try: + if os.path.exists(uml_modules_path): + modules = subprocess.check_output(['ls', + uml_modules_path]).decode('utf-8').split() + except OSError as e: + raise ConfigError('Could not list directory ' + e) + except subprocess.CalledProcessError as e: + raise ConfigError(e.output) + + with open(os.path.join(uml_rootfs_path, 'etc/modules'), 'w') as f: + for i in modules: + f.write(i.replace('.ko', '')) + class ConfigError(Exception): """Represents an error trying to configure the Linux kernel.""" @@ -70,20 +121,28 @@ class LinuxSourceTreeOperations(object): kunit_parser.print_with_timestamp( 'Starting Kernel with all configs takes a few minutes...') - def make(self, jobs, build_dir, make_options): - command = ['make', 'ARCH=um', '--jobs=' + str(jobs)] + def make(self, cmd, jobs, build_dir, uml_rootfs_path, make_options): + command = MAKE_CMD[cmd]['command'] + ['--jobs=' + str(jobs)] + if make_options: command.extend(make_options) if build_dir: command += ['O=' + build_dir] + + if cmd == 'make_modules_install': + command += ['INSTALL_MOD_PATH=' + uml_rootfs_path] + try: - subprocess.check_output(command) + output = subprocess.check_output(command) + if cmd == 'make_modules_install': + enable_uml_modules_on_boot(output, + uml_rootfs_path) except OSError as e: - raise BuildError('Could not call execute make: ' + e) + raise BuildError(MAKE_CMD[cmd]['msg_error'] + e) except subprocess.CalledProcessError as e: raise BuildError(e.output) - def linux_bin(self, params, timeout, build_dir, outfile): + def linux_bin(self, params, timeout, build_dir, uml_rootfs_path, outfile): """Runs the Linux UML binary. Must be named 'linux'.""" linux_bin = './linux' if build_dir: @@ -92,7 +151,11 @@ class LinuxSourceTreeOperations(object): process = subprocess.Popen([linux_bin] + params, stdout=output, stderr=subprocess.STDOUT) - process.wait(timeout) + if uml_rootfs_path: + time.sleep(timeout) + halt_uml() + else: + process.wait(timeout) def get_kconfig_path(build_dir): @@ -132,11 +195,16 @@ class LinuxSourceTree(object): return False return True - def build_config(self, build_dir, make_options): + def build_config(self, build_dir, uml_rootfs_path, make_options): kconfig_path = get_kconfig_path(build_dir) if build_dir and not os.path.exists(build_dir): os.mkdir(build_dir) self._kconfig.write_to_file(kconfig_path) + + if uml_rootfs_path: + with open(kconfig_path, 'a') as fw: + with open(X86_64_DEFCONFIG_PATH, 'r') as fr: + fw.write(fr.read()) try: self._ops.make_olddefconfig(build_dir, make_options) except ConfigError as e: @@ -144,7 +212,7 @@ class LinuxSourceTree(object): return False return self.validate_config(build_dir) - def build_reconfig(self, build_dir, make_options): + def build_reconfig(self, build_dir, uml_rootfs_path, make_options): """Creates a new .config if it is not a subset of the .kunitconfig.""" kconfig_path = get_kconfig_path(build_dir) if os.path.exists(kconfig_path): @@ -153,28 +221,38 @@ class LinuxSourceTree(object): if not self._kconfig.is_subset_of(existing_kconfig): print('Regenerating .config ...') os.remove(kconfig_path) - return self.build_config(build_dir, make_options) + return self.build_config(build_dir, uml_rootfs_path, + make_options) else: return True else: print('Generating .config ...') - return self.build_config(build_dir, make_options) + return self.build_config(build_dir, uml_rootfs_path, make_options) - def build_um_kernel(self, alltests, jobs, build_dir, make_options): + def build_um_kernel(self, alltests, jobs, build_dir, + uml_rootfs_path, make_options): if alltests: self._ops.make_allyesconfig() try: self._ops.make_olddefconfig(build_dir, make_options) - self._ops.make(jobs, build_dir, make_options) + self._ops.make('make', jobs, build_dir, uml_rootfs_path, make_options) + if uml_rootfs_path: + self._ops.make('make_modules', jobs, build_dir, + uml_rootfs_path, make_options) + self._ops.make('make_modules_install', jobs, + build_dir, uml_rootfs_path, make_options) except (ConfigError, BuildError) as e: logging.error(e) return False return self.validate_config(build_dir) - def run_kernel(self, args=[], build_dir='', timeout=None): + def run_kernel(self, args=[], build_dir='', uml_rootfs_path=None, timeout=None): args.extend(['mem=1G']) + if uml_rootfs_path: + args.extend(['root=/dev/root', 'rootfstype=hostfs', + 'rootflags=' + uml_rootfs_path, 'umid=kunitid']) outfile = 'test.log' - self._ops.linux_bin(args, timeout, build_dir, outfile) + self._ops.linux_bin(args, timeout, build_dir, uml_rootfs_path, outfile) subprocess.call(['stty', 'sane']) with open(outfile, 'r') as file: for line in file: diff --git a/tools/testing/kunit/kunit_tool_test.py b/tools/testing/kunit/kunit_tool_test.py index ebedfc57a39b..2359c77c8efb 100755 --- a/tools/testing/kunit/kunit_tool_test.py +++ b/tools/testing/kunit/kunit_tool_test.py @@ -247,14 +247,14 @@ class KUnitMainTest(unittest.TestCase): def test_build_passes_args_pass(self): kunit.main(['build'], self.linux_source_mock) assert self.linux_source_mock.build_reconfig.call_count == 0 - self.linux_source_mock.build_um_kernel.assert_called_once_with(False, 8, '.kunit', None) + self.linux_source_mock.build_um_kernel.assert_called_once_with(False, 8, '.kunit', None, None) assert self.linux_source_mock.run_kernel.call_count == 0 def test_exec_passes_args_pass(self): kunit.main(['exec'], self.linux_source_mock) assert self.linux_source_mock.build_reconfig.call_count == 0 assert self.linux_source_mock.run_kernel.call_count == 1 - self.linux_source_mock.run_kernel.assert_called_once_with(build_dir='.kunit', timeout=300) + self.linux_source_mock.run_kernel.assert_called_once_with(build_dir='.kunit', uml_rootfs_path=None, timeout=300) self.print_mock.assert_any_call(StrContains('Testing complete.')) def test_run_passes_args_pass(self): @@ -262,7 +262,7 @@ class KUnitMainTest(unittest.TestCase): assert self.linux_source_mock.build_reconfig.call_count == 1 assert self.linux_source_mock.run_kernel.call_count == 1 self.linux_source_mock.run_kernel.assert_called_once_with( - build_dir='.kunit', timeout=300) + build_dir='.kunit', uml_rootfs_path=None, timeout=300) self.print_mock.assert_any_call(StrContains('Testing complete.')) def test_exec_passes_args_fail(self): @@ -302,7 +302,7 @@ class KUnitMainTest(unittest.TestCase): def test_exec_timeout(self): timeout = 3453 kunit.main(['exec', '--timeout', str(timeout)], self.linux_source_mock) - self.linux_source_mock.run_kernel.assert_called_once_with(build_dir='.kunit', timeout=timeout) + self.linux_source_mock.run_kernel.assert_called_once_with(build_dir='.kunit', uml_rootfs_path=None, timeout=timeout) self.print_mock.assert_any_call(StrContains('Testing complete.')) def test_run_timeout(self): @@ -310,7 +310,7 @@ class KUnitMainTest(unittest.TestCase): kunit.main(['run', '--timeout', str(timeout)], self.linux_source_mock) assert self.linux_source_mock.build_reconfig.call_count == 1 self.linux_source_mock.run_kernel.assert_called_once_with( - build_dir='.kunit', timeout=timeout) + build_dir='.kunit', uml_rootfs_path=None, timeout=timeout) self.print_mock.assert_any_call(StrContains('Testing complete.')) def test_run_builddir(self): @@ -318,7 +318,7 @@ class KUnitMainTest(unittest.TestCase): kunit.main(['run', '--build_dir=.kunit'], self.linux_source_mock) assert self.linux_source_mock.build_reconfig.call_count == 1 self.linux_source_mock.run_kernel.assert_called_once_with( - build_dir=build_dir, timeout=300) + build_dir=build_dir, uml_rootfs_path=None, timeout=300) self.print_mock.assert_any_call(StrContains('Testing complete.')) def test_config_builddir(self): @@ -329,12 +329,12 @@ class KUnitMainTest(unittest.TestCase): def test_build_builddir(self): build_dir = '.kunit' kunit.main(['build', '--build_dir', build_dir], self.linux_source_mock) - self.linux_source_mock.build_um_kernel.assert_called_once_with(False, 8, build_dir, None) + self.linux_source_mock.build_um_kernel.assert_called_once_with(False, 8, build_dir, None, None) def test_exec_builddir(self): build_dir = '.kunit' kunit.main(['exec', '--build_dir', build_dir], self.linux_source_mock) - self.linux_source_mock.run_kernel.assert_called_once_with(build_dir=build_dir, timeout=300) + self.linux_source_mock.run_kernel.assert_called_once_with(build_dir=build_dir, uml_rootfs_path=None, timeout=300) self.print_mock.assert_any_call(StrContains('Testing complete.')) if __name__ == '__main__': base-commit: 725aca9585956676687c4cb803e88f770b0df2b2 prerequisite-patch-id: 5e5f9a8a05c5680fda1b04c9ab1b95ce91dc88b2 prerequisite-patch-id: 4d997940f4a9f303424af9bac412de1af861f9d9 prerequisite-patch-id: 582b6d9d28ce4b71628890ec832df6522ca68de0 -- 2.26.2

4 years, 6 months

1
0
0 0

[PATCH v2] selftests: net: ip_defrag: modprobe missing nf_defrag_ipv6 support

by Paolo Pisati

Fix ip_defrag.sh when CONFIG_NF_DEFRAG_IPV6=m: $ sudo ./ip_defrag.sh + set -e + mktemp -u XXXXXX + readonly NETNS=ns-rGlXcw + trap cleanup EXIT + setup + ip netns add ns-rGlXcw + ip -netns ns-rGlXcw link set lo up + ip netns exec ns-rGlXcw sysctl -w net.ipv4.ipfrag_high_thresh=9000000 + ip netns exec ns-rGlXcw sysctl -w net.ipv4.ipfrag_low_thresh=7000000 + ip netns exec ns-rGlXcw sysctl -w net.ipv4.ipfrag_time=1 + ip netns exec ns-rGlXcw sysctl -w net.ipv6.ip6frag_high_thresh=9000000 + ip netns exec ns-rGlXcw sysctl -w net.ipv6.ip6frag_low_thresh=7000000 + ip netns exec ns-rGlXcw sysctl -w net.ipv6.ip6frag_time=1 + ip netns exec ns-rGlXcw sysctl -w net.netfilter.nf_conntrack_frag6_high_thresh=9000000 + cleanup + ip netns del ns-rGlXcw $ ls -la /proc/sys/net/netfilter/nf_conntrack_frag6_high_thresh ls: cannot access '/proc/sys/net/netfilter/nf_conntrack_frag6_high_thresh': No such file or directory $ sudo modprobe nf_defrag_ipv6 $ ls -la /proc/sys/net/netfilter/nf_conntrack_frag6_high_thresh -rw-r--r-- 1 root root 0 Jul 14 12:34 /proc/sys/net/netfilter/nf_conntrack_frag6_high_thresh Signed-off-by: Paolo Pisati <paolo.pisati(a)canonical.com> --- tools/testing/selftests/net/ip_defrag.sh | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tools/testing/selftests/net/ip_defrag.sh b/tools/testing/selftests/net/ip_defrag.sh index 15d3489ecd9c..ceb7ad4dbd94 100755 --- a/tools/testing/selftests/net/ip_defrag.sh +++ b/tools/testing/selftests/net/ip_defrag.sh @@ -6,6 +6,8 @@ set +x set -e +modprobe -q nf_defrag_ipv6 + readonly NETNS="ns-$(mktemp -u XXXXXX)" setup() { -- 2.25.1

4 years, 6 months

3
2
0 0

[RFC 0/3] kunit: add support to use modules

by Vitor Massaru Iha

Currently, KUnit does not allow the use of tests as a module. This prevents the implementation of tests that require userspace. This patchset makes this possible by introducing the use of the root filesystem in KUnit. And it allows the use of tests that can be compiled as a module Vitor Massaru Iha (3): kunit: tool: Add support root filesystem in kunit-tool lib: Allows to borrow mm in userspace on KUnit lib: Convert test_user_copy to KUnit test include/kunit/test.h | 1 + lib/Kconfig.debug | 17 ++ lib/Makefile | 2 +- lib/kunit/try-catch.c | 15 +- lib/{test_user_copy.c => user_copy_kunit.c} | 196 +++++++++----------- tools/testing/kunit/kunit.py | 37 +++- tools/testing/kunit/kunit_kernel.py | 105 +++++++++-- 7 files changed, 238 insertions(+), 135 deletions(-) rename lib/{test_user_copy.c => user_copy_kunit.c} (55%) base-commit: 725aca9585956676687c4cb803e88f770b0df2b2 prerequisite-patch-id: 582b6d9d28ce4b71628890ec832df6522ca68de0 -- 2.26.2

4 years, 6 months

4
16
0 0

[PATCH] selftests: net: ip_defrag: modprobe missing nf_defrag_ipv6 support

by Paolo Pisati

Fix ip_defrag.sh when CONFIG_NF_DEFRAG_IPV6=m: $ sudo ./ip_defrag.sh + set -e + mktemp -u XXXXXX + readonly NETNS=ns-rGlXcw + trap cleanup EXIT + setup + ip netns add ns-rGlXcw + ip -netns ns-rGlXcw link set lo up + ip netns exec ns-rGlXcw sysctl -w net.ipv4.ipfrag_high_thresh=9000000 + ip netns exec ns-rGlXcw sysctl -w net.ipv4.ipfrag_low_thresh=7000000 + ip netns exec ns-rGlXcw sysctl -w net.ipv4.ipfrag_time=1 + ip netns exec ns-rGlXcw sysctl -w net.ipv6.ip6frag_high_thresh=9000000 + ip netns exec ns-rGlXcw sysctl -w net.ipv6.ip6frag_low_thresh=7000000 + ip netns exec ns-rGlXcw sysctl -w net.ipv6.ip6frag_time=1 + ip netns exec ns-rGlXcw sysctl -w net.netfilter.nf_conntrack_frag6_high_thresh=9000000 + cleanup + ip netns del ns-rGlXcw $ ls -la /proc/sys/net/netfilter/nf_conntrack_frag6_high_thresh ls: cannot access '/proc/sys/net/netfilter/nf_conntrack_frag6_high_thresh': No such file or directory $ sudo modprobe nf_defrag_ipv6 $ ls -la /proc/sys/net/netfilter/nf_conntrack_frag6_high_thresh -rw-r--r-- 1 root root 0 Jul 14 12:34 /proc/sys/net/netfilter/nf_conntrack_frag6_high_thresh Signed-off-by: Paolo Pisati <paolo.pisati(a)canonical.com> --- tools/testing/selftests/net/ip_defrag.sh | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tools/testing/selftests/net/ip_defrag.sh b/tools/testing/selftests/net/ip_defrag.sh index 15d3489ecd9c..6919afe47e0a 100755 --- a/tools/testing/selftests/net/ip_defrag.sh +++ b/tools/testing/selftests/net/ip_defrag.sh @@ -3,6 +3,8 @@ # # Run a couple of IP defragmentation tests. +modprobe -q nf_defrag_ipv6 + set +x set -e -- 2.25.1

4 years, 6 months

2
4
0 0

[PATCH v3 0/3] selftests: powerpc: Fixes and execute-disable test for pkeys

by Sandipan Das

This fixes the way the Authority Mask Register (AMR) is updated by the existing pkey tests and adds a new test to verify the functionality of execute-disabled pkeys. Previous versions can be found at: v2: https://lore.kernel.org/linuxppc-dev/20200527030342.13712-1-sandipan@linux.… v1: https://lore.kernel.org/linuxppc-dev/20200508162332.65316-1-sandipan@linux.… Changes in v3: - Fixed AMR writes for existing pkey tests (new patch). - Moved Hash MMU check under utilities (new patch) and removed duplicate code. - Fixed comments on why the pkey permission bits were redefined. - Switched to existing mfspr() macro for reading AMR. - Switched to sig_atomic_t as data type for variables updated in the signal handlers. - Switched to exit()-ing if the signal handlers come across an unexpected condition instead of trying to reset page and pkey permissions. - Switched to write() from printf() for printing error messages from the signal handlers. - Switched to getpagesize(). - Renamed fault counter to denote remaining faults. - Dropped unnecessary randomization for choosing an address to fault at. - Added additional information on change in permissions due to AMR and IAMR bits in comments. - Switched the first instruction word of the executable region to a trap to test if it is actually overwritten by a no-op later. - Added an new test scenario where the pkey imposes no restrictions and an attempt is made to jump to the executable region again. Changes in v2: - Added .gitignore entry for test binary. - Fixed builds for older distros where siginfo_t might not have si_pkey as a formal member based on discussion with Michael. Sandipan Das (3): selftests: powerpc: Fix pkey access right updates selftests: powerpc: Move Hash MMU check to utilities selftests: powerpc: Add test for execute-disabled pkeys tools/testing/selftests/powerpc/include/reg.h | 6 + .../testing/selftests/powerpc/include/utils.h | 1 + tools/testing/selftests/powerpc/mm/.gitignore | 1 + tools/testing/selftests/powerpc/mm/Makefile | 5 +- .../selftests/powerpc/mm/bad_accesses.c | 28 -- .../selftests/powerpc/mm/pkey_exec_prot.c | 388 ++++++++++++++++++ .../selftests/powerpc/ptrace/core-pkey.c | 2 +- .../selftests/powerpc/ptrace/ptrace-pkey.c | 2 +- tools/testing/selftests/powerpc/utils.c | 28 ++ 9 files changed, 429 insertions(+), 32 deletions(-) create mode 100644 tools/testing/selftests/powerpc/mm/pkey_exec_prot.c -- 2.25.1

4 years, 6 months

2
4
0 0

Re: [PATCH 2/4] KVM: x86: Introduce paravirt feature CR0/CR4 pinning

by Andersen, John

On Thu, Jul 09, 2020 at 09:27:43AM -0700, Andy Lutomirski wrote: > On Thu, Jul 9, 2020 at 9:22 AM Dave Hansen <dave.hansen(a)intel.com> wrote: > > > > On 7/9/20 9:07 AM, Andy Lutomirski wrote: > > > On Thu, Jul 9, 2020 at 8:56 AM Dave Hansen <dave.hansen(a)intel.com> wrote: > > >> On 7/9/20 8:44 AM, Andersen, John wrote: > > >>> Bits which are allowed to be pinned default to WP for CR0 and SMEP, > > >>> SMAP, and UMIP for CR4. > > >> I think it also makes sense to have FSGSBASE in this set. > > >> > > >> I know it hasn't been tested, but I think we should do the legwork to > > >> test it. If not in this set, can we agree that it's a logical next step? > > > I have no objection to pinning FSGSBASE, but is there a clear > > > description of the threat model that this whole series is meant to > > > address? The idea is to provide a degree of protection against an > > > attacker who is able to convince a guest kernel to write something > > > inappropriate to CR4, right? How realistic is this? > > > > If a quick search can find this: > > > > > https://googleprojectzero.blogspot.com/2017/05/exploiting-linux-kernel-via-… > > > > I'd pretty confident that the guys doing actual bad things have it in > > their toolbox too. > > > > True, but we have the existing software CR4 pinning. I suppose the > virtualization version is stronger. > Yes, as Kees said this will be stronger because it stops ROP and other gadget based techniques which avoid the use of native_write_cr0/4(). With regards to what should be done in this patchset and what in other patchsets. I have a fix for kexec thanks to Arvind's note about TRAMPOLINE_32BIT_CODE_SIZE. The physical host boots fine now and the virtual one can kexec fine. What remains to be done on that front is to add some identifying information to the kernel image to declare that it supports paravirtualized control register pinning or not. Liran suggested adding a section to the built image acting as a flag to signify support for being kexec'd by a kernel with pinning enabled. If anyone has any opinions on how they'd like to see this implemented please let me know. Otherwise I'll just take a stab at it and you'll all see it hopefully in the next version. With regards to FSGSBASE, are we open to validating and adding that to the DEFAULT set as a part of a separate patchset? This patchset is focused on replicating the functionality we already have natively. (If anyone got this email twice, sorry I messed up the From: field the first time around)

4 years, 6 months

2
2
0 0

Re: [PATCH 0/3] readfile(2): a new syscall to make open/read/close faster

by Jan Ziak

Hello At first, I thought that the proposed system call is capable of reading *multiple* small files using a single system call - which would help increase HDD/SSD queue utilization and increase IOPS (I/O operations per second) - but that isn't the case and the proposed system call can read just a single file. Without the ability to read multiple small files using a single system call, it is impossible to increase IOPS (unless an application is using multiple reader threads or somehow instructs the kernel to prefetch multiple files into memory). While you are at it, why not also add a readfiles system call to read multiple, presumably small, files? The initial unoptimized implementation of readfiles syscall can simply call readfile sequentially. Sincerely Jan (atomsymbol)

4 years, 6 months

8
26
0 0

[PATCH v2] selftests/livepatch: adopt to newer sysctl error format

by Petr Mladek

With procfs v3.3.16, the sysctl command doesn't print the set key and value on error. This change breaks livepatch selftest test-ftrace.sh, that tests the interaction of sysctl ftrace_enabled: Make it work with all sysctl versions using '-q' option. Explicitly print the final status on success so that it can be verified in the log. The error message is enough on failure. Reported-by: Kamalesh Babulal <kamalesh(a)linux.vnet.ibm.com> Signed-off-by: Petr Mladek <pmladek(a)suse.com> --- The patch has been created against livepatch.git, branch for-5.9/selftests-cleanup. But it applies also against the current Linus' tree. tools/testing/selftests/livepatch/functions.sh | 3 ++- tools/testing/selftests/livepatch/test-ftrace.sh | 2 +- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/tools/testing/selftests/livepatch/functions.sh b/tools/testing/selftests/livepatch/functions.sh index 408529d94ddb..1aba83c87ad3 100644 --- a/tools/testing/selftests/livepatch/functions.sh +++ b/tools/testing/selftests/livepatch/functions.sh @@ -75,7 +75,8 @@ function set_dynamic_debug() { } function set_ftrace_enabled() { - result=$(sysctl kernel.ftrace_enabled="$1" 2>&1 | paste --serial --delimiters=' ') + result=$(sysctl -q kernel.ftrace_enabled="$1" 2>&1 && \ + sysctl kernel.ftrace_enabled 2>&1) echo "livepatch: $result" > /dev/kmsg } diff --git a/tools/testing/selftests/livepatch/test-ftrace.sh b/tools/testing/selftests/livepatch/test-ftrace.sh index 9160c9ec3b6f..552e165512f4 100755 --- a/tools/testing/selftests/livepatch/test-ftrace.sh +++ b/tools/testing/selftests/livepatch/test-ftrace.sh @@ -51,7 +51,7 @@ livepatch: '$MOD_LIVEPATCH': initializing patching transition livepatch: '$MOD_LIVEPATCH': starting patching transition livepatch: '$MOD_LIVEPATCH': completing patching transition livepatch: '$MOD_LIVEPATCH': patching complete -livepatch: sysctl: setting key \"kernel.ftrace_enabled\": Device or resource busy kernel.ftrace_enabled = 0 +livepatch: sysctl: setting key \"kernel.ftrace_enabled\": Device or resource busy % echo 0 > /sys/kernel/livepatch/$MOD_LIVEPATCH/enabled livepatch: '$MOD_LIVEPATCH': initializing unpatching transition livepatch: '$MOD_LIVEPATCH': starting unpatching transition -- 2.26.2

4 years, 6 months

4
4
0 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

Linux-kselftest-mirror