May 2022 - Linux-kselftest-mirror

[PATCH v2] kunit: tool: cosmetic: don't specify duplicate kernel cmdline options

by Daniel Latypov

Context: When using a non-UML arch, kunit.py will boot the test kernel with options like these by default (this is x86_64): > mem=1G console=tty kunit_shutdown=halt console=ttyS0 kunit_shutdown=reboot The first three options are added unconditionally but are only intended for UML. 1. 'mem=1G' is redundant with the '-m 1024' that we hard-code into the qemu commandline. 2. We specify a 'console' for all tools/testing/kunit/qemu_configs/*.py already, so 'console=tty' gets overwritten. 3. For QEMU, we need to use 'reboot', and for UML we need to use 'halt'. If you switch them, kunit.py will hang until the --timeout expires. This patch: Having these duplicate options is a bit noisy. Switch so we only add UML-specific options for UML. I.e. we now get UML: 'mem=1G console=tty kunit_shutdown=halt' (unchanged) x86_64: 'console=ttyS0 kunit_shutdown=reboot' Side effect: you can't overwrite these options on UML w/ --kernel_arg. But you already couldn't for QEMU (console, kunit_shutdown), and why would you want to? Signed-off-by: Daniel Latypov <dlatypov(a)google.com> Reviewed-by: David Gow <davidgow(a)google.com> --- v1 -> v2: Remove other UML-specific args from other arches. I.e. also only specify "mem=1G and console=tty" for UML. --- tools/testing/kunit/kunit_kernel.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/testing/kunit/kunit_kernel.py b/tools/testing/kunit/kunit_kernel.py index 1b9c4922a675..8945640b5063 100644 --- a/tools/testing/kunit/kunit_kernel.py +++ b/tools/testing/kunit/kunit_kernel.py @@ -160,6 +160,7 @@ class LinuxSourceTreeOperationsUml(LinuxSourceTreeOperations): def start(self, params: List[str], build_dir: str) -> subprocess.Popen: """Runs the Linux UML binary. Must be named 'linux'.""" linux_bin = os.path.join(build_dir, 'linux') + params.extend(['mem=1G', 'console=tty', 'kunit_shutdown=halt']) return subprocess.Popen([linux_bin] + params, stdin=subprocess.PIPE, stdout=subprocess.PIPE, @@ -334,7 +335,6 @@ class LinuxSourceTree(object): def run_kernel(self, args=None, build_dir='', filter_glob='', timeout=None) -> Iterator[str]: if not args: args = [] - args.extend(['mem=1G', 'console=tty', 'kunit_shutdown=halt']) if filter_glob: args.append('kunit.filter_glob='+filter_glob) base-commit: 9660209d9418f2295d31fea0d32e313e9b2c1200 -- 2.36.0.550.gb090851708-goog

2 years, 5 months

2
1
0 0

[PATCH 1/3] kunit: tool: drop unused load_config argument

by Daniel Latypov

It's always set to true except in one test case. And in that test case it can safely be set to true anyways. Signed-off-by: Daniel Latypov <dlatypov(a)google.com> --- tools/testing/kunit/kunit_kernel.py | 4 ---- tools/testing/kunit/kunit_tool_test.py | 2 +- 2 files changed, 1 insertion(+), 5 deletions(-) diff --git a/tools/testing/kunit/kunit_kernel.py b/tools/testing/kunit/kunit_kernel.py index 3539efaf99ba..8bc8305ba817 100644 --- a/tools/testing/kunit/kunit_kernel.py +++ b/tools/testing/kunit/kunit_kernel.py @@ -219,7 +219,6 @@ class LinuxSourceTree: def __init__( self, build_dir: str, - load_config=True, kunitconfig_path='', kconfig_add: Optional[List[str]]=None, arch=None, @@ -233,9 +232,6 @@ class LinuxSourceTree: self._arch = 'um' if arch is None else arch self._ops = get_source_tree_ops(self._arch, cross_compile) - if not load_config: - return - if kunitconfig_path: if os.path.isdir(kunitconfig_path): kunitconfig_path = os.path.join(kunitconfig_path, KUNITCONFIG_PATH) diff --git a/tools/testing/kunit/kunit_tool_test.py b/tools/testing/kunit/kunit_tool_test.py index 25a2eb3bf114..b9158017ece6 100755 --- a/tools/testing/kunit/kunit_tool_test.py +++ b/tools/testing/kunit/kunit_tool_test.py @@ -393,7 +393,7 @@ class LinuxSourceTreeTest(unittest.TestCase): return subprocess.Popen(['echo "hi\nbye"'], shell=True, text=True, stdout=subprocess.PIPE) with tempfile.TemporaryDirectory('') as build_dir: - tree = kunit_kernel.LinuxSourceTree(build_dir, load_config=False) + tree = kunit_kernel.LinuxSourceTree(build_dir) mock.patch.object(tree._ops, 'start', side_effect=fake_start).start() with self.assertRaises(ValueError): base-commit: 8a7ccad38f8b25c8202efd69371a022357286400 -- 2.36.1.124.g0e6072fb45-goog

2 years, 5 months

3
10
0 0

[PATCH] kunit: use kmemdup in kunit_filter_tests(), take suite as const

by Daniel Latypov

kmemdup() is easier than kmalloc() + memcpy(), per lkp bot. Also make the input `suite` as const since we're now always making copies after commit a127b154a8f2 ("kunit: tool: allow filtering test cases via glob"). Reported-by: kernel test robot <lkp(a)intel.com> Signed-off-by: Daniel Latypov <dlatypov(a)google.com> --- lib/kunit/executor.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/lib/kunit/executor.c b/lib/kunit/executor.c index 2f73a6a35a7e..43e15112460a 100644 --- a/lib/kunit/executor.c +++ b/lib/kunit/executor.c @@ -55,7 +55,7 @@ static void kunit_parse_filter_glob(struct kunit_test_filter *parsed, /* Create a copy of suite with only tests that match test_glob. */ static struct kunit_suite * -kunit_filter_tests(struct kunit_suite *const suite, const char *test_glob) +kunit_filter_tests(const struct kunit_suite *const suite, const char *test_glob) { int n = 0; struct kunit_case *filtered, *test_case; @@ -69,11 +69,9 @@ kunit_filter_tests(struct kunit_suite *const suite, const char *test_glob) if (n == 0) return NULL; - /* Use memcpy to workaround copy->name being const. */ - copy = kmalloc(sizeof(*copy), GFP_KERNEL); + copy = kmemdup(suite, sizeof(*copy), GFP_KERNEL); if (!copy) return ERR_PTR(-ENOMEM); - memcpy(copy, suite, sizeof(*copy)); filtered = kcalloc(n + 1, sizeof(*filtered), GFP_KERNEL); if (!filtered) base-commit: 9660209d9418f2295d31fea0d32e313e9b2c1200 -- 2.36.0.550.gb090851708-goog

2 years, 5 months

3
2
0 0

Re: [PATCH 2/2] kvm: rename KVM_MAX_VCPU_ID to, KVM_MAX_VCPU_IDS

by Christian Zigotzky

Hello Juergen, Hello All, Since the RC1 of kernel 5.13, -smp 2 and -smp 4 don't work with a virtual e5500 QEMU KVM-HV machine anymore. [1] I see in the serial console, that the uImage doesn't load. I use the following QEMU command for booting: qemu-system-ppc64 -M ppce500 -cpu e5500 -enable-kvm -m 1024 -kernel uImage -drive format=raw,file=MintPPC32-X5000.img,index=0,if=virtio -netdev user,id=mynet0 -device virtio-net,netdev=mynet0 -append "rw root=/dev/vda" -device virtio-vga -device virtio-mouse-pci -device virtio-keyboard-pci -device pci-ohci,id=newusb -device usb-audio,bus=newusb.0 -smp 4 The kernels boot without KVM-HV. Summary for KVM-HV: -smp 1 -> works -smp 2 -> doesn't work -smp 3 -> works -smp 4 -> doesn't work I used -smp 4 before the RC1 of kernel 5.13 because my FSL P5040 BookE machine [2] has 4 cores. Does this patch solve this issue? [3] Thanks, Christian [1] https://lists.ozlabs.org/pipermail/linuxppc-dev/2021-May/229103.html [2] http://wiki.amiga.org/index.php?title=X5000 [3] https://lists.ozlabs.org/pipermail/linuxppc-dev/2021-September/234152.html

2 years, 5 months

1
1
0 0

[PATCH v11 0/8] cgroup/cpuset: cpu partition code enhancements

by Waiman Long

v11: - Fix incorrect spacing in patch 7 and include documentation suggestions by Michal. - Move partition_is_populated() check to the last one in list of conditions to be checked. v10: - Relax constraints for changes made to "cpuset.cpus" and "cpuset.cpus.partition" as suggested. Now almost all changes are allowed. - Add patch 1 to signal that we may need to do additional work in the future to relax the constraint that tasks' cpumask may need some adjustment if child partitions are present. - Add patch 2 for miscellaneous cleanups. This patchset include the following enhancements to the cpuset v2 partition code. 1) Allow partitions that have no task to have empty effective cpus. 2) Relax the constraints on what changes are allowed in cpuset.cpus and cpuset.cpus.partition. However, the partition remain invalid until the constraints of a valid partition root is satisfied. 3) Add a new "isolated" partition type for partitions with no load balancing which is available in v1 but not yet in v2. 4) Allow the reading of cpuset.cpus.partition to include a reason string as to why the partition remain invalid. In addition, the cgroup-v2.rst documentation file is updated and a self test is added to verify the correctness the partition code. The code diff from v10 is listed below. Waiman Long (8): cgroup/cpuset: Add top_cpuset check in update_tasks_cpumask() cgroup/cpuset: Miscellaneous cleanups & add helper functions cgroup/cpuset: Allow no-task partition to have empty cpuset.cpus.effective cgroup/cpuset: Relax constraints to partition & cpus changes cgroup/cpuset: Add a new isolated cpus.partition type cgroup/cpuset: Show invalid partition reason string cgroup/cpuset: Update description of cpuset.cpus.partition in cgroup-v2.rst kselftest/cgroup: Add cpuset v2 partition root state test Documentation/admin-guide/cgroup-v2.rst | 149 ++-- kernel/cgroup/cpuset.c | 718 +++++++++++------- tools/testing/selftests/cgroup/Makefile | 5 +- .../selftests/cgroup/test_cpuset_prs.sh | 674 ++++++++++++++++ tools/testing/selftests/cgroup/wait_inotify.c | 87 +++ 5 files changed, 1304 insertions(+), 329 deletions(-) create mode 100755 tools/testing/selftests/cgroup/test_cpuset_prs.sh create mode 100644 tools/testing/selftests/cgroup/wait_inotify.c -- diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst index 94e1e3771830..9184a09e0fc9 100644 --- a/Documentation/admin-guide/cgroup-v2.rst +++ b/Documentation/admin-guide/cgroup-v2.rst @@ -2130,10 +2130,10 @@ Cpuset Interface Files CPUs should be carefully distributed and bound to each of the individual CPUs for optimal performance. - The value shown in "cpuset.cpus.effective" of a partition root is - the CPUs that the parent partition root can dedicate to the new - partition root. They are subtracted from "cpuset.cpus.effective" - of the parent and may be different from "cpuset.cpus" + The value shown in "cpuset.cpus.effective" of a partition root + is the CPUs that the partition root can dedicate to a potential + new child partition root. The new child subtracts available + CPUs from its parent "cpuset.cpus.effective". A partition root ("root" or "isolated") can be in one of the two possible states - valid or invalid. An invalid partition @@ -2165,24 +2165,28 @@ Cpuset Interface Files 2) The parent cgroup is a valid partition root. 3) The "cpuset.cpus" is not empty and must contain at least one of the CPUs from parent's "cpuset.cpus", i.e. they overlap. - 4) The "cpuset.cpus.effective" must be a subset of "cpuset.cpus" - and cannot be empty unless there is no task associated with - this partition. + 4) The "cpuset.cpus.effective" must be a subset of "cpuset.cpus" + and cannot be empty unless there is no task associated with + this partition. External events like hotplug or changes to "cpuset.cpus" can cause a valid partition root to become invalid and vice versa. Note that a task cannot be moved to a cgroup with empty "cpuset.cpus.effective". - For a valid partition root or an invalid partition root with - the exclusivity rule enabled, changes made to "cpuset.cpus" - that violate the exclusivity rule will not be allowed. + For a valid partition root or an invalid partition root with + the exclusivity rule enabled, changes made to "cpuset.cpus" + that violate the exclusivity rule will not be allowed. A valid non-root parent partition may distribute out all its CPUs to its child partitions when there is no task associated with it. - Care must be taken to change a valid partition root to "member" - as all its child partitions, if present, will become invalid. + Care must be taken to change a valid partition root to + "member" as all its child partitions, if present, will become + invalid causing disruption to tasks running in those child + partitions. These inactivated partitions could be recovered if + their parent is switched back to a partition root with a proper + set of "cpuset.cpus". Poll and inotify events are triggered whenever the state of "cpuset.cpus.partition" changes. That includes changes caused diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c index 90ee0e4d8d7e..261974f5bb3c 100644 --- a/kernel/cgroup/cpuset.c +++ b/kernel/cgroup/cpuset.c @@ -1283,9 +1283,12 @@ static int update_flag(cpuset_flagbits_t bit, struct cpuset *cs, * invalid to valid violates the exclusivity rule. * * The partcmd_enable and partcmd_disable commands are used by - * update_prstate(). The partcmd_update command is used by - * update_cpumasks_hier() with newmask NULL and update_cpumask() with - * newmask set. + * update_prstate(). An error code may be returned and the caller will check + * for error. + * + * The partcmd_update command is used by update_cpumasks_hier() with newmask + * NULL and update_cpumask() with newmask set. The callers won't check for + * error and so partition_root_state and prs_error will be updated directly. */ static int update_parent_subparts_cpumask(struct cpuset *cs, int cmd, struct cpumask *newmask, @@ -1326,8 +1329,8 @@ static int update_parent_subparts_cpumask(struct cpuset *cs, int cmd, * A parent can be left with no CPU as long as there is no * task directly associated with the parent partition. */ - if (partition_is_populated(parent, cs) && - !cpumask_intersects(cs->cpus_allowed, parent->effective_cpus)) + if (!cpumask_intersects(cs->cpus_allowed, parent->effective_cpus) && + partition_is_populated(parent, cs)) return PERR_NOCPUS; cpumask_copy(tmp->addmask, cs->cpus_allowed); @@ -1361,9 +1364,10 @@ static int update_parent_subparts_cpumask(struct cpuset *cs, int cmd, * Make partition invalid if parent's effective_cpus could * become empty and there are tasks in the parent. */ - if (adding && partition_is_populated(parent, cs) && + if (adding && cpumask_subset(parent->effective_cpus, tmp->addmask) && - !cpumask_intersects(tmp->delmask, cpu_active_mask)) { + !cpumask_intersects(tmp->delmask, cpu_active_mask) && + partition_is_populated(parent, cs)) { part_error = PERR_NOCPUS; adding = false; deleting = cpumask_and(tmp->delmask, cs->cpus_allowed, @@ -1749,13 +1753,13 @@ static int update_cpumask(struct cpuset *cs, struct cpuset *trialcs, /* * Make sure that subparts_cpus, if not empty, is a subset of - * cpus_allowed. Clear subparts_cpus if there is an error or + * cpus_allowed. Clear subparts_cpus if partition not valid or * empty effective cpus with tasks. */ if (cs->nr_subparts_cpus) { - if (cs->prs_err || - (partition_is_populated(cs, NULL) && - cpumask_subset(trialcs->effective_cpus, cs->subparts_cpus))) { + if (!is_partition_valid(cs) || + (cpumask_subset(trialcs->effective_cpus, cs->subparts_cpus) && + partition_is_populated(cs, NULL))) { cs->nr_subparts_cpus = 0; cpumask_clear(cs->subparts_cpus); } else {

2 years, 5 months

5
35
0 0

[PATCH] Documentation: kunit: eliminate code-block warnings

by Randy Dunlap

Fix Sphinx complaints about code-block directive missing an argument. For start.rst, add "none" since that is already heavily used in that file. For run_wrapper.rst, use the simpler "::" literal block instead. dev-tools/kunit/start.rst:83: WARNING: Error in "code-block" directive: 1 argument(s) required, 0 supplied. dev-tools/kunit/run_wrapper.rst:17: WARNING: Error in "code-block" directive: 1 argument(s) required, 0 supplied. dev-tools/kunit/run_wrapper.rst:23: WARNING: Error in "code-block" directive: 1 argument(s) required, 0 supplied. dev-tools/kunit/run_wrapper.rst:31: WARNING: Error in "code-block" directive: 1 argument(s) required, 0 supplied. dev-tools/kunit/run_wrapper.rst:51: WARNING: Error in "code-block" directive: 1 argument(s) required, 0 supplied. dev-tools/kunit/run_wrapper.rst:57: WARNING: Error in "code-block" directive: 1 argument(s) required, 0 supplied. dev-tools/kunit/run_wrapper.rst:78: WARNING: Error in "code-block" directive: 1 argument(s) required, 0 supplied. dev-tools/kunit/run_wrapper.rst:85: WARNING: Error in "code-block" directive: 1 argument(s) required, 0 supplied. dev-tools/kunit/run_wrapper.rst:109: WARNING: Error in "code-block" directive: 1 argument(s) required, 0 supplied. dev-tools/kunit/run_wrapper.rst:116: WARNING: Error in "code-block" directive: 1 argument(s) required, 0 supplied. dev-tools/kunit/run_wrapper.rst:124: WARNING: Error in "code-block" directive: 1 argument(s) required, 0 supplied. dev-tools/kunit/run_wrapper.rst:139: WARNING: Error in "code-block" directive: 1 argument(s) required, 0 supplied. dev-tools/kunit/run_wrapper.rst:162: WARNING: Error in "code-block" directive: 1 argument(s) required, 0 supplied. Fixes: c48b9ef1f794 ("Documentation: KUnit: Rewrite getting started") Fixes: 46201d47d6c4 ("Documentation: kunit: Reorganize documentation related to running tests") Signed-off-by: Randy Dunlap <rdunlap(a)infradead.org> Cc: Brendan Higgins <brendanhiggins(a)google.com> Cc: linux-kselftest(a)vger.kernel.org Cc: kunit-dev(a)googlegroups.com Cc: Jonathan Corbet <corbet(a)lwn.net> Cc: linux-doc(a)vger.kernel.org Cc: Harinder Singh <sharinder(a)google.com> Cc: Tim Bird <tim.bird(a)sony.com> --- Documentation/dev-tools/kunit/run_wrapper.rst | 24 ++++++++-------- Documentation/dev-tools/kunit/start.rst | 2 - 2 files changed, 13 insertions(+), 13 deletions(-) --- linux-next-20220331.orig/Documentation/dev-tools/kunit/run_wrapper.rst +++ linux-next-20220331/Documentation/dev-tools/kunit/run_wrapper.rst @@ -14,13 +14,13 @@ tests, and formats the test results. Run command: -.. code-block:: +:: ./tools/testing/kunit/kunit.py run We should see the following: -.. code-block:: +:: Generating .config... Building KUnit kernel... @@ -28,7 +28,7 @@ We should see the following: We may want to use the following options: -.. code-block:: +:: ./tools/testing/kunit/kunit.py run --timeout=30 --jobs=`nproc --all @@ -48,13 +48,13 @@ test configs for certain subsystems. To use a different ``.kunitconfig`` file (such as one provided to test a particular subsystem), pass it as an option: -.. code-block:: +:: ./tools/testing/kunit/kunit.py run --kunitconfig=fs/ext4/.kunitconfig To view kunit_tool flags (optional command-line arguments), run: -.. code-block:: +:: ./tools/testing/kunit/kunit.py run --help @@ -75,14 +75,14 @@ certain code blocks, arch configs and so To create a ``.kunitconfig``, using the KUnit ``defconfig``: -.. code-block:: +:: cd $PATH_TO_LINUX_REPO cp tools/testing/kunit/configs/default.config .kunit/.kunitconfig We can then add any other Kconfig options. For example: -.. code-block:: +:: CONFIG_LIST_KUNIT_TEST=y @@ -106,14 +106,14 @@ can run part of the KUnit build process When running kunit_tool, from a ``.kunitconfig``, we can generate a ``.config`` by using the ``config`` argument: -.. code-block:: +:: ./tools/testing/kunit/kunit.py config To build a KUnit kernel from the current ``.config``, we can use the ``build`` argument: -.. code-block:: +:: ./tools/testing/kunit/kunit.py build @@ -121,7 +121,7 @@ If we already have built UML kernel with can run the kernel, and display the test results with the ``exec`` argument: -.. code-block:: +:: ./tools/testing/kunit/kunit.py exec @@ -136,7 +136,7 @@ format. When running tests, kunit_tool p a summary. To see the raw test results in TAP format, we can pass the ``--raw_output`` argument: -.. code-block:: +:: ./tools/testing/kunit/kunit.py run --raw_output @@ -159,7 +159,7 @@ By passing a bash style glob filter to t commands, we can run a subset of the tests built into a kernel . For example: if we only want to run KUnit resource tests, use: -.. code-block:: +:: ./tools/testing/kunit/kunit.py run 'kunit-resource*' --- linux-next-20220331.orig/Documentation/dev-tools/kunit/start.rst +++ linux-next-20220331/Documentation/dev-tools/kunit/start.rst @@ -80,7 +80,7 @@ Running Tests (KUnit Wrapper) If everything worked correctly, you should see the following: -.. code-block:: +.. code-block:: none Generating .config ... Building KUnit Kernel ...

2 years, 6 months

4
7
0 0

[Bug Report] - kselftest build fails if output directory is first level sub-directory

by Muhammad Usama Anjum

Hello, kselftest can be built using the kernel's top most Makefile without using kselftest's Makefile directly. But there is bug in the top most Makefile. The build fails if the specified output directory is first level sub-directory. Here is a example to reproduce this bug: make kselftest-all O=build "The Make is working in a wrong directory, that is why the relative path does not work." Masahiro Yamada. Feel free to fix it if someone pin the bug. It should be noted that the build works in some other combinations: make kselftest-all (works) make kselftest-all O=/tmp (works) make kselftest-all O=build/build2 (works) My unsuccessful attempt to fix this bug can be found here: https://lore.kernel.org/lkml/20220223191016.1658728-1-usama.anjum@collabora… Thanks, Muhammad Usama Anjum

2 years, 6 months

3
5
0 0

[PATCH v2] kunit: tool: refactor internal kconfig handling, allow overriding

by Daniel Latypov

Currently, you cannot ovewrwrite what's in your kunitconfig via --kconfig_add. Nor can you override something in a qemu_config via either means. This patch makes it so we have this level of priority * --kconfig_add * kunitconfig file (the default or the one from --kunitconfig) * qemu_config The rationale for this order is that the more "dynamic" sources of kconfig options should take priority. --kconfig_add is obviously the most dynamic. And for kunitconfig, users probably tweak the file manually or specify --kunitconfig more often than they delve into qemu_config python files. And internally, we convert the kconfigs from a python list into a set or dict fairly often. We should just use a dict internally. We exposed the set transform in the past since we didn't define __eq__, so also take the chance to shore up the kunit_kconfig.Kconfig interface. Example ======= Let's consider the unrealistic example where someone would want to disable CONFIG_KUNIT. I.e. they run $ ./tools/testing/kunit/kunit.py config --kconfig_add=CONFIG_KUNIT=n Before ------ We'd write the following > # CONFIG_KUNIT is not set > CONFIG_KUNIT_ALL_TESTS=y > CONFIG_KUNIT_TEST=y > CONFIG_KUNIT=y > CONFIG_KUNIT_EXAMPLE_TEST=y And we'd error out with > ERROR:root:Not all Kconfig options selected in kunitconfig were in the generated .config. > This is probably due to unsatisfied dependencies. > Missing: # CONFIG_KUNIT is not set After ----- We'd write the following > # CONFIG_KUNIT is not set > CONFIG_KUNIT_TEST=y > CONFIG_KUNIT_ALL_TESTS=y > CONFIG_KUNIT_EXAMPLE_TEST=y And we'd error out with > ERROR:root:Not all Kconfig options selected in kunitconfig were in the generated .config. > This is probably due to unsatisfied dependencies. > Missing: CONFIG_KUNIT_EXAMPLE_TEST=y, CONFIG_KUNIT_TEST=y, CONFIG_KUNIT_ALL_TESTS=y Signed-off-by: Daniel Latypov <dlatypov(a)google.com> --- v1 -> v2: fix validate_config() func. There was a bug found by David, see https://lore.kernel.org/linux-kselftest/CAGS_qxpF338dvbB+6QW1n8_agddeS10+nk… --- tools/testing/kunit/kunit_config.py | 49 +++++++++++++++----------- tools/testing/kunit/kunit_kernel.py | 18 +++++----- tools/testing/kunit/kunit_tool_test.py | 45 ++++++++++------------- 3 files changed, 57 insertions(+), 55 deletions(-) diff --git a/tools/testing/kunit/kunit_config.py b/tools/testing/kunit/kunit_config.py index 75a8dc1683d4..89443400b17e 100644 --- a/tools/testing/kunit/kunit_config.py +++ b/tools/testing/kunit/kunit_config.py @@ -8,7 +8,7 @@ from dataclasses import dataclass import re -from typing import List, Set +from typing import Dict, Iterable, Set CONFIG_IS_NOT_SET_PATTERN = r'^# CONFIG_(\w+) is not set$' CONFIG_PATTERN = r'^CONFIG_(\w+)=(\S+|".*")$' @@ -32,35 +32,46 @@ class Kconfig: """Represents defconfig or .config specified using the Kconfig language.""" def __init__(self) -> None: - self._entries = [] # type: List[KconfigEntry] + self._entries = {} # type: Dict[str, str] - def entries(self) -> Set[KconfigEntry]: - return set(self._entries) + def __eq__(self, other) -> bool: + if not isinstance(other, self.__class__): + return False + return self._entries == other._entries - def add_entry(self, entry: KconfigEntry) -> None: - self._entries.append(entry) + def __repr__(self) -> str: + return ','.join(str(e) for e in self._as_entries()) + + + def _as_entries(self) -> Iterable[KconfigEntry]: + for name, value in self._entries.items(): + yield KconfigEntry(name, value) + + def add_entry(self, name: str, value: str) -> None: + self._entries[name] = value def is_subset_of(self, other: 'Kconfig') -> bool: - other_dict = {e.name: e.value for e in other.entries()} - for a in self.entries(): - b = other_dict.get(a.name) + for name, value in self._entries.items(): + b = other._entries.get(name) if b is None: - if a.value == 'n': + if value == 'n': continue return False - if a.value != b: + if value != b: return False return True + def set_diff(self, other: 'Kconfig') -> Set[KconfigEntry]: + return set(self._as_entries()) - set(other._as_entries()) + def merge_in_entries(self, other: 'Kconfig') -> None: - if other.is_subset_of(self): - return - self._entries = list(self.entries().union(other.entries())) + for name, value in other._entries.items(): + self._entries[name] = value def write_to_file(self, path: str) -> None: with open(path, 'a+') as f: - for entry in self.entries(): - f.write(str(entry) + '\n') + for e in self._as_entries(): + f.write(str(e) + '\n') def parse_file(path: str) -> Kconfig: with open(path, 'r') as f: @@ -78,14 +89,12 @@ def parse_from_string(blob: str) -> Kconfig: match = config_matcher.match(line) if match: - entry = KconfigEntry(match.group(1), match.group(2)) - kconfig.add_entry(entry) + kconfig.add_entry(match.group(1), match.group(2)) continue empty_match = is_not_set_matcher.match(line) if empty_match: - entry = KconfigEntry(empty_match.group(1), 'n') - kconfig.add_entry(entry) + kconfig.add_entry(empty_match.group(1), 'n') continue if line[0] == '#': diff --git a/tools/testing/kunit/kunit_kernel.py b/tools/testing/kunit/kunit_kernel.py index 3539efaf99ba..6d994bb24999 100644 --- a/tools/testing/kunit/kunit_kernel.py +++ b/tools/testing/kunit/kunit_kernel.py @@ -53,8 +53,8 @@ class LinuxSourceTreeOperations: except subprocess.CalledProcessError as e: raise ConfigError(e.output.decode()) - def make_arch_qemuconfig(self, base_kunitconfig: kunit_config.Kconfig) -> None: - pass + def make_arch_qemuconfig(self, base_kunitconfig: kunit_config.Kconfig) -> kunit_config.Kconfig: + return base_kunitconfig def make_allyesconfig(self, build_dir: str, make_options) -> None: raise ConfigError('Only the "um" arch is supported for alltests') @@ -109,9 +109,10 @@ class LinuxSourceTreeOperationsQemu(LinuxSourceTreeOperations): self._kernel_command_line = qemu_arch_params.kernel_command_line + ' kunit_shutdown=reboot' self._extra_qemu_params = qemu_arch_params.extra_qemu_params - def make_arch_qemuconfig(self, base_kunitconfig: kunit_config.Kconfig) -> None: + def make_arch_qemuconfig(self, base_kunitconfig: kunit_config.Kconfig) -> kunit_config.Kconfig: kconfig = kunit_config.parse_from_string(self._kconfig) - base_kunitconfig.merge_in_entries(kconfig) + kconfig.merge_in_entries(base_kunitconfig) + return kconfig def start(self, params: List[str], build_dir: str) -> subprocess.Popen: kernel_path = os.path.join(build_dir, self._kernel_path) @@ -267,7 +268,7 @@ class LinuxSourceTree: validated_kconfig = kunit_config.parse_file(kconfig_path) if self._kconfig.is_subset_of(validated_kconfig): return True - invalid = self._kconfig.entries() - validated_kconfig.entries() + invalid = self._kconfig.set_diff(validated_kconfig) message = 'Not all Kconfig options selected in kunitconfig were in the generated .config.\n' \ 'This is probably due to unsatisfied dependencies.\n' \ 'Missing: ' + ', '.join([str(e) for e in invalid]) @@ -282,7 +283,7 @@ class LinuxSourceTree: if build_dir and not os.path.exists(build_dir): os.mkdir(build_dir) try: - self._ops.make_arch_qemuconfig(self._kconfig) + self._kconfig = self._ops.make_arch_qemuconfig(self._kconfig) self._kconfig.write_to_file(kconfig_path) self._ops.make_olddefconfig(build_dir, make_options) except ConfigError as e: @@ -303,7 +304,7 @@ class LinuxSourceTree: return True old_kconfig = kunit_config.parse_file(old_path) - return old_kconfig.entries() != self._kconfig.entries() + return old_kconfig != self._kconfig def build_reconfig(self, build_dir: str, make_options) -> bool: """Creates a new .config if it is not a subset of the .kunitconfig.""" @@ -313,7 +314,8 @@ class LinuxSourceTree: return self.build_config(build_dir, make_options) existing_kconfig = kunit_config.parse_file(kconfig_path) - self._ops.make_arch_qemuconfig(self._kconfig) + self._kconfig = self._ops.make_arch_qemuconfig(self._kconfig) + if self._kconfig.is_subset_of(existing_kconfig) and not self._kunitconfig_changed(build_dir): return True print('Regenerating .config ...') diff --git a/tools/testing/kunit/kunit_tool_test.py b/tools/testing/kunit/kunit_tool_test.py index 25a2eb3bf114..3a8f638ff092 100755 --- a/tools/testing/kunit/kunit_tool_test.py +++ b/tools/testing/kunit/kunit_tool_test.py @@ -45,7 +45,7 @@ class KconfigTest(unittest.TestCase): self.assertTrue(kconfig0.is_subset_of(kconfig0)) kconfig1 = kunit_config.Kconfig() - kconfig1.add_entry(kunit_config.KconfigEntry('TEST', 'y')) + kconfig1.add_entry('TEST', 'y') self.assertTrue(kconfig1.is_subset_of(kconfig1)) self.assertTrue(kconfig0.is_subset_of(kconfig1)) self.assertFalse(kconfig1.is_subset_of(kconfig0)) @@ -56,40 +56,28 @@ class KconfigTest(unittest.TestCase): kconfig = kunit_config.parse_file(kconfig_path) expected_kconfig = kunit_config.Kconfig() - expected_kconfig.add_entry( - kunit_config.KconfigEntry('UML', 'y')) - expected_kconfig.add_entry( - kunit_config.KconfigEntry('MMU', 'y')) - expected_kconfig.add_entry( - kunit_config.KconfigEntry('TEST', 'y')) - expected_kconfig.add_entry( - kunit_config.KconfigEntry('EXAMPLE_TEST', 'y')) - expected_kconfig.add_entry( - kunit_config.KconfigEntry('MK8', 'n')) - - self.assertEqual(kconfig.entries(), expected_kconfig.entries()) + expected_kconfig.add_entry('UML', 'y') + expected_kconfig.add_entry('MMU', 'y') + expected_kconfig.add_entry('TEST', 'y') + expected_kconfig.add_entry('EXAMPLE_TEST', 'y') + expected_kconfig.add_entry('MK8', 'n') + + self.assertEqual(kconfig, expected_kconfig) def test_write_to_file(self): kconfig_path = os.path.join(test_tmpdir, '.config') expected_kconfig = kunit_config.Kconfig() - expected_kconfig.add_entry( - kunit_config.KconfigEntry('UML', 'y')) - expected_kconfig.add_entry( - kunit_config.KconfigEntry('MMU', 'y')) - expected_kconfig.add_entry( - kunit_config.KconfigEntry('TEST', 'y')) - expected_kconfig.add_entry( - kunit_config.KconfigEntry('EXAMPLE_TEST', 'y')) - expected_kconfig.add_entry( - kunit_config.KconfigEntry('MK8', 'n')) + expected_kconfig.add_entry('UML', 'y') + expected_kconfig.add_entry('MMU', 'y') + expected_kconfig.add_entry('TEST', 'y') + expected_kconfig.add_entry('EXAMPLE_TEST', 'y') + expected_kconfig.add_entry('MK8', 'n') expected_kconfig.write_to_file(kconfig_path) actual_kconfig = kunit_config.parse_file(kconfig_path) - - self.assertEqual(actual_kconfig.entries(), - expected_kconfig.entries()) + self.assertEqual(actual_kconfig, expected_kconfig) class KUnitParserTest(unittest.TestCase): @@ -381,8 +369,11 @@ class LinuxSourceTreeTest(unittest.TestCase): kunit_kernel.LinuxSourceTree('', kunitconfig_path=dir) def test_kconfig_add(self): + want_kconfig = kunit_config.Kconfig() + want_kconfig.add_entry('NOT_REAL', 'y') + tree = kunit_kernel.LinuxSourceTree('', kconfig_add=['CONFIG_NOT_REAL=y']) - self.assertIn(kunit_config.KconfigEntry('NOT_REAL', 'y'), tree._kconfig.entries()) + self.assertFalse(want_kconfig.set_diff(tree._kconfig)) def test_invalid_arch(self): with self.assertRaisesRegex(kunit_kernel.ConfigError, 'not a valid arch, options are.*x86_64'): base-commit: 1b11063d32d7e11366e48be64215ff517ce32217 -- 2.36.1.124.g0e6072fb45-goog

2 years, 6 months

2
3
0 0

[PATCH v2] docs/kselftest: add more guidelines for adding new tests

by Muhammad Usama Anjum

Improve and add instructions to add new tests. Add build commands to test before sending the new test patch. Signed-off-by: Muhammad Usama Anjum <usama.anjum(a)collabora.com> --- Changes in v2: - Updated commit message - Removed dependence of this patch from other patch - Updated instructions --- Documentation/dev-tools/kselftest.rst | 27 ++++++++++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-) diff --git a/Documentation/dev-tools/kselftest.rst b/Documentation/dev-tools/kselftest.rst index a833ecf12fbc1..ee6467ca8293f 100644 --- a/Documentation/dev-tools/kselftest.rst +++ b/Documentation/dev-tools/kselftest.rst @@ -208,6 +208,14 @@ In general, the rules for selftests are Contributing new tests (details) ================================ + * In your Makefile, use facilities from lib.mk by including it instead of + reinventing the wheel. Specify flags and binaries generation flags on + need basis before including lib.mk. :: + + CFLAGS = $(KHDR_INCLUDES) + TEST_GEN_PROGS := close_range_test + include ../lib.mk + * Use TEST_GEN_XXX if such binaries or files are generated during compiling. @@ -230,13 +238,30 @@ Contributing new tests (details) * First use the headers inside the kernel source and/or git repo, and then the system headers. Headers for the kernel release as opposed to headers installed by the distro on the system should be the primary focus to be able - to find regressions. + to find regressions. Use KHDR_INCLUDES in Makefile to include headers from + the kernel source. * If a test needs specific kernel config options enabled, add a config file in the test directory to enable them. e.g: tools/testing/selftests/android/config + * Create a .gitignore file inside test directory and add all generated objects + in it. + + * Add new test name in TARGETS in selftests/Makefile:: + + TARGETS += android + + * All changes should pass:: + + kselftest-{all,install,clean,gen_tar} + kselftest-{all,install,clean,gen_tar} O=abo_path + kselftest-{all,install,clean,gen_tar} O=rel_path + make -C tools/testing/selftests {all,install,clean,gen_tar} + make -C tools/testing/selftests {all,install,clean,gen_tar} O=abs_path + make -C tools/testing/selftests {all,install,clean,gen_tar} O=rel_path + Test Module =========== -- 2.30.2

2 years, 6 months

3
6
0 0

[PATCH v7 0/6] Proposal for a GPU cgroup controller

by T.J. Mercier

This patch series revisits the proposal for a GPU cgroup controller to track and limit memory allocations by various device/allocator subsystems. The patch series also contains a simple prototype to illustrate how Android intends to implement DMA-BUF allocator attribution using the GPU cgroup controller. The prototype does not include resource limit enforcements. Changelog: v7: Hide gpucg and gpucg_bucket struct definitions per Michal Koutný. This means gpucg_register_bucket now returns an internally allocated struct gpucg_bucket. Move all public function documentation to the cgroup_gpu.h header. Remove comment in documentation about duplicate name rejection which is not relevant to cgroups users per Michal Koutný. v6: Move documentation into cgroup-v2.rst per Tejun Heo. Rename BINDER_FD{A}_FLAG_SENDER_NO_NEED -> BINDER_FD{A}_FLAG_XFER_CHARGE per Carlos Llamas. Return error on transfer failure per Carlos Llamas. v5: Rebase on top of v5.18-rc3 Drop the global GPU cgroup "total" (sum of all device totals) portion of the design since there is no currently known use for this per Tejun Heo. Fix commit message which still contained the old name for dma_buf_transfer_charge per Michal Koutný. Remove all GPU cgroup code except what's necessary to support charge transfer from dma_buf. Previously charging was done in export, but for non-Android graphics use-cases this is not ideal since there may be a delay between allocation and export, during which time there is no accounting. Merge dmabuf: Use the GPU cgroup charge/uncharge APIs patch into dmabuf: heaps: export system_heap buffers with GPU cgroup charging as a result of above. Put the charge and uncharge code in the same file (system_heap_allocate, system_heap_dma_buf_release) instead of splitting them between the heap and the dma_buf_release. This avoids asymmetric management of the gpucg charges. Modify the dma_buf_transfer_charge API to accept a task_struct instead of a gpucg. This avoids requiring the caller to manage the refcount of the gpucg upon failure and confusing ownership transfer logic. Support all strings for gpucg_register_bucket instead of just string literals. Enforce globally unique gpucg_bucket names. Constrain gpucg_bucket name lengths to 64 bytes. Append "-heap" to gpucg_bucket names from dmabuf-heaps. Drop patch 7 from the series, which changed the types of binder_transaction_data's sender_pid and sender_euid fields. This was done in another commit here: https://lore.kernel.org/all/20220210021129.3386083-4-masahiroy@kernel.org/ Rename: gpucg_try_charge -> gpucg_charge find_cg_rpool_locked -> cg_rpool_find_locked init_cg_rpool -> cg_rpool_init get_cg_rpool_locked -> cg_rpool_get_locked "gpu cgroup controller" -> "GPU controller" gpucg_device -> gpucg_bucket usage -> size Tests: Support both binder_fd_array_object and binder_fd_object. This is necessary because new versions of Android will use binder_fd_object instead of binder_fd_array_object, and we need to support both. Tests for both binder_fd_array_object and binder_fd_object. For binder_utils return error codes instead of struct binder{fs}_ctx. Use ifdef __ANDROID__ to choose platform-dependent temp path instead of a runtime fallback. Ensure binderfs_mntpt ends with a trailing '/' character instead of prepending it where used. v4: Skip test if not run as root per Shuah Khan Add better test logging for abnormal child termination per Shuah Khan Adjust ordering of charge/uncharge during transfer to avoid potentially hitting cgroup limit per Michal Koutný Adjust gpucg_try_charge critical section for charge transfer functionality Fix uninitialized return code error for dmabuf_try_charge error case v3: Remove Upstreaming Plan from gpu-cgroup.rst per John Stultz Use more common dual author commit message format per John Stultz Remove android from binder changes title per Todd Kjos Add a kselftest for this new behavior per Greg Kroah-Hartman Include details on behavior for all combinations of kernel/userspace versions in changelog (thanks Suren Baghdasaryan) per Greg Kroah-Hartman. Fix pid and uid types in binder UAPI header v2: See the previous revision of this change submitted by Hridya Valsaraju at: https://lore.kernel.org/all/20220115010622.3185921-1-hridya@google.com/ Move dma-buf cgroup charge transfer from a dma_buf_op defined by every heap to a single dma-buf function for all heaps per Daniel Vetter and Christian König. Pointers to struct gpucg and struct gpucg_device tracking the current associations were added to the dma_buf struct to achieve this. Fix incorrect Kconfig help section indentation per Randy Dunlap. History of the GPU cgroup controller ==================================== The GPU/DRM cgroup controller came into being when a consensus[1] was reached that the resources it tracked were unsuitable to be integrated into memcg. Originally, the proposed controller was specific to the DRM subsystem and was intended to track GEM buffers and GPU-specific resources[2]. In order to help establish a unified memory accounting model for all GPU and all related subsystems, Daniel Vetter put forth a suggestion to move it out of the DRM subsystem so that it can be used by other DMA-BUF exporters as well[3]. This RFC proposes an interface that does the same. [1]: https://patchwork.kernel.org/project/dri-devel/cover/20190501140438.9506-1-… [2]: https://lore.kernel.org/amd-gfx/20210126214626.16260-1-brian.welty@intel.co… [3]: https://lore.kernel.org/amd-gfx/YCVOl8%2F87bqRSQei@phenom.ffwll.local/ Hridya Valsaraju (3): gpu: rfc: Proposal for a GPU cgroup controller cgroup: gpu: Add a cgroup controller for allocator attribution of GPU memory binder: Add flags to relinquish ownership of fds T.J. Mercier (3): dmabuf: heaps: export system_heap buffers with GPU cgroup charging dmabuf: Add gpu cgroup charge transfer function selftests: Add binder cgroup gpu memory transfer tests Documentation/admin-guide/cgroup-v2.rst | 23 + drivers/android/binder.c | 31 +- drivers/dma-buf/dma-buf.c | 80 ++- drivers/dma-buf/dma-heap.c | 38 ++ drivers/dma-buf/heaps/system_heap.c | 28 +- include/linux/cgroup_gpu.h | 146 +++++ include/linux/cgroup_subsys.h | 4 + include/linux/dma-buf.h | 49 +- include/linux/dma-heap.h | 15 + include/uapi/linux/android/binder.h | 23 +- init/Kconfig | 7 + kernel/cgroup/Makefile | 1 + kernel/cgroup/gpu.c | 390 +++++++++++++ .../selftests/drivers/android/binder/Makefile | 8 + .../drivers/android/binder/binder_util.c | 250 +++++++++ .../drivers/android/binder/binder_util.h | 32 ++ .../selftests/drivers/android/binder/config | 4 + .../binder/test_dmabuf_cgroup_transfer.c | 526 ++++++++++++++++++ 18 files changed, 1632 insertions(+), 23 deletions(-) create mode 100644 include/linux/cgroup_gpu.h create mode 100644 kernel/cgroup/gpu.c create mode 100644 tools/testing/selftests/drivers/android/binder/Makefile create mode 100644 tools/testing/selftests/drivers/android/binder/binder_util.c create mode 100644 tools/testing/selftests/drivers/android/binder/binder_util.h create mode 100644 tools/testing/selftests/drivers/android/binder/config create mode 100644 tools/testing/selftests/drivers/android/binder/test_dmabuf_cgroup_transfer.c -- 2.36.0.512.ge40c2bad7a-goog

2 years, 6 months

7
19
0 0

2024

2023

2022

2021

2020

2019

2018

2017

Linux-kselftest-mirror May 2022