I was working on fixing the cross-compilation for the selftests/vm tests.
Currently, there are two issues in my testing:
1) problem: required library missing from some cross-compile environments:
tools/testing/selftests/vm/mlock-random-test.c requires libcap
be installed. The target line for mlock-random-test in
tools/testing/selftests/vm/Makefile looks like this:
$(OUTPUT)/mlock-random-test: LDLIBS += -lcap
and mlock-random-test.c has this include line:
#include <sys/capability.h>
this is confusing, since this is different from the header file
linux/capability.h. It is associated with the capability library (libcap)
and not the kernel. In any event, on some distros and in some
cross-compile SDKs the package containing these files is not installed
by default.
Once this library is installed, things progress farther. Using an Ubuntu
system, you can install the cross version of this library (for arm64) by doing:
$ sudo apt install libcap-dev:arm64
1) solution:
I would like to add some meta-data about this build dependency, by putting
something in the settings file as a hint to CI build systems. Specifically, I'd like to
create the file 'tools/testing/selftests/vm/settings', with the content:
NEED_LIB=cap
We already use settings for other meta-data about a test (right now, just a
non-default timeout value), but I don't want to create a new file or syntax
for this build dependency data.
Let me know what you think.
I may follow up with some script in the kernel source tree to check these
dependencies, independent of any CI system. I have such a script in Fuego
that I could submit, but it would need some work to fit into the kernel build
flow for kselftest. The goal would be to provide a nicely formatted warning,
with a recommendation for a package install. But that's more work than
I think is needed right now just to let developers know there's a build dependency
here.
2) problem: reference to source-relative header file
the Makefile for vm uses a relative path for include directories.
Specifically, it has the line:
CFLAGS = -Wall -I ../../../../../usr/include $(EXTRA_CFLAGS)
I believe this needs to reference kernel include files from the
output directory, not the source directory.
With the relative include directory path, the program userfaultfd.c
gets compilation error like this:
userfaultfd.c:267:21: error: 'UFFD_API_RANGE_IOCTLS_BASIC' undeclared here (not in a function)
.expected_ioctls = UFFD_API_RANGE_IOCTLS_BASIC,
^
userfaultfd.c: In function 'uffd_poll_thread':
userfaultfd.c:529:8: error: 'UFFD_EVENT_FORK' undeclared (first use in this function)
case UFFD_EVENT_FORK:
^
userfaultfd.c:529:8: note: each undeclared identifier is reported only once for each function it appears in
userfaultfd.c:531:18: error: 'union <anonymous>' has no member named 'fork'
uffd = msg.arg.fork.ufd;
^
2) incomplete solution:
I originally changed this line to read:
CFLAGS = -Wall -I $(KBUILD_OUTPUT)/usr/include $(EXTRA_CFLAGS)
This works when the output directory is specified using KBUILD_OUTPUT,
but not when the output directory is specified using O=
I'm not sure what happens when the output directory is specified
with a non-source-tree current working directory.
In any event, while researching a proper solution to this, I found
the following in tools/testing/selftests/Makefile:
If compiling with ifneq ($(O),)
BUILD := $(O)
else
ifneq ($(KBUILD_OUTPUT),)
BUILD := $(KBUILD_OUTPUT)/kselftest
else
BUILD := $(shell pwd)
DEFAULT_INSTALL_HDR_PATH := 1
endif
endif
This doesn't seem right. It looks like the selftests Makefile treats a directory
passed in using O= different from one specified using KBUILD_OUTPUT
or the current working directory.
In the KBUILD_OUTPUT case, you get an extra 'kselftest' directory layer
that you don't get for the other two.
In contrast, the kernel top-level Makefile has this:
ifeq ("$(origin O)", "command line")
KBUILD_OUTPUT := $(O)
endif
(and from then on, the top-level Makefile appears to only use KBUILD_OUTPUT)
This makes it look like the rest of the kernel build system treats O= and KBUILD_OUTPUT
identically.
Am I missing something, or is there a flaw in the O=/KBUILD_OUTPUT handling in
kselftest? Please let me know and I'll try to work out an appropriate fix for
cross-compiling the vm tests.
-- Tim
Hi Shuah,
We discussed collecting and uploading all syzkaller reproducers
somewhere. You wanted to see how they look. I've uploaded all current
reproducers here:
https://github.com/dvyukov/syzkaller-repros
Minimalistic build/run scripts are included.
+some testing mailing lists too as this can be used as a test suite
If you have any potential uses for this, you are welcome to use it.
But then we probably need to find some more official and shared place
for them than my private github.
The test programs can also be bulk updated if necessary, because all
of this is auto-generated.
Thanks
From: SeongJae Park <sjpark(a)amazon.de>
Deletions of configs in the '.kunitconfig' is not applied because kunit
rebuilds '.config' only if the '.config' is not a subset of the
'.kunitconfig'. To allow the deletions to applied, this commit modifies
the '.config' rebuild condition to addtionally check the modified times
of those files.
Signed-off-by: SeongJae Park <sjpark(a)amazon.de>
---
tools/testing/kunit/kunit_kernel.py | 17 +++++++++++------
1 file changed, 11 insertions(+), 6 deletions(-)
diff --git a/tools/testing/kunit/kunit_kernel.py b/tools/testing/kunit/kunit_kernel.py
index cc5d844ecca1..a3a5d6c7e66d 100644
--- a/tools/testing/kunit/kunit_kernel.py
+++ b/tools/testing/kunit/kunit_kernel.py
@@ -111,17 +111,22 @@ class LinuxSourceTree(object):
return True
def build_reconfig(self, build_dir):
- """Creates a new .config if it is not a subset of the .kunitconfig."""
+ """Creates a new .config if it is not a subset of, or older than the .kunitconfig."""
kconfig_path = get_kconfig_path(build_dir)
if os.path.exists(kconfig_path):
existing_kconfig = kunit_config.Kconfig()
existing_kconfig.read_from_file(kconfig_path)
- if not self._kconfig.is_subset_of(existing_kconfig):
- print('Regenerating .config ...')
- os.remove(kconfig_path)
- return self.build_config(build_dir)
- else:
+ subset = self._kconfig.is_subset_of(existing_kconfig)
+
+ kunitconfig_mtime = os.path.getmtime(kunitconfig_path)
+ kconfig_mtime = os.path.getmtime(kconfig_path)
+ older = kconfig_mtime < kunitconfig_mtime
+
+ if subset and not older:
return True
+ print('Regenerating .config ...')
+ os.remove(kconfig_path)
+ return self.build_config(build_dir)
else:
print('Generating .config ...')
return self.build_config(build_dir)
--
2.17.1
A recent RFC patch set [1] suggests some additional functionality
may be needed around kunit resources. It seems to require
1. support for resources without allocation
2. support for lookup of such resources
3. support for access to resources across multiple kernel threads
The proposed changes here are designed to address these needs.
The idea is we first generalize the API to support adding
resources with static data; then from there we support named
resources. The latter support is needed because if we are
in a different thread context and only have the "struct kunit *"
to work with, we need a way to identify a resource in lookup.
[1] https://lkml.org/lkml/2020/2/26/1286
Alan Maguire (2):
kunit: generalize kunit_resource API beyond allocated resources
kunit: add support for named resources
include/kunit/test.h | 145 ++++++++++++++++++++++------
lib/kunit/kunit-test.c | 103 ++++++++++++++++----
lib/kunit/string-stream.c | 14 ++-
lib/kunit/test.c | 234 +++++++++++++++++++++++++++++++++-------------
4 files changed, 375 insertions(+), 121 deletions(-)
--
1.8.3.1
When kunit tests are run on native (i.e. non-UML) environments, the results
of test execution are often intermixed with dmesg output. This patch
series attempts to solve this by providing a debugfs representation
of the results of the last test run, available as
/sys/kernel/debug/kunit/<testsuite>/results
Changes since v5:
- replaced undefined behaviour use of snprintf(buf, ..., buf) in kunit_log()
with a function to append string to existing log (Frank, patch 1)
- added clarification on log size limitations to documentation
(Frank, patch 4)
Changes since v4:
- added suite-level log expectations to kunit log test (Brendan, patch 2)
- added log expectations (of it being NULL) for case where
CONFIG_KUNIT_DEBUGFS=n to kunit log test (patch 2)
- added patch 3 which replaces subtest tab indentation with 4 space
indentation as per TAP 14 spec (Frank, patch 3)
Changes since v3:
- added CONFIG_KUNIT_DEBUGFS to support conditional compilation of debugfs
representation, including string logging (Frank, patch 1)
- removed unneeded NULL check for test_case in
kunit_suite_for_each_test_case() (Frank, patch 1)
- added kunit log test to verify logging multiple strings works
(Frank, patch 2)
- rephrased description of results file (Frank, patch 3)
Changes since v2:
- updated kunit_status2str() to kunit_status_to_string() and made it
static inline in include/kunit/test.h (Brendan)
- added log string to struct kunit_suite and kunit_case, with log
pointer in struct kunit pointing at the case log. This allows us
to collect kunit_[err|info|warning]() messages at the same time
as we printk() them. This solves for the most part the sharing
of log messages between test execution and debugfs since we
just print the suite log (which contains the test suite preamble)
and the individual test logs. The only exception is the suite-level
status, which we cannot store in the suite log as it would mean
we'd print the suite and its status prior to the suite's results.
(Brendan, patch 1)
- dropped debugfs-based kunit run patch for now so as not to cause
problems with tests currently under development (Brendan)
- fixed doc issues with code block (Brendan, patch 3)
Changes since v1:
- trimmed unneeded include files in lib/kunit/debugfs.c (Greg)
- renamed global debugfs functions to be prefixed with kunit_ (Greg)
- removed error checking for debugfs operations (Greg)
Alan Maguire (4):
kunit: add debugfs /sys/kernel/debug/kunit/<suite>/results display
kunit: add log test
kunit: subtests should be indented 4 spaces according to TAP
kunit: update documentation to describe debugfs representation
Documentation/dev-tools/kunit/usage.rst | 14 +++
include/kunit/test.h | 59 +++++++++++--
lib/kunit/Kconfig | 8 ++
lib/kunit/Makefile | 4 +
lib/kunit/assert.c | 79 ++++++++---------
lib/kunit/debugfs.c | 116 +++++++++++++++++++++++++
lib/kunit/debugfs.h | 30 +++++++
lib/kunit/kunit-test.c | 45 +++++++++-
lib/kunit/test.c | 147 +++++++++++++++++++++++++-------
9 files changed, 421 insertions(+), 81 deletions(-)
create mode 100644 lib/kunit/debugfs.c
create mode 100644 lib/kunit/debugfs.h
--
1.8.3.1