When writing tests, it'd often be very useful to be able to intercept
calls to a function in the code being tested and replace it with a
test-specific stub. This has always been an obviously missing piece of
KUnit, and the solutions always involve some tradeoffs with cleanliness,
performance, or impact on non-test code. See the folowing document for
some of the challenges:
https://kunit.dev/mocking.html
This series consists of two prototype patches which add support for this
sort of redirection to KUnit tests:
1: static_stub: Any function which might want to be intercepted adds a
call to a macro which checks if a test has redirected calls to it, and
calls the corresponding replacement.
2: ftrace_stub: Functions are intercepted using ftrace.
This doesn't require adding a new prologue to each function being
replaced, but does have more dependencies (which restricts it to a small
number of architectures, not including UML), and doesn't work well with
inline functions.
The API for both implementations is very similar, so it should be easy
to migrate from one to the other if necessary. Both of these
implementations restrict the redirection to the test context: it is
automatically undone after the KUnit test completes, and does not affect
calls in other threads. If CONFIG_KUNIT is not enabled, there should be
no overhead in either implementation.
Does either (or both) of these features sound useful, and is this
sort-of API the right model? (Personally, I think there's a reasonable
scope for both.) Is anything obviously missing or wrong? Do the names,
descriptions etc. make any sense?
Note that these patches are definitely still at the "prototype" level,
and things like error-handling, documentation, and testing are still
pretty sparse. There is also quite a bit of room for optimisation.
These'll all be improved for v1 if the concept seems good.
We're going to be talking about this again at LPC, so it's worth having
another look before then if you're interested and/or will be attending:
https://lpc.events/event/16/contributions/1308/
Cheers,
-- David
---
Changes since RFC v1:
https://lore.kernel.org/lkml/20220318021314.3225240-1-davidgow@google.com/
- Fix some typos (thanks Daniel)
- Use typecheck_fn() to fix typechecking in some cases (thanks Brendan)
- Use ftrace_instruction_pointer_set() in place of kernel livepatch,
which seems to have disappeared:
https://lore.kernel.org/lkml/0a76550d-008d-0364-8244-4dae2981ea05@csgroup.e…
- Fix a copy-paste name error in the resource finding function.
- Rebase on top of torvalds/master, as it wasn't applying cleanly.
Note that the Kernel Livepatch -> ftrace change seems to allow more
architectures to work, but while they compile, there still seems to be
issues. So, this will compile on (e.g.) arm64, but fails:
$ ./tools/testing/kunit/kunit.py run 'example*' --kunitconfig lib/kunit/stubs_example.kunitconfig --arch arm64 --make_options LLVM=1
[05:00:13] # example_ftrace_stub_test: initializing
[05:00:13] # example_ftrace_stub_test: EXPECTATION FAILED at lib/kunit/kunit-example-test.c:179
[05:00:13] Expected add_one(1) == 0, but
[05:00:13] add_one(1) == 2
[05:00:13] not ok 6 - example_ftrace_stub_test
[05:00:13] [FAILED] example_ftrace_stub_test
Daniel Latypov (1):
kunit: expose ftrace-based API for stubbing out functions during tests
David Gow (1):
kunit: Expose 'static stub' API to redirect functions
include/kunit/ftrace_stub.h | 84 +++++++++++++++++
include/kunit/static_stub.h | 106 +++++++++++++++++++++
lib/kunit/Kconfig | 11 +++
lib/kunit/Makefile | 5 +
lib/kunit/ftrace_stub.c | 137 ++++++++++++++++++++++++++++
lib/kunit/kunit-example-test.c | 63 +++++++++++++
lib/kunit/static_stub.c | 125 +++++++++++++++++++++++++
lib/kunit/stubs_example.kunitconfig | 10 ++
8 files changed, 541 insertions(+)
create mode 100644 include/kunit/ftrace_stub.h
create mode 100644 include/kunit/static_stub.h
create mode 100644 lib/kunit/ftrace_stub.c
create mode 100644 lib/kunit/static_stub.c
create mode 100644 lib/kunit/stubs_example.kunitconfig
--
2.37.2.789.g6183377224-goog
kunit_tool's --alltests option was changed in commit
980ac3ad0512 ("kunit: tool: rename all_test_uml.config, use it for --alltests")
to use a manually curated list of architecture-indpendent Kconfig
options, rather than attempting to use make allyesconfig on UML, which
was broken.
Update the kunit_tool documentation to reflect the new behaviour of
--alltests.
Signed-off-by: David Gow <davidgow(a)google.com>
---
Documentation/dev-tools/kunit/run_wrapper.rst | 17 +++++++++--------
1 file changed, 9 insertions(+), 8 deletions(-)
diff --git a/Documentation/dev-tools/kunit/run_wrapper.rst b/Documentation/dev-tools/kunit/run_wrapper.rst
index 6b33caf6c8ab..dafe8eb28d30 100644
--- a/Documentation/dev-tools/kunit/run_wrapper.rst
+++ b/Documentation/dev-tools/kunit/run_wrapper.rst
@@ -251,14 +251,15 @@ command line arguments:
compiling a kernel (using ``build`` or ``run`` commands). For example:
to enable compiler warnings, we can pass ``--make_options W=1``.
-- ``--alltests``: Builds a UML kernel with all config options enabled
- using ``make allyesconfig``. This allows us to run as many tests as
- possible.
-
- .. note:: It is slow and prone to breakage as new options are
- added or modified. Instead, enable all tests
- which have satisfied dependencies by adding
- ``CONFIG_KUNIT_ALL_TESTS=y`` to your ``.kunitconfig``.
+- ``--alltests``: Enable a predefined set of options in order to build
+ as many tests as possible.
+
+ .. note:: The list of enabled options can be found in
+ ``tools/testing/kunit/configs/all_tests.config``.
+
+ If you only want to enable all tests with otherwise satisfied
+ dependencies, instead add ``CONFIG_KUNIT_ALL_TESTS=y`` to your
+ ``.kunitconfig``.
- ``--kunitconfig``: Specifies the path or the directory of the ``.kunitconfig``
file. For example:
--
2.38.0.rc1.362.ged0d419d3c-goog
As suggested by Thomas Gleixner, I'm following up to move on with
the SPDX tag needed for copyleft-next-0.3.1. I've split this out
from the test_sysfs selftest so to separate review from that.
Changes on this v10:
o embraced paragraph from Thomas Gleixner which helps explain why
the OR operator in the SPDX license name
o dropped the GPL-2.0 and GPL-2.0+ tags as suggested by Thomas Gleixner
as these are outdated (still valid) in the SPDX spec
o trimmed the Cc list to remove the test_sysfs / block layer / fs folks as
the test_sysfs stuff is now dropped from consideration in this series
The last series was at v9 but it also had the test_sysfs and its
changes, its history can be found here:
https://lore.kernel.org/all/20211029184500.2821444-1-mcgrof@kernel.org/
Luis Chamberlain (2):
LICENSES: Add the copyleft-next-0.3.1 license
testing: use the copyleft-next-0.3.1 SPDX tag
LICENSES/dual/copyleft-next-0.3.1 | 236 +++++++++++++++++++++++
lib/test_kmod.c | 12 +-
lib/test_sysctl.c | 12 +-
tools/testing/selftests/kmod/kmod.sh | 13 +-
tools/testing/selftests/sysctl/sysctl.sh | 12 +-
5 files changed, 240 insertions(+), 45 deletions(-)
create mode 100644 LICENSES/dual/copyleft-next-0.3.1
--
2.35.1
In the test_icr() function in xapic_state_test, one of the for loops is
initialized with vcpu->id. Fix this assumption that vcpu->id is 0 so
that IPIs are correctly sent to non-existent vCPUs [1].
Suggested-by: Sean Christopherson <seanjc(a)google.com>
Signed-off-by: Gautam Menghani <gautammenghani201(a)gmail.com>
---
[1] https://lore.kernel.org/kvm/YyoZr9rXSSMEtdh5@google.com/
tools/testing/selftests/kvm/x86_64/xapic_state_test.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/tools/testing/selftests/kvm/x86_64/xapic_state_test.c b/tools/testing/selftests/kvm/x86_64/xapic_state_test.c
index 6f7a5ef66718..d7d37dae3eeb 100644
--- a/tools/testing/selftests/kvm/x86_64/xapic_state_test.c
+++ b/tools/testing/selftests/kvm/x86_64/xapic_state_test.c
@@ -114,7 +114,9 @@ static void test_icr(struct xapic_vcpu *x)
* vCPUs, not vcpu.id + 1. Arbitrarily use vector 0xff.
*/
icr = APIC_INT_ASSERT | 0xff;
- for (i = vcpu->id + 1; i < 0xff; i++) {
+ for (i = 0; i < 0xff; i++) {
+ if (i == vcpu->id)
+ continue;
for (j = 0; j < 8; j++)
__test_icr(x, i << (32 + 24) | icr | (j << 8));
}
--
2.34.1
The wordings of step-by-step instructions on writing the first Kunit test
are instructing readers to write codes without knowing what these are about.
Rewrite these instructions to include the purpose of written code.
While at it, align the code blocks of these contents.
Signed-off-by: Bagas Sanjaya <bagasdotme(a)gmail.com>
---
Changes since v1 [1]:
- Fix jumped list numbering on writing the feature
This patch is based on Khalid's full path to .kunitconfig patch [2].
[1]: https://lore.kernel.org/linux-doc/20220929125458.52979-1-bagasdotme@gmail.c…
[2]: https://lore.kernel.org/linux-doc/20220929085332.4155-1-khalid.masum.92@gma…
Documentation/dev-tools/kunit/start.rst | 40 ++++++++++++++-----------
1 file changed, 22 insertions(+), 18 deletions(-)
diff --git a/Documentation/dev-tools/kunit/start.rst b/Documentation/dev-tools/kunit/start.rst
index 7999874dc4ddb3..c0a5adf6d8d665 100644
--- a/Documentation/dev-tools/kunit/start.rst
+++ b/Documentation/dev-tools/kunit/start.rst
@@ -131,17 +131,19 @@ are built-in. Otherwise the module will need to be loaded.
Writing Your First Test
=======================
-In your kernel repository, let's add some code that we can test.
+In your kernel repository, let's add some code that we can test. For this
+purpose, we are going to add simple addition driver.
-1. Create a file ``drivers/misc/example.h``, which includes:
+1. Write the feature that will be tested. First, write the declaration
+ for ``misc_example_add()`` in ``drivers/misc/example.h``:
-.. code-block:: c
+ .. code-block:: c
int misc_example_add(int left, int right);
-2. Create a file ``drivers/misc/example.c``, which includes:
+ Then implement the function in ``drivers/misc/example.c``:
-.. code-block:: c
+ .. code-block:: c
#include <linux/errno.h>
@@ -152,24 +154,25 @@ In your kernel repository, let's add some code that we can test.
return left + right;
}
-3. Add the following lines to ``drivers/misc/Kconfig``:
+2. Add Kconfig menu entry for the feature to ``drivers/misc/Kconfig``:
-.. code-block:: kconfig
+ .. code-block:: kconfig
config MISC_EXAMPLE
bool "My example"
-4. Add the following lines to ``drivers/misc/Makefile``:
+3. Add the kbuild goal that will build the feature to
+ ``drivers/misc/Makefile``:
-.. code-block:: make
+ .. code-block:: make
obj-$(CONFIG_MISC_EXAMPLE) += example.o
Now we are ready to write the test cases.
-1. Add the below test case in ``drivers/misc/example_test.c``:
+1. Write the test in ``drivers/misc/example_test.c``:
-.. code-block:: c
+ .. code-block:: c
#include <kunit/test.h>
#include "example.h"
@@ -202,31 +205,32 @@ Now we are ready to write the test cases.
};
kunit_test_suite(misc_example_test_suite);
-2. Add the following lines to ``drivers/misc/Kconfig``:
+2. Add following Kconfig entry for the test to ``drivers/misc/Kconfig``:
-.. code-block:: kconfig
+ .. code-block:: kconfig
config MISC_EXAMPLE_TEST
tristate "Test for my example" if !KUNIT_ALL_TESTS
depends on MISC_EXAMPLE && KUNIT=y
default KUNIT_ALL_TESTS
-3. Add the following lines to ``drivers/misc/Makefile``:
+3. Add kbuild goal of the test to ``drivers/misc/Makefile``:
-.. code-block:: make
+ .. code-block:: make
obj-$(CONFIG_MISC_EXAMPLE_TEST) += example_test.o
-4. Add following configuration fragments to ``.kunit/.kunitconfig``:
+4. Add following configuration fragments for the test to
+ ``.kunit/.kunitconfig``:
-.. code-block:: none
+ .. code-block:: none
CONFIG_MISC_EXAMPLE=y
CONFIG_MISC_EXAMPLE_TEST=y
5. Run the test:
-.. code-block:: bash
+ .. code-block:: bash
./tools/testing/kunit/kunit.py run
--
An old man doll... just what I always wanted! - Clara
There a couple of spelling mistakes, one in a literal string and one
in a comment. Fix them.
Signed-off-by: Colin Ian King <colin.i.king(a)gmail.com>
---
tools/testing/selftests/bpf/verifier/calls.c | 2 +-
tools/testing/selftests/bpf/verifier/var_off.c | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/tools/testing/selftests/bpf/verifier/calls.c b/tools/testing/selftests/bpf/verifier/calls.c
index 3fb4f69b1962..e1a937277b54 100644
--- a/tools/testing/selftests/bpf/verifier/calls.c
+++ b/tools/testing/selftests/bpf/verifier/calls.c
@@ -284,7 +284,7 @@
.result = ACCEPT,
},
{
- "calls: not on unpriviledged",
+ "calls: not on unprivileged",
.insns = {
BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, 1, 0, 2),
BPF_MOV64_IMM(BPF_REG_0, 1),
diff --git a/tools/testing/selftests/bpf/verifier/var_off.c b/tools/testing/selftests/bpf/verifier/var_off.c
index 187c6f6e32bc..d37f512fad16 100644
--- a/tools/testing/selftests/bpf/verifier/var_off.c
+++ b/tools/testing/selftests/bpf/verifier/var_off.c
@@ -121,7 +121,7 @@
BPF_EXIT_INSN(),
},
.fixup_map_hash_8b = { 1 },
- /* The unpriviledged case is not too interesting; variable
+ /* The unprivileged case is not too interesting; variable
* stack access is rejected.
*/
.errstr_unpriv = "R2 variable stack access prohibited for !root",
--
2.37.1
From: Roberto Sassu <roberto.sassu(a)huawei.com>
===
All credits of this patch set go to Lorenz Bauer <oss(a)lmb.io>, as he
identified this issue and proposed a number of solutions.
===
Lorenz presented at the Linux Plumbers EU 2022 a talk with title 'Closing
the BPF map permission loophole', where he reported that read-only fds can
be used for map update operations, if they were provided to eBPF programs.
This work initially started as PoC to reproduce the reported bug, and
became the test for validating an idea on how to fix the bug.
Patch 1 adds a dependency necessary for the tests.
The actual fix, in patch 2, is relatively simple. It is based on an already
existing enforcement mechanism in the eBPF verifier for map flags. As
Lorenz mentioned, a problem would be backporting this fix to stable kernels
which don't have that enforcement mechanism. However, backporting just the
enforcement mechanism itself (without introducing the new map flags and
allowing user space to use them) could meet the stable kernel criteria.
Alternatively, a completely different fix can be developed for older stable
kernels, like what Lorenz suggested, to refuse fds which are not
read/write.
Finally, patch 3 introduces the tests.
Roberto Sassu (3):
libbpf: Define bpf_get_fd_opts and introduce
bpf_map_get_fd_by_id_opts()
bpf: Enforce granted permissions in a map fd at verifier level
selftests/bpf: Test enforcement of map fd permissions at verifier
level
include/linux/bpf.h | 13 +
include/linux/bpf_verifier.h | 1 +
kernel/bpf/verifier.c | 26 +-
tools/lib/bpf/bpf.c | 12 +-
tools/lib/bpf/bpf.h | 10 +
tools/lib/bpf/libbpf.map | 3 +-
.../selftests/bpf/prog_tests/map_fd_perm.c | 227 ++++++++++++++++++
7 files changed, 288 insertions(+), 4 deletions(-)
create mode 100644 tools/testing/selftests/bpf/prog_tests/map_fd_perm.c
--
2.25.1
Hi!
>
> On 30.09.22 08:35, Zhao Gongyi wrote:
> > Some momory will be left in offline state when calling
> > offline_memory_expect_fail() failed. Restore it before exit.
> >
> > Signed-off-by: Zhao Gongyi <zhaogongyi(a)huawei.com>
> > ---
> > .../memory-hotplug/mem-on-off-test.sh | 21
> ++++++++++++++-----
> > 1 file changed, 16 insertions(+), 5 deletions(-)
> >
> > diff --git a/tools/testing/selftests/memory-hotplug/mem-on-off-test.sh
> b/tools/testing/selftests/memory-hotplug/mem-on-off-test.sh
> > index 1d87611a7d52..91a7457616bb 100755
> > --- a/tools/testing/selftests/memory-hotplug/mem-on-off-test.sh
> > +++ b/tools/testing/selftests/memory-hotplug/mem-on-off-test.sh
> > @@ -134,6 +134,16 @@ offline_memory_expect_fail()
> > return 0
> > }
> >
> > +online_all_offline_memory()
> > +{
> > + for memory in `hotpluggable_offline_memory`; do
> > + if ! online_memory_expect_success $memory; then
> > + echo "$FUNCNAME $memory: unexpected fail" >&2
>
> Do we need that output?
In my opinion, if online a memory node failed ,it should be a kernel bug catched, so, I think the output here is needed.
Thanks!
Gongyi