This series adds new functions to the mmu interval notifier API to
allow device drivers with MMUs to dynamically mirror a process' page
tables based on device faults and invalidation callbacks. The Nouveau
driver is updated to use the extended API and a set of stand alone self
tests is added to help validate and maintain correctness.
The patches are based on linux-5.5.0-rc6 and are for Jason's rdma/hmm tree
since I believe he is planning some interval notifier changes.
Changes v5 -> v6:
Rebase to linux-5.5.0-rc6
Refactored mmu interval notifier patches
Converted nouveau to use the new mmu interval notifier API
Changes v4 -> v5:
Added mmu interval notifier insert/remove/update callable from the
invalidate() callback
Updated HMM tests to use the new core interval notifier API
Changes v1 -> v4:
https://lore.kernel.org/linux-mm/20191104222141.5173-1-rcampbell@nvidia.com
Ralph Campbell (6):
mm/mmu_notifier: add mmu_interval_notifier_insert_safe()
mm/mmu_notifier: add mmu_interval_notifier_put()
mm/notifier: add mmu_interval_notifier_update()
mm/mmu_notifier: add mmu_interval_notifier_find()
nouveau: use new mmu interval notifiers
mm/hmm/test: add self tests for HMM
MAINTAINERS | 3 +
drivers/gpu/drm/nouveau/nouveau_svm.c | 313 ++++--
include/linux/mmu_notifier.h | 29 +
lib/Kconfig.debug | 11 +
lib/Makefile | 1 +
lib/test_hmm.c | 1368 ++++++++++++++++++++++++
mm/mmu_notifier.c | 223 +++-
tools/testing/selftests/vm/.gitignore | 1 +
tools/testing/selftests/vm/Makefile | 3 +
tools/testing/selftests/vm/config | 2 +
tools/testing/selftests/vm/hmm-tests.c | 1354 +++++++++++++++++++++++
tools/testing/selftests/vm/run_vmtests | 16 +
tools/testing/selftests/vm/test_hmm.sh | 97 ++
13 files changed, 3289 insertions(+), 132 deletions(-)
create mode 100644 lib/test_hmm.c
create mode 100644 tools/testing/selftests/vm/hmm-tests.c
create mode 100755 tools/testing/selftests/vm/test_hmm.sh
--
2.20.1
On 2/19/20 3:38 PM, Heidi Fahim wrote:
> On Wed, Feb 19, 2020 at 2:18 PM shuah <shuah(a)kernel.org
> <mailto:shuah@kernel.org>> wrote:
>
> Hi Heidi,
>
> On 2/18/20 3:19 PM, Heidi Fahim wrote:
> > Implemented small fix so that the script changes work directories
> to the
> > root of the linux kernel source tree from which kunit.py is run. This
> > enables the user to run kunit from any working directory. Originally
> > considered using os.path.join but this is more error prone as we
> would
> > have to find all file path usages and modify them accordingly. Using
> > os.chdir ensures that the entire script is run within /linux.
> >
> > Signed-off-by: Heidi Fahim <heidifahim(a)google.com
> <mailto:heidifahim@google.com>>
> > Reviewed-by: Brendan Higgins <brendanhiggins(a)google.com
> <mailto:brendanhiggins@google.com>>
>
> Thanks for the patch. In the future please include changes from v1
> to v2.
>
> I am assuming this v2 addresses Frank's comments.
>
>
> Sorry about that! Yes the only change here was in the commit message
> addressing Frank's comment.
>
Great. I will pull this in for 5.6-rc4.
thanks,
-- Shuah
Implemented small fix so that the script changes work directories to the
root of the linux kernel source tree from which kunit.py is run. This
enables the user to run kunit from any working directory. Originally
considered using os.path.join but this is more error prone as we would
have to find all file path usages and modify them accordingly. Using
os.chdir ensures that the entire script is run within /linux.
Signed-off-by: Heidi Fahim <heidifahim(a)google.com>
Reviewed-by: Brendan Higgins <brendanhiggins(a)google.com>
---
tools/testing/kunit/kunit.py | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/tools/testing/kunit/kunit.py b/tools/testing/kunit/kunit.py
index 3f552e847a14..060d960a7029 100755
--- a/tools/testing/kunit/kunit.py
+++ b/tools/testing/kunit/kunit.py
@@ -26,6 +26,8 @@ KunitResult = namedtuple('KunitResult', ['status','result'])
KunitRequest = namedtuple('KunitRequest', ['raw_output','timeout', 'jobs',
'build_dir', 'defconfig', 'json'])
+KernelDirectoryPath = sys.argv[0].split('tools/testing/kunit/')[0]
+
class KunitStatus(Enum):
SUCCESS = auto()
CONFIG_FAILURE = auto()
@@ -37,6 +39,13 @@ def create_default_kunitconfig():
shutil.copyfile('arch/um/configs/kunit_defconfig',
kunit_kernel.kunitconfig_path)
+def get_kernel_root_path():
+ parts = sys.argv[0] if not __file__ else __file__
+ parts = os.path.realpath(parts).split('tools/testing/kunit')
+ if len(parts) != 2:
+ sys.exit(1)
+ return parts[0]
+
def run_tests(linux: kunit_kernel.LinuxSourceTree,
request: KunitRequest) -> KunitResult:
config_start = time.time()
@@ -130,6 +139,9 @@ def main(argv, linux=None):
cli_args = parser.parse_args(argv)
if cli_args.subcommand == 'run':
+ if get_kernel_root_path():
+ os.chdir(get_kernel_root_path())
+
if cli_args.build_dir:
if not os.path.exists(cli_args.build_dir):
os.mkdir(cli_args.build_dir)
--
2.25.0.265.gbab2e86ba0-goog
This patch series extend the "bpftool feature" subcommand with the
new positional arguments:
- "section", which allows to select a specific section of probes (i.e.
"system_config", "program_types", "map_types");
- "filter_in", which allows to select only probes which matches the
given regex pattern;
- "filter_out", which allows to filter out probes which do not match the
given regex pattern.
The main motivation behind those changes is ability the fact that some
probes (for example those related to "trace" or "write_user" helpers)
emit dmesg messages which might be confusing for people who are running
on production environments. For details see the Cilium issue[0].
[0] https://github.com/cilium/cilium/issues/10048
Michal Rostecki (6):
bpftool: Move out sections to separate functions
bpftool: Allow to select a specific section to probe
bpftool: Add arguments for filtering in and filtering out probes
bpftool: Update documentation of "bpftool feature" command
bpftool: Update bash completion for "bpftool feature" command
selftests/bpf: Add test for "bpftool feature" command
.../bpftool/Documentation/bpftool-feature.rst | 37 +-
tools/bpf/bpftool/bash-completion/bpftool | 32 +-
tools/bpf/bpftool/feature.c | 592 +++++++++++++-----
tools/testing/selftests/.gitignore | 5 +-
tools/testing/selftests/bpf/Makefile | 3 +-
tools/testing/selftests/bpf/test_bpftool.py | 294 +++++++++
tools/testing/selftests/bpf/test_bpftool.sh | 5 +
7 files changed, 811 insertions(+), 157 deletions(-)
create mode 100644 tools/testing/selftests/bpf/test_bpftool.py
create mode 100755 tools/testing/selftests/bpf/test_bpftool.sh
--
2.25.0
When running the ftrace selftests, 2 failures and 6 unresolved
cases were observed. The failures can be avoided by setting
a sysctl prior to test execution (fixed in patch 1) and the
unresolved cases result from absence of testing modules which
are built based on CONFIG options being set and program
availability (fixed in patch 2).
These seem more like "unsupported" than "unresolved" errors,
since for the ftrace tests "unresolved" cases cause the test
(and thus kselftest) to report failure. With these changes
in place, the unresolved cases become unsupported and the
test failures disappear, resulting in the ftracetest program
exiting with "ok" status.
Alan Maguire (2):
ftrace/selftests: workaround cgroup RT scheduling issues
ftrace/selftest: absence of modules/programs should trigger
unsupported errors
tools/testing/selftests/ftrace/ftracetest | 23 ++++++++++++++++++++++
.../ftrace/test.d/direct/ftrace-direct.tc | 2 +-
.../ftrace/test.d/direct/kprobe-direct.tc | 2 +-
.../selftests/ftrace/test.d/event/trace_printk.tc | 2 +-
.../ftrace/test.d/ftrace/func_mod_trace.tc | 2 +-
.../ftrace/test.d/kprobe/kprobe_module.tc | 2 +-
.../selftests/ftrace/test.d/selftest/bashisms.tc | 2 +-
7 files changed, 29 insertions(+), 6 deletions(-)
--
1.8.3.1
When kunit tests are run on native (i.e. non-UML) environments, the results
of test execution are often intermixed with dmesg output. This patch
series attempts to solve this by providing a debugfs representation
of the results of the last test run, available as
/sys/kernel/debug/kunit/<testsuite>/results
Changes since v2:
- updated kunit_status2str() to kunit_status_to_string() and made it
static inline in include/kunit/test.h (Brendan)
- added log string to struct kunit_suite and kunit_case, with log
pointer in struct kunit pointing at the case log. This allows us
to collect kunit_[err|info|warning]() messages at the same time
as we printk() them. This solves for the most part the sharing
of log messages between test execution and debugfs since we
just print the suite log (which contains the test suite preamble)
and the individual test logs. The only exception is the suite-level
status, which we cannot store in the suite log as it would mean
we'd print the suite and its status prior to the suite's results.
(Brendan, patch 1)
- dropped debugfs-based kunit run patch for now so as not to cause
problems with tests currently under development (Brendan)
- fixed doc issues with code block (Brendan, patch 3)
Changes since v1:
- trimmed unneeded include files in lib/kunit/debugfs.c (Greg)
- renamed global debugfs functions to be prefixed with kunit_ (Greg)
- removed error checking for debugfs operations (Greg)
Alan Maguire (2):
kunit: add debugfs /sys/kernel/debug/kunit/<suite>/results display
kunit: update documentation to describe debugfs representation
Documentation/dev-tools/kunit/usage.rst | 12 +++
include/kunit/test.h | 52 ++++++++++--
lib/kunit/Makefile | 3 +-
lib/kunit/debugfs.c | 105 ++++++++++++++++++++++++
lib/kunit/debugfs.h | 16 ++++
lib/kunit/kunit-test.c | 4 +-
lib/kunit/test.c | 136 ++++++++++++++++++++++++--------
7 files changed, 286 insertions(+), 42 deletions(-)
create mode 100644 lib/kunit/debugfs.c
create mode 100644 lib/kunit/debugfs.h
--
1.8.3.1
Implemented small fix so that the script changes work directories to the
linux directory where kunit.py is run. This enables the user to run
kunit from any working directory. Originally considered using
os.path.join but this is more error prone as we would have to find all
file path usages and modify them accordingly. Using os.chdir ensures
that the entire script is run within /linux.
Signed-off-by: Heidi Fahim <heidifahim(a)google.com>
---
tools/testing/kunit/kunit.py | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/tools/testing/kunit/kunit.py b/tools/testing/kunit/kunit.py
index e59eb9e7f923..3cc7be7b28a0 100755
--- a/tools/testing/kunit/kunit.py
+++ b/tools/testing/kunit/kunit.py
@@ -35,6 +35,13 @@ def create_default_kunitconfig():
shutil.copyfile('arch/um/configs/kunit_defconfig',
kunit_kernel.kunitconfig_path)
+def get_kernel_root_path():
+ parts = sys.argv[0] if not __file__ else __file__
+ parts = os.path.realpath(parts).split('tools/testing/kunit')
+ if len(parts) != 2:
+ sys.exit(1)
+ return parts[0]
+
def run_tests(linux: kunit_kernel.LinuxSourceTree,
request: KunitRequest) -> KunitResult:
config_start = time.time()
@@ -114,6 +121,9 @@ def main(argv, linux=None):
cli_args = parser.parse_args(argv)
if cli_args.subcommand == 'run':
+ if get_kernel_root_path():
+ os.chdir(get_kernel_root_path())
+
if cli_args.build_dir:
if not os.path.exists(cli_args.build_dir):
os.mkdir(cli_args.build_dir)
--
2.25.0.341.g760bfbb309-goog
There's four patches here, but only one of them actually does anything. The
first patch fixes a BPF selftests build failure on my machine and has already
been sent to the list separately. The next three are just staged such that
there are some patches that avoid changing any functionality pulled out from
the whole point of those refactorings, with two cleanups and then the idea.
Maybe this is an odd thing to say in a cover letter, but I'm not actually sure
this patch set is a good idea. The issue of extra moves after calls came up as
I was reviewing some unrelated performance optimizations to the RISC-V BPF JIT.
I figured I'd take a whack at performing the optimization in the context of the
arm64 port just to get a breath of fresh air, and I'm not convinced I like the
results.
That said, I think I would accept something like this for the RISC-V port
because we're already doing a multi-pass optimization for shrinking function
addresses so it's not as much extra complexity over there. If we do that we
should probably start puling some of this code into the shared BPF compiler,
but we're also opening the doors to more complicated BPF JIT optimizations.
Given that the BPF JIT appears to have been designed explicitly to be
simple/fast as opposed to perform complex optimization, I'm not sure this is a
sane way to move forward.
I figured I'd send the patch set out as more of a question than anything else.
Specifically:
* How should I go about measuring the performance of these sort of
optimizations? I'd like to balance the time it takes to run the JIT with the
time spent executing the program, but I don't have any feel for what real BPF
programs look like or have any benchmark suite to run. Is there something
out there this should be benchmarked against? (I'd also like to know that to
run those benchmarks on the RISC-V port.)
* Is this the sort of thing that makes sense in a BPF JIT? I guess I've just
realized I turned "review this patch" into a way bigger rabbit hole than I
really want to go down...
I worked on top of 5.4 for these, but trivially different versions of the
patches applied on Linus' master a few days ago when I tried. LMK if those
aren't sane places to start from over here, I'm new to both arm64 and BPF so I
might be a bit lost.
[PATCH 1/4] selftests/bpf: Elide a check for LLVM versions that can't
[PATCH 2/4] arm64: bpf: Convert bpf2a64 to a function
[PATCH 3/4] arm64: bpf: Split the read and write halves of dst
[PATCH 4/4] arm64: bpf: Elide some moves to a0 after calls
When kunit tests are run on native (i.e. non-UML) environments, the results
of test execution are often intermixed with dmesg output. This patch
series attempts to solve this by providing a debugfs representation
of the results of the last test run, available as
/sys/kernel/debug/kunit/<testsuite>/results
Changes since v3:
- added CONFIG_KUNIT_DEBUGFS to support conditional compilation of debugfs
representation, including string logging (Frank, patch 1)
- removed unneeded NULL check for test_case in kunit_suite_for_each_test_case()
(Frank, patch 1)
- added kunit log test to verify logging multiple strings works
(Frank, patch 2)
- rephrased description of results file (Frank, patch 3)
Changes since v2:
- updated kunit_status2str() to kunit_status_to_string() and made it
static inline in include/kunit/test.h (Brendan)
- added log string to struct kunit_suite and kunit_case, with log
pointer in struct kunit pointing at the case log. This allows us
to collect kunit_[err|info|warning]() messages at the same time
as we printk() them. This solves for the most part the sharing
of log messages between test execution and debugfs since we
just print the suite log (which contains the test suite preamble)
and the individual test logs. The only exception is the suite-level
status, which we cannot store in the suite log as it would mean
we'd print the suite and its status prior to the suite's results.
(Brendan, patch 1)
- dropped debugfs-based kunit run patch for now so as not to cause
problems with tests currently under development (Brendan)
- fixed doc issues with code block (Brendan, patch 3)
Changes since v1:
- trimmed unneeded include files in lib/kunit/debugfs.c (Greg)
- renamed global debugfs functions to be prefixed with kunit_ (Greg)
- removed error checking for debugfs operations (Greg)
Alan Maguire (3):
kunit: add debugfs /sys/kernel/debug/kunit/<suite>/results display
kunit: add log test
kunit: update documentation to describe debugfs representation
Documentation/dev-tools/kunit/usage.rst | 13 ++++
include/kunit/test.h | 54 +++++++++++---
lib/kunit/Kconfig | 8 +++
lib/kunit/Makefile | 4 ++
lib/kunit/debugfs.c | 116 ++++++++++++++++++++++++++++++
lib/kunit/debugfs.h | 30 ++++++++
lib/kunit/kunit-test.c | 31 +++++++-
lib/kunit/test.c | 122 ++++++++++++++++++++++++--------
8 files changed, 336 insertions(+), 42 deletions(-)
create mode 100644 lib/kunit/debugfs.c
create mode 100644 lib/kunit/debugfs.h
--
1.8.3.1