Hi Linus,
Please pull the following second Kselftest fixes update for Linux 6.6-rc2.
This kselftest fixes update for Linux 6.6-rc2 consists of important
fixes to user_events test and ftrace test.
user_events test has been enabled for default run in Linux 6.6-rc1. The
following fixes are for bugs found since then:
- adds checks for dependencies and skips the test. user_events test
requires root access, and tracefs and user_events enabled. It leaves
tracefs mounted and a fix is in progress for that missing piece.
- creates user_events test-specific Kconfig fragments.
ftrace test fixes:
- unmounts tracefs for recovering environment. Fix identified during the
above mentioned user_events dependencies fix.
- adds softlink to latest log directory improving usage.
diff is attached.
thanks,
-- Shuah
----------------------------------------------------------------
The following changes since commit 0bb80ecc33a8fb5a682236443c1e740d5c917d1d:
Linux 6.6-rc1 (2023-09-10 16:28:41 -0700)
are available in the Git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest tags/linux-kselftest-fixes-6.6-rc2
for you to fetch changes up to 7e021da80f48582171029714f8a487347f29dddb:
selftests: tracing: Fix to unmount tracefs for recovering environment (2023-09-12 09:34:20 -0600)
----------------------------------------------------------------
linux-kselftest-fixes-6.6-rc2
This kselftest fixes update for Linux 6.6-rc2 consists of important
fixes to user_events test and ftrace test.
user_events test has been enabled for default run in Linux 6.6-rc1. The
following fixes are for bugs found since then:
- adds checks for dependencies and skips the test. user_events test
requires root access, and tracefs and user_events enabled. It leaves
tracefs mounted and a fix is in progress for that missing piece.
- creates user_events test-specific Kconfig fragments.
ftrace test fixes:
- unmounts tracefs for recovering environment. Fix identified during the
above mentioned user_events dependencies fix.
- adds softlink to latest log directory improving usage.
----------------------------------------------------------------
Beau Belgrave (1):
selftests/user_events: Fix failures when user_events is not installed
Masami Hiramatsu (Google) (1):
selftests: tracing: Fix to unmount tracefs for recovering environment
Naresh Kamboju (1):
selftests: user_events: create test-specific Kconfig fragments
Steven Rostedt (Google) (1):
ftrace/selftests: Add softlink to latest log directory
tools/testing/selftests/ftrace/ftracetest | 18 +++-
tools/testing/selftests/user_events/abi_test.c | 3 +
tools/testing/selftests/user_events/config | 1 +
tools/testing/selftests/user_events/dyn_test.c | 2 +
tools/testing/selftests/user_events/ftrace_test.c | 3 +
tools/testing/selftests/user_events/perf_test.c | 3 +
.../selftests/user_events/user_events_selftests.h | 100 +++++++++++++++++++++
7 files changed, 129 insertions(+), 1 deletion(-)
create mode 100644 tools/testing/selftests/user_events/config
create mode 100644 tools/testing/selftests/user_events/user_events_selftests.h
----------------------------------------------------------------
Changes from RFC
(https://lore.kernel.org/damon/20230910034048.59191-1-sj@kernel.org/)
- Add kselftest for damos_apply_interval_us sysfs file
- Rebase on latest mm-unstable
DAMON-based operation schemes are applied for every aggregation
interval. That is mainly because schemes are using nr_accesses, which
be complete to be used for every aggregation interval.
This makes some DAMOS use cases be tricky. Quota setting under long
aggregation interval is one such example. Suppose the aggregation
interval is ten seconds, and there is a scheme having CPU quota 100ms
per 1s. The scheme will actually uses 100ms per ten seconds, since it
cannobe be applied before next aggregation interval. The feature is
working as intended, but the results might not that intuitive for some
users. This could be fixed by updating the quota to 1s per 10s. But,
in the case, the CPU usage of DAMOS could look like spikes, and actually
make a bad effect to other CPU-sensitive workloads.
Also, with such huge aggregation interval, users may want schemes to be
applied more frequently.
DAMON provides nr_accesses_bp, which is updated for each sampling
interval in a way that reasonable to be used. By using that instead of
nr_accesses, DAMOS can have its own time interval and mitigate abovely
mentioned issues.
This patchset makes DAMOS schemes to use nr_accesses_bp instead of
nr_accesses, and have their own timing intervals. Also update DAMOS
tried regions sysfs files and DAMOS before_apply tracepoint to use the
new data as their source. Note that the interval is zero by default,
and it is interpreted to use the aggregation interval instead. This
avoids making user-visible behavioral changes.
Patches Seuqeunce
-----------------
The first patch (patch 1/9) makes DAMOS uses nr_accesses_bp instead of
nr_accesses, and following two patches (patches 2/9 and 3/9) updates
DAMON sysfs interface for DAMOS tried regions and the DAMOS before_apply
tracespoint to use nr_accesses_bp instead of nr_accesses, respectively.
The following two patches (patches 4/9 and 5/9) implements the
scheme-specific apply interval for DAMON kernel API users and update the
design document for the new feature.
Finally, the following four patches (patches 6/9, 7/9, 8/9 and 9/9) add
support of the feature in DAMON sysfs interface, add a simple selftest
test case, and document the new file on the usage and the ABI documents,
repsectively.
SeongJae Park (9):
mm/damon/core: make DAMOS uses nr_accesses_bp instead of nr_accesses
mm/damon/sysfs-schemes: use nr_accesses_bp as the source of
tried_regions/<N>/nr_accesses
mm/damon/core: use nr_accesses_bp as a source of damos_before_apply
tracepoint
mm/damon/core: implement scheme-specific apply interval
Docs/mm/damon/design: document DAMOS apply intervals
mm/damon/sysfs-schemes: support DAMOS apply interval
selftests/damon/sysfs: test DAMOS apply intervals
Docs/admin-guide/mm/damon/usage: update for DAMOS apply intervals
Docs/ABI/damon: update for DAMOS apply intervals
.../ABI/testing/sysfs-kernel-mm-damon | 7 ++
Documentation/admin-guide/mm/damon/usage.rst | 9 ++-
Documentation/mm/damon/design.rst | 3 +-
include/linux/damon.h | 17 +++-
include/trace/events/damon.h | 2 +-
mm/damon/core.c | 77 ++++++++++++++++---
mm/damon/dbgfs.c | 3 +-
mm/damon/lru_sort.c | 2 +
mm/damon/reclaim.c | 2 +
mm/damon/sysfs-schemes.c | 40 ++++++++--
tools/testing/selftests/damon/sysfs.sh | 1 +
11 files changed, 141 insertions(+), 22 deletions(-)
base-commit: abf99d088da21843246382c7a95f21e886193c31
--
2.25.1
Hi Al, Linus,
These patches make some changes to the kunit tests previously added for
iov_iter testing, in particular adding support for testing UBUF/IOVEC
iterators:
(1) Clean up a couple of checkpatch style complaints.
(2) Consolidate some repeated bits of code into helper functions and use
the same struct to represent straight offset/address ranges and
partial page lists.
(3) Add a function to set up a userspace VM, attach the VM to the kunit
testing thread, create an anonymous file, stuff some pages into the
file and map the file into the VM to act as a buffer that can be used
with UBUF/IOVEC iterators.
I map an anonymous file with pages attached rather than using MAP_ANON
so that I can check the pages obtained from iov_iter_extract_pages()
without worrying about them changing due to swap, migrate, etc..
[?] Is this the best way to do things? Mirroring execve, it requires
a number of extra core symbols to be exported. Should this be done in
the core code?
(4) Add tests for copying into and out of UBUF and IOVEC iterators.
(5) Add tests for extracting pages from UBUF and IOVEC iterators.
(6) Add tests to benchmark copying 256MiB to UBUF, IOVEC, KVEC, BVEC and
XARRAY iterators.
[!] Note that this requires 256MiB of memory for UBUF and IOVEC; the
KVEC, BVEC and XARRAY benchmarking maps a single page multiple times.
I might be able to shrink that if I can add the same page multiple
times to the anon file's pagecache. I'm sure this is not recommended,
but I might be able to get away with it for this particular
application.
(7) Add a test to benchmark copying 256MiB through dynamically allocated
256-page bvecs to simulate bio construction.
Example benchmarks output:
iov_kunit_benchmark_ubuf: avg 26899 uS, stddev 142 uS
iov_kunit_benchmark_iovec: avg 26897 uS, stddev 74 uS
iov_kunit_benchmark_kvec: avg 2688 uS, stddev 35 uS
iov_kunit_benchmark_bvec: avg 3139 uS, stddev 21 uS
iov_kunit_benchmark_bvec_split: avg 3379 uS, stddev 15 uS
iov_kunit_benchmark_xarray: avg 3582 uS, stddev 13 uS
I've pushed the patches here also:
https://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git/log/?…
David
David Howells (9):
iov_iter: Fix some checkpatch complaints in kunit tests
iov_iter: Consolidate some of the repeated code into helpers
iov_iter: Consolidate the test vector struct in the kunit tests
iov_iter: Consolidate bvec pattern checking
iov_iter: Create a function to prepare userspace VM for UBUF/IOVEC
tests
iov_iter: Add copy kunit tests for ITER_UBUF and ITER_IOVEC
iov_iter: Add extract kunit tests for ITER_UBUF and ITER_IOVEC
iov_iter: Add benchmarking kunit tests
iov_iter: Add benchmarking kunit tests for UBUF/IOVEC
fs/anon_inodes.c | 1 +
kernel/fork.c | 2 +
lib/kunit_iov_iter.c | 1211 +++++++++++++++++++++++++++++++++++-------
mm/mmap.c | 1 +
mm/util.c | 1 +
5 files changed, 1024 insertions(+), 192 deletions(-)
Write_schemata() uses fprintf() to write a bitmask into a schemata file
inside resctrl FS. It checks fprintf() return value but it doesn't check
fclose() return value. Error codes from fprintf() such as write errors,
are buffered and flushed back to the user only after fclose() is executed
which means any invalid bitmask can be written into the schemata file.
Rewrite write_schemata() to use syscalls instead of stdio file
operations to avoid the buffering.
The resctrlfs.c file defines functions that interact with the resctrl FS
while resctrl_val.c file defines functions that perform measurements on
the cache. Run_benchmark() fits logically into the second file before
resctrl_val() function that uses it.
Move run_benchmark() from resctrlfs.c to resctrl_val.c and remove
redundant part of the kernel-doc comment. Make run_benchmark() static
and remove it from the header file.
Series is based on kselftest next branch.
Changelog v3:
- Use snprintf() return value instead of strlen() in write_schemata().
(Ilpo)
- Make run_benchmark() static and remove it from the header file.
(Reinette)
- Added Ilpo's reviewed-by tag to Patch 2/2.
- Patch messages and cover letter rewording.
Changelog v2:
- Change sprintf() to snprintf() in write_schemata().
- Redo write_schemata() with syscalls instead of stdio functions.
- Fix typos and missing dots in patch messages.
- Branch printf attribute patch to a separate series.
[v1] https://lore.kernel.org/all/cover.1692880423.git.maciej.wieczor-retman@inte…
[v2] https://lore.kernel.org/all/cover.1693213468.git.maciej.wieczor-retman@inte…
Wieczor-Retman Maciej (2):
selftests/resctrl: Fix schemata write error check
selftests/resctrl: Move run_benchmark() to a more fitting file
tools/testing/selftests/resctrl/resctrl.h | 1 -
tools/testing/selftests/resctrl/resctrl_val.c | 50 ++++++++++++
tools/testing/selftests/resctrl/resctrlfs.c | 78 ++++---------------
3 files changed, 64 insertions(+), 65 deletions(-)
base-commit: 9b1db732866bee060b9bca9493e5ebf5e8874c48
--
2.42.0
Changes from RFC
(https://lore.kernel.org/damon/20230909033711.55794-1-sj@kernel.org/)
- Rebase on latest mm-unstable
- Minor wordsmithing of coverletter
DAMON checks the access to each region for every sampling interval, increase
the access rate counter of the region, namely nr_accesses, if the access was
made. For every aggregation interval, the counter is reset. The counter is
exposed to users to be used as a metric showing the relative access rate
(frequency) of each region. In other words, DAMON provides access rate of each
region in every aggregation interval. The aggregation avoids temporal access
pattern changes making things confusing. However, this also makes a few
DAMON-related operations to unnecessarily need to be aligned to the aggregation
interval. This can restrict the flexibility of DAMON applications, especially
when the aggregation interval is huge.
To provide the monitoring results in finer-grained timing while keeping
handling of temporal access pattern change, this patchset implements a
pseudo-moving sum based access rate metric. It is pseudo-moving sum because
strict moving sum implementation would need to keep all values for last time
window, and that could incur high overhead of there could be arbitrary number
of values in a time window. Especially in case of the nr_accesses, since the
sampling interval and aggregation interval can arbitrarily set and the past
values should be maintained for every region, it could be risky. The
pseudo-moving sum assumes there were no temporal access pattern change in last
discrete time window to remove the needs for keeping the list of the last time
window values. As a result, it beocmes not strict moving sum implementation,
but provides a reasonable accuracy.
Also, it keeps an important property of the moving sum. That is, the moving
sum becomes same to discrete-window based sum at the time that aligns to the
time window. This means using the pseudo moving sum based nr_accesses makes no
change to users who shows the value for every aggregation interval.
Patches Sequence
----------------
The sequence of the patches is as follows. The first four patches are
for preparation of the change. The first two (patches 1 and 2)
implements a helper function for nr_accesses update and eliminate corner
case that skips use of the function, respectively. Following two
(patches 3 and 4) respectively implement the pseudo-moving sum function
and its simple unit test case.
Two patches for making DAMON to use the pseudo-moving sum follow. The
fifthe one (patch 5) introduces a new field for representing the
pseudo-moving sum-based access rate of each region, and the sixth one
makes the new representation to actually updated with the pseudo-moving
sum function.
Last two patches (patches 7 and 8) makes followup fixes for skipping
unnecessary updates and marking the moving sum function as static,
respectively.
SeongJae Park (8):
mm/damon/core: define and use a dedicated function for region access
rate update
mm/damon/vaddr: call damon_update_region_access_rate() always
mm/damon/core: implement a pseudo-moving sum function
mm/damon/core-test: add a unit test for damon_moving_sum()
mm/damon/core: introduce nr_accesses_bp
mm/damon/core: use pseudo-moving sum for nr_accesses_bp
mm/damon/core: skip updating nr_accesses_bp for each aggregation
interval
mm/damon/core: mark damon_moving_sum() as a static function
include/linux/damon.h | 16 +++++++++-
mm/damon/core-test.h | 21 ++++++++++++
mm/damon/core.c | 74 +++++++++++++++++++++++++++++++++++++++++++
mm/damon/paddr.c | 11 +++----
mm/damon/vaddr.c | 22 +++++++------
5 files changed, 128 insertions(+), 16 deletions(-)
base-commit: a5b7405a0eaa74d23547ede9c3820f01ee0a2c13
--
2.25.1
Add parsing of attributes as diagnostic data. Fixes issue with test plan
being parsed incorrectly as diagnostic data when located after
suite-level attributes.
Note that if there does not exist a test plan line, the diagnostic lines
between the suite header and the first result will be saved in the suite
log rather than the first test log.
This could be changed to do the opposite if preferred.
Signed-off-by: Rae Moar <rmoar(a)google.com>
---
tools/testing/kunit/kunit_parser.py | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/tools/testing/kunit/kunit_parser.py b/tools/testing/kunit/kunit_parser.py
index 79d8832c862a..ce34be15c929 100644
--- a/tools/testing/kunit/kunit_parser.py
+++ b/tools/testing/kunit/kunit_parser.py
@@ -450,7 +450,7 @@ def parse_diagnostic(lines: LineStream) -> List[str]:
Log of diagnostic lines
"""
log = [] # type: List[str]
- non_diagnostic_lines = [TEST_RESULT, TEST_HEADER, KTAP_START, TAP_START]
+ non_diagnostic_lines = [TEST_RESULT, TEST_HEADER, KTAP_START, TAP_START, TEST_PLAN]
while lines and not any(re.match(lines.peek())
for re in non_diagnostic_lines):
log.append(lines.pop())
@@ -726,6 +726,7 @@ def parse_test(lines: LineStream, expected_num: int, log: List[str], is_subtest:
# test plan
test.name = "main"
ktap_line = parse_ktap_header(lines, test)
+ test.log.extend(parse_diagnostic(lines))
parse_test_plan(lines, test)
parent_test = True
else:
@@ -737,6 +738,7 @@ def parse_test(lines: LineStream, expected_num: int, log: List[str], is_subtest:
if parent_test:
# If KTAP version line and/or subtest header is found, attempt
# to parse test plan and print test header
+ test.log.extend(parse_diagnostic(lines))
parse_test_plan(lines, test)
print_test_header(test)
expected_count = test.expected_count
base-commit: 9076bc476d7ebf0565903c4b048442131825c1c3
--
2.42.0.459.ge4e396fd5e-goog