The default timeout for selftests tests is 45 seconds. Although
we already have 13 settings for tests of about 96 sefltests which
use a timeout greater than this, we want to try to avoid encouraging
more tests to forcing a higher test timeout as selftests strives to
run all tests quickly. Selftests also uses the timeout as a non-fatal
error. Only tests runners which have control over a system would know
if to treat a timeout as fatal or not.
To help with all this:
o Enhance documentation to avoid future increases of insane timeouts
o Add the option to allow overriding the default timeout with test
runners with a command line option
Suggested-by: Shuah Khan <skhan(a)linuxfoundation.org>
Signed-off-by: Luis Chamberlain <mcgrof(a)kernel.org>
---
Documentation/dev-tools/kselftest.rst | 22 +++++++++++++++++++++
tools/testing/selftests/kselftest/runner.sh | 11 ++++++++++-
tools/testing/selftests/run_kselftest.sh | 5 +++++
3 files changed, 37 insertions(+), 1 deletion(-)
diff --git a/Documentation/dev-tools/kselftest.rst b/Documentation/dev-tools/kselftest.rst
index 12b575b76b20..dd214af7b7ff 100644
--- a/Documentation/dev-tools/kselftest.rst
+++ b/Documentation/dev-tools/kselftest.rst
@@ -168,6 +168,28 @@ the `-t` option for specific single tests. Either can be used multiple times::
For other features see the script usage output, seen with the `-h` option.
+Timeout for selftests
+=====================
+
+Selftests are designed to be quick and so a default timeout is used of 45
+seconds for each test. Tests can override the default timeout by adding
+a settings file in their directory and set a timeout variable there to the
+configured a desired upper timeout for the test. Only a few tests override
+the timeout with a value higher than 45 seconds, selftests strives to keep
+it that way. Timeouts in selftests are not considered fatal because the
+system under which a test runs may change and this can also modify the
+expected time it takes to run a test. If you have control over the systems
+which will run the tests you can configure a test runner on those systems to
+use a greater or lower timeout on the command line as with the `-o` or
+the `--override-timeout` argument. For example to use 165 seconds instead
+one would use:
+
+ $ ./run_kselftest.sh --override-timeout 165
+
+You can look at the TAP output to see if you ran into the timeout. Test
+runners which know a test must run under a specific time can then optionally
+treat these timeouts then as fatal.
+
Packaging selftests
===================
diff --git a/tools/testing/selftests/kselftest/runner.sh b/tools/testing/selftests/kselftest/runner.sh
index 294619ade49f..1c952d1401d4 100644
--- a/tools/testing/selftests/kselftest/runner.sh
+++ b/tools/testing/selftests/kselftest/runner.sh
@@ -8,7 +8,8 @@ export logfile=/dev/stdout
export per_test_logging=
# Defaults for "settings" file fields:
-# "timeout" how many seconds to let each test run before failing.
+# "timeout" how many seconds to let each test run before running
+# over our soft timeout limit.
export kselftest_default_timeout=45
# There isn't a shell-agnostic way to find the path of a sourced file,
@@ -90,6 +91,14 @@ run_one()
done < "$settings"
fi
+ # Command line timeout overrides the settings file
+ if [ -n "$kselftest_override_timeout" ]; then
+ kselftest_timeout="$kselftest_override_timeout"
+ echo "# overriding timeout to $kselftest_timeout" >> "$logfile"
+ else
+ echo "# timeout set to $kselftest_timeout" >> "$logfile"
+ fi
+
TEST_HDR_MSG="selftests: $DIR: $BASENAME_TEST"
echo "# $TEST_HDR_MSG"
if [ ! -e "$TEST" ]; then
diff --git a/tools/testing/selftests/run_kselftest.sh b/tools/testing/selftests/run_kselftest.sh
index 97165a83df63..9a981b36bd7f 100755
--- a/tools/testing/selftests/run_kselftest.sh
+++ b/tools/testing/selftests/run_kselftest.sh
@@ -26,6 +26,7 @@ Usage: $0 [OPTIONS]
-l | --list List the available collection:test entries
-d | --dry-run Don't actually run any tests
-h | --help Show this usage info
+ -o | --override-timeout Number of seconds after which we timeout
EOF
exit $1
}
@@ -33,6 +34,7 @@ EOF
COLLECTIONS=""
TESTS=""
dryrun=""
+kselftest_override_timeout=""
while true; do
case "$1" in
-s | --summary)
@@ -51,6 +53,9 @@ while true; do
-d | --dry-run)
dryrun="echo"
shift ;;
+ -o | --override-timeout)
+ kselftest_override_timeout="$2"
+ shift 2 ;;
-h | --help)
usage 0 ;;
"")
--
2.39.2
It turned out that an even dozen patches were required in order to get the
selftests building cleanly, and all running, once again. I made it worse on
myself by insisting on using clang, which seems to uncover a few more warnings
than gcc these days.
So I still haven't gotten to my original goal of running a new HMM test that
Alistair handed me (it's not here yet), but at least this fixes everything I ran
into just now.
John Hubbard (12):
selftests/mm: fix uffd-stress unused function warning
selftests/mm: fix unused variable warning in hugetlb-madvise.c
selftests/mm: fix unused variable warning in migration.c
selftests/mm: fix a char* assignment in mlock2-tests.c
selftests/mm: fix invocation of tests that are run via shell scripts
selftests/mm: .gitignore: add mkdirty, va_high_addr_switch
selftests/mm: set -Wno-format-security to avoid uffd build warnings
selftests/mm: fix a "possibly uninitialized" warning in pkey-x86.h
selftests/mm: move psize(), pshift() into vm_utils.c
selftests/mm: move uffd* routines from vm_util.c to uffd-common.c
selftests/mm: fix missing UFFDIO_CONTINUE_MODE_WP and similar build
failures
selftests/mm: fix uffd-unit-tests.c build failure due to missing
MADV_COLLAPSE
tools/testing/selftests/mm/.gitignore | 2 +
tools/testing/selftests/mm/Makefile | 9 +-
tools/testing/selftests/mm/cow.c | 7 --
tools/testing/selftests/mm/hugepage-mremap.c | 2 +-
tools/testing/selftests/mm/hugetlb-madvise.c | 2 +-
tools/testing/selftests/mm/khugepaged.c | 10 --
.../selftests/mm/ksm_functional_tests.c | 2 +-
tools/testing/selftests/mm/migration.c | 2 +-
tools/testing/selftests/mm/mlock2-tests.c | 2 +-
tools/testing/selftests/mm/pkey-x86.h | 2 +-
tools/testing/selftests/mm/run_vmtests.sh | 6 +-
tools/testing/selftests/mm/uffd-common.c | 105 +++++++++++++++++
tools/testing/selftests/mm/uffd-common.h | 29 ++++-
tools/testing/selftests/mm/uffd-stress.c | 10 --
tools/testing/selftests/mm/vm_util.c | 106 ++----------------
tools/testing/selftests/mm/vm_util.h | 36 ++----
16 files changed, 170 insertions(+), 162 deletions(-)
base-commit: 929ed21dfdb6ee94391db51c9eedb63314ef6847
--
2.40.1
On Sun, Jun 4, 2023, at 10:29, 吴章金 wrote:
>
> Sorry for missing part of your feedbacks, I will check if -nostdlib
> stops the linking of libgcc_s or my own separated test script forgot
> linking the libgcc_s manually.
According to the gcc documentation, -nostdlib drops libgcc.a, but
adding -lgcc is the recommended way to bring it back.
> And as suggestion from Thomas' reply,
>
>>> Perhaps we really need to add the missing __divdi3 and __aeabi_ldivmod and the
>>> ones for the other architectures, or get one from lib/math/div64.c.
>
>>No, these ones come from the compiler via libgcc_s, we must not try to
> reimplement them. And we should do our best to avoid depending on them
> to avoid the error you got above.
>
> So, the explicit conversion is used instead in the patch.
I think a cast to a 32-bit type is ideal when converting the
clock_gettime() result into microseconds, since the kernel guarantees
that the timespec value is normalized, with all zeroes in the
upper 34 bits. Going through __aeabi_ldivmod would make the
conversion much slower.
For user supplied non-normalized timeval values, it's not obvious
whether we need the full 64-bit division
Arnd
This patchset consolidates a number of disparate items that can all be
considered cleanups. They are all related to mlxsw in that they are
directly in mlxsw code, or in selftests that mlxsw heavily uses.
- patch #1 fixes a comment, patch #2 propagates an extack
- patches #3 and #4 tweak several loops to query a resource once and cache
in a local variable instead of querying on each iteration
- patches #5 and #6 fix selftest diagrams, and #7 adds a missing diagram
into an existing test
- patch #8 disables a PVID on a bridge in a selftest that should not need
said PVID
Petr Machata (8):
mlxsw: spectrum_router: Clarify a comment
mlxsw: spectrum_router: Use extack in
mlxsw_sp~_rif_ipip_lb_configure()
mlxsw: spectrum_router: Do not query MAX_RIFS on each iteration
mlxsw: spectrum_router: Do not query MAX_VRS on each iteration
selftests: mlxsw: ingress_rif_conf_1d: Fix the diagram
selftests: mlxsw: egress_vid_classification: Fix the diagram
selftests: router_bridge_vlan: Add a diagram
selftests: router_bridge_vlan: Set vlan_default_pvid 0 on the bridge
.../ethernet/mellanox/mlxsw/spectrum_router.c | 26 ++++++++++++-------
.../net/mlxsw/egress_vid_classification.sh | 5 ++--
.../drivers/net/mlxsw/ingress_rif_conf_1d.sh | 5 ++--
.../net/forwarding/router_bridge_vlan.sh | 24 ++++++++++++++++-
4 files changed, 43 insertions(+), 17 deletions(-)
--
2.40.1
Hi, Willy
When I worked on adding new syscalls and the related library routines,
I have seen most of the library routines share the same syscall call and
return logic, this patchset adds two macros to simplify and shrink them.
All of them have been tested on arm, aarch64, rv32 and rv64, no new
regressions found.
If this is ok, I will rebase the new syscalls and library routines on
this patchset.
Best regards,
Zhangjin
---
Zhangjin Wu (4):
tools/nolibc: unistd.h: add __syscall() and __syscall_ret() helpers
tools/nolibc: unistd.h: apply __syscall_ret() helper
tools/nolibc: sys.h: apply __syscall_ret() helper
tools/nolibc: sys.h: apply __syscall() helper
tools/include/nolibc/sys.h | 369 ++++++----------------------------
tools/include/nolibc/unistd.h | 12 +-
2 files changed, 65 insertions(+), 316 deletions(-)
--
2.25.1
Hi, Willy
This is the v3 generic part1 for rv32, all of the found issues of v2
part1 [1] have been fixed up, several generic patches have been fixed up
and merged from v2 part2 [2] to this series, the standalone test_fork
patch [4] is merged with a Reviewed-by line into this series too.
This series is based on 20230528-nolibc-rv32+stkp5 branch of [5].
Changes from v2 -> v3:
* selftests/nolibc: fix up compile warning with glibc on x86_64
Use simpler 'long long' conversion instead of old #ifdef ...
(Suggestion from Willy)
* tools/nolibc: add missing nanoseconds support for __NR_statx
Split the compound assignment into two single assignments
(Suggestion from Thomas)
* selftests/nolibc: add new gettimeofday test cases
Removed the gettimeofday(NULL, &tz)
(Suggestion from Thomas)
All of the commit messages have been re-checked, some missing
Suggested-by lines are added.
The whole patchset have been tested on arm, aarch64, rv32 and rv64, no
regressions (the next compile patchset is required to do rv32 test).
The nolibc-test has been tested with glibc on x86_64 too.
Btw, we have found such poll failures on arm (not introduced by this
patchset), this will be fixed in our coming ppoll_time64 patchset:
48 poll_null = -1 ENOSYS [FAIL]
49 poll_stdout = -1 ENOSYS [FAIL]
50 poll_fault = -1 ENOSYS != (-1 EFAULT) [FAIL]
And the gettimeofday_null removal patch from Thomas [3] may conflicts
with the gettimeofday removal and addition patches, but it is not hard
to fix.
Best regards,
Zhangjin
---
[1]: https://lore.kernel.org/linux-riscv/cover.1685362482.git.falcon@tinylab.org…
[2]: https://lore.kernel.org/linux-riscv/cover.1685387484.git.falcon@tinylab.org…
[3]: https://lore.kernel.org/lkml/20230530-nolibc-gettimeofday-v1-1-7307441a002b…
[4]: https://lore.kernel.org/lkml/61bdfe7bacebdef8aa9195f6f2550a5b0d33aab3.16854…
[5]: https://git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git
Zhangjin Wu (12):
selftests/nolibc: syscall_args: use generic __NR_statx
tools/nolibc: add missing nanoseconds support for __NR_statx
selftests/nolibc: allow specify extra arguments for qemu
selftests/nolibc: fix up compile warning with glibc on x86_64
selftests/nolibc: not include limits.h for nolibc
selftests/nolibc: use INT_MAX instead of __INT_MAX__
tools/nolibc: arm: add missing my_syscall6
tools/nolibc: open: fix up compile warning for arm
selftests/nolibc: support two errnos with EXPECT_SYSER2()
selftests/nolibc: remove gettimeofday_bad1/2 completely
selftests/nolibc: add new gettimeofday test cases
selftests/nolibc: test_fork: fix up duplicated print
tools/include/nolibc/arch-arm.h | 23 +++++++++++
tools/include/nolibc/stdint.h | 14 +++++++
tools/include/nolibc/sys.h | 39 +++++++++---------
tools/testing/selftests/nolibc/Makefile | 2 +-
tools/testing/selftests/nolibc/nolibc-test.c | 42 ++++++++++++--------
5 files changed, 85 insertions(+), 35 deletions(-)
--
2.25.1