When running the ftrace selftests, 2 failures and 6 unresolved cases were observed. The failures can be avoided by setting a sysctl prior to test execution (fixed in patch 1) and the unresolved cases result from absence of testing modules which are built based on CONFIG options being set and program availability (fixed in patch 2).
These seem more like "unsupported" than "unresolved" errors, since for the ftrace tests "unresolved" cases cause the test (and thus kselftest) to report failure. With these changes in place, the unresolved cases become unsupported and the test failures disappear, resulting in the ftracetest program exiting with "ok" status.
Alan Maguire (2): ftrace/selftests: workaround cgroup RT scheduling issues ftrace/selftest: absence of modules/programs should trigger unsupported errors
tools/testing/selftests/ftrace/ftracetest | 23 ++++++++++++++++++++++ .../ftrace/test.d/direct/ftrace-direct.tc | 2 +- .../ftrace/test.d/direct/kprobe-direct.tc | 2 +- .../selftests/ftrace/test.d/event/trace_printk.tc | 2 +- .../ftrace/test.d/ftrace/func_mod_trace.tc | 2 +- .../ftrace/test.d/kprobe/kprobe_module.tc | 2 +- .../selftests/ftrace/test.d/selftest/bashisms.tc | 2 +- 7 files changed, 29 insertions(+), 6 deletions(-)
wakeup_rt.tc and wakeup.tc tests in tracers/ subdirectory fail due to the chrt command returning:
chrt: failed to set pid 0's policy: Operation not permitted.
To work around this, temporarily disable grout RT scheduling during ftracetest execution. Restore original value on test run completion. With these changes in place, both tests consistently pass.
Fixes: c575dea2c1a5 ("selftests/ftrace: Add wakeup_rt tracer testcase") Fixes: c1edd060b413 ("selftests/ftrace: Add wakeup tracer testcase") Signed-off-by: Alan Maguire alan.maguire@oracle.com --- tools/testing/selftests/ftrace/ftracetest | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+)
diff --git a/tools/testing/selftests/ftrace/ftracetest b/tools/testing/selftests/ftrace/ftracetest index 063ecb2..3207bbf 100755 --- a/tools/testing/selftests/ftrace/ftracetest +++ b/tools/testing/selftests/ftrace/ftracetest @@ -29,8 +29,26 @@ err_ret=1 # kselftest skip code is 4 err_skip=4
+# cgroup RT scheduling prevents chrt commands from succeeding, which +# induces failures in test wakeup tests. Disable for the duration of +# the tests. +sched_rt_runtime=$(sysctl -n kernel.sched_rt_runtime_us) + +set_sysctl() { + sysctl -qw ${1}=${2} >/dev/null 2>&1 +} + +setup() { + set_sysctl kernel.sched_rt_runtime_us -1 +} + +cleanup() { + set_sysctl kernel.sched_rt_runtime_us $sched_rt_runtime +} + errexit() { # message echo "Error: $1" 1>&2 + cleanup exit $err_ret }
@@ -39,6 +57,8 @@ if [ `id -u` -ne 0 ]; then errexit "this must be run by root user" fi
+setup + # Utilities absdir() { # file_path (cd `dirname $1`; pwd) @@ -235,6 +255,7 @@ TOTAL_RESULT=0
INSTANCE= CASENO=0 + testcase() { # testfile CASENO=$((CASENO+1)) desc=`grep "^#[ \t]*description:" $1 | cut -f2 -d:` @@ -406,5 +427,7 @@ prlog "# of unsupported: " `echo $UNSUPPORTED_CASES | wc -w` prlog "# of xfailed: " `echo $XFAILED_CASES | wc -w` prlog "# of undefined(test bug): " `echo $UNDEFINED_CASES | wc -w`
+cleanup + # if no error, return 0 exit $TOTAL_RESULT
On Thu, 6 Feb 2020 15:09:19 +0000 Alan Maguire alan.maguire@oracle.com wrote:
wakeup_rt.tc and wakeup.tc tests in tracers/ subdirectory fail due to the chrt command returning:
chrt: failed to set pid 0's policy: Operation not permitted.
To work around this, temporarily disable grout RT scheduling during ftracetest execution. Restore original value on test run completion. With these changes in place, both tests consistently pass.
OK, this looks good to me.
Acked-by: Masami Hiramatsu mhiramat@kernel.org
Thanks!
Fixes: c575dea2c1a5 ("selftests/ftrace: Add wakeup_rt tracer testcase") Fixes: c1edd060b413 ("selftests/ftrace: Add wakeup tracer testcase") Signed-off-by: Alan Maguire alan.maguire@oracle.com
tools/testing/selftests/ftrace/ftracetest | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+)
diff --git a/tools/testing/selftests/ftrace/ftracetest b/tools/testing/selftests/ftrace/ftracetest index 063ecb2..3207bbf 100755 --- a/tools/testing/selftests/ftrace/ftracetest +++ b/tools/testing/selftests/ftrace/ftracetest @@ -29,8 +29,26 @@ err_ret=1 # kselftest skip code is 4 err_skip=4 +# cgroup RT scheduling prevents chrt commands from succeeding, which +# induces failures in test wakeup tests. Disable for the duration of +# the tests. +sched_rt_runtime=$(sysctl -n kernel.sched_rt_runtime_us)
OK, but can you
+set_sysctl() {
- sysctl -qw ${1}=${2} >/dev/null 2>&1
+}
+setup() {
- set_sysctl kernel.sched_rt_runtime_us -1
+}
+cleanup() {
- set_sysctl kernel.sched_rt_runtime_us $sched_rt_runtime
+}
errexit() { # message echo "Error: $1" 1>&2
- cleanup exit $err_ret
} @@ -39,6 +57,8 @@ if [ `id -u` -ne 0 ]; then errexit "this must be run by root user" fi +setup
# Utilities absdir() { # file_path (cd `dirname $1`; pwd) @@ -235,6 +255,7 @@ TOTAL_RESULT=0 INSTANCE= CASENO=0
testcase() { # testfile CASENO=$((CASENO+1)) desc=`grep "^#[ \t]*description:" $1 | cut -f2 -d:` @@ -406,5 +427,7 @@ prlog "# of unsupported: " `echo $UNSUPPORTED_CASES | wc -w` prlog "# of xfailed: " `echo $XFAILED_CASES | wc -w` prlog "# of undefined(test bug): " `echo $UNDEFINED_CASES | wc -w` +cleanup
# if no error, return 0 exit $TOTAL_RESULT -- 1.8.3.1
On Fri, 7 Feb 2020 15:14:56 +0900 Masami Hiramatsu mhiramat@kernel.org wrote:
On Thu, 6 Feb 2020 15:09:19 +0000 Alan Maguire alan.maguire@oracle.com wrote:
wakeup_rt.tc and wakeup.tc tests in tracers/ subdirectory fail due to the chrt command returning:
chrt: failed to set pid 0's policy: Operation not permitted.
To work around this, temporarily disable grout RT scheduling during ftracetest execution. Restore original value on test run completion. With these changes in place, both tests consistently pass.
OK, this looks good to me.
Acked-by: Masami Hiramatsu mhiramat@kernel.org
Thanks!
Fixes: c575dea2c1a5 ("selftests/ftrace: Add wakeup_rt tracer testcase") Fixes: c1edd060b413 ("selftests/ftrace: Add wakeup tracer testcase") Signed-off-by: Alan Maguire alan.maguire@oracle.com
tools/testing/selftests/ftrace/ftracetest | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+)
diff --git a/tools/testing/selftests/ftrace/ftracetest b/tools/testing/selftests/ftrace/ftracetest index 063ecb2..3207bbf 100755 --- a/tools/testing/selftests/ftrace/ftracetest +++ b/tools/testing/selftests/ftrace/ftracetest @@ -29,8 +29,26 @@ err_ret=1 # kselftest skip code is 4 err_skip=4 +# cgroup RT scheduling prevents chrt commands from succeeding, which +# induces failures in test wakeup tests. Disable for the duration of +# the tests. +sched_rt_runtime=$(sysctl -n kernel.sched_rt_runtime_us)
OK, but can you
??
Masami?
-- Steve
+set_sysctl() {
- sysctl -qw ${1}=${2} >/dev/null 2>&1
+}
On Mon, 10 Feb 2020 17:18:01 -0500 Steven Rostedt rostedt@goodmis.org wrote:
On Fri, 7 Feb 2020 15:14:56 +0900 Masami Hiramatsu mhiramat@kernel.org wrote:
On Thu, 6 Feb 2020 15:09:19 +0000 Alan Maguire alan.maguire@oracle.com wrote:
wakeup_rt.tc and wakeup.tc tests in tracers/ subdirectory fail due to the chrt command returning:
chrt: failed to set pid 0's policy: Operation not permitted.
To work around this, temporarily disable grout RT scheduling during ftracetest execution. Restore original value on test run completion. With these changes in place, both tests consistently pass.
OK, this looks good to me.
Acked-by: Masami Hiramatsu mhiramat@kernel.org
Thanks!
Fixes: c575dea2c1a5 ("selftests/ftrace: Add wakeup_rt tracer testcase") Fixes: c1edd060b413 ("selftests/ftrace: Add wakeup tracer testcase") Signed-off-by: Alan Maguire alan.maguire@oracle.com
tools/testing/selftests/ftrace/ftracetest | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+)
diff --git a/tools/testing/selftests/ftrace/ftracetest b/tools/testing/selftests/ftrace/ftracetest index 063ecb2..3207bbf 100755 --- a/tools/testing/selftests/ftrace/ftracetest +++ b/tools/testing/selftests/ftrace/ftracetest @@ -29,8 +29,26 @@ err_ret=1 # kselftest skip code is 4 err_skip=4 +# cgroup RT scheduling prevents chrt commands from succeeding, which +# induces failures in test wakeup tests. Disable for the duration of +# the tests. +sched_rt_runtime=$(sysctl -n kernel.sched_rt_runtime_us)
OK, but can you
??
Masami?
Oops, I missed to fill the comment. I meant
"but can you consider to use /proc/sys directly instead of sysctl command, because other test cases uses /proc/sys (ftrace/fgraph-filter-stack.tc and ftrace/func_stack_tracer.tc)?"
Thank you,
-- Steve
+set_sysctl() {
- sysctl -qw ${1}=${2} >/dev/null 2>&1
+}
In a number of cases, the ftrace tests check for the presence of ftrace testing-related modules (ftrace-direct, trace-printk) and programs (checkbashisms), returning exit_unresolved if these are not found. The problem is, exit_unresolved causes execution of ftracetest to return an error, when really our tests are failing due to not having the requisite kernel configuration/tools present, which is I think more of an unsupported error condition. With these fixed, we see no unresolved test cases and ftracetest returns success ("ok" when run via kselftest).
Fixes: 646f01ccdd59 ("ftrace/selftest: Add tests to test register_ftrace_direct()") Fixes: 4d23e9b4fd2e ("selftests/ftrace: Add trace_printk sample module test") Fixes: 7bc026d6c032 ("selftests/ftrace: Add function filter on module testcase") Fixes: ff431b1390cb ("selftests/ftrace: Add a test to probe module functions") Fixes: 4a075bd4e13f ("selftests/ftrace: Add checkbashisms meta-testcase") Signed-off-by: Alan Maguire alan.maguire@oracle.com --- tools/testing/selftests/ftrace/test.d/direct/ftrace-direct.tc | 2 +- tools/testing/selftests/ftrace/test.d/direct/kprobe-direct.tc | 2 +- tools/testing/selftests/ftrace/test.d/event/trace_printk.tc | 2 +- tools/testing/selftests/ftrace/test.d/ftrace/func_mod_trace.tc | 2 +- tools/testing/selftests/ftrace/test.d/kprobe/kprobe_module.tc | 2 +- tools/testing/selftests/ftrace/test.d/selftest/bashisms.tc | 2 +- 6 files changed, 6 insertions(+), 6 deletions(-)
diff --git a/tools/testing/selftests/ftrace/test.d/direct/ftrace-direct.tc b/tools/testing/selftests/ftrace/test.d/direct/ftrace-direct.tc index d75a869..3d6189e 100644 --- a/tools/testing/selftests/ftrace/test.d/direct/ftrace-direct.tc +++ b/tools/testing/selftests/ftrace/test.d/direct/ftrace-direct.tc @@ -5,7 +5,7 @@ rmmod ftrace-direct ||: if ! modprobe ftrace-direct ; then echo "No ftrace-direct sample module - please make CONFIG_SAMPLE_FTRACE_DIRECT=m" - exit_unresolved; + exit_unsupported; fi
echo "Let the module run a little" diff --git a/tools/testing/selftests/ftrace/test.d/direct/kprobe-direct.tc b/tools/testing/selftests/ftrace/test.d/direct/kprobe-direct.tc index 801ecb6..3d0e3ca 100644 --- a/tools/testing/selftests/ftrace/test.d/direct/kprobe-direct.tc +++ b/tools/testing/selftests/ftrace/test.d/direct/kprobe-direct.tc @@ -5,7 +5,7 @@ rmmod ftrace-direct ||: if ! modprobe ftrace-direct ; then echo "No ftrace-direct sample module - please build with CONFIG_SAMPLE_FTRACE_DIRECT=m" - exit_unresolved; + exit_unsupported; fi
if [ ! -f kprobe_events ]; then diff --git a/tools/testing/selftests/ftrace/test.d/event/trace_printk.tc b/tools/testing/selftests/ftrace/test.d/event/trace_printk.tc index b02550b..dd8b10d 100644 --- a/tools/testing/selftests/ftrace/test.d/event/trace_printk.tc +++ b/tools/testing/selftests/ftrace/test.d/event/trace_printk.tc @@ -5,7 +5,7 @@ rmmod trace-printk ||: if ! modprobe trace-printk ; then echo "No trace-printk sample module - please make CONFIG_SAMPLE_TRACE_PRINTK=m" - exit_unresolved; + exit_unsupported; fi
echo "Waiting for irq work" diff --git a/tools/testing/selftests/ftrace/test.d/ftrace/func_mod_trace.tc b/tools/testing/selftests/ftrace/test.d/ftrace/func_mod_trace.tc index 9330c87..fc22ac0 100644 --- a/tools/testing/selftests/ftrace/test.d/ftrace/func_mod_trace.tc +++ b/tools/testing/selftests/ftrace/test.d/ftrace/func_mod_trace.tc @@ -13,7 +13,7 @@ echo '*:mod:trace_printk' > set_ftrace_filter if ! modprobe trace-printk ; then echo "No trace-printk sample module - please make CONFIG_SAMPLE_TRACE_PRINTK= m" - exit_unresolved; + exit_unsupported; fi
: "Wildcard should be resolved after loading module" diff --git a/tools/testing/selftests/ftrace/test.d/kprobe/kprobe_module.tc b/tools/testing/selftests/ftrace/test.d/kprobe/kprobe_module.tc index d861bd7..4e07c69 100644 --- a/tools/testing/selftests/ftrace/test.d/kprobe/kprobe_module.tc +++ b/tools/testing/selftests/ftrace/test.d/kprobe/kprobe_module.tc @@ -8,7 +8,7 @@ rmmod trace-printk ||: if ! modprobe trace-printk ; then echo "No trace-printk sample module - please make CONFIG_SAMPLE_TRACE_PRINTK= m" - exit_unresolved; + exit_unsupported; fi
MOD=trace_printk diff --git a/tools/testing/selftests/ftrace/test.d/selftest/bashisms.tc b/tools/testing/selftests/ftrace/test.d/selftest/bashisms.tc index 1b081e9..1b339bd 100644 --- a/tools/testing/selftests/ftrace/test.d/selftest/bashisms.tc +++ b/tools/testing/selftests/ftrace/test.d/selftest/bashisms.tc @@ -9,7 +9,7 @@ fi
if ! which checkbashisms > /dev/null 2>&1 ; then echo "No checkbashisms found. skipped." - exit_unresolved + exit_unsupported fi
checkbashisms $FTRACETEST_ROOT/ftracetest
Hi Alan,
On Thu, 6 Feb 2020 15:09:20 +0000 Alan Maguire alan.maguire@oracle.com wrote:
In a number of cases, the ftrace tests check for the presence of ftrace testing-related modules (ftrace-direct, trace-printk) and programs (checkbashisms), returning exit_unresolved if these are not found. The problem is, exit_unresolved causes execution of ftracetest to return an error, when really our tests are failing due to not having the requisite kernel configuration/tools present, which is I think more of an unsupported error condition. With these fixed, we see no unresolved test cases and ftracetest returns success ("ok" when run via kselftest).
If your problem is to pass the test even if you don't test the feature, please change the ftracetest itself instead of replacing unresolved with unsupported. Those notice different situation.
unresolved - Testcase can not find some tools or helper drivers which are required for this testcase.
unsupported - Kernel does not have tested feature because of the version or the configuration.
Obviously the unresolved is a test environment issue. No test-module doesn't mean no feature to be tested. Could you tell me the reason why you can't install those required tools and modules on the test environment?
Thank you,
Fixes: 646f01ccdd59 ("ftrace/selftest: Add tests to test register_ftrace_direct()") Fixes: 4d23e9b4fd2e ("selftests/ftrace: Add trace_printk sample module test") Fixes: 7bc026d6c032 ("selftests/ftrace: Add function filter on module testcase") Fixes: ff431b1390cb ("selftests/ftrace: Add a test to probe module functions") Fixes: 4a075bd4e13f ("selftests/ftrace: Add checkbashisms meta-testcase") Signed-off-by: Alan Maguire alan.maguire@oracle.com
tools/testing/selftests/ftrace/test.d/direct/ftrace-direct.tc | 2 +- tools/testing/selftests/ftrace/test.d/direct/kprobe-direct.tc | 2 +- tools/testing/selftests/ftrace/test.d/event/trace_printk.tc | 2 +- tools/testing/selftests/ftrace/test.d/ftrace/func_mod_trace.tc | 2 +- tools/testing/selftests/ftrace/test.d/kprobe/kprobe_module.tc | 2 +- tools/testing/selftests/ftrace/test.d/selftest/bashisms.tc | 2 +- 6 files changed, 6 insertions(+), 6 deletions(-)
diff --git a/tools/testing/selftests/ftrace/test.d/direct/ftrace-direct.tc b/tools/testing/selftests/ftrace/test.d/direct/ftrace-direct.tc index d75a869..3d6189e 100644 --- a/tools/testing/selftests/ftrace/test.d/direct/ftrace-direct.tc +++ b/tools/testing/selftests/ftrace/test.d/direct/ftrace-direct.tc @@ -5,7 +5,7 @@ rmmod ftrace-direct ||: if ! modprobe ftrace-direct ; then echo "No ftrace-direct sample module - please make CONFIG_SAMPLE_FTRACE_DIRECT=m"
- exit_unresolved;
- exit_unsupported;
fi echo "Let the module run a little" diff --git a/tools/testing/selftests/ftrace/test.d/direct/kprobe-direct.tc b/tools/testing/selftests/ftrace/test.d/direct/kprobe-direct.tc index 801ecb6..3d0e3ca 100644 --- a/tools/testing/selftests/ftrace/test.d/direct/kprobe-direct.tc +++ b/tools/testing/selftests/ftrace/test.d/direct/kprobe-direct.tc @@ -5,7 +5,7 @@ rmmod ftrace-direct ||: if ! modprobe ftrace-direct ; then echo "No ftrace-direct sample module - please build with CONFIG_SAMPLE_FTRACE_DIRECT=m"
- exit_unresolved;
- exit_unsupported;
fi if [ ! -f kprobe_events ]; then diff --git a/tools/testing/selftests/ftrace/test.d/event/trace_printk.tc b/tools/testing/selftests/ftrace/test.d/event/trace_printk.tc index b02550b..dd8b10d 100644 --- a/tools/testing/selftests/ftrace/test.d/event/trace_printk.tc +++ b/tools/testing/selftests/ftrace/test.d/event/trace_printk.tc @@ -5,7 +5,7 @@ rmmod trace-printk ||: if ! modprobe trace-printk ; then echo "No trace-printk sample module - please make CONFIG_SAMPLE_TRACE_PRINTK=m"
- exit_unresolved;
- exit_unsupported;
fi echo "Waiting for irq work" diff --git a/tools/testing/selftests/ftrace/test.d/ftrace/func_mod_trace.tc b/tools/testing/selftests/ftrace/test.d/ftrace/func_mod_trace.tc index 9330c87..fc22ac0 100644 --- a/tools/testing/selftests/ftrace/test.d/ftrace/func_mod_trace.tc +++ b/tools/testing/selftests/ftrace/test.d/ftrace/func_mod_trace.tc @@ -13,7 +13,7 @@ echo '*:mod:trace_printk' > set_ftrace_filter if ! modprobe trace-printk ; then echo "No trace-printk sample module - please make CONFIG_SAMPLE_TRACE_PRINTK= m"
- exit_unresolved;
- exit_unsupported;
fi : "Wildcard should be resolved after loading module" diff --git a/tools/testing/selftests/ftrace/test.d/kprobe/kprobe_module.tc b/tools/testing/selftests/ftrace/test.d/kprobe/kprobe_module.tc index d861bd7..4e07c69 100644 --- a/tools/testing/selftests/ftrace/test.d/kprobe/kprobe_module.tc +++ b/tools/testing/selftests/ftrace/test.d/kprobe/kprobe_module.tc @@ -8,7 +8,7 @@ rmmod trace-printk ||: if ! modprobe trace-printk ; then echo "No trace-printk sample module - please make CONFIG_SAMPLE_TRACE_PRINTK= m"
- exit_unresolved;
- exit_unsupported;
fi MOD=trace_printk diff --git a/tools/testing/selftests/ftrace/test.d/selftest/bashisms.tc b/tools/testing/selftests/ftrace/test.d/selftest/bashisms.tc index 1b081e9..1b339bd 100644 --- a/tools/testing/selftests/ftrace/test.d/selftest/bashisms.tc +++ b/tools/testing/selftests/ftrace/test.d/selftest/bashisms.tc @@ -9,7 +9,7 @@ fi if ! which checkbashisms > /dev/null 2>&1 ; then echo "No checkbashisms found. skipped."
- exit_unresolved
- exit_unsupported
fi checkbashisms $FTRACETEST_ROOT/ftracetest -- 1.8.3.1
On Fri, 7 Feb 2020, Masami Hiramatsu wrote:
Hi Alan,
On Thu, 6 Feb 2020 15:09:20 +0000 Alan Maguire alan.maguire@oracle.com wrote:
In a number of cases, the ftrace tests check for the presence of ftrace testing-related modules (ftrace-direct, trace-printk) and programs (checkbashisms), returning exit_unresolved if these are not found. The problem is, exit_unresolved causes execution of ftracetest to return an error, when really our tests are failing due to not having the requisite kernel configuration/tools present, which is I think more of an unsupported error condition. With these fixed, we see no unresolved test cases and ftracetest returns success ("ok" when run via kselftest).
If your problem is to pass the test even if you don't test the feature, please change the ftracetest itself instead of replacing unresolved with unsupported. Those notice different situation.
unresolved - Testcase can not find some tools or helper drivers which are required for this testcase.
unsupported - Kernel does not have tested feature because of the version or the configuration.
Obviously the unresolved is a test environment issue. No test-module doesn't mean no feature to be tested. Could you tell me the reason why you can't install those required tools and modules on the test environment?
Sure! In my case, I'm testing a distro production kernel, where I can't control the CONFIG variable settings. In this case, ideally I'd like the tests to return success if no problems with ftrace were detected, even if some of the tests could not be run due to missing modules and programs. As you suggest above (unless I'm misunderstanding), this could be accomplished by modifying ftracetest itself. Would doing something like what is done for UNSUPPORTED_RESULT (defaults to 0, but can be set to 1 via --fail-unsupported, such that ftracetest returns 1 if we encounter unsupported results) make sense for the unresolved case too?
Thanks!
Alan
Thank you,
Fixes: 646f01ccdd59 ("ftrace/selftest: Add tests to test register_ftrace_direct()") Fixes: 4d23e9b4fd2e ("selftests/ftrace: Add trace_printk sample module test") Fixes: 7bc026d6c032 ("selftests/ftrace: Add function filter on module testcase") Fixes: ff431b1390cb ("selftests/ftrace: Add a test to probe module functions") Fixes: 4a075bd4e13f ("selftests/ftrace: Add checkbashisms meta-testcase") Signed-off-by: Alan Maguire alan.maguire@oracle.com
tools/testing/selftests/ftrace/test.d/direct/ftrace-direct.tc | 2 +- tools/testing/selftests/ftrace/test.d/direct/kprobe-direct.tc | 2 +- tools/testing/selftests/ftrace/test.d/event/trace_printk.tc | 2 +- tools/testing/selftests/ftrace/test.d/ftrace/func_mod_trace.tc | 2 +- tools/testing/selftests/ftrace/test.d/kprobe/kprobe_module.tc | 2 +- tools/testing/selftests/ftrace/test.d/selftest/bashisms.tc | 2 +- 6 files changed, 6 insertions(+), 6 deletions(-)
diff --git a/tools/testing/selftests/ftrace/test.d/direct/ftrace-direct.tc b/tools/testing/selftests/ftrace/test.d/direct/ftrace-direct.tc index d75a869..3d6189e 100644 --- a/tools/testing/selftests/ftrace/test.d/direct/ftrace-direct.tc +++ b/tools/testing/selftests/ftrace/test.d/direct/ftrace-direct.tc @@ -5,7 +5,7 @@ rmmod ftrace-direct ||: if ! modprobe ftrace-direct ; then echo "No ftrace-direct sample module - please make CONFIG_SAMPLE_FTRACE_DIRECT=m"
- exit_unresolved;
- exit_unsupported;
fi echo "Let the module run a little" diff --git a/tools/testing/selftests/ftrace/test.d/direct/kprobe-direct.tc b/tools/testing/selftests/ftrace/test.d/direct/kprobe-direct.tc index 801ecb6..3d0e3ca 100644 --- a/tools/testing/selftests/ftrace/test.d/direct/kprobe-direct.tc +++ b/tools/testing/selftests/ftrace/test.d/direct/kprobe-direct.tc @@ -5,7 +5,7 @@ rmmod ftrace-direct ||: if ! modprobe ftrace-direct ; then echo "No ftrace-direct sample module - please build with CONFIG_SAMPLE_FTRACE_DIRECT=m"
- exit_unresolved;
- exit_unsupported;
fi if [ ! -f kprobe_events ]; then diff --git a/tools/testing/selftests/ftrace/test.d/event/trace_printk.tc b/tools/testing/selftests/ftrace/test.d/event/trace_printk.tc index b02550b..dd8b10d 100644 --- a/tools/testing/selftests/ftrace/test.d/event/trace_printk.tc +++ b/tools/testing/selftests/ftrace/test.d/event/trace_printk.tc @@ -5,7 +5,7 @@ rmmod trace-printk ||: if ! modprobe trace-printk ; then echo "No trace-printk sample module - please make CONFIG_SAMPLE_TRACE_PRINTK=m"
- exit_unresolved;
- exit_unsupported;
fi echo "Waiting for irq work" diff --git a/tools/testing/selftests/ftrace/test.d/ftrace/func_mod_trace.tc b/tools/testing/selftests/ftrace/test.d/ftrace/func_mod_trace.tc index 9330c87..fc22ac0 100644 --- a/tools/testing/selftests/ftrace/test.d/ftrace/func_mod_trace.tc +++ b/tools/testing/selftests/ftrace/test.d/ftrace/func_mod_trace.tc @@ -13,7 +13,7 @@ echo '*:mod:trace_printk' > set_ftrace_filter if ! modprobe trace-printk ; then echo "No trace-printk sample module - please make CONFIG_SAMPLE_TRACE_PRINTK= m"
- exit_unresolved;
- exit_unsupported;
fi : "Wildcard should be resolved after loading module" diff --git a/tools/testing/selftests/ftrace/test.d/kprobe/kprobe_module.tc b/tools/testing/selftests/ftrace/test.d/kprobe/kprobe_module.tc index d861bd7..4e07c69 100644 --- a/tools/testing/selftests/ftrace/test.d/kprobe/kprobe_module.tc +++ b/tools/testing/selftests/ftrace/test.d/kprobe/kprobe_module.tc @@ -8,7 +8,7 @@ rmmod trace-printk ||: if ! modprobe trace-printk ; then echo "No trace-printk sample module - please make CONFIG_SAMPLE_TRACE_PRINTK= m"
- exit_unresolved;
- exit_unsupported;
fi MOD=trace_printk diff --git a/tools/testing/selftests/ftrace/test.d/selftest/bashisms.tc b/tools/testing/selftests/ftrace/test.d/selftest/bashisms.tc index 1b081e9..1b339bd 100644 --- a/tools/testing/selftests/ftrace/test.d/selftest/bashisms.tc +++ b/tools/testing/selftests/ftrace/test.d/selftest/bashisms.tc @@ -9,7 +9,7 @@ fi if ! which checkbashisms > /dev/null 2>&1 ; then echo "No checkbashisms found. skipped."
- exit_unresolved
- exit_unsupported
fi checkbashisms $FTRACETEST_ROOT/ftracetest -- 1.8.3.1
-- Masami Hiramatsu mhiramat@kernel.org
On Fri, 7 Feb 2020 08:27:13 +0000 (GMT) Alan Maguire alan.maguire@oracle.com wrote:
On Fri, 7 Feb 2020, Masami Hiramatsu wrote:
Hi Alan,
On Thu, 6 Feb 2020 15:09:20 +0000 Alan Maguire alan.maguire@oracle.com wrote:
In a number of cases, the ftrace tests check for the presence of ftrace testing-related modules (ftrace-direct, trace-printk) and programs (checkbashisms), returning exit_unresolved if these are not found. The problem is, exit_unresolved causes execution of ftracetest to return an error, when really our tests are failing due to not having the requisite kernel configuration/tools present, which is I think more of an unsupported error condition. With these fixed, we see no unresolved test cases and ftracetest returns success ("ok" when run via kselftest).
If your problem is to pass the test even if you don't test the feature, please change the ftracetest itself instead of replacing unresolved with unsupported. Those notice different situation.
unresolved - Testcase can not find some tools or helper drivers which are required for this testcase.
unsupported - Kernel does not have tested feature because of the version or the configuration.
Obviously the unresolved is a test environment issue. No test-module doesn't mean no feature to be tested. Could you tell me the reason why you can't install those required tools and modules on the test environment?
Sure! In my case, I'm testing a distro production kernel, where I can't control the CONFIG variable settings. In this case, ideally I'd like the tests to return success if no problems with ftrace were detected, even if some of the tests could not be run due to missing modules and programs.
OK, for modules, we need to find another way to solve the issue. But how about checkbashisms? you can download and build it.
https://sources.debian.org/src/devscripts/2.20.2/
For the modules, you might be able to build it from kernel source code as out-of-tree modules, or not? (hmm, how do the other test handle it...?)
As you suggest above (unless I'm misunderstanding), this could be accomplished by modifying ftracetest itself. Would doing something like what is done for UNSUPPORTED_RESULT (defaults to 0, but can be set to 1 via --fail-unsupported, such that ftracetest returns 1 if we encounter unsupported results) make sense for the unresolved case too?
Yes, but at first could you try to setup your testing environment? If you are officially testing your distro kernel, the distro might need to be tested with full-set of testcases.
If not (like you are testing kernel for fun :)), you can just make your custom set of testcases. (just remove those test files)
Thank you,
On Fri, 7 Feb 2020, Masami Hiramatsu wrote:
On Fri, 7 Feb 2020 08:27:13 +0000 (GMT) Alan Maguire alan.maguire@oracle.com wrote:
On Fri, 7 Feb 2020, Masami Hiramatsu wrote:
Hi Alan,
On Thu, 6 Feb 2020 15:09:20 +0000 Alan Maguire alan.maguire@oracle.com wrote:
In a number of cases, the ftrace tests check for the presence of ftrace testing-related modules (ftrace-direct, trace-printk) and programs (checkbashisms), returning exit_unresolved if these are not found. The problem is, exit_unresolved causes execution of ftracetest to return an error, when really our tests are failing due to not having the requisite kernel configuration/tools present, which is I think more of an unsupported error condition. With these fixed, we see no unresolved test cases and ftracetest returns success ("ok" when run via kselftest).
If your problem is to pass the test even if you don't test the feature, please change the ftracetest itself instead of replacing unresolved with unsupported. Those notice different situation.
unresolved - Testcase can not find some tools or helper drivers which are required for this testcase.
unsupported - Kernel does not have tested feature because of the version or the configuration.
Obviously the unresolved is a test environment issue. No test-module doesn't mean no feature to be tested. Could you tell me the reason why you can't install those required tools and modules on the test environment?
Sure! In my case, I'm testing a distro production kernel, where I can't control the CONFIG variable settings. In this case, ideally I'd like the tests to return success if no problems with ftrace were detected, even if some of the tests could not be run due to missing modules and programs.
OK, for modules, we need to find another way to solve the issue. But how about checkbashisms? you can download and build it.
Yep, I should have said that this one isn't a big issue, it's also packaged in some distros in rpmdevtools.
For the modules, you might be able to build it from kernel source code as out-of-tree modules, or not? (hmm, how do the other test handle it...?)
Ideally (from my perspective at least) the tests would build the modules if needed, but I think the constraints on kselftests are that sometimes the kselftests are packaged and installed without the rest of the source tree, so I _think_ given that the module source is in other parts of the tree (that may not be present) we're probably stuck. It'd be great to have have a solution to this though, as I have a feeling there may be more and more cases like this with the growth of kunit.
Looks like the official way to do this sort of thing is described in Documentation/dev-tools/kselftest.rst:
# Assumes you have booted a fresh build of this kernel tree cd /path/to/linux/tree make kselftest-merge make modules sudo make modules_install make TARGETS=lib kselftest
It seems to merge existing config with the config files in the kselftest dirs and rebuild modules; I'll give this a try.
I was hoping for a lightweight version of the above which just builds the modules needed without rebuilding everything.
As you suggest above (unless I'm misunderstanding), this could be accomplished by modifying ftracetest itself. Would doing something like what is done for UNSUPPORTED_RESULT (defaults to 0, but can be set to 1 via --fail-unsupported, such that ftracetest returns 1 if we encounter unsupported results) make sense for the unresolved case too?
Yes, but at first could you try to setup your testing environment? If you are officially testing your distro kernel, the distro might need to be tested with full-set of testcases.
Absolutely! I'll see if I can convince the modules to build using the above scheme.
Thanks for the review and advice!
Alan
If not (like you are testing kernel for fun :)), you can just make your custom set of testcases. (just remove those test files)
Thank you,
-- Masami Hiramatsu mhiramat@kernel.org
linux-kselftest-mirror@lists.linaro.org
-
Alan Maguire -
Masami Hiramatsu -
Steven Rostedt