On Sat, 2025-01-11 at 15:42 -0300, Filipe Xavier wrote:

This new test makes sure that ftrace can trace a function that was introduced by a livepatch.

Signed-off-by: Filipe Xavier felipeaggger@gmail.com

Thanks for the new test Filipe!

I have some nits below, but these don't need to be addressed for the test to be merged. Either way,

Reviewed-by: Marcos Paulo de Souza mpdesouza@suse.com Tested-by: Marcos Paulo de Souza mpdesouza@suse.com

---

Changes in v2:

• functions.sh: added reset tracing on push and pop_config.
• test-ftrace.sh: enabled tracing_on before test init.
• nitpick: added double quotations on filenames and fixed some

wording.

• Link to v1:

https://lore.kernel.org/r/20250102-ftrace-selftest-livepatch-v1-1-84880baefc...

tools/testing/selftests/livepatch/functions.sh   | 14 ++++++++++  tools/testing/selftests/livepatch/test-ftrace.sh | 33 ++++++++++++++++++++++++  2 files changed, 47 insertions(+)

diff --git a/tools/testing/selftests/livepatch/functions.sh b/tools/testing/selftests/livepatch/functions.sh index e5d06fb402335d85959bafe099087effc6ddce12..e6c13514002dae5f8d7461f90b8 241ab43024ea4 100644 --- a/tools/testing/selftests/livepatch/functions.sh +++ b/tools/testing/selftests/livepatch/functions.sh @@ -62,6 +62,9 @@ function push_config() {   awk -F'[: ]' '{print "file " $1 " line " $2 " " $4}')   FTRACE_ENABLED=$(sysctl --values kernel.ftrace_enabled)   KPROBE_ENABLED=$(cat "$SYSFS_KPROBES_DIR/enabled")

• TRACING_ON=$(cat "$SYSFS_DEBUG_DIR/tracing/tracing_on")
• CURRENT_TRACER=$(cat

"$SYSFS_DEBUG_DIR/tracing/current_tracer")

• FTRACE_FILTER=$(cat

"$SYSFS_DEBUG_DIR/tracing/set_ftrace_filter")  }    function pop_config() { @@ -74,6 +77,17 @@ function pop_config() {   if [[ -n "$KPROBE_ENABLED" ]]; then   echo "$KPROBE_ENABLED" > "$SYSFS_KPROBES_DIR/enabled"   fi

• if [[ -n "$TRACING_ON" ]]; then

• echo "$TRACING_ON" >
  

"$SYSFS_DEBUG_DIR/tracing/tracing_on"

• fi
• if [[ -n "$CURRENT_TRACER" ]]; then

• echo "$CURRENT_TRACER" >
  

"$SYSFS_DEBUG_DIR/tracing/current_tracer"

• fi
• if [[ "$FTRACE_FILTER" == *"#"* ]]; then

• echo > "$SYSFS_DEBUG_DIR/tracing/set_ftrace_filter"
  

• elif [[ -n "$FTRACE_FILTER" ]]; then

• echo "$FTRACE_FILTER" >
  

"$SYSFS_DEBUG_DIR/tracing/set_ftrace_filter"

• fi

}

I believe that this could be a separate patch, since this is new functionality that's being added to functions.sh, and not exactly related to the new test.

function set_dynamic_debug() { diff --git a/tools/testing/selftests/livepatch/test-ftrace.sh b/tools/testing/selftests/livepatch/test-ftrace.sh index fe14f248913acbec46fb6c0fec38a2fc84209d39..66af5d726c52e48e5177804e182 b4ff31784d5ac 100755 --- a/tools/testing/selftests/livepatch/test-ftrace.sh +++ b/tools/testing/selftests/livepatch/test-ftrace.sh @@ -61,4 +61,37 @@ livepatch: '$MOD_LIVEPATCH': unpatching complete  % rmmod $MOD_LIVEPATCH"     +# - verify livepatch can load +# - check if traces have a patched function +# - unload livepatch and reset trace

+start_test "trace livepatched function and check that the live patch remains in effect"

+TRACE_FILE="$SYSFS_DEBUG_DIR/tracing/trace" +FUNCTION_NAME="livepatch_cmdline_proc_show"

+load_lp $MOD_LIVEPATCH

+echo 1 > "$SYSFS_DEBUG_DIR/tracing/tracing_on" +echo $FUNCTION_NAME > "$SYSFS_DEBUG_DIR/tracing/set_ftrace_filter" +echo "function" > "$SYSFS_DEBUG_DIR/tracing/current_tracer" +echo "" > "$TRACE_FILE"

+if [[ "$(cat /proc/cmdline)" != "$MOD_LIVEPATCH: this has been live patched" ]] ; then

• echo -e "FAIL\n\n"
• die "livepatch kselftest(s) failed"

+fi

+grep -q $FUNCTION_NAME "$TRACE_FILE" +FOUND=$?

+disable_lp $MOD_LIVEPATCH +unload_lp $MOD_LIVEPATCH

+if [ "$FOUND" -eq 1 ]; then

• echo -e "FAIL\n\n"
• die "livepatch kselftest(s) failed"

+fi

exit 0

The test works, and that's very cool. But when running locally, I find the if we miss check_result call it doesn't add a newline after the "ok":

... # timeout set to 0 # selftests: livepatch: test-ftrace.sh # TEST: livepatch interaction with ftrace_enabled sysctl ... ok # TEST: trace livepatched function and check that the live patch remains in effect ... ok 5 selftests: livepatch: test-ftrace.sh # timeout set to 0 # selftests: livepatch: test-sysfs.sh ...

If the check_result below is added the output if sane again:

... # selftests: livepatch: test-ftrace.sh # TEST: livepatch interaction with ftrace_enabled sysctl ... ok # TEST: trace livepatched function and check that the live patch remains in effect ... ok ok 5 selftests: livepatch: test-ftrace.sh ...

I checked and this would be the only one test without using check_result, so maybe we should add this either way? I'm not sure what you guys think about it.

diff --git a/tools/testing/selftests/livepatch/test-ftrace.sh b/tools/testing/selftests/livepatch/test-ftrace.sh index 66af5d726c52..135c0fb17a98 100755 --- a/tools/testing/selftests/livepatch/test-ftrace.sh +++ b/tools/testing/selftests/livepatch/test-ftrace.sh @@ -93,5 +93,18 @@ if [ "$FOUND" -eq 1 ]; then die "livepatch kselftest(s) failed" fi

+check_result "% insmod test_modules/$MOD_LIVEPATCH.ko +livepatch: enabling patch '$MOD_LIVEPATCH' +livepatch: '$MOD_LIVEPATCH': initializing patching transition +livepatch: '$MOD_LIVEPATCH': starting patching transition +livepatch: '$MOD_LIVEPATCH': completing patching transition +livepatch: '$MOD_LIVEPATCH': patching complete +% echo 0 > $SYSFS_KLP_DIR/$MOD_LIVEPATCH/enabled +livepatch: '$MOD_LIVEPATCH': initializing unpatching transition +livepatch: '$MOD_LIVEPATCH': starting unpatching transition +livepatch: '$MOD_LIVEPATCH': completing unpatching transition +livepatch: '$MOD_LIVEPATCH': unpatching complete +% rmmod $MOD_LIVEPATCH" +

exit 0

---

base-commit: fc033cf25e612e840e545f8d5ad2edd6ba613ed5 change-id: 20250101-ftrace-selftest-livepatch-161fb77dbed8

Best regards,

Hi Filipe,

first, thanks for the test. It is good. I just suggest a lot of clean up ;-)

On Sat 2025-01-11 15:42:27, Filipe Xavier wrote:

This new test makes sure that ftrace can trace a function that was introduced by a livepatch.

---

Changes in v2:

• functions.sh: added reset tracing on push and pop_config.
• test-ftrace.sh: enabled tracing_on before test init.
• nitpick: added double quotations on filenames and fixed some wording.
• Link to v1: https://lore.kernel.org/r/20250102-ftrace-selftest-livepatch-v1-1-84880baefc...

---

tools/testing/selftests/livepatch/functions.sh | 14 ++++++++++ tools/testing/selftests/livepatch/test-ftrace.sh | 33 ++++++++++++++++++++++++ 2 files changed, 47 insertions(+)

diff --git a/tools/testing/selftests/livepatch/functions.sh b/tools/testing/selftests/livepatch/functions.sh index e5d06fb402335d85959bafe099087effc6ddce12..e6c13514002dae5f8d7461f90b8241ab43024ea4 100644 --- a/tools/testing/selftests/livepatch/functions.sh +++ b/tools/testing/selftests/livepatch/functions.sh @@ -62,6 +62,9 @@ function push_config() { awk -F'[: ]' '{print "file " $1 " line " $2 " " $4}') FTRACE_ENABLED=$(sysctl --values kernel.ftrace_enabled) KPROBE_ENABLED=$(cat "$SYSFS_KPROBES_DIR/enabled")

• TRACING_ON=$(cat "$SYSFS_DEBUG_DIR/tracing/tracing_on")

The $SYSFS_DEBUG_DIR/tracing directory is used many times. It would make sense to define:

SYSFS_TRACING_DIR="$SYSFS_DEBUG_DIR/tracing"

and use:

TRACING_ON=$(cat "$SYSFS_TRACING_DIR/tracing_on")

• CURRENT_TRACER=$(cat "$SYSFS_DEBUG_DIR/tracing/current_tracer")
• FTRACE_FILTER=$(cat "$SYSFS_DEBUG_DIR/tracing/set_ftrace_filter")

} function pop_config() { @@ -74,6 +77,17 @@ function pop_config() { if [[ -n "$KPROBE_ENABLED" ]]; then echo "$KPROBE_ENABLED" > "$SYSFS_KPROBES_DIR/enabled" fi

• if [[ -n "$TRACING_ON" ]]; then

• echo "$TRACING_ON" > "$SYSFS_DEBUG_DIR/tracing/tracing_on"
  

• fi
• if [[ -n "$CURRENT_TRACER" ]]; then

• echo "$CURRENT_TRACER" > "$SYSFS_DEBUG_DIR/tracing/current_tracer"
  

• fi
• if [[ "$FTRACE_FILTER" == *"#"* ]]; then

• echo > "$SYSFS_DEBUG_DIR/tracing/set_ftrace_filter"
  

My understanding is that this is handling the situation when the filter is:

#### all functions enabled ####

I was curious about the pattern *"#"*. The quotation looked pretty non-standard to me. It made me to read the bash documentation and the rules for extressions in [[ ]] looked quite complicated.

I wonder if we could make this more for "dummies". The following seems to work:

if [[ -n "$FTRACE_FILTER" ]]; then echo "$FTRACE_FILTER" | sed -e "/#### all functions enabled ####/d" > "$SYSFS_DEBUG_DIR/tracing/set_ftrace_filter" fi

Note that the string "#### all functions enabled ####" is a very stable thing. It has never been modified since it was introduced by the commit 0c75a3ed633419d75d ("ftrace: state that all functions are enabled in set_ftrace_filter") in Feb 2009.

The string is used also in ftrace selftests, see tools/testing/selftests/ftrace/test.d/ftrace/func_set_ftrace_file.tc

• elif [[ -n "$FTRACE_FILTER" ]]; then

• echo "$FTRACE_FILTER" > "$SYSFS_DEBUG_DIR/tracing/set_ftrace_filter"
  

• fi

} function set_dynamic_debug() { diff --git a/tools/testing/selftests/livepatch/test-ftrace.sh b/tools/testing/selftests/livepatch/test-ftrace.sh index fe14f248913acbec46fb6c0fec38a2fc84209d39..66af5d726c52e48e5177804e182b4ff31784d5ac 100755 --- a/tools/testing/selftests/livepatch/test-ftrace.sh +++ b/tools/testing/selftests/livepatch/test-ftrace.sh @@ -61,4 +61,37 @@ livepatch: '$MOD_LIVEPATCH': unpatching complete % rmmod $MOD_LIVEPATCH" +# - verify livepatch can load +# - check if traces have a patched function +# - unload livepatch and reset trace

+start_test "trace livepatched function and check that the live patch remains in effect"

+TRACE_FILE="$SYSFS_DEBUG_DIR/tracing/trace" +FUNCTION_NAME="livepatch_cmdline_proc_show"

+load_lp $MOD_LIVEPATCH

+echo 1 > "$SYSFS_DEBUG_DIR/tracing/tracing_on" +echo $FUNCTION_NAME > "$SYSFS_DEBUG_DIR/tracing/set_ftrace_filter"

The above mentioned ftrace selftest shows that the behavior gets more complicated when "traceoff" is used:

1. The "traceoff" entries are not cleared by the above command. 2. The "traceoff" entries might cause that the tracing will stop before we finish this test.

+echo "function" > "$SYSFS_DEBUG_DIR/tracing/current_tracer" +echo "" > "$TRACE_FILE"

I suggest to change the ordering and make sure that nothing is running before we start:

# Start with the clean up echo 0 > "$SYSFS_TRACING_DIR/tracing_on" echo '!*:traceoff' > $SYSFS_TRACING_DIR/set_ftrace_filter echo "" > "$SYSFS_TRACING_DIR/current_tracer" echo "" > $SYSFS_TRACING_DIR/trace # Setup and start our tracing echo "function" > "$SYSFS_TRACING_DIR/current_tracer" echo $FUNCTION_NAME > "$SYSFS_TRACING_DIR/set_ftrace_filter" echo 1 > "$SYSFS_TRACING_DIR/tracing_on"

I suggest to this into helper functions which might be used by more tests:

# cleanup_tracing() - stop and clean up function tracing function cleanup_tracing() { echo 0 > "$SYSFS_TRACING_DIR/tracing_on" echo '!*:traceoff' > $SYSFS_TRACING_DIR/set_ftrace_filter echo "" > "$SYSFS_TRACING_DIR/current_tracer" echo "" > $SYSFS_TRACING_DIR/trace }

# trace_function(function) - start tracing of a function # function - to be traced function function trace_function() { local function="$1"; shift

cleanup_tracing

echo "function" > "$SYSFS_TRACING_DIR/current_tracer" echo "$functions" > $SYSFS_TRACING_DIR/set_ftrace_filter echo 1 > "$SYSFS_TRACING_DIR/tracing_on" }

# check_traced_function(function) - check whether function appeared in trace log # function - to be traced function function check_trace_function() { local function="$1"; shift

if ! grep -q "$function" $SYSFS_TRACING_DIR/trace" ; then die "Function ($function) did not appear in the trace" fi }

As Marcos suggested, it would make sense to add the above mentioned helper functions in a separate patch.

+if [[ "$(cat /proc/cmdline)" != "$MOD_LIVEPATCH: this has been live patched" ]] ; then

• echo -e "FAIL\n\n"
• die "livepatch kselftest(s) failed"

+fi

+grep -q $FUNCTION_NAME "$TRACE_FILE" +FOUND=$?

We could replace this with the above proposed

check_traced_function "$FUNCTION_NAME"

Also I would personally stop the tracing after this sub-test finishes. We might use the cleanup_tracing function here.

cleanup_tracing

+disable_lp $MOD_LIVEPATCH +unload_lp $MOD_LIVEPATCH

+if [ "$FOUND" -eq 1 ]; then

• echo -e "FAIL\n\n"
• die "livepatch kselftest(s) failed"

+fi

As mentioned by Marcos, we should also print "ok" on success. I would personally use the "check_result" function here. It could catch non-expected entries in dmesg.

exit 0

Best Regards, Petr