We need to pass the arguments provided to --kmake-arg to all make invocations. In particular, the make invocations generating the configs need to see the final make arguments, e.g. if config variables depend on particular variables that are passed to make.
For example, when using '--kcsan --kmake-arg CC=clang-11', we would lose CONFIG_KCSAN=y due to 'make oldconfig' not seeing that we want to use a compiler that supports KCSAN.
Signed-off-by: Marco Elver elver@google.com --- tools/testing/selftests/rcutorture/bin/configinit.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/tools/testing/selftests/rcutorture/bin/configinit.sh b/tools/testing/selftests/rcutorture/bin/configinit.sh index 93e80a42249a..d6e5ce084b1c 100755 --- a/tools/testing/selftests/rcutorture/bin/configinit.sh +++ b/tools/testing/selftests/rcutorture/bin/configinit.sh @@ -32,11 +32,11 @@ if test -z "$TORTURE_TRUST_MAKE" then make clean > $resdir/Make.clean 2>&1 fi -make $TORTURE_DEFCONFIG > $resdir/Make.defconfig.out 2>&1 +make $TORTURE_KMAKE_ARG $TORTURE_DEFCONFIG > $resdir/Make.defconfig.out 2>&1 mv .config .config.sav sh $T/upd.sh < .config.sav > .config cp .config .config.new -yes '' | make oldconfig > $resdir/Make.oldconfig.out 2> $resdir/Make.oldconfig.err +yes '' | make $TORTURE_KMAKE_ARG oldconfig > $resdir/Make.oldconfig.out 2> $resdir/Make.oldconfig.err
# verify new config matches specification. configcheck.sh .config $c
On Tue, Jun 16, 2020 at 11:49:24AM +0200, Marco Elver wrote:
We need to pass the arguments provided to --kmake-arg to all make invocations. In particular, the make invocations generating the configs need to see the final make arguments, e.g. if config variables depend on particular variables that are passed to make.
For example, when using '--kcsan --kmake-arg CC=clang-11', we would lose CONFIG_KCSAN=y due to 'make oldconfig' not seeing that we want to use a compiler that supports KCSAN.
Signed-off-by: Marco Elver elver@google.com
Queued and pushed, thank you!
Would the following patch make sense, at least until such time as some other compiler supports KCSAN?
Thanx, Paul
------------------------------------------------------------------------
commit 88bcaa730b6d40ddf69b09ed6f0a14803d087d99 Author: Paul E. McKenney paulmck@kernel.org Date: Tue Jun 16 09:02:34 2020 -0700
torture: Make --kcsan default to using Clang 11
Currently, Clang 11 is the only compiler that can support KCSAN. Therefore, as a convenience to the KCSAN user, this commit causes --kcsan to specify Clang 11 unless a "CC=" argument was already specified via the --kmake-arg argument.
Signed-off-by: Paul E. McKenney paulmck@kernel.org
diff --git a/tools/testing/selftests/rcutorture/bin/kvm.sh b/tools/testing/selftests/rcutorture/bin/kvm.sh index 446f680..f8c6cfa 100755 --- a/tools/testing/selftests/rcutorture/bin/kvm.sh +++ b/tools/testing/selftests/rcutorture/bin/kvm.sh @@ -34,6 +34,7 @@ TORTURE_KCONFIG_ARG="" TORTURE_KCONFIG_KASAN_ARG="" TORTURE_KCONFIG_KCSAN_ARG="" TORTURE_KMAKE_ARG="" +TORTURE_KMAKE_KCSAN_ARG="" TORTURE_QEMU_MEM=512 TORTURE_SHUTDOWN_GRACE=180 TORTURE_SUITE=rcu @@ -144,6 +145,7 @@ do ;; --kcsan) TORTURE_KCONFIG_KCSAN_ARG="CONFIG_DEBUG_INFO=y CONFIG_KCSAN=y CONFIG_KCSAN_ASSUME_PLAIN_WRITES_ATOMIC=n CONFIG_KCSAN_REPORT_VALUE_CHANGE_ONLY=n CONFIG_KCSAN_REPORT_ONCE_IN_MS=100000 CONFIG_KCSAN_VERBOSE=y CONFIG_KCSAN_INTERRUPT_WATCHER=y"; export TORTURE_KCONFIG_KCSAN_ARG + TORTURE_KMAKE_KCSAN_ARG="CC=clang-11" ;; --kmake-arg) checkarg --kmake-arg "(kernel make arguments)" $# "$2" '.*' '^error$' @@ -214,6 +216,11 @@ else exit 1 fi
+if test -n "$TORTURE_KMAKE_KCSAN_ARG" && ! echo "$TORTURE_KMAKE_ARG" | grep -q 'CC=' +then + TORTURE_KMAKE_ARG="$TORTURE_KMAKE_ARG $TORTURE_KMAKE_KCSAN_ARG" +fi + CONFIGFRAG=${KVM}/configs/${TORTURE_SUITE}; export CONFIGFRAG
defaultconfigs="`tr '\012' ' ' < $CONFIGFRAG/CFLIST`"
On Tue, Jun 16 2020 at 09:05AM -0700, Paul E. McKenney wrote:
On Tue, Jun 16, 2020 at 11:49:24AM +0200, Marco Elver wrote:
We need to pass the arguments provided to --kmake-arg to all make invocations. In particular, the make invocations generating the configs need to see the final make arguments, e.g. if config variables depend on particular variables that are passed to make.
For example, when using '--kcsan --kmake-arg CC=clang-11', we would lose CONFIG_KCSAN=y due to 'make oldconfig' not seeing that we want to use a compiler that supports KCSAN.
Signed-off-by: Marco Elver elver@google.com
Queued and pushed, thank you!
Would the following patch make sense, at least until such time as some other compiler supports KCSAN?
Thanx, Paul
commit 88bcaa730b6d40ddf69b09ed6f0a14803d087d99 Author: Paul E. McKenney paulmck@kernel.org Date: Tue Jun 16 09:02:34 2020 -0700
torture: Make --kcsan default to using Clang 11Currently, Clang 11 is the only compiler that can support KCSAN. Therefore, as a convenience to the KCSAN user, this commit causes --kcsan to specify Clang 11 unless a "CC=" argument was already specified via the --kmake-arg argument.
As soon as more compilers support KCSAN (e.g. clang-12, etc...) we run the risk of actually inconveniencing ourselves more because then we really need to say '--kmake-arg CC=clang-1X' to not use the old compiler. Or revert this in time.
My command-line looks more like this right now:
kvm.sh ... --kmake-arg "CC="${HOME}/local/<gcc-or-clang>-11.kcsan/local/bin/<gcc-or-clang>" ...
I think the safer alternative would be to error if CONFIG_KCSAN=y is not in the config, and simply suggest "Did you forget to switch your compiler with '--kmake-arg CC=<cc-that-supports-kcsan>'?" (of course, a 'gcc' in $PATH that supports KCSAN would also be fine -- see below). Eventually, when the default compilers support KCSAN, this will resolve itself gracefully.
Also, I'm going to send a series later this week to re-enable GCC support. ;-)
Thanks, -- Marco
On Tue, Jun 16, 2020 at 06:42:02PM +0200, Marco Elver wrote:
On Tue, Jun 16 2020 at 09:05AM -0700, Paul E. McKenney wrote:
On Tue, Jun 16, 2020 at 11:49:24AM +0200, Marco Elver wrote:
We need to pass the arguments provided to --kmake-arg to all make invocations. In particular, the make invocations generating the configs need to see the final make arguments, e.g. if config variables depend on particular variables that are passed to make.
For example, when using '--kcsan --kmake-arg CC=clang-11', we would lose CONFIG_KCSAN=y due to 'make oldconfig' not seeing that we want to use a compiler that supports KCSAN.
Signed-off-by: Marco Elver elver@google.com
Queued and pushed, thank you!
Would the following patch make sense, at least until such time as some other compiler supports KCSAN?
Thanx, Paul
commit 88bcaa730b6d40ddf69b09ed6f0a14803d087d99 Author: Paul E. McKenney paulmck@kernel.org Date: Tue Jun 16 09:02:34 2020 -0700
torture: Make --kcsan default to using Clang 11Currently, Clang 11 is the only compiler that can support KCSAN. Therefore, as a convenience to the KCSAN user, this commit causes --kcsan to specify Clang 11 unless a "CC=" argument was already specified via the --kmake-arg argument.
As soon as more compilers support KCSAN (e.g. clang-12, etc...) we run the risk of actually inconveniencing ourselves more because then we really need to say '--kmake-arg CC=clang-1X' to not use the old compiler. Or revert this in time.
My command-line looks more like this right now:
kvm.sh ... --kmake-arg "CC="${HOME}/local/<gcc-or-clang>-11.kcsan/local/bin/<gcc-or-clang>" ...
I think the safer alternative would be to error if CONFIG_KCSAN=y is not in the config, and simply suggest "Did you forget to switch your compiler with '--kmake-arg CC=<cc-that-supports-kcsan>'?" (of course, a 'gcc' in $PATH that supports KCSAN would also be fine -- see below). Eventually, when the default compilers support KCSAN, this will resolve itself gracefully.
Also, I'm going to send a series later this week to re-enable GCC support. ;-)
OK, sounds like I should leave well enough alone, then. ;-)
In its current state, specifying "--kcsan" without a KCSAN-capable compiler does get you this:
:CONFIG_KCSAN=y: improperly set :CONFIG_KCSAN_REPORT_ONCE_IN_MS=100000: improperly set :CONFIG_KCSAN_VERBOSE=y: improperly set :CONFIG_KCSAN_INTERRUPT_WATCHER=y: improperly set Clean KCSAN run in /home/git/linux-rcu/tools/testing/selftests/rcutorture/res/2020.06.16-09.53.16
Which admittedly is a bit obtuse, especially that last line. So how about the following patch, which instead results in this?
:CONFIG_KCSAN=y: improperly set :CONFIG_KCSAN_REPORT_ONCE_IN_MS=100000: improperly set :CONFIG_KCSAN_VERBOSE=y: improperly set :CONFIG_KCSAN_INTERRUPT_WATCHER=y: improperly set Did you forget to switch your compiler with --kmake-arg CC=<cc-that-supports-kcsan>?
Thanx, Paul
------------------------------------------------------------------------
commit f571795b1146007407851675a258b6685ea2d589 Author: Paul E. McKenney paulmck@kernel.org Date: Tue Jun 16 10:38:57 2020 -0700
torture: Improve diagnostic for KCSAN-incapable compilers
Using --kcsan when the compiler does not support KCSAN results in this:
:CONFIG_KCSAN=y: improperly set :CONFIG_KCSAN_REPORT_ONCE_IN_MS=100000: improperly set :CONFIG_KCSAN_VERBOSE=y: improperly set :CONFIG_KCSAN_INTERRUPT_WATCHER=y: improperly set Clean KCSAN run in /home/git/linux-rcu/tools/testing/selftests/rcutorture/res/2020.06.16-09.53.16
This is a bit obtuse, so this commit adds checks resulting in this:
:CONFIG_KCSAN=y: improperly set :CONFIG_KCSAN_REPORT_ONCE_IN_MS=100000: improperly set :CONFIG_KCSAN_VERBOSE=y: improperly set :CONFIG_KCSAN_INTERRUPT_WATCHER=y: improperly set Did you forget to switch your compiler with --kmake-arg CC=<cc-that-supports-kcsan>?
Suggested-by: Marco Elver elver@google.com Signed-off-by: Paul E. McKenney paulmck@kernel.org
diff --git a/tools/testing/selftests/rcutorture/bin/kvm-recheck.sh b/tools/testing/selftests/rcutorture/bin/kvm-recheck.sh index 357899c..837643a 100755 --- a/tools/testing/selftests/rcutorture/bin/kvm-recheck.sh +++ b/tools/testing/selftests/rcutorture/bin/kvm-recheck.sh @@ -44,7 +44,8 @@ do then echo QEMU killed fi - configcheck.sh $i/.config $i/ConfigFragment + configcheck.sh $i/.config $i/ConfigFragment > $T 2>&1 + cat $T if test -r $i/Make.oldconfig.err then cat $i/Make.oldconfig.err @@ -73,7 +74,10 @@ do done if test -f "$rd/kcsan.sum" then - if test -s "$rd/kcsan.sum" + if grep -q CONFIG_KCSAN=y $T + then + echo Did you forget to switch your compiler with '--kmake-arg CC=<cc-that-supports-kcsan>'? + elif test -s "$rd/kcsan.sum" then echo KCSAN summary in $rd/kcsan.sum else
On Tue, Jun 16 2020 at 10:42AM -0700, Paul E. McKenney wrote:
On Tue, Jun 16, 2020 at 06:42:02PM +0200, Marco Elver wrote:
On Tue, Jun 16 2020 at 09:05AM -0700, Paul E. McKenney wrote:
On Tue, Jun 16, 2020 at 11:49:24AM +0200, Marco Elver wrote:
We need to pass the arguments provided to --kmake-arg to all make invocations. In particular, the make invocations generating the configs need to see the final make arguments, e.g. if config variables depend on particular variables that are passed to make.
For example, when using '--kcsan --kmake-arg CC=clang-11', we would lose CONFIG_KCSAN=y due to 'make oldconfig' not seeing that we want to use a compiler that supports KCSAN.
Signed-off-by: Marco Elver elver@google.com
Queued and pushed, thank you!
Would the following patch make sense, at least until such time as some other compiler supports KCSAN?
Thanx, Paul
commit 88bcaa730b6d40ddf69b09ed6f0a14803d087d99 Author: Paul E. McKenney paulmck@kernel.org Date: Tue Jun 16 09:02:34 2020 -0700
torture: Make --kcsan default to using Clang 11Currently, Clang 11 is the only compiler that can support KCSAN. Therefore, as a convenience to the KCSAN user, this commit causes --kcsan to specify Clang 11 unless a "CC=" argument was already specified via the --kmake-arg argument.
As soon as more compilers support KCSAN (e.g. clang-12, etc...) we run the risk of actually inconveniencing ourselves more because then we really need to say '--kmake-arg CC=clang-1X' to not use the old compiler. Or revert this in time.
My command-line looks more like this right now:
kvm.sh ... --kmake-arg "CC="${HOME}/local/<gcc-or-clang>-11.kcsan/local/bin/<gcc-or-clang>" ...
I think the safer alternative would be to error if CONFIG_KCSAN=y is not in the config, and simply suggest "Did you forget to switch your compiler with '--kmake-arg CC=<cc-that-supports-kcsan>'?" (of course, a 'gcc' in $PATH that supports KCSAN would also be fine -- see below). Eventually, when the default compilers support KCSAN, this will resolve itself gracefully.
Also, I'm going to send a series later this week to re-enable GCC support. ;-)
OK, sounds like I should leave well enough alone, then. ;-)
In its current state, specifying "--kcsan" without a KCSAN-capable compiler does get you this:
:CONFIG_KCSAN=y: improperly set :CONFIG_KCSAN_REPORT_ONCE_IN_MS=100000: improperly set :CONFIG_KCSAN_VERBOSE=y: improperly set :CONFIG_KCSAN_INTERRUPT_WATCHER=y: improperly set Clean KCSAN run in /home/git/linux-rcu/tools/testing/selftests/rcutorture/res/2020.06.16-09.53.16
Which admittedly is a bit obtuse, especially that last line. So how about the following patch, which instead results in this?
:CONFIG_KCSAN=y: improperly set :CONFIG_KCSAN_REPORT_ONCE_IN_MS=100000: improperly set :CONFIG_KCSAN_VERBOSE=y: improperly set
Noticed that CONFIG_KCSAN_VERBOSE=y warning still appears, because it wants CONFIG_PROVE_LOCKING=y.
:CONFIG_KCSAN_INTERRUPT_WATCHER=y: improperly set Did you forget to switch your compiler with --kmake-arg CC=<cc-that-supports-kcsan>?
Sounds good, with another suggestion below, but otherwise
Acked-by: Marco Elver elver@google.com
Thanks, -- Marco
Thanx, Paul
commit f571795b1146007407851675a258b6685ea2d589 Author: Paul E. McKenney paulmck@kernel.org Date: Tue Jun 16 10:38:57 2020 -0700
torture: Improve diagnostic for KCSAN-incapable compilersUsing --kcsan when the compiler does not support KCSAN results in this: :CONFIG_KCSAN=y: improperly set :CONFIG_KCSAN_REPORT_ONCE_IN_MS=100000: improperly set :CONFIG_KCSAN_VERBOSE=y: improperly set :CONFIG_KCSAN_INTERRUPT_WATCHER=y: improperly set Clean KCSAN run in /home/git/linux-rcu/tools/testing/selftests/rcutorture/res/2020.06.16-09.53.16 This is a bit obtuse, so this commit adds checks resulting in this: :CONFIG_KCSAN=y: improperly set :CONFIG_KCSAN_REPORT_ONCE_IN_MS=100000: improperly set :CONFIG_KCSAN_VERBOSE=y: improperly set :CONFIG_KCSAN_INTERRUPT_WATCHER=y: improperly set Did you forget to switch your compiler with --kmake-arg CC=<cc-that-supports-kcsan>? Suggested-by: Marco Elver elver@google.com Signed-off-by: Paul E. McKenney paulmck@kernel.org
diff --git a/tools/testing/selftests/rcutorture/bin/kvm-recheck.sh b/tools/testing/selftests/rcutorture/bin/kvm-recheck.sh index 357899c..837643a 100755 --- a/tools/testing/selftests/rcutorture/bin/kvm-recheck.sh +++ b/tools/testing/selftests/rcutorture/bin/kvm-recheck.sh @@ -44,7 +44,8 @@ do then echo QEMU killed fi
configcheck.sh $i/.config $i/ConfigFragment
configcheck.sh $i/.config $i/ConfigFragment > $T 2>&1cat $T if test -r $i/Make.oldconfig.err then cat $i/Make.oldconfig.err@@ -73,7 +74,10 @@ do done if test -f "$rd/kcsan.sum" then
if test -s "$rd/kcsan.sum"
if grep -q CONFIG_KCSAN=y $Tthen
For completeness, could add this above:
echo "Compiler or architecture does not support KCSAN!"
Because we might also be on an unsupported architecture.
echo Did you forget to switch your compiler with '--kmake-arg CC=<cc-that-supports-kcsan>'?elif test -s "$rd/kcsan.sum"
On Tue, Jun 16, 2020 at 07:57:20PM +0200, Marco Elver wrote:
On Tue, Jun 16 2020 at 10:42AM -0700, Paul E. McKenney wrote:
On Tue, Jun 16, 2020 at 06:42:02PM +0200, Marco Elver wrote:
On Tue, Jun 16 2020 at 09:05AM -0700, Paul E. McKenney wrote:
On Tue, Jun 16, 2020 at 11:49:24AM +0200, Marco Elver wrote:
We need to pass the arguments provided to --kmake-arg to all make invocations. In particular, the make invocations generating the configs need to see the final make arguments, e.g. if config variables depend on particular variables that are passed to make.
For example, when using '--kcsan --kmake-arg CC=clang-11', we would lose CONFIG_KCSAN=y due to 'make oldconfig' not seeing that we want to use a compiler that supports KCSAN.
Signed-off-by: Marco Elver elver@google.com
Queued and pushed, thank you!
Would the following patch make sense, at least until such time as some other compiler supports KCSAN?
Thanx, Paul
commit 88bcaa730b6d40ddf69b09ed6f0a14803d087d99 Author: Paul E. McKenney paulmck@kernel.org Date: Tue Jun 16 09:02:34 2020 -0700
torture: Make --kcsan default to using Clang 11Currently, Clang 11 is the only compiler that can support KCSAN. Therefore, as a convenience to the KCSAN user, this commit causes --kcsan to specify Clang 11 unless a "CC=" argument was already specified via the --kmake-arg argument.
As soon as more compilers support KCSAN (e.g. clang-12, etc...) we run the risk of actually inconveniencing ourselves more because then we really need to say '--kmake-arg CC=clang-1X' to not use the old compiler. Or revert this in time.
My command-line looks more like this right now:
kvm.sh ... --kmake-arg "CC="${HOME}/local/<gcc-or-clang>-11.kcsan/local/bin/<gcc-or-clang>" ...
I think the safer alternative would be to error if CONFIG_KCSAN=y is not in the config, and simply suggest "Did you forget to switch your compiler with '--kmake-arg CC=<cc-that-supports-kcsan>'?" (of course, a 'gcc' in $PATH that supports KCSAN would also be fine -- see below). Eventually, when the default compilers support KCSAN, this will resolve itself gracefully.
Also, I'm going to send a series later this week to re-enable GCC support. ;-)
OK, sounds like I should leave well enough alone, then. ;-)
In its current state, specifying "--kcsan" without a KCSAN-capable compiler does get you this:
:CONFIG_KCSAN=y: improperly set :CONFIG_KCSAN_REPORT_ONCE_IN_MS=100000: improperly set :CONFIG_KCSAN_VERBOSE=y: improperly set :CONFIG_KCSAN_INTERRUPT_WATCHER=y: improperly set Clean KCSAN run in /home/git/linux-rcu/tools/testing/selftests/rcutorture/res/2020.06.16-09.53.16
Which admittedly is a bit obtuse, especially that last line. So how about the following patch, which instead results in this?
:CONFIG_KCSAN=y: improperly set :CONFIG_KCSAN_REPORT_ONCE_IN_MS=100000: improperly set :CONFIG_KCSAN_VERBOSE=y: improperly set
Noticed that CONFIG_KCSAN_VERBOSE=y warning still appears, because it wants CONFIG_PROVE_LOCKING=y.
:CONFIG_KCSAN_INTERRUPT_WATCHER=y: improperly set Did you forget to switch your compiler with --kmake-arg CC=<cc-that-supports-kcsan>?
Sounds good, with another suggestion below, but otherwise
Acked-by: Marco Elver elver@google.com
Applied and updated as suggested below, so that the output now looks like this:
:CONFIG_KCSAN=y: improperly set :CONFIG_KCSAN_REPORT_ONCE_IN_MS=100000: improperly set :CONFIG_KCSAN_VERBOSE=y: improperly set :CONFIG_KCSAN_INTERRUPT_WATCHER=y: improperly set Compiler or architecture does not support KCSAN! Did you forget to switch your compiler with --kmake-arg CC=<cc-that-supports-kcsan>?
And agreed, much better!
Thanx, Paul
Thanks, -- Marco
Thanx, Paul
commit f571795b1146007407851675a258b6685ea2d589 Author: Paul E. McKenney paulmck@kernel.org Date: Tue Jun 16 10:38:57 2020 -0700
torture: Improve diagnostic for KCSAN-incapable compilersUsing --kcsan when the compiler does not support KCSAN results in this: :CONFIG_KCSAN=y: improperly set :CONFIG_KCSAN_REPORT_ONCE_IN_MS=100000: improperly set :CONFIG_KCSAN_VERBOSE=y: improperly set :CONFIG_KCSAN_INTERRUPT_WATCHER=y: improperly set Clean KCSAN run in /home/git/linux-rcu/tools/testing/selftests/rcutorture/res/2020.06.16-09.53.16 This is a bit obtuse, so this commit adds checks resulting in this: :CONFIG_KCSAN=y: improperly set :CONFIG_KCSAN_REPORT_ONCE_IN_MS=100000: improperly set :CONFIG_KCSAN_VERBOSE=y: improperly set :CONFIG_KCSAN_INTERRUPT_WATCHER=y: improperly set Did you forget to switch your compiler with --kmake-arg CC=<cc-that-supports-kcsan>? Suggested-by: Marco Elver elver@google.com Signed-off-by: Paul E. McKenney paulmck@kernel.org
diff --git a/tools/testing/selftests/rcutorture/bin/kvm-recheck.sh b/tools/testing/selftests/rcutorture/bin/kvm-recheck.sh index 357899c..837643a 100755 --- a/tools/testing/selftests/rcutorture/bin/kvm-recheck.sh +++ b/tools/testing/selftests/rcutorture/bin/kvm-recheck.sh @@ -44,7 +44,8 @@ do then echo QEMU killed fi
configcheck.sh $i/.config $i/ConfigFragment
configcheck.sh $i/.config $i/ConfigFragment > $T 2>&1cat $T if test -r $i/Make.oldconfig.err then cat $i/Make.oldconfig.err@@ -73,7 +74,10 @@ do done if test -f "$rd/kcsan.sum" then
if test -s "$rd/kcsan.sum"
if grep -q CONFIG_KCSAN=y $TthenFor completeness, could add this above:
echo "Compiler or architecture does not support KCSAN!"
Because we might also be on an unsupported architecture.
echo Did you forget to switch your compiler with '--kmake-arg CC=<cc-that-supports-kcsan>'?elif test -s "$rd/kcsan.sum"
linux-kselftest-mirror@lists.linaro.org
-
Marco Elver -
Paul E. McKenney