Clang uses a different set of CLI args for coverage, and the output needs to be processed by a different set of tools. Update the Makefile and add an example of usage in kunit docs.
Michał Winiarski (2): arch: um: Add Clang coverage support Documentation: kunit: Add clang UML coverage example
Documentation/dev-tools/kunit/running_tips.rst | 11 +++++++++++ arch/um/Makefile-skas | 5 +++++ 2 files changed, 16 insertions(+)
Clang uses a different set of command line arguments for enabling coverage.
Signed-off-by: Michał Winiarski michal.winiarski@intel.com --- arch/um/Makefile-skas | 5 +++++ 1 file changed, 5 insertions(+)
diff --git a/arch/um/Makefile-skas b/arch/um/Makefile-skas index ac35de5316a6..67323b028999 100644 --- a/arch/um/Makefile-skas +++ b/arch/um/Makefile-skas @@ -4,7 +4,12 @@ #
GPROF_OPT += -pg + +ifdef CONFIG_CC_IS_CLANG +GCOV_OPT += -fprofile-instr-generate -fcoverage-mapping +else GCOV_OPT += -fprofile-arcs -ftest-coverage +endif
CFLAGS-$(CONFIG_GCOV) += $(GCOV_OPT) CFLAGS-$(CONFIG_GPROF) += $(GPROF_OPT)
On Fri, 20 Oct 2023 at 17:22, Michał Winiarski michal.winiarski@intel.com wrote:
Clang uses a different set of command line arguments for enabling coverage.
Signed-off-by: Michał Winiarski michal.winiarski@intel.com
This works brilliantly here -- I'm very glad to finally be able to use something newer than gcc 6!
I assume this will go in via the UML tree, but if you want, we can take it via KUnit.
Tested-by: David Gow davidgow@google.com
Cheers, -- David
arch/um/Makefile-skas | 5 +++++ 1 file changed, 5 insertions(+)
diff --git a/arch/um/Makefile-skas b/arch/um/Makefile-skas index ac35de5316a6..67323b028999 100644 --- a/arch/um/Makefile-skas +++ b/arch/um/Makefile-skas @@ -4,7 +4,12 @@ #
GPROF_OPT += -pg
+ifdef CONFIG_CC_IS_CLANG +GCOV_OPT += -fprofile-instr-generate -fcoverage-mapping +else GCOV_OPT += -fprofile-arcs -ftest-coverage +endif
CFLAGS-$(CONFIG_GCOV) += $(GCOV_OPT) CFLAGS-$(CONFIG_GPROF) += $(GPROF_OPT) -- 2.42.0
LLVM-based toolchain is using a different set of tools for coverage. Add an example that produces output in lcov format.
Signed-off-by: Michał Winiarski michal.winiarski@intel.com --- Documentation/dev-tools/kunit/running_tips.rst | 11 +++++++++++ 1 file changed, 11 insertions(+)
diff --git a/Documentation/dev-tools/kunit/running_tips.rst b/Documentation/dev-tools/kunit/running_tips.rst index 766f9cdea0fa..9f69c122dee7 100644 --- a/Documentation/dev-tools/kunit/running_tips.rst +++ b/Documentation/dev-tools/kunit/running_tips.rst @@ -139,6 +139,17 @@ If your installed version of gcc doesn't work, you can tweak the steps: $ ./tools/testing/kunit/kunit.py run --make_options=CC=/usr/bin/gcc-6 $ lcov -t "my_kunit_tests" -o coverage.info -c -d .kunit/ --gcov-tool=/usr/bin/gcov-6
+Alternatively, LLVM-based toolchain can also be used: + +.. code-block:: bash + + # Build with LLVM and append coverage options to the current config + $ $ ./tools/testing/kunit/kunit.py run --make_options LLVM=1 --kunitconfig=.kunit/ --kunitconfig=tools/testing/kunit/configs/coverage_uml.config + $ llvm-profdata merge -sparse default.profraw -o default.profdata + $ llvm-cov export --format=lcov .kunit/vmlinux -instr-profile default.profdata > coverage.info + # The coverage.info file is in lcov-compatible format and it can be used to e.g. generate HTML report + $ genhtml -o /tmp/coverage_html coverage.info +
Running tests manually ======================
On Fri, 20 Oct 2023 at 17:22, Michał Winiarski michal.winiarski@intel.com wrote:
LLVM-based toolchain is using a different set of tools for coverage. Add an example that produces output in lcov format.
Signed-off-by: Michał Winiarski michal.winiarski@intel.com
This looks good to me, minus a couple of very, very minor typos, and the fact that this whole section could probably do with some tidying up. We'll deal with that separately, though.
UML folks: do you want to take this via the UML branch (alongside patch 1), or should we take one or both of them via KUnit? (Or, this could go via the docs tree, too, I suppose.) There shouldn't be any merge conflicts on our side.
Reviewed-by: David Gow davidgow@google.com
Cheers, -- David
Documentation/dev-tools/kunit/running_tips.rst | 11 +++++++++++ 1 file changed, 11 insertions(+)
diff --git a/Documentation/dev-tools/kunit/running_tips.rst b/Documentation/dev-tools/kunit/running_tips.rst index 766f9cdea0fa..9f69c122dee7 100644 --- a/Documentation/dev-tools/kunit/running_tips.rst +++ b/Documentation/dev-tools/kunit/running_tips.rst @@ -139,6 +139,17 @@ If your installed version of gcc doesn't work, you can tweak the steps: $ ./tools/testing/kunit/kunit.py run --make_options=CC=/usr/bin/gcc-6 $ lcov -t "my_kunit_tests" -o coverage.info -c -d .kunit/ --gcov-tool=/usr/bin/gcov-6
+Alternatively, LLVM-based toolchain can also be used:
Nit: should this be "an LLVM-based toolchain" or "LLVM-based toolchains"?
+.. code-block:: bash
# Build with LLVM and append coverage options to the current config$ $ ./tools/testing/kunit/kunit.py run --make_options LLVM=1 --kunitconfig=.kunit/ --kunitconfig=tools/testing/kunit/configs/coverage_uml.config
I'm not a big fan of the --kunitconfig=.kunit/ bit here, but since we're doing it in the gcc version above, let's leave it for now for consistency.
Also, Nit: two '$' starting the line.
$ llvm-profdata merge -sparse default.profraw -o default.profdata$ llvm-cov export --format=lcov .kunit/vmlinux -instr-profile default.profdata > coverage.info# The coverage.info file is in lcov-compatible format and it can be used to e.g. generate HTML report$ genhtml -o /tmp/coverage_html coverage.infoRunning tests manually
-- 2.42.0
On 20/10/23 06:21, Michał Winiarski wrote:
Clang uses a different set of CLI args for coverage, and the output needs to be processed by a different set of tools. Update the Makefile and add an example of usage in kunit docs.
Great change! It's great not to rely on older versions of GCC for that.
I was able to generate coverage reports, but I have a few of things to note:
When I ran the kunit.py it generated this warning several times:
WARNING: modpost: vmlinux (__llvm_covfun): unexpected non-allocatable section. Did you forget to use "ax"/"aw" in a .S file? Note that for example <linux/init.h> contains section definitions for use in .S files.
Maybe it would be great to know why this is happening.
Also, the linker consumed a lot of RAM to build the kernel with those additional flags, but maybe this is expected :P.
Best Regards, ~Arthur Grillo
Michał Winiarski (2): arch: um: Add Clang coverage support Documentation: kunit: Add clang UML coverage example
Documentation/dev-tools/kunit/running_tips.rst | 11 +++++++++++ arch/um/Makefile-skas | 5 +++++ 2 files changed, 16 insertions(+)
linux-kselftest-mirror@lists.linaro.org
-
Arthur Grillo -
David Gow -
Michał Winiarski