The kunit_tool documentation page was pretty minimal, and a bit outdated. Update it and flesh it out a bit.
In particular, - Mention that .kunitconfig is now in the build directory - Describe the use of --kunitconfig to specify a different config framgent - Mention the split functionality (i.e., commands other than 'run') - Describe --raw_output and kunit.py parse - Mention the globbing support - Provide a quick overview of other options, including --build_dir and --alltests
Note that this does overlap a little with the new running_tips page. I don't think it's a problem having both: this page is supposed to be a bit more of a reference, rather than a list of useful tips, so the fact that they both describe the same features isn't a problem.
Signed-off-by: David Gow davidgow@google.com --- Documentation/dev-tools/kunit/kunit-tool.rst | 132 ++++++++++++++++++- 1 file changed, 128 insertions(+), 4 deletions(-)
diff --git a/Documentation/dev-tools/kunit/kunit-tool.rst b/Documentation/dev-tools/kunit/kunit-tool.rst index 29ae2fee8123..0b45affcd65c 100644 --- a/Documentation/dev-tools/kunit/kunit-tool.rst +++ b/Documentation/dev-tools/kunit/kunit-tool.rst @@ -22,14 +22,19 @@ not require any virtualization support: it is just a regular program. What is a .kunitconfig? =======================
-It's just a defconfig that kunit_tool looks for in the base directory. +It's just a defconfig that kunit_tool looks for in the build directory. kunit_tool uses it to generate a .config as you might expect. In addition, it verifies that the generated .config contains the CONFIG options in the .kunitconfig; the reason it does this is so that it is easy to be sure that a CONFIG that enables a test actually ends up in the .config.
-How do I use kunit_tool? -======================== +It's also possible to pass a separate .kunitconfig fragment to kunit_tool, +which is useful if you have several different groups of tests you wish +to run independently, or if you want to use pre-defined test configs for +certain subsystems. + +Getting Started with kunit_tool +===============================
If a kunitconfig is present at the root directory, all you have to do is:
@@ -48,10 +53,129 @@ However, you most likely want to use it with the following options:
.. note:: This command will work even without a .kunitconfig file: if no - .kunitconfig is present, a default one will be used instead. + .kunitconfig is present, a default one will be used instead. + +If you wish to use a different .kunitconfig file (such as one provided for +testing a particular subsystem), you can pass it as an option. + +.. code-block:: bash + + ./tools/testing/kunit/kunit.py run --kunitconfig=fs/ext4/.kunitconfig
For a list of all the flags supported by kunit_tool, you can run:
.. code-block:: bash
./tools/testing/kunit/kunit.py run --help + +Configuring, Building, and Running Tests +======================================== + +It's also possible to run just parts of the KUnit build process independently, +which is useful if you want to make manual changes to part of the process. + +A .config can be generated from a .kunitconfig by using the ``config`` argument +when running kunit_tool: + +.. code-block:: bash + + ./tools/testing/kunit/kunit.py config + +Similarly, if you just want to build a KUnit kernel from the current .config, +you can use the ``build`` argument: + +.. code-block:: bash + + ./tools/testing/kunit/kunit.py build + +And, if you already have a built UML kernel with built-in KUnit tests, you can +run the kernel and display the test results with the ``exec`` argument: + +.. code-block:: bash + + ./tools/testing/kunit/kunit.py exec + +The ``run`` command which is discussed above is equivalent to running all three +of these in sequence. + +All of these commands accept a number of optional command-line arguments. The +``--help`` flag will give a complete list of these, or keep reading this page +for a guide to some of the more useful ones. + +Parsing Test Results +==================== + +KUnit tests output their results in TAP (Test Anything Protocol) format. +kunit_tool will, when running tests, parse this output and print a summary +which is much more pleasant to read. If you wish to look at the raw test +results in TAP format, you can pass the ``--raw_output`` argument. + +.. code-block:: bash + + ./tools/testing/kunit/kunit.py run --raw_output + +.. note:: + The raw output from test runs may contain other, non-KUnit kernel log + lines. + +If you have KUnit results in their raw TAP format, you can parse them and print +the human-readable summary with the ``parse`` command for kunit_tool. This +accepts a filename for an argument, or will read from standard input. + +.. code-block:: bash + + # Reading from a file + ./tools/testing/kunit/kunit.py parse /var/log/dmesg + # Reading from stdin + dmesg | ./tools/testing/kunit/kunit.py parse + +This is very useful if you wish to run tests in a configuration not supported +by kunit_tool (such as on real hardware, or an unsupported architecture). + +Filtering Tests +=============== + +It's possible to run only a subset of the tests built into a kernel by passing +a filter to the ``exec`` or ``run`` commands. For example, if you only wanted +to run KUnit resource tests, you could use: + +.. code-block:: bash + + ./tools/testing/kunit/kunit.py run 'kunit-resource*' + +This uses the standard glob format for wildcards. + +Other Useful Options +==================== + +kunit_tool has a number of other command-line arguments which can be useful +when adapting it to fit your environment or needs. + +Some of the more useful ones are: + +``--help`` + Lists all of the available options. Note that different commands + (``config``, ``build``, ``run``, etc) will have different supported + options. Place ``--help`` before the command to list common options, + and after the command for options specific to that command. + +``--build_dir`` + Specifies the build directory that kunit_tool will use. This is where + the .kunitconfig file is located, as well as where the .config and + compiled kernel will be placed. Defaults to ``.kunit``. + +``--make_options`` + Specifies additional options to pass to ``make`` when compiling a + kernel (with the ``build`` or ``run`` commands). For example, to enable + compiler warnings, you can pass ``--make_options W=1``. + +``--alltests`` + Builds a UML kernel with all config options enabled using + ``make allyesconfig``. This allows you to run as many tests as is + possible, but is very slow and prone to breakage as new options are + added or modified. Most people should add ``CONFIG_KUNIT_ALL_TESTS=1`` + to their .kunitconfig instead if they wish to run "all tests". + + +There are several other options (and new ones are often added), so do check +``--help`` if you're looking for something not mentioned here.
On Thu, Apr 15, 2021 at 8:40 PM David Gow davidgow@google.com wrote:
The kunit_tool documentation page was pretty minimal, and a bit outdated. Update it and flesh it out a bit.
In particular,
- Mention that .kunitconfig is now in the build directory
- Describe the use of --kunitconfig to specify a different config framgent
- Mention the split functionality (i.e., commands other than 'run')
- Describe --raw_output and kunit.py parse
- Mention the globbing support
- Provide a quick overview of other options, including --build_dir and --alltests
Note that this does overlap a little with the new running_tips page. I don't think it's a problem having both: this page is supposed to be a bit more of a reference, rather than a list of useful tips, so the fact that they both describe the same features isn't a problem.
Signed-off-by: David Gow davidgow@google.com
Reviewed-by: Daniel Latypov dlatypov@google.com
Looks good to me. I completely forgot to update this page when adding blobs about the new features in running_tips.rst... Two minor, optional nits.
Documentation/dev-tools/kunit/kunit-tool.rst | 132 ++++++++++++++++++- 1 file changed, 128 insertions(+), 4 deletions(-)
diff --git a/Documentation/dev-tools/kunit/kunit-tool.rst b/Documentation/dev-tools/kunit/kunit-tool.rst index 29ae2fee8123..0b45affcd65c 100644 --- a/Documentation/dev-tools/kunit/kunit-tool.rst +++ b/Documentation/dev-tools/kunit/kunit-tool.rst @@ -22,14 +22,19 @@ not require any virtualization support: it is just a regular program. What is a .kunitconfig? =======================
-It's just a defconfig that kunit_tool looks for in the base directory. +It's just a defconfig that kunit_tool looks for in the build directory.
nit: should we mention this is .kunit here? We don't mention this till the very bottom right now, which seems suboptimal.
I assume most people are going to still be fiddling with that .kunitconfig rather than passing in --kunitconfig (I know I am).
kunit_tool uses it to generate a .config as you might expect. In addition, it verifies that the generated .config contains the CONFIG options in the .kunitconfig; the reason it does this is so that it is easy to be sure that a CONFIG that enables a test actually ends up in the .config.
-How do I use kunit_tool?
+It's also possible to pass a separate .kunitconfig fragment to kunit_tool, +which is useful if you have several different groups of tests you wish +to run independently, or if you want to use pre-defined test configs for +certain subsystems.
+Getting Started with kunit_tool +===============================
If a kunitconfig is present at the root directory, all you have to do is:
@@ -48,10 +53,129 @@ However, you most likely want to use it with the following options:
.. note:: This command will work even without a .kunitconfig file: if no
.kunitconfig is present, a default one will be used instead.
.kunitconfig is present, a default one will be used instead.+If you wish to use a different .kunitconfig file (such as one provided for +testing a particular subsystem), you can pass it as an option.
+.. code-block:: bash
./tools/testing/kunit/kunit.py run --kunitconfig=fs/ext4/.kunitconfigFor a list of all the flags supported by kunit_tool, you can run:
.. code-block:: bash
./tools/testing/kunit/kunit.py run --help
+Configuring, Building, and Running Tests +========================================
+It's also possible to run just parts of the KUnit build process independently, +which is useful if you want to make manual changes to part of the process.
+A .config can be generated from a .kunitconfig by using the ``config`` argument +when running kunit_tool:
+.. code-block:: bash
./tools/testing/kunit/kunit.py config+Similarly, if you just want to build a KUnit kernel from the current .config, +you can use the ``build`` argument:
+.. code-block:: bash
./tools/testing/kunit/kunit.py build+And, if you already have a built UML kernel with built-in KUnit tests, you can +run the kernel and display the test results with the ``exec`` argument:
+.. code-block:: bash
./tools/testing/kunit/kunit.py exec+The ``run`` command which is discussed above is equivalent to running all three +of these in sequence.
+All of these commands accept a number of optional command-line arguments. The +``--help`` flag will give a complete list of these, or keep reading this page +for a guide to some of the more useful ones.
+Parsing Test Results +====================
+KUnit tests output their results in TAP (Test Anything Protocol) format. +kunit_tool will, when running tests, parse this output and print a summary +which is much more pleasant to read. If you wish to look at the raw test +results in TAP format, you can pass the ``--raw_output`` argument.
+.. code-block:: bash
./tools/testing/kunit/kunit.py run --raw_output+.. note::
The raw output from test runs may contain other, non-KUnit kernel loglines.+If you have KUnit results in their raw TAP format, you can parse them and print +the human-readable summary with the ``parse`` command for kunit_tool. This +accepts a filename for an argument, or will read from standard input.
+.. code-block:: bash
# Reading from a file./tools/testing/kunit/kunit.py parse /var/log/dmesg# Reading from stdindmesg | ./tools/testing/kunit/kunit.py parse+This is very useful if you wish to run tests in a configuration not supported +by kunit_tool (such as on real hardware, or an unsupported architecture).
+Filtering Tests +===============
+It's possible to run only a subset of the tests built into a kernel by passing +a filter to the ``exec`` or ``run`` commands. For example, if you only wanted +to run KUnit resource tests, you could use:
+.. code-block:: bash
./tools/testing/kunit/kunit.py run 'kunit-resource*'+This uses the standard glob format for wildcards.
+Other Useful Options +====================
+kunit_tool has a number of other command-line arguments which can be useful +when adapting it to fit your environment or needs.
+Some of the more useful ones are:
+``--help``
Lists all of the available options. Note that different commands(``config``, ``build``, ``run``, etc) will have different supportedoptions. Place ``--help`` before the command to list common options,and after the command for options specific to that command.+``--build_dir``
Specifies the build directory that kunit_tool will use. This is wherethe .kunitconfig file is located, as well as where the .config andcompiled kernel will be placed. Defaults to ``.kunit``.+``--make_options``
Specifies additional options to pass to ``make`` when compiling akernel (with the ``build`` or ``run`` commands). For example, to enablecompiler warnings, you can pass ``--make_options W=1``.+``--alltests``
Builds a UML kernel with all config options enabled using``make allyesconfig``. This allows you to run as many tests as ispossible, but is very slow and prone to breakage as new options areadded or modified. Most people should add ``CONFIG_KUNIT_ALL_TESTS=1``to their .kunitconfig instead if they wish to run "all tests".
Should we note here that CONFIG_KUNIT_ALL_TESTS will not select tests that have unmet dependencies?
+There are several other options (and new ones are often added), so do check
+``--help`` if you're looking for something not mentioned here.
2.31.1.368.gbe11c130af-goog
linux-kselftest-mirror@lists.linaro.org
-
Daniel Latypov -
David Gow