[PATCH RESEND 3/7] Documentation: kunit: rewrite the rest of "Getting Started" documentation

Rewrite the rest of "Getting Started" documentation for clarity. Major points:

* link to "Build directory for the kernel" section of kernel README for the caveats. * To change kernel configurations in .kunitconfig, use configuration interfaces just like configuring normal kernel. * Remove "Next steps" section as it will be placed on the introduction page.

Signed-off-by: Bagas Sanjaya bagasdotme@gmail.com --- Documentation/admin-guide/README.rst | 2 + Documentation/dev-tools/kunit/start.rst | 210 ++++++++++-------------- 2 files changed, 93 insertions(+), 119 deletions(-)

diff --git a/Documentation/admin-guide/README.rst b/Documentation/admin-guide/README.rst index 9a969c0157f1e5..f3cdc8496da03f 100644 --- a/Documentation/admin-guide/README.rst +++ b/Documentation/admin-guide/README.rst @@ -123,6 +123,8 @@ Software requirements you can just update packages when obvious problems arise during build or operation.

+.. _kernel-build-directory: + Build directory for the kernel ------------------------------

diff --git a/Documentation/dev-tools/kunit/start.rst b/Documentation/dev-tools/kunit/start.rst index cdf043b6550e66..1195d5de53343e 100644 --- a/Documentation/dev-tools/kunit/start.rst +++ b/Documentation/dev-tools/kunit/start.rst @@ -4,178 +4,164 @@ Getting Started ===============

-This page contains an overview of the kunit_tool and KUnit framework, -teaching how to run existing tests and then how to write a simple test case, -and covers common problems users face when using KUnit for the first time. +This guide is an overview of KUnit framework. It teaches how to run tests +with kunit_tool as well as writing a simple test case.

-Installing Dependencies -======================= -KUnit has the same dependencies as the Linux kernel. As long as you can -build the kernel, you can run KUnit. +Prerequisites +============= +No extra dependencies are required in order to use KUnit. See +:doc:`/process/changes` for details.

Running tests with kunit_tool ============================= -kunit_tool is a Python script, which configures and builds a kernel, runs -tests, and formats the test results. From the kernel repository, you -can run kunit_tool: +kunit_tool is a wrapper script written in Python. It configures and builds +the kernel, runs tests, and formats the results. To run the script:

.. code-block:: bash

./tools/testing/kunit/kunit.py run

.. note :: - You may see the following error: - "The source tree is not clean, please run 'make ARCH=um mrproper'" + You may see the following error::

- This happens because internally kunit.py specifies ``.kunit`` - (default option) as the build directory in the command ``make O=output/dir`` - through the argument ``--build_dir``. Hence, before starting an - out-of-tree build, the source tree must be clean. + The source tree is not clean, please run 'make ARCH=um mrproper'

- There is also the same caveat mentioned in the "Build directory for - the kernel" section of the :doc:`admin-guide </admin-guide/README>`, - that is, its use, it must be used for all invocations of ``make``. - The good news is that it can indeed be solved by running - ``make ARCH=um mrproper``, just be aware that this will delete the - current configuration and all generated files. + This happens because internally kunit_tool passes the default build + directory ``.kunit`` as environment variable to make (which is invoked + as ``make O=.kunit``). A different build directory can be specified by + passing ``--build_dir`` option. Hence, before starting the build, + the source tree must be clean.

-If everything worked correctly, you should see the following: + The caveat from the :ref:`kernel-build-directory` + also applies to running kunit_tool.

-.. code-block:: +If everything worked correctly, you should see the following output::

Configuring KUnit Kernel ... Building KUnit Kernel ... Starting KUnit Kernel ...

-The tests will pass or fail. +.. note :: + Depending on configurations enabled in the kernel, the build process + may take a while. + +See Documentation/dev-tools/kunit/run_wrapper.rst for details. + +Selecting tests +--------------- + +By default, kunit_tool runs all tests using default configuration (defconfig). +The following subsections allow you to customize the configuration as well as +filtering the test that will be run.

.. note :: - Because it is building a lot of sources for the first time, - the ``Building KUnit Kernel`` step may take a while. + KUnit by default runs tests for UML architecture (``ARCH=um``). If you + need to run test on other architectures see :ref:`kunit-on-qemu`.

-For detailed information on this wrapper, see: -Documentation/dev-tools/kunit/run_wrapper.rst. - -Selecting which tests to run ----------------------------- - -By default, kunit_tool runs all tests reachable with minimal configuration, -that is, using default values for most of the kconfig options. However, -you can select which tests to run by: - -- `Customizing Kconfig`_ used to compile the kernel, or -- `Filtering tests by name`_ to select specifically which compiled tests to run. - -Customizing Kconfig -~~~~~~~~~~~~~~~~~~~ -A good starting point for the ``.kunitconfig`` is the KUnit default config. -If you didn't run ``kunit.py run`` yet, you can generate it by running: +Kernel configuration +~~~~~~~~~~~~~~~~~~~~ +A good starting point for the ``.kunitconfig`` is the KUnit default config, +which can be generated by:

.. code-block:: bash

- cd $PATH_TO_LINUX_REPO - tools/testing/kunit/kunit.py config - cat .kunit/.kunitconfig + ./tools/testing/kunit/kunit.py config

.. note :: - ``.kunitconfig`` lives in the ``--build_dir`` used by kunit.py, which is - ``.kunit`` by default. + ``.kunitconfig`` is located in the directory specified by ``--build_dir`` + argument. The default build directory is ``.kunit``.

-Before running the tests, kunit_tool ensures that all config options -set in ``.kunitconfig`` are set in the kernel ``.config``. It will warn -you if you have not included dependencies for the options used. +Before running the tests, kunit_tool ensures that all options set in +``.kunitconfig`` are set in the ``.config`` used for kernel build process. It +will errored if you have not enabled option dependencies for all selected +options.

There are many ways to customize the configurations:

-a. Edit ``.kunit/.kunitconfig``. The file should contain the list of kconfig +a. Edit the configuration. ``.kunitconfig`` should contains configuration options required to run the desired tests, including their dependencies. - You may want to remove CONFIG_KUNIT_ALL_TESTS from the ``.kunitconfig`` as - it will enable a number of additional tests that you may not want. - If you need to run on an architecture other than UML see :ref:`kunit-on-qemu`. - -b. Enable additional kconfig options on top of ``.kunit/.kunitconfig``. - For example, to include the kernel's linked-list test you can run:: + As with other kernel configurations, it is recommended to use :doc:`one of + configuration interfaces </kbuild/kconfig>`, passing + ``KCONFIG_CONFIG=.kunit/.kunitconfig`` target variable to make. You may + want to disable ``CONFIG_KUNIT_ALL_TESTS`` as it will enable a number of + additional tests that you may not want to be run. + +b. Add options with ``--kconfig_add``. For example, to include the + linked-list test you can run::

./tools/testing/kunit/kunit.py run \ --kconfig_add CONFIG_LIST_KUNIT_TEST=y

-c. Provide the path of one or more .kunitconfig files from the tree. - For example, to run only ``FAT_FS`` and ``EXT4`` tests you can run:: +c. Pass the path to one or more .kunitconfig files from the tree. + For example, to run only ``FAT_FS`` and ``EXT4`` tests::

./tools/testing/kunit/kunit.py run \ --kunitconfig ./fs/fat/.kunitconfig \ --kunitconfig ./fs/ext4/.kunitconfig

-d. If you change the ``.kunitconfig``, kunit.py will trigger a rebuild of the - ``.config`` file. But you can edit the ``.config`` file directly or with - tools like ``make menuconfig O=.kunit``. As long as its a superset of - ``.kunitconfig``, kunit.py won't overwrite your changes. - - .. note ::

- To save a .kunitconfig after finding a satisfactory configuration:: - - make savedefconfig O=.kunit - cp .kunit/defconfig .kunit/.kunitconfig + If you change the ``.kunitconfig``, kunit_tool will resynchronize + ``.config``, which can be edited. As long as it is a superset of + ``.kunitconfig``, kunit.py won't overwrite your changes in ``.config``

Filtering tests by name ~~~~~~~~~~~~~~~~~~~~~~~ -If you want to be more specific than Kconfig can provide, it is also possible -to select which tests to execute at boot-time by passing a glob filter -(read instructions regarding the pattern in the manpage :manpage:`glob(7)`). -If there is a ``"."`` (period) in the filter, it will be interpreted as a -separator between the name of the test suite and the test case, -otherwise, it will be interpreted as the name of the test suite. -For example, let's assume we are using the default config: +It is also possible to select which tests to execute at boot-time by passing +a glob filter (see :manpage:`glob(7)` for details). +If there is a period (``.``) in the filter, it will be interpreted as a +separator between the name of the test suite and the test case; otherwise it +will be interpreted as the name of the test suite.

-a. inform the name of a test suite, like ``"kunit_executor_test"``, - to run every test case it contains:: +The complete ways to specify the tests are: + +a. Pass the test suite name to run every case in that test::

./tools/testing/kunit/kunit.py run "kunit_executor_test"

-b. inform the name of a test case prefixed by its test suite, - like ``"example.example_simple_test"``, to run specifically that test case:: +b. Pass the fully qualified name of test case to run the specific case::

./tools/testing/kunit/kunit.py run "example.example_simple_test"

-c. use wildcard characters (``*?[``) to run any test case that matches the pattern, - like ``"*.*64*"`` to run test cases containing ``"64"`` in the name inside - any test suite:: +c. Use wildcard characters (``*?[``) to run any test case that matches the + pattern. For example, ``"*.*64*"`` matches test case names that contain + "64" across any test suite::

./tools/testing/kunit/kunit.py run "*.*64*"

-Running Tests without the KUnit Wrapper -======================================= -If you do not want to use the KUnit Wrapper (for example: you want code -under test to integrate with other systems, or use a different/ +Running tests without kunit_tool +================================ +If you do not want to use kunit_tool (e.g. you want the tested code +to be integrated with other systems, or use different/ unsupported architecture or configuration), KUnit can be included in -any kernel, and the results are read out and parsed manually. +any kernel code. The results can then be read out and parsed manually.

.. note :: ``CONFIG_KUNIT`` should not be enabled in a production environment. Enabling KUnit disables Kernel Address-Space Layout Randomization - (KASLR), and tests may affect the state of the kernel in ways not - suitable for production. + (KASLR) and tests may affect the state of the kernel in ways not + suitable for production (like performance issues).

-Configuring the Kernel +Configuring the kernel ---------------------- -To enable KUnit itself, you need to enable the ``CONFIG_KUNIT`` Kconfig -option (under Kernel Hacking/Kernel Testing and Coverage in -``menuconfig``). From there, you can enable any KUnit tests. They -usually have config options ending in ``_KUNIT_TEST``. +To enable KUnit itself, you need to enable the ``CONFIG_KUNIT`` +option (the entry is under +:menuselection:`Kernel Hacking -> Kernel Testing and Coverage`). +You can enable any tests that you want to be run. These +configuration usually have ``_KUNIT_TEST`` suffix.

-KUnit and KUnit tests can be compiled as modules. The tests in a module -will run when the module is loaded. +Just like any other configuration options, KUnit and KUnit tests can be +built as modules. The tests in the module will be run when the module is +loaded.

-Running Tests (without KUnit Wrapper) -------------------------------------- -Build and run your kernel. In the kernel log, the test output is printed -out in the TAP format. This will only happen by default if KUnit/tests -are built-in. Otherwise the module will need to be loaded. +Running tests +------------- +Build and run your kernel as usual. If KUnit and the tests are built as +modules, these need to be loaded first. The test results are logged (printed +out) in the TAP format.

.. note :: - Some lines and/or data may get interspersed in the TAP output. + Some lines and/or data may get interspersed in the output.

Writing Your First Test ======================= @@ -298,17 +284,3 @@ You should see the following output: ...

Congrats! You have just written your first KUnit test. - -Next Steps -========== - -* Documentation/dev-tools/kunit/architecture.rst - KUnit architecture. -* Documentation/dev-tools/kunit/run_wrapper.rst - run kunit_tool. -* Documentation/dev-tools/kunit/run_manual.rst - run tests without kunit_tool. -* Documentation/dev-tools/kunit/usage.rst - write tests. -* Documentation/dev-tools/kunit/tips.rst - best practices with - examples. -* Documentation/dev-tools/kunit/api/index.rst - KUnit APIs - used for testing. -* Documentation/dev-tools/kunit/faq.rst - KUnit common questions and - answers.