On Mon, Feb 10, 2025 at 3:20 PM Yury Norov yury.norov@gmail.com wrote:
On Mon, Feb 10, 2025 at 11:35:48AM -0800, John Hubbard wrote:
On 2/9/25 11:54 PM, Geert Uytterhoeven wrote:
On Sat, 8 Feb 2025 at 18:53, Yury Norov yury.norov@gmail.com wrote:
On Fri, Feb 07, 2025 at 03:14:01PM -0500, Tamir Duberstein wrote:
On 7/27/24 12:35 AM, Shuah Khan wrote:
...
The crux of the argument seems to be that the config help text is taken to describe the author's intent with the fragment "at boot". I think
IMO, "at boot" is a misnomer, as most tests can be either builtin or modular.
Right.
KUNIT is disabled in defconfig, at least on x86_64. It is also disabled on my Ubuntu 24.04 machine. If I take your patches, I'll be unable to
OK so I just bought a shiny new test machine, and installed one of the big name distros on it, hoping they've moved ahead and bought into the kunit story...
$ grep KUNIT /boot/config-6.8.0-52-generic # CONFIG_KUNIT is not set
...gagghh! No such luck. One more data point, in support of Yuri's complaint. :)
I think distros should start setting CONFIG_KUNIT=m.
Yes they should! kunit really does have important advantages for many use cases, including bitmaps here, and "CONFIG_KUNIT is not set" is the main obstacle.
Hi John, Geert, Tamir,
Can you please explain in details which advantages KUNIT brings to the test_bitmap.c, find_bit_benchmark.c and other low-level tests?
I can try, but I'm not the expert, and David Gow can probably elaborate further.
As I understand it the main benefit of KUnit is standardization and speed (and standardization _is_ speed). KUnit makes it very easy for me, a person who has not previously contributed to any of the bitmap code, to run those tests, and it requires zero configuration, it all just works. It's basically just `tools/testing/kunit/kunit.py run bitmap`, and I get the test results in a human-readable format. The same benefit applies on the author side: test facilities are standardized, so once you get to know the tools, all the tests start to look the same: you can jump in and contribute without having to first learn the so-called local "testing framework".
The important part is that this all applies to ~all other tests written in KUnit. I can even run them *all* trivially: `tools/testing/kunit/kunit.py run`.
Anecdotally I've also noticed there are bots running those KUnit tests e.g. see https://lore.kernel.org/all/20250207-blackholedev-kunit-convert-v1-1-8ef0dc1... where a test I converted was immediately flagged by a robot as having dubious type coercion.
None of these are must-haves, they are just (to me) a nice way to make the kernel more approachable for new contributors.
As for pure benchmarks like find_bit_benchmark.c and test_bitmap.c::`test_bitmap_{read,write}_perf` specifically: I believe the benefits are super limited or even negative: AFAIK KUnit is designed to generally suppress output (in the userspace reporter, not in the kernel) unless a test fails, so I wouldn't hurry to use it for these.
I'm not strongly against moving under KUNIT's hat, but I do:
- respect commitment of my contributors, so I don't want to wipe git history for no serious reason;
- respect time of my testers, so no extra dependencies;
- respect time of reviewers.
These are valid concerns. Certainly the testing case is the most compelling and folks are clearly interested in lowering those barriers. I don't have any influence in this area, but I am grateful to John for starting the conversation.
As I mentioned in the previous thread: I think we could keep `test_bitmap_{read,write}_perf` in test_bitmap.c and get the best of both worlds. WDYT?
Tamir,
If it comes to v2, can you please begin your series with an exhaustive and clear answer to the following questions:
- What do the tests miss now?
- What do _you_ need from the tests? Describe your test scenario.
- How exactly KUNIT helps you testing bitmaps and friends better?
- Is there a way to meet your needs with a less invasive approach, particularly without run-time dependencies?
Hopefully I've answered these above. I can include some of it in a v2, but perhaps the general pitch for KUnit is better placed in documentation or slides from a conference?
Cheers.
Tamir