- Linux-kselftest-mirror - lists.linaro.org

[PATCH 0/3] kunit: support running subsets of test suites from

by Daniel Latypov

When using `kunit.py run` to run tests, users must populate a `kunitconfig` file to select the options the tests are hidden behind and all their dependencies. The patch [1] to allow specifying a path to kunitconfig promises to make this nicer as we can have checked in files corresponding to different sets of tests. But it's still annoying 1) when trying to run a subet of tests 2) when you want to run tests that don't have such a pre-existing kunitconfig and selecting all the necessary options is tricky. This patch series aims to alleviate both: 1) `kunit.py run 'my-suite-*'` I.e. use my current kunitconfig, but just run suites that match this glob 2) `kunit.py run --alltests 'my-suite-*'` I.e. use allyesconfig so I don't have to worry about writing a kunitconfig at all (this is a bit overkill, but it works!) See the first commit message for more details and discussion about future work. This patch series also includes a bugfix for a latent bug that can't be triggered right now but has worse consequences as a result of the changes needed to plumb in this suite name glob. [1] https://lore.kernel.org/linux-kselftest/20210201205514.3943096-1-dlatypov@g… Daniel Latypov (3): kunit: add kunit.filter_glob cmdline option to filter suites kunit: tool: add support for filtering suites by glob kunit: tool: fix unintentional statefulness in run_kernel() lib/kunit/Kconfig | 1 + lib/kunit/executor.c | 85 ++++++++++++++++++++++++++--- tools/testing/kunit/kunit.py | 21 +++++-- tools/testing/kunit/kunit_kernel.py | 6 +- 4 files changed, 99 insertions(+), 14 deletions(-) base-commit: 88bb507a74ea7d75fa49edd421eaa710a7d80598 -- 2.30.0.365.g02bc693789-goog

4 years, 7 months

2
6
0 0

[PATCH 1/2] KVM: selftests: Keep track of memslots more efficiently

by Maciej S. Szmigiero

From: "Maciej S. Szmigiero" <maciej.szmigiero(a)oracle.com> The KVM selftest framework was using a simple list for keeping track of the memslots currently in use. This resulted in lookups and adding a single memslot being O(n), the later due to linear scanning of the existing memslot set to check for the presence of any conflicting entries. Before this change, benchmarking high count of memslots was more or less impossible as pretty much all the benchmark time was spent in the selftest framework code. We can simply use a rbtree for keeping track of both of gfn and hva. We don't need an interval tree for hva here as we can't have overlapping memslots because we allocate a completely new memory chunk for each new memslot. Signed-off-by: Maciej S. Szmigiero <maciej.szmigiero(a)oracle.com> --- tools/testing/selftests/kvm/Makefile | 2 +- tools/testing/selftests/kvm/lib/kvm_util.c | 141 ++++++++++++++---- .../selftests/kvm/lib/kvm_util_internal.h | 15 +- tools/testing/selftests/kvm/lib/rbtree.c | 1 + 4 files changed, 124 insertions(+), 35 deletions(-) create mode 100644 tools/testing/selftests/kvm/lib/rbtree.c diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile index fe41c6a0fa67..e7c6237d7383 100644 --- a/tools/testing/selftests/kvm/Makefile +++ b/tools/testing/selftests/kvm/Makefile @@ -33,7 +33,7 @@ ifeq ($(ARCH),s390) UNAME_M := s390x endif -LIBKVM = lib/assert.c lib/elf.c lib/io.c lib/kvm_util.c lib/sparsebit.c lib/test_util.c lib/guest_modes.c lib/perf_test_util.c +LIBKVM = lib/assert.c lib/elf.c lib/io.c lib/kvm_util.c lib/rbtree.c lib/sparsebit.c lib/test_util.c lib/guest_modes.c lib/perf_test_util.c LIBKVM_x86_64 = lib/x86_64/processor.c lib/x86_64/vmx.c lib/x86_64/svm.c lib/x86_64/ucall.c lib/x86_64/handlers.S LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c LIBKVM_s390x = lib/s390x/processor.c lib/s390x/ucall.c lib/s390x/diag318_test_handler.c diff --git a/tools/testing/selftests/kvm/lib/kvm_util.c b/tools/testing/selftests/kvm/lib/kvm_util.c index fa5a90e6c6f0..632433dbfa25 100644 --- a/tools/testing/selftests/kvm/lib/kvm_util.c +++ b/tools/testing/selftests/kvm/lib/kvm_util.c @@ -195,7 +195,9 @@ struct kvm_vm *vm_create(enum vm_guest_mode mode, uint64_t phy_pages, int perm) TEST_ASSERT(vm != NULL, "Insufficient Memory"); INIT_LIST_HEAD(&vm->vcpus); - INIT_LIST_HEAD(&vm->userspace_mem_regions); + vm->regions.gpa_tree = RB_ROOT; + vm->regions.hva_tree = RB_ROOT; + hash_init(vm->regions.slot_hash); vm->mode = mode; vm->type = 0; @@ -347,13 +349,14 @@ struct kvm_vm *vm_create_default(uint32_t vcpuid, uint64_t extra_mem_pages, */ void kvm_vm_restart(struct kvm_vm *vmp, int perm) { + int ctr; struct userspace_mem_region *region; vm_open(vmp, perm); if (vmp->has_irqchip) vm_create_irqchip(vmp); - list_for_each_entry(region, &vmp->userspace_mem_regions, list) { + hash_for_each(vmp->regions.slot_hash, ctr, region, slot_node) { int ret = ioctl(vmp->fd, KVM_SET_USER_MEMORY_REGION, &region->region); TEST_ASSERT(ret == 0, "KVM_SET_USER_MEMORY_REGION IOCTL failed,\n" " rc: %i errno: %i\n" @@ -416,14 +419,21 @@ uint32_t kvm_vm_reset_dirty_ring(struct kvm_vm *vm) static struct userspace_mem_region * userspace_mem_region_find(struct kvm_vm *vm, uint64_t start, uint64_t end) { - struct userspace_mem_region *region; + struct rb_node *node; - list_for_each_entry(region, &vm->userspace_mem_regions, list) { + for (node = vm->regions.gpa_tree.rb_node; node; ) { + struct userspace_mem_region *region = + container_of(node, struct userspace_mem_region, gpa_node); uint64_t existing_start = region->region.guest_phys_addr; uint64_t existing_end = region->region.guest_phys_addr + region->region.memory_size - 1; if (start <= existing_end && end >= existing_start) return region; + + if (start < existing_start) + node = node->rb_left; + else + node = node->rb_right; } return NULL; @@ -538,11 +548,16 @@ void kvm_vm_release(struct kvm_vm *vmp) } static void __vm_mem_region_delete(struct kvm_vm *vm, - struct userspace_mem_region *region) + struct userspace_mem_region *region, + bool unlink) { int ret; - list_del(&region->list); + if (unlink) { + rb_erase(&region->gpa_node, &vm->regions.gpa_tree); + rb_erase(&region->hva_node, &vm->regions.hva_tree); + hash_del(&region->slot_node); + } region->region.memory_size = 0; ret = ioctl(vm->fd, KVM_SET_USER_MEMORY_REGION, &region->region); @@ -561,14 +576,16 @@ static void __vm_mem_region_delete(struct kvm_vm *vm, */ void kvm_vm_free(struct kvm_vm *vmp) { - struct userspace_mem_region *region, *tmp; + int ctr; + struct hlist_node *node; + struct userspace_mem_region *region; if (vmp == NULL) return; /* Free userspace_mem_regions. */ - list_for_each_entry_safe(region, tmp, &vmp->userspace_mem_regions, list) - __vm_mem_region_delete(vmp, region); + hash_for_each_safe(vmp->regions.slot_hash, ctr, node, region, slot_node) + __vm_mem_region_delete(vmp, region, false); /* Free sparsebit arrays. */ sparsebit_free(&vmp->vpages_valid); @@ -650,6 +667,57 @@ int kvm_memcmp_hva_gva(void *hva, struct kvm_vm *vm, vm_vaddr_t gva, size_t len) return 0; } +static void vm_userspace_mem_region_gpa_insert(struct rb_root *gpa_tree, + struct userspace_mem_region *region) +{ + struct rb_node **cur, *parent; + + for (cur = &gpa_tree->rb_node, parent = NULL; *cur; ) { + struct userspace_mem_region *cregion; + + cregion = container_of(*cur, typeof(*cregion), gpa_node); + parent = *cur; + if (region->region.guest_phys_addr < + cregion->region.guest_phys_addr) + cur = &(*cur)->rb_left; + else { + TEST_ASSERT(region->region.guest_phys_addr != + cregion->region.guest_phys_addr, + "Duplicate GPA in region tree"); + + cur = &(*cur)->rb_right; + } + } + + rb_link_node(&region->gpa_node, parent, cur); + rb_insert_color(&region->gpa_node, gpa_tree); +} + +static void vm_userspace_mem_region_hva_insert(struct rb_root *hva_tree, + struct userspace_mem_region *region) +{ + struct rb_node **cur, *parent; + + for (cur = &hva_tree->rb_node, parent = NULL; *cur; ) { + struct userspace_mem_region *cregion; + + cregion = container_of(*cur, typeof(*cregion), hva_node); + parent = *cur; + if (region->host_mem < cregion->host_mem) + cur = &(*cur)->rb_left; + else { + TEST_ASSERT(region->host_mem != + cregion->host_mem, + "Duplicate HVA in region tree"); + + cur = &(*cur)->rb_right; + } + } + + rb_link_node(&region->hva_node, parent, cur); + rb_insert_color(&region->hva_node, hva_tree); +} + /* * VM Userspace Memory Region Add * @@ -714,7 +782,8 @@ void vm_userspace_mem_region_add(struct kvm_vm *vm, (uint64_t) region->region.memory_size); /* Confirm no region with the requested slot already exists. */ - list_for_each_entry(region, &vm->userspace_mem_regions, list) { + hash_for_each_possible(vm->regions.slot_hash, region, slot_node, + slot) { if (region->region.slot != slot) continue; @@ -794,8 +863,10 @@ void vm_userspace_mem_region_add(struct kvm_vm *vm, ret, errno, slot, flags, guest_paddr, (uint64_t) region->region.memory_size); - /* Add to linked-list of memory regions. */ - list_add(&region->list, &vm->userspace_mem_regions); + /* Add to quick lookup data structures */ + vm_userspace_mem_region_gpa_insert(&vm->regions.gpa_tree, region); + vm_userspace_mem_region_hva_insert(&vm->regions.hva_tree, region); + hash_add(vm->regions.slot_hash, &region->slot_node, slot); } /* @@ -818,10 +889,10 @@ memslot2region(struct kvm_vm *vm, uint32_t memslot) { struct userspace_mem_region *region; - list_for_each_entry(region, &vm->userspace_mem_regions, list) { + hash_for_each_possible(vm->regions.slot_hash, region, slot_node, + memslot) if (region->region.slot == memslot) return region; - } fprintf(stderr, "No mem region with the requested slot found,\n" " requested slot: %u\n", memslot); @@ -906,7 +977,7 @@ void vm_mem_region_move(struct kvm_vm *vm, uint32_t slot, uint64_t new_gpa) */ void vm_mem_region_delete(struct kvm_vm *vm, uint32_t slot) { - __vm_mem_region_delete(vm, memslot2region(vm, slot)); + __vm_mem_region_delete(vm, memslot2region(vm, slot), true); } /* @@ -1178,16 +1249,14 @@ void *addr_gpa2hva(struct kvm_vm *vm, vm_paddr_t gpa) { struct userspace_mem_region *region; - list_for_each_entry(region, &vm->userspace_mem_regions, list) { - if ((gpa >= region->region.guest_phys_addr) - && (gpa <= (region->region.guest_phys_addr - + region->region.memory_size - 1))) - return (void *) ((uintptr_t) region->host_mem - + (gpa - region->region.guest_phys_addr)); + region = userspace_mem_region_find(vm, gpa, gpa); + if (!region) { + TEST_FAIL("No vm physical memory at 0x%lx", gpa); + return NULL; } - TEST_FAIL("No vm physical memory at 0x%lx", gpa); - return NULL; + return (void *)((uintptr_t)region->host_mem + + (gpa - region->region.guest_phys_addr)); } /* @@ -1209,15 +1278,22 @@ void *addr_gpa2hva(struct kvm_vm *vm, vm_paddr_t gpa) */ vm_paddr_t addr_hva2gpa(struct kvm_vm *vm, void *hva) { - struct userspace_mem_region *region; + struct rb_node *node; + + for (node = vm->regions.hva_tree.rb_node; node; ) { + struct userspace_mem_region *region = + container_of(node, struct userspace_mem_region, hva_node); + + if (hva >= region->host_mem) { + if (hva <= (region->host_mem + + region->region.memory_size - 1)) + return (vm_paddr_t)((uintptr_t) + region->region.guest_phys_addr + + (hva - (uintptr_t)region->host_mem)); - list_for_each_entry(region, &vm->userspace_mem_regions, list) { - if ((hva >= region->host_mem) - && (hva <= (region->host_mem - + region->region.memory_size - 1))) - return (vm_paddr_t) ((uintptr_t) - region->region.guest_phys_addr - + (hva - (uintptr_t) region->host_mem)); + node = node->rb_right; + } else + node = node->rb_left; } TEST_FAIL("No mapping to a guest physical address, hva: %p", hva); @@ -1743,6 +1819,7 @@ int _kvm_ioctl(struct kvm_vm *vm, unsigned long cmd, void *arg) */ void vm_dump(FILE *stream, struct kvm_vm *vm, uint8_t indent) { + int ctr; struct userspace_mem_region *region; struct vcpu *vcpu; @@ -1750,7 +1827,7 @@ void vm_dump(FILE *stream, struct kvm_vm *vm, uint8_t indent) fprintf(stream, "%*sfd: %i\n", indent, "", vm->fd); fprintf(stream, "%*spage_size: 0x%x\n", indent, "", vm->page_size); fprintf(stream, "%*sMem Regions:\n", indent, ""); - list_for_each_entry(region, &vm->userspace_mem_regions, list) { + hash_for_each(vm->regions.slot_hash, ctr, region, slot_node) { fprintf(stream, "%*sguest_phys: 0x%lx size: 0x%lx " "host_virt: %p\n", indent + 2, "", (uint64_t) region->region.guest_phys_addr, diff --git a/tools/testing/selftests/kvm/lib/kvm_util_internal.h b/tools/testing/selftests/kvm/lib/kvm_util_internal.h index 34465dc562d8..af310110602b 100644 --- a/tools/testing/selftests/kvm/lib/kvm_util_internal.h +++ b/tools/testing/selftests/kvm/lib/kvm_util_internal.h @@ -8,6 +8,9 @@ #ifndef SELFTEST_KVM_UTIL_INTERNAL_H #define SELFTEST_KVM_UTIL_INTERNAL_H +#include "linux/hashtable.h" +#include "linux/rbtree.h" + #include "sparsebit.h" #define KVM_DEV_PATH "/dev/kvm" @@ -20,7 +23,9 @@ struct userspace_mem_region { void *host_mem; void *mmap_start; size_t mmap_size; - struct list_head list; + struct rb_node gpa_node; + struct rb_node hva_node; + struct hlist_node slot_node; }; struct vcpu { @@ -33,6 +38,12 @@ struct vcpu { uint32_t dirty_gfns_count; }; +struct userspace_mem_regions { + struct rb_root gpa_tree; + struct rb_root hva_tree; + DECLARE_HASHTABLE(slot_hash, 9); +}; + struct kvm_vm { int mode; unsigned long type; @@ -45,7 +56,7 @@ struct kvm_vm { unsigned int va_bits; uint64_t max_gfn; struct list_head vcpus; - struct list_head userspace_mem_regions; + struct userspace_mem_regions regions; struct sparsebit *vpages_valid; struct sparsebit *vpages_mapped; bool has_irqchip; diff --git a/tools/testing/selftests/kvm/lib/rbtree.c b/tools/testing/selftests/kvm/lib/rbtree.c new file mode 100644 index 000000000000..a703f0194ea3 --- /dev/null +++ b/tools/testing/selftests/kvm/lib/rbtree.c @@ -0,0 +1 @@ +#include "../../../../lib/rbtree.c"

4 years, 7 months

2
4
0 0

[PATCH v3 0/7] Introduce Sequence Number Ops

by Shuah Khan

Sequence Number api provides interfaces for unsigned atomic up counters. There are a number of atomic_t usages in the kernel where atomic_t api is used for counting sequence numbers and other statistical counters. Several of these usages, convert atomic_read() and atomic_inc_return() return values to unsigned. Introducing sequence number ops supports these use-cases with a standard core-api. Sequence Number ops provide interfaces to initialize, increment and get the sequence number. These ops also check for overflow and log message to indicate when overflow occurs. This check is intended to help catch cases where overflow could lead to problems. Since v2: - Uses atomic_inc_return() for incrementing the sequence number. - No longer uses atomic_read() Shuah Khan (7): seqnum_ops: Introduce Sequence Number Ops selftests: lib:test_seqnum_ops: add new test for seqnum_ops drivers/acpi: convert seqno to use seqnum_ops drivers/acpi/apei: convert seqno to seqnum_ops drivers/staging/rtl8723bs: convert event_seq to use seqnum_ops drivers/staging/rtl8188eu: convert event_seq to use seqnum_ops kobject: convert uevent_seqnum to seqnum_ops Documentation/core-api/index.rst | 1 + Documentation/core-api/seqnum_ops.rst | 62 ++++++++ MAINTAINERS | 8 ++ drivers/acpi/acpi_extlog.c | 8 +- drivers/acpi/apei/ghes.c | 8 +- drivers/staging/rtl8188eu/core/rtw_mlme_ext.c | 23 ++- .../staging/rtl8188eu/include/rtw_mlme_ext.h | 3 +- drivers/staging/rtl8723bs/core/rtw_cmd.c | 3 +- drivers/staging/rtl8723bs/core/rtw_mlme_ext.c | 33 +++-- drivers/staging/rtl8723bs/include/rtw_cmd.h | 3 +- .../staging/rtl8723bs/include/rtw_mlme_ext.h | 3 +- include/linux/kobject.h | 3 +- include/linux/seqnum_ops.h | 131 +++++++++++++++++ kernel/ksysfs.c | 3 +- lib/Kconfig | 9 ++ lib/Makefile | 1 + lib/kobject_uevent.c | 9 +- lib/test_seqnum_ops.c | 133 ++++++++++++++++++ tools/testing/selftests/lib/Makefile | 1 + tools/testing/selftests/lib/config | 1 + .../testing/selftests/lib/test_seqnum_ops.sh | 10 ++ 21 files changed, 423 insertions(+), 33 deletions(-) create mode 100644 Documentation/core-api/seqnum_ops.rst create mode 100644 include/linux/seqnum_ops.h create mode 100644 lib/test_seqnum_ops.c create mode 100755 tools/testing/selftests/lib/test_seqnum_ops.sh -- 2.27.0

4 years, 7 months

6
13
0 0

[PATCH v3 0/3] kunit: support running subsets of test suites from kunit.py

by Daniel Latypov

When using `kunit.py run` to run tests, users must populate a `kunitconfig` file to select the options the tests are hidden behind and all their dependencies. The patch [1] to allow specifying a path to kunitconfig promises to make this nicer as we can have checked in files corresponding to different sets of tests. But it's still annoying 1) when trying to run a subet of tests 2) when you want to run tests that don't have such a pre-existing kunitconfig and selecting all the necessary options is tricky. This patch series aims to alleviate both: 1) `kunit.py run 'my-suite-*'` I.e. use my current kunitconfig, but just run suites that match this glob 2) `kunit.py run --alltests 'my-suite-*'` I.e. use allyesconfig so I don't have to worry about writing a kunitconfig at all. See the first commit message for more details and discussion about future work. This patch series also includes a bugfix for a latent bug that can't be triggered right now but has worse consequences as a result of the changes needed to plumb in this suite name glob. [1] https://lore.kernel.org/linux-kselftest/20210201205514.3943096-1-dlatypov@g… --- v1 -> v2: Fix free of `suites` subarray in suite_set. Found by Dan Carpenter and kernel test robot. v2 -> v3: Add MODULE_PARM_DESC() for kunit.filter_glob. Daniel Latypov (3): kunit: add kunit.filter_glob cmdline option to filter suites kunit: tool: add support for filtering suites by glob kunit: tool: fix unintentional statefulness in run_kernel() lib/kunit/Kconfig | 1 + lib/kunit/executor.c | 93 ++++++++++++++++++++++++++--- tools/testing/kunit/kunit.py | 21 +++++-- tools/testing/kunit/kunit_kernel.py | 6 +- 4 files changed, 106 insertions(+), 15 deletions(-) base-commit: 88bb507a74ea7d75fa49edd421eaa710a7d80598 -- 2.30.0.478.g8a0d178c01-goog

4 years, 7 months

2
4
0 0

[PATCH v2] kunit: make kunit_tool accept optional path to .kunitconfig fragment

by Daniel Latypov

Currently running tests via KUnit tool means tweaking a .kunitconfig file, which you'd keep around locally and never commit. This changes makes it so users can pass in a path to a kunitconfig. One of the imagined use cases is having kunitconfig fragments in-tree to formalize interesting sets of tests for features/subsystems, e.g. $ ./tools/testing/kunit/kunit.py run --kunticonfig=fs/ext4/kunitconfig For now, this hypothetical fs/ext4/kunitconfig would contain CONFIG_KUNIT=y CONFIG_EXT4_FS=y CONFIG_EXT4_KUNIT_TESTS=y At the moment, it's not hard to manually whip up this file, but as more and more tests get added, this will get tedious. It also opens the door to documenting how to run all the tests relevant to a specific subsystem or feature as a simple one-liner. This can be seen as an analogue to tools/testing/selftests/*/config But in the case of KUnit, the tests live in the same directory as the code-under-test, so it feels more natural to allow the kunitconfig fragments to live anywhere. (Though, people could create a separate directory if wanted; this patch imposes no restrictions on the path). Signed-off-by: Daniel Latypov <dlatypov(a)google.com> --- Changes since v1: change from a positional arg to a flag --kunitconfig. Ensure that it gets added for `kunit.py config` and all other commands. --- tools/testing/kunit/kunit.py | 9 +++++--- tools/testing/kunit/kunit_kernel.py | 12 ++++++---- tools/testing/kunit/kunit_tool_test.py | 32 ++++++++++++++++++++++++++ 3 files changed, 46 insertions(+), 7 deletions(-) diff --git a/tools/testing/kunit/kunit.py b/tools/testing/kunit/kunit.py index e808a47c839b..02871a363f76 100755 --- a/tools/testing/kunit/kunit.py +++ b/tools/testing/kunit/kunit.py @@ -182,6 +182,9 @@ def add_common_opts(parser) -> None: parser.add_argument('--alltests', help='Run all KUnit tests through allyesconfig', action='store_true') + parser.add_argument('--kunitconfig', + help='Path to Kconfig fragment that enables KUnit tests', + metavar='kunitconfig') def add_build_opts(parser) -> None: parser.add_argument('--jobs', @@ -256,7 +259,7 @@ def main(argv, linux=None): os.mkdir(cli_args.build_dir) if not linux: - linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir) + linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir, kunitconfig_path=cli_args.kunitconfig) request = KunitRequest(cli_args.raw_output, cli_args.timeout, @@ -274,7 +277,7 @@ def main(argv, linux=None): os.mkdir(cli_args.build_dir) if not linux: - linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir) + linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir, kunitconfig_path=cli_args.kunitconfig) request = KunitConfigRequest(cli_args.build_dir, cli_args.make_options) @@ -286,7 +289,7 @@ def main(argv, linux=None): sys.exit(1) elif cli_args.subcommand == 'build': if not linux: - linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir) + linux = kunit_kernel.LinuxSourceTree(cli_args.build_dir, kunitconfig_path=cli_args.kunitconfig) request = KunitBuildRequest(cli_args.jobs, cli_args.build_dir, diff --git a/tools/testing/kunit/kunit_kernel.py b/tools/testing/kunit/kunit_kernel.py index 2076a5a2d060..0b461663e7d9 100644 --- a/tools/testing/kunit/kunit_kernel.py +++ b/tools/testing/kunit/kunit_kernel.py @@ -123,7 +123,7 @@ def get_outfile_path(build_dir) -> str: class LinuxSourceTree(object): """Represents a Linux kernel source tree with KUnit tests.""" - def __init__(self, build_dir: str, load_config=True, defconfig=DEFAULT_KUNITCONFIG_PATH) -> None: + def __init__(self, build_dir: str, load_config=True, kunitconfig_path='') -> None: signal.signal(signal.SIGINT, self.signal_handler) self._ops = LinuxSourceTreeOperations() @@ -131,9 +131,13 @@ class LinuxSourceTree(object): if not load_config: return - kunitconfig_path = get_kunitconfig_path(build_dir) - if not os.path.exists(kunitconfig_path): - shutil.copyfile(defconfig, kunitconfig_path) + if kunitconfig_path: + if not os.path.exists(kunitconfig_path): + raise ConfigError(f'Specified kunitconfig ({kunitconfig_path}) does not exist') + else: + kunitconfig_path = get_kunitconfig_path(build_dir) + if not os.path.exists(kunitconfig_path): + shutil.copyfile(DEFAULT_KUNITCONFIG_PATH, kunitconfig_path) self._kconfig = kunit_config.Kconfig() self._kconfig.read_from_file(kunitconfig_path) diff --git a/tools/testing/kunit/kunit_tool_test.py b/tools/testing/kunit/kunit_tool_test.py index b593f4448e83..22f50b931138 100755 --- a/tools/testing/kunit/kunit_tool_test.py +++ b/tools/testing/kunit/kunit_tool_test.py @@ -12,6 +12,7 @@ from unittest import mock import tempfile, shutil # Handling test_tmpdir import json +import signal import os import kunit_config @@ -250,6 +251,23 @@ class KUnitParserTest(unittest.TestCase): result.status) self.assertEqual('kunit-resource-test', result.suites[0].name) +class LinuxSourceTreeTest(unittest.TestCase): + + def setUp(self): + mock.patch.object(signal, 'signal').start() + self.addCleanup(mock.patch.stopall) + + def test_invalid_kunitconfig(self): + with self.assertRaisesRegex(kunit_kernel.ConfigError, 'nonexistent.* does not exist'): + kunit_kernel.LinuxSourceTree('', kunitconfig_path='/nonexistent_file') + + def test_valid_kunitconfig(self): + with tempfile.NamedTemporaryFile('wt') as kunitconfig: + tree = kunit_kernel.LinuxSourceTree('', kunitconfig_path=kunitconfig.name) + + # TODO: add more test cases. + + class KUnitJsonTest(unittest.TestCase): def _json_for(self, log_file): @@ -399,5 +417,19 @@ class KUnitMainTest(unittest.TestCase): self.linux_source_mock.run_kernel.assert_called_once_with(build_dir=build_dir, timeout=300) self.print_mock.assert_any_call(StrContains('Testing complete.')) + @mock.patch.object(kunit_kernel, 'LinuxSourceTree') + def test_run_kunitconfig(self, mock_linux_init): + mock_linux_init.return_value = self.linux_source_mock + kunit.main(['run', '--kunitconfig=mykunitconfig']) + # Just verify that we parsed and initialized it correctly here. + mock_linux_init.assert_called_once_with('.kunit', kunitconfig_path='mykunitconfig') + + @mock.patch.object(kunit_kernel, 'LinuxSourceTree') + def test_config_kunitconfig(self, mock_linux_init): + mock_linux_init.return_value = self.linux_source_mock + kunit.main(['config', '--kunitconfig=mykunitconfig']) + # Just verify that we parsed and initialized it correctly here. + mock_linux_init.assert_called_once_with('.kunit', kunitconfig_path='mykunitconfig') + if __name__ == '__main__': unittest.main() base-commit: 88bb507a74ea7d75fa49edd421eaa710a7d80598 -- 2.30.0.365.g02bc693789-goog

4 years, 7 months

2
2
0 0

[PATCH] Documentation: kunit: add tips.rst for small examples

by Daniel Latypov

./usage.rst contains fairly long examples and explanations of things like how to fake a class and how to use parameterized tests (and how you could do table-driven tests yourself). It's not exactly necessary information, so we add a new page with more digestible tips like "use kunit_kzalloc() instead of kzalloc() so you don't have to worry about calling kfree() yourself" and the like. Change start.rst to point users to this new page first and let them know that usage.rst is more of optional further reading. Signed-off-by: Daniel Latypov <dlatypov(a)google.com> --- Documentation/dev-tools/kunit/index.rst | 2 + Documentation/dev-tools/kunit/start.rst | 4 +- Documentation/dev-tools/kunit/tips.rst | 115 ++++++++++++++++++++++++ 3 files changed, 120 insertions(+), 1 deletion(-) create mode 100644 Documentation/dev-tools/kunit/tips.rst diff --git a/Documentation/dev-tools/kunit/index.rst b/Documentation/dev-tools/kunit/index.rst index c234a3ab3c34..848478838347 100644 --- a/Documentation/dev-tools/kunit/index.rst +++ b/Documentation/dev-tools/kunit/index.rst @@ -13,6 +13,7 @@ KUnit - Unit Testing for the Linux Kernel api/index style faq + tips What is KUnit? ============== @@ -88,6 +89,7 @@ How do I use it? ================ * :doc:`start` - for new users of KUnit +* :doc:`tips` - for short examples of best practices * :doc:`usage` - for a more detailed explanation of KUnit features * :doc:`api/index` - for the list of KUnit APIs used for testing * :doc:`kunit-tool` - for more information on the kunit_tool helper script diff --git a/Documentation/dev-tools/kunit/start.rst b/Documentation/dev-tools/kunit/start.rst index 454f307813ea..c09e2747c958 100644 --- a/Documentation/dev-tools/kunit/start.rst +++ b/Documentation/dev-tools/kunit/start.rst @@ -233,5 +233,7 @@ Congrats! You just wrote your first KUnit test! Next Steps ========== -* Check out the :doc:`usage` page for a more +* Check out the :doc:`tips` page for tips on + writing idiomatic KUnit tests. +* Optional: see the :doc:`usage` page for a more in-depth explanation of KUnit. diff --git a/Documentation/dev-tools/kunit/tips.rst b/Documentation/dev-tools/kunit/tips.rst new file mode 100644 index 000000000000..a6ca0af14098 --- /dev/null +++ b/Documentation/dev-tools/kunit/tips.rst @@ -0,0 +1,115 @@ +.. SPDX-License-Identifier: GPL-2.0 + +============================ +Tips For Writing KUnit Tests +============================ + +Exiting early on failed expectations +------------------------------------ + +``KUNIT_EXPECT_EQ`` and friends will mark the test as failed and continue +execution. In some cases, it's unsafe to continue and you can use the +``KUNIT_ASSERT`` variant to exit on failure. + +.. code-block:: c + + void example_test_user_alloc_function(struct kunit *test) + { + void *object = alloc_some_object_for_me(); + + /* Make sure we got a valid pointer back. */ + KUNIT_ASSERT_NOT_ERR_OR_NULL(test, object); + do_something_with_object(object); + } + +Allocating memory +----------------- + +Where you would use ``kzalloc``, you should prefer ``kunit_kzalloc`` instead. +KUnit will ensure the memory is freed once the test completes. + +This is particularly useful since it lets you use the ``KUNIT_ASSERT_EQ`` +macros to exit early from a test without having to worry about remembering to +call ``kfree``. + +Example: + +.. code-block:: c + + void example_test_allocation(struct kunit *test) + { + char *buffer = kunit_kzalloc(test, 16, GFP_KERNEL); + /* Ensure allocation succeeded. */ + KUNIT_ASSERT_NOT_ERR_OR_NULL(test, buffer); + + KUNIT_ASSERT_STREQ(test, buffer, ""); + } + + +Testing static functions +------------------------ + +If you don't want to expose functions or variables just for testing, one option +is to conditionally ``#include`` the test file at the end of your .c file, e.g. + +.. code-block:: c + + /* In my_file.c */ + + static int do_interesting_thing(); + + #ifdef CONFIG_MY_KUNIT_TEST + #include "my_kunit_test.c" + #endif + +Injecting test-only code +------------------------ + +Similarly to the above, it can be useful to add test-specific logic. + +.. code-block:: c + + /* In my_file.h */ + + #ifdef CONFIG_MY_KUNIT_TEST + /* Defined in my_kunit_test.c */ + void test_only_hook(void); + #else + void test_only_hook(void) { } + #endif + +TODO(dlatypov(a)google.com): add an example of using ``current->kunit_test`` in +such a hook when it's not only updated for ``CONFIG_KASAN=y``. + +Customizing error messages +-------------------------- + +Each of the ``KUNIT_EXPECT`` and ``KUNIT_ASSERT`` macros have a ``_MSG`` variant. +These take a format string and arguments to provide additional context to the automatically generated error messages. + +.. code-block:: c + + char some_str[41]; + generate_sha1_hex_string(some_str); + + /* Before. Not easy to tell why the test failed. */ + KUNIT_EXPECT_EQ(test, strlen(some_str), 40); + + /* After. Now we see the offending string. */ + KUNIT_EXPECT_EQ_MSG(test, strlen(some_str), 40, "some_str='%s'", some_str); + +Alternatively, one can take full control over the error message by using ``KUNIT_FAIL()``, e.g. + +.. code-block:: c + + /* Before */ + KUNIT_EXPECT_EQ(test, some_setup_function(), 0); + + /* After: full control over the failure message. */ + if (some_setup_function()) + KUNIT_FAIL(test, "Failed to setup thing for testing"); + +Next Steps +========== +* Optional: see the :doc:`usage` page for a more + in-depth explanation of KUnit. base-commit: 6ee1d745b7c9fd573fba142a2efdad76a9f1cb04 -- 2.30.0.280.ga3ce27912f-goog

4 years, 7 months

2
1
0 0

[PATCH] KUnit: Docs: make start.rst example Kconfig follow style.rst

by Daniel Latypov

The primary change is that we want to encourage people to respect KUNIT_ALL_TESTS to make it easy to run all the relevant tests for a given config. Signed-off-by: Daniel Latypov <dlatypov(a)google.com> --- Documentation/dev-tools/kunit/start.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/Documentation/dev-tools/kunit/start.rst b/Documentation/dev-tools/kunit/start.rst index 454f307813ea..560f27af4619 100644 --- a/Documentation/dev-tools/kunit/start.rst +++ b/Documentation/dev-tools/kunit/start.rst @@ -196,8 +196,9 @@ Now add the following to ``drivers/misc/Kconfig``: .. code-block:: kconfig config MISC_EXAMPLE_TEST - bool "Test for my example" + tristate "Test for my example" if !KUNIT_ALL_TESTS depends on MISC_EXAMPLE && KUNIT=y + default KUNIT_ALL_TESTS and the following to ``drivers/misc/Makefile``: base-commit: 146620506274bd24d52fb1c589110a30eed8240b -- 2.30.0.296.g2bfb1c46d8-goog

4 years, 7 months

2
1
0 0

[PATCH] kunit: Print test statistics on failure

by David Gow

When a number of tests fail, it can be useful to get higher-level statistics of how many tests are failing (or how many parameters are failing in parameterised tests), and in what cases or suites. This is already done by some non-KUnit tests, so add support for automatically generating these for KUnit tests. This change adds a 'kunit_stats_enabled' switch which has three values: - 0: No stats are printed (current behaviour) - 1: Stats are printed only for tests/suites with more than one subtests, and at least one failure (new default) - 2: Always print test statistics For parameterised tests, the summary line looks as follows: " # inode_test_xtimestamp_decoding: 0 / 16 test parameters failed" For test suites, it looks like this: "# ext4_inode_test: (0 / 1) tests failed (0 / 16 test parameters)" kunit_tool is also updated to correctly ignore diagnostic lines, so that these statistics do not prevent the result from parsing. Signed-off-by: David Gow <davidgow(a)google.com> --- This is largely a follow-up to the discussion here: https://lore.kernel.org/linux-kselftest/CABVgOSmy4n_LGwDS7yWfoLftcQzxv6S+iX… Does this seem like a sensible addition? Cheers, -- David lib/kunit/test.c | 71 +++++++++++++++++++++++++++++ tools/testing/kunit/kunit_parser.py | 2 +- 2 files changed, 72 insertions(+), 1 deletion(-) diff --git a/lib/kunit/test.c b/lib/kunit/test.c index ec9494e914ef..711e269366a7 100644 --- a/lib/kunit/test.c +++ b/lib/kunit/test.c @@ -9,6 +9,7 @@ #include <kunit/test.h> #include <linux/kernel.h> #include <linux/kref.h> +#include <linux/moduleparam.h> #include <linux/sched/debug.h> #include <linux/sched.h> @@ -16,6 +17,40 @@ #include "string-stream.h" #include "try-catch-impl.h" +/* + * KUnit statistic mode: + * 0 - disabled + * 1 - only when there is at least one failure, and more than one subtest + * 2 - enabled + */ +static int kunit_stats_enabled = 1; +core_param(kunit_stats_enabled, kunit_stats_enabled, int, 0644); + +static bool kunit_should_print_stats(int num_failures, int num_subtests) +{ + if (kunit_stats_enabled == 0) + return false; + + if (kunit_stats_enabled == 2) + return true; + + return (num_failures > 0 && num_subtests > 1); +} + +static void kunit_print_test_stats(struct kunit *test, + size_t num_failures, size_t num_subtests) +{ + if (!kunit_should_print_stats(num_failures, num_subtests)) + return; + + kunit_log(KERN_INFO, test, + KUNIT_SUBTEST_INDENT + "# %s: %lu / %lu test parameters failed", + test->name, + num_failures, + num_subtests); +} + /* * Append formatted message to log, size of which is limited to * KUNIT_LOG_SIZE bytes (including null terminating byte). @@ -346,15 +381,37 @@ static void kunit_run_case_catch_errors(struct kunit_suite *suite, test_case->success = test->success; } +static void kunit_print_suite_stats(struct kunit_suite *suite, + size_t num_failures, + size_t total_param_failures, + size_t total_params) +{ + size_t num_cases = kunit_suite_num_test_cases(suite); + + if (!kunit_should_print_stats(num_failures, num_cases)) + return; + + kunit_log(KERN_INFO, suite, + "# %s: (%lu / %lu) tests failed (%lu / %lu test parameters)", + suite->name, + num_failures, + num_cases, + total_param_failures, + total_params); +} + int kunit_run_tests(struct kunit_suite *suite) { char param_desc[KUNIT_PARAM_DESC_SIZE]; struct kunit_case *test_case; + size_t num_suite_failures = 0; + size_t total_param_failures = 0, total_params = 0; kunit_print_subtest_start(suite); kunit_suite_for_each_test_case(suite, test_case) { struct kunit test = { .param_value = NULL, .param_index = 0 }; + size_t num_params = 0, num_failures = 0; bool test_success = true; if (test_case->generate_params) { @@ -385,13 +442,27 @@ int kunit_run_tests(struct kunit_suite *suite) test.param_value = test_case->generate_params(test.param_value, param_desc); test.param_index++; } + + if (!test.success) + num_failures++; + num_params++; + } while (test.param_value); + kunit_print_test_stats(&test, num_failures, num_params); + kunit_print_ok_not_ok(&test, true, test_success, kunit_test_case_num(suite, test_case), test_case->name); + + if (!test_success) + num_suite_failures++; + total_params += num_params; + total_param_failures += num_failures; } + kunit_print_suite_stats(suite, num_suite_failures, + total_param_failures, total_params); kunit_print_subtest_end(suite); return 0; diff --git a/tools/testing/kunit/kunit_parser.py b/tools/testing/kunit/kunit_parser.py index 6614ec4d0898..88ee2b2668ad 100644 --- a/tools/testing/kunit/kunit_parser.py +++ b/tools/testing/kunit/kunit_parser.py @@ -95,7 +95,7 @@ def print_log(log): for m in log: print_with_timestamp(m) -TAP_ENTRIES = re.compile(r'^(TAP|[\s]*ok|[\s]*not ok|[\s]*[0-9]+\.\.[0-9]+|[\s]*#).*$') +TAP_ENTRIES = re.compile(r'^(TAP|[\s]*ok|[\s]*not ok|[\s]*[0-9]+\.\.[0-9]+|[\s]*# Subtest:).*$') def consume_non_diagnositic(lines: List[str]) -> None: while lines and not TAP_ENTRIES.match(lines[0]): base-commit: 5f6b99d0287de2c2d0b5e7abcb0092d553ad804a -- 2.29.2.576.ga3fc446d84-goog

4 years, 7 months

4
4
0 0

[PATCH] kunit: tool: simplify kconfig is_subset_of() logic

by Daniel Latypov

Don't use an O(nm) algorithm* and make it more readable by using a dict. *Most obviously, it does a nested for-loop over the entire other config. A bit more subtle, it calls .entries(), which constructs a set from the list for _every_ outer iteration. Signed-off-by: Daniel Latypov <dlatypov(a)google.com> --- tools/testing/kunit/kunit_config.py | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/tools/testing/kunit/kunit_config.py b/tools/testing/kunit/kunit_config.py index 02ffc3a3e5dc..f1101075d458 100644 --- a/tools/testing/kunit/kunit_config.py +++ b/tools/testing/kunit/kunit_config.py @@ -40,15 +40,14 @@ class Kconfig(object): self._entries.append(entry) def is_subset_of(self, other: 'Kconfig') -> bool: + other_dict = {e.name: e.value for e in other.entries()} for a in self.entries(): - found = False - for b in other.entries(): - if a.name != b.name: + b = other_dict.get(a.name) + if b is None: + if a.value == 'n': continue - if a.value != b.value: - return False - found = True - if a.value != 'n' and found == False: + return False + elif a.value != b: return False return True base-commit: c6f7e1510b872c281ff603a3108c084b6548d35c -- 2.29.2.576.ga3fc446d84-goog

4 years, 7 months

3
2
0 0

[PATCH v2] kunit: don't show `1 == 1` in failed assertion messages

by Daniel Latypov

Currently, given something (fairly dystopian) like > KUNIT_EXPECT_EQ(test, 2 + 2, 5) KUnit will prints a failure message like this. > Expected 2 + 2 == 5, but > 2 + 2 == 4 > 5 == 5 With this patch, the output just becomes > Expected 2 + 2 == 5, but > 2 + 2 == 4 This patch is slightly hacky, but it's quite common* to compare an expression to a literal integer value, so this can make KUnit less chatty in many cases. (This patch also fixes variants like KUNIT_EXPECT_GT, LE, et al.). It also allocates an additional string briefly, but given this only happens on test failures, it doesn't seem too bad a tradeoff. Also, in most cases it'll realize the lengths are unequal and bail out before the allocation. We could save the result of the formatted string to avoid wasting this extra work, but it felt cleaner to leave it as-is. Edge case: for something silly and unrealistic like > KUNIT_EXPECT_EQ(test, 4, 5); It'll generate this message with a trailing "but" > Expected 4 == 5, but > <next line of normal output> It didn't feel worth adding a check up-front to see if both sides are literals to handle this better. *A quick grep suggests 100+ comparisons to an integer literal as the right hand side. Signed-off-by: Daniel Latypov <dlatypov(a)google.com> Tested-by: David Gow <davidgow(a)google.com> Reviewed-by: Brendan Higgins <brendanhiggins(a)google.com> --- lib/kunit/assert.c | 39 +++++++++++++++++++++++++++++++++------ 1 file changed, 33 insertions(+), 6 deletions(-) diff --git a/lib/kunit/assert.c b/lib/kunit/assert.c index 33acdaa28a7d..e0ec7d6fed6f 100644 --- a/lib/kunit/assert.c +++ b/lib/kunit/assert.c @@ -85,6 +85,29 @@ void kunit_ptr_not_err_assert_format(const struct kunit_assert *assert, } EXPORT_SYMBOL_GPL(kunit_ptr_not_err_assert_format); +/* Checks if `text` is a literal representing `value`, e.g. "5" and 5 */ +static bool is_literal(struct kunit *test, const char *text, long long value, + gfp_t gfp) +{ + char *buffer; + int len; + bool ret; + + len = snprintf(NULL, 0, "%lld", value); + if (strlen(text) != len) + return false; + + buffer = kunit_kmalloc(test, len+1, gfp); + if (!buffer) + return false; + + snprintf(buffer, len+1, "%lld", value); + ret = strncmp(buffer, text, len) == 0; + + kunit_kfree(test, buffer); + return ret; +} + void kunit_binary_assert_format(const struct kunit_assert *assert, struct string_stream *stream) { @@ -97,12 +120,16 @@ void kunit_binary_assert_format(const struct kunit_assert *assert, binary_assert->left_text, binary_assert->operation, binary_assert->right_text); - string_stream_add(stream, KUNIT_SUBSUBTEST_INDENT "%s == %lld\n", - binary_assert->left_text, - binary_assert->left_value); - string_stream_add(stream, KUNIT_SUBSUBTEST_INDENT "%s == %lld", - binary_assert->right_text, - binary_assert->right_value); + if (!is_literal(stream->test, binary_assert->left_text, + binary_assert->left_value, stream->gfp)) + string_stream_add(stream, KUNIT_SUBSUBTEST_INDENT "%s == %lld\n", + binary_assert->left_text, + binary_assert->left_value); + if (!is_literal(stream->test, binary_assert->right_text, + binary_assert->right_value, stream->gfp)) + string_stream_add(stream, KUNIT_SUBSUBTEST_INDENT "%s == %lld", + binary_assert->right_text, + binary_assert->right_value); kunit_assert_print_msg(assert, stream); } EXPORT_SYMBOL_GPL(kunit_binary_assert_format); base-commit: 1e0d27fce010b0a4a9e595506b6ede75934c31be -- 2.30.0.478.g8a0d178c01-goog

4 years, 7 months

1
0
0 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

Linux-kselftest-mirror