January 2024 - Linux-kselftest-mirror

[RFC PATCH v2 00/29] NT synchronization primitive driver

by Elizabeth Figura

This patch series introduces a new char misc driver, /dev/ntsync, which is used to implement Windows NT synchronization primitives. == Background == The Wine project emulates the Windows API in user space. One particular part of that API, namely the NT synchronization primitives, have historically been implemented via RPC to a dedicated "kernel" process. However, more recent applications use these APIs more strenuously, and the overhead of RPC has become a bottleneck. The NT synchronization APIs are too complex to implement on top of existing primitives without sacrificing correctness. Certain operations, such as NtPulseEvent() or the "wait-for-all" mode of NtWaitForMultipleObjects(), require direct control over the underlying wait queue, and implementing a wait queue sufficiently robust for Wine in user space is not possible. This proposed driver, therefore, implements the problematic interfaces directly in the Linux kernel. This driver was presented at Linux Plumbers Conference 2023. For those further interested in the history of synchronization in Wine and past attempts to solve this problem in user space, a recording of the presentation can be viewed here: https://www.youtube.com/watch?v=NjU4nyWyhU8 == Performance == The gain in performance varies wildly depending on the application in question and the user's hardware. For some games NT synchronization is not a bottleneck and no change can be observed, but for others frame rate improvements of 50 to 150 percent are not atypical. The following table lists frame rate measurements from a variety of games on a variety of hardware, taken by users Dmitry Skvortsov, FuzzyQuils, OnMars, and myself: Game Upstream ntsync improvement =========================================================================== Anger Foot 69 99 43% Call of Juarez 99.8 224.1 125% Dirt 3 110.6 860.7 678% Forza Horizon 5 108 160 48% Lara Croft: Temple of Osiris 141 326 131% Metro 2033 164.4 199.2 21% Resident Evil 2 26 77 196% The Crew 26 51 96% Tiny Tina's Wonderlands 130 360 177% Total War Saga: Troy 109 146 34% =========================================================================== == Patches == The intended semantics of the patches are broadly intended to match those of the corresponding Windows functions. For those not already familiar with the Windows functions (or their undocumented behaviour), patch 29/29 provides a detailed specification, and individual patches also include a brief description of the API they are implementing. The patches making use of this driver in Wine can be retrieved or browsed here: https://repo.or.cz/wine/zf.git/shortlog/refs/heads/ntsync5 == Implementation == Some aspects of the implementation may deserve particular comment: * In the interest of performance, each object is governed only by a single spinlock. However, NTSYNC_IOC_WAIT_ALL requires that the state of multiple objects be changed as a single atomic operation. In order to achieve this, we first take a device-wide lock ("wait_all_lock") any time we are going to lock more than one object at a time. The maximum number of objects that can be used in a vectored wait, and therefore the maximum that can be locked simultaneously, is 64. This number is NT's own limit. The acquisition of multiple spinlocks will degrade performance. This is a conscious choice, however. Wait-for-all is known to be a very rare operation in practice, especially with counts that approach the maximum, and it is the intent of the ntsync driver to optimize wait-for-any at the expense of wait-for-all as much as possible. * NT mutexes are tied to their threads on an OS level, and the kernel includes builtin support for "robust" mutexes. In order to keep the ntsync driver self-contained and avoid touching more code than necessary, it does not hook into task exit nor use pids. Instead, the user space emulator is expected to manage thread IDs and pass them as an argument to any relevant functions; this is the "owner" field of ntsync_wait_args and ntsync_mutex_args. When the emulator detects that a thread dies, it should therefore call NTSYNC_IOC_KILL_OWNER, which will mark mutexes owned by that thread (if any) as abandoned. * This implementation uses a misc device mostly because it seemed like the simplest and least obtrusive option. Besides simplicitly of implementation, the only particularly interesting advantage is the ability to create an arbitrary number of "contexts" (corresponding to Windows virtual machines) which are self-contained and shareable across multiple processes; this maps nicely to file descriptions (i.e. struct file). This is not impossible with syscalls of course but would require an extra argument. On the other hand, there is no reason to forbid using ntsync by default from user-mode processes, and (as far as I understand) to do so with a char device requires explicit configuration by e.g. udev or init. Since this is done with e.g. fuse, I assume this is the model to follow, but I may have chosen something deprecated. * ntsync is module-capable mostly because there was nothing preventing it, and because it aided development. It is not a hard requirement, though. == Previous versions == Changes in v2: * Send the whole series instead of just the first few patches. * Try to add more description to each patch, as a short documentation of the functions to be implemented. A more complete documentation of all aspects of the driver is provided in the contents of the last patch. * Objects are now files rather than indices into a table. This prevents a process from changing the state of an object which it should not have access to. Suggested by Andy Lutorminski. * Because the device no longer inherently has a table of all objects, marking a thread's owned mutexes as abandoned is now done through an ioctl on the mutex. * Change the names of a couple ioctls to be a bit less odd (PUT_SEM -> SEM_POST, PUT_MUTEX -> MUTEX_UNLOCK), and to reflect that they are ioctls on an object rather than on the device. * Pass the timeout for wait functions as a bare u64 (in ns), per Arnd Bergmann, with U64_MAX used to indicate no timeout. I originally indicated that I would change the timeout to be relative, but on reflection ended up keeping it as absolute, as this results in the least number of calls to get the current time (i.e. one). * Use compat_ptr_ioctl(), per Arnd Bergmann. * Remove the fixed minor number and module alias, per Greg Kroah-Hartman. * Allocate the fds array on stack in setup_wait(). This array takes up 260 bytes. * Link to v1: https://lore.kernel.org/lkml/20240124004028.16826-1-zfigura@codeweavers.com/ Elizabeth Figura (29): ntsync: Introduce the ntsync driver and character device. ntsync: Introduce NTSYNC_IOC_CREATE_SEM. ntsync: Introduce NTSYNC_IOC_SEM_POST. ntsync: Introduce NTSYNC_IOC_WAIT_ANY. ntsync: Introduce NTSYNC_IOC_WAIT_ALL. ntsync: Introduce NTSYNC_IOC_CREATE_MUTEX. ntsync: Introduce NTSYNC_IOC_MUTEX_UNLOCK. ntsync: Introduce NTSYNC_IOC_MUTEX_KILL. ntsync: Introduce NTSYNC_IOC_CREATE_EVENT. ntsync: Introduce NTSYNC_IOC_EVENT_SET. ntsync: Introduce NTSYNC_IOC_EVENT_RESET. ntsync: Introduce NTSYNC_IOC_EVENT_PULSE. ntsync: Introduce NTSYNC_IOC_SEM_READ. ntsync: Introduce NTSYNC_IOC_MUTEX_READ. ntsync: Introduce NTSYNC_IOC_EVENT_READ. ntsync: Introduce alertable waits. selftests: ntsync: Add some tests for semaphore state. selftests: ntsync: Add some tests for mutex state. selftests: ntsync: Add some tests for NTSYNC_IOC_WAIT_ANY. selftests: ntsync: Add some tests for NTSYNC_IOC_WAIT_ALL. selftests: ntsync: Add some tests for wakeup signaling with WINESYNC_IOC_WAIT_ANY. selftests: ntsync: Add some tests for wakeup signaling with WINESYNC_IOC_WAIT_ALL. selftests: ntsync: Add some tests for manual-reset event state. selftests: ntsync: Add some tests for auto-reset event state. selftests: ntsync: Add some tests for wakeup signaling with events. selftests: ntsync: Add tests for alertable waits. selftests: ntsync: Add some tests for wakeup signaling via alerts. maintainers: Add an entry for ntsync. docs: ntsync: Add documentation for the ntsync uAPI. Documentation/userspace-api/index.rst | 1 + .../userspace-api/ioctl/ioctl-number.rst | 2 + Documentation/userspace-api/ntsync.rst | 390 +++++ MAINTAINERS | 9 + drivers/misc/Kconfig | 9 + drivers/misc/Makefile | 1 + drivers/misc/ntsync.c | 1132 ++++++++++++++ include/uapi/linux/ntsync.h | 58 + tools/testing/selftests/Makefile | 1 + .../testing/selftests/drivers/ntsync/Makefile | 8 + tools/testing/selftests/drivers/ntsync/config | 1 + .../testing/selftests/drivers/ntsync/ntsync.c | 1300 +++++++++++++++++ 12 files changed, 2912 insertions(+) create mode 100644 Documentation/userspace-api/ntsync.rst create mode 100644 drivers/misc/ntsync.c create mode 100644 include/uapi/linux/ntsync.h create mode 100644 tools/testing/selftests/drivers/ntsync/Makefile create mode 100644 tools/testing/selftests/drivers/ntsync/config create mode 100644 tools/testing/selftests/drivers/ntsync/ntsync.c -- 2.43.0

10 months, 3 weeks

2
31
0 0

[PATCH bpf-next v3 0/6] Improvements for tracking scalars in the BPF verifier

by Maxim Mikityanskiy

From: Maxim Mikityanskiy <maxim(a)isovalent.com> The goal of this series is to extend the verifier's capabilities of tracking scalars when they are spilled to stack, especially when the spill or fill is narrowing. It also contains a fix by Eduard for infinite loop detection and a state pruning optimization by Eduard that compensates for a verification complexity regression introduced by tracking unbounded scalars. These improvements reduce the surface of false rejections that I saw while working on Cilium codebase. Patches 1-9 of the original series were previously applied in v2. Patches 1-2 (Maxim): Support the case when boundary checks are first performed after the register was spilled to the stack. Patches 3-4 (Maxim): Support narrowing fills. Patches 5-6 (Eduard): Optimization for state pruning in stacksafe() to mitigate the verification complexity regression. veristat -e file,prog,states -f '!states_diff<50' -f '!states_pct<10' -f '!states_a<10' -f '!states_b<10' -C ... * Without patch 5: File Program States (A) States (B) States (DIFF) -------------------- -------- ---------- ---------- ---------------- pyperf100.bpf.o on_event 4878 6528 +1650 (+33.83%) pyperf180.bpf.o on_event 6936 11032 +4096 (+59.05%) pyperf600.bpf.o on_event 22271 39455 +17184 (+77.16%) pyperf600_iter.bpf.o on_event 400 490 +90 (+22.50%) strobemeta.bpf.o on_event 4895 14028 +9133 (+186.58%) * With patch 5: File Program States (A) States (B) States (DIFF) ----------------------- ------------- ---------- ---------- --------------- bpf_xdp.o tail_lb_ipv4 2770 2224 -546 (-19.71%) pyperf100.bpf.o on_event 4878 5848 +970 (+19.89%) pyperf180.bpf.o on_event 6936 8868 +1932 (+27.85%) pyperf600.bpf.o on_event 22271 29656 +7385 (+33.16%) pyperf600_iter.bpf.o on_event 400 450 +50 (+12.50%) xdp_synproxy_kern.bpf.o syncookie_tc 280 226 -54 (-19.29%) xdp_synproxy_kern.bpf.o syncookie_xdp 302 228 -74 (-24.50%) v2 changes: Fixed comments in patch 1, moved endianness checks to header files in patch 12 where possible, added Eduard's ACKs. v3 changes: Maxim: Removed __is_scalar_unbounded altogether, addressed Andrii's comments. Eduard: Patch #5 (#14 in v2) changed significantly: - Logical changes: - Handling of STACK_{MISC,ZERO} mix turned out to be incorrect: a mix of MISC and ZERO in old state is not equivalent to e.g. just MISC is current state, because verifier could have deduced zero scalars from ZERO slots in old state for some loads. - There is no reason to limit the change only to cases when old or current stack is a spill of unbounded scalar, it is valid to compare any 64-bit scalar spill with fake register impersonating MISC. - STACK_ZERO vs spilled zero case was dropped, after recent changes for zero handling by Andrii and Yonghong it is hard (impossible?) to conjure all ZERO slots for an spi. => the case does not make any difference in veristat results. - Use global static variable for unbound_reg (Andrii) - Code shuffling to remove duplication in stacksafe() (Andrii) Eduard Zingerman (2): bpf: handle scalar spill vs all MISC in stacksafe() selftests/bpf: states pruning checks for scalar vs STACK_MISC Maxim Mikityanskiy (4): bpf: Track spilled unbounded scalars selftests/bpf: Test tracking spilled unbounded scalars bpf: Preserve boundaries and track scalars on narrowing fill selftests/bpf: Add test cases for narrowing fill include/linux/bpf_verifier.h | 9 + kernel/bpf/verifier.c | 103 ++++-- .../selftests/bpf/progs/verifier_spill_fill.c | 324 +++++++++++++++++- 3 files changed, 404 insertions(+), 32 deletions(-) -- 2.43.0

10 months, 3 weeks

2
7
0 0

[PATCH v3 0/5] selftests/resctrl: Add non-contiguous CBMs in Intel CAT selftest

by Maciej Wieczor-Retman

Non-contiguous CBM support for Intel CAT has been merged into the kernel with Commit 0e3cd31f6e90 ("x86/resctrl: Enable non-contiguous CBMs in Intel CAT") but there is no selftest that would validate if this feature works correctly. The selftest needs to verify if writing non-contiguous CBMs to the schemata file behaves as expected in comparison to the information about non-contiguous CBMs support. The patch series is based on a rework of resctrl selftests that's currently in review [1]. The patch also implements a similar functionality presented in the bash script included in the cover letter of the original non-contiguous CBMs in Intel CAT series [3]. Changelog v3: - Rebase onto v4 of Ilpo's series [1]. - Split old patch 3/4 into two parts. One doing refactoring and one adding a new function. - Some changes to all the patches after Reinette's review. Changelog v2: - Rebase onto v4 of Ilpo's series [2]. - Add two patches that prepare helpers for the new test. - Move Ilpo's patch that adds test grouping to this series. - Apply Ilpo's suggestion to the patch that adds a new test. [1] https://lore.kernel.org/all/20231215150515.36983-1-ilpo.jarvinen@linux.inte… [2] https://lore.kernel.org/all/20231211121826.14392-1-ilpo.jarvinen@linux.inte… [3] https://lore.kernel.org/all/cover.1696934091.git.maciej.wieczor-retman@inte… Older versions of this series: [v1] https://lore.kernel.org/all/20231109112847.432687-1-maciej.wieczor-retman@i… [v2] https://lore.kernel.org/all/cover.1702392177.git.maciej.wieczor-retman@inte… Ilpo Järvinen (1): selftests/resctrl: Add test groups and name L3 CAT test L3_CAT Maciej Wieczor-Retman (4): selftests/resctrl: Add helpers for the non-contiguous test selftests/resctrl: Split validate_resctrl_feature_request() selftests/resctrl: Add resource_info_file_exists() selftests/resctrl: Add non-contiguous CBMs CAT test tools/testing/selftests/resctrl/cat_test.c | 84 +++++++++++++++- tools/testing/selftests/resctrl/cmt_test.c | 4 +- tools/testing/selftests/resctrl/mba_test.c | 4 +- tools/testing/selftests/resctrl/mbm_test.c | 6 +- tools/testing/selftests/resctrl/resctrl.h | 11 ++- .../testing/selftests/resctrl/resctrl_tests.c | 18 +++- tools/testing/selftests/resctrl/resctrlfs.c | 98 ++++++++++++++++--- 7 files changed, 199 insertions(+), 26 deletions(-) -- 2.43.0

10 months, 3 weeks

3
21
0 0

[PATCH v2 0/7] of: populate of_root node if bootloader doesn't

by Stephen Boyd

Arch maintainers, please ack/review patches. This is a resend of a series from Frank last year[1]. I worked in Rob's review comments to unconditionally call unflatten_device_tree() and fixup/audit calls to of_have_populated_dt() so that behavior doesn't change. I need this series so I can add DT based tests in the clk framework. Either I can merge it through the clk tree once everyone is happy, or Rob can merge it through the DT tree and provide some branch so I can base clk patches on it. Changes from Frank's series[1]: * Add a DTB loaded kunit test * Make of_have_populated_dt() return false if the DTB isn't from the bootloader * Architecture calls made unconditional so that a root node is always made Changes from v1 (https://lore.kernel.org/r/20240112200750.4062441-1-sboyd@kernel.org): * x86 patch included * arm64 knocks out initial dtb if acpi is in use * keep Kconfig hidden but def_bool enabled otherwise Frank Rowand (2): of: Create of_root if no dtb provided by firmware of: unittest: treat missing of_root as error instead of fixing up Stephen Boyd (5): arm64: Unconditionally call unflatten_device_tree() um: Unconditionally call unflatten_device_tree() x86/of: Unconditionally call unflatten_and_copy_device_tree() of: Always unflatten in unflatten_and_copy_device_tree() of: Add KUnit test to confirm DTB is loaded arch/arm64/kernel/setup.c | 7 +++-- arch/um/kernel/dtb.c | 14 +++++----- arch/x86/kernel/devicetree.c | 24 +++++++++-------- drivers/of/.kunitconfig | 3 +++ drivers/of/Kconfig | 11 +++++++- drivers/of/Makefile | 4 ++- drivers/of/empty_root.dts | 6 +++++ drivers/of/fdt.c | 52 +++++++++++++++++++++++++----------- drivers/of/of_test.c | 48 +++++++++++++++++++++++++++++++++ drivers/of/platform.c | 3 --- drivers/of/unittest.c | 16 +++-------- include/linux/of.h | 25 ++++++++++------- 12 files changed, 151 insertions(+), 62 deletions(-) create mode 100644 drivers/of/.kunitconfig create mode 100644 drivers/of/empty_root.dts create mode 100644 drivers/of/of_test.c Cc: Anton Ivanov <anton.ivanov(a)cambridgegreys.com> Cc: Brendan Higgins <brendan.higgins(a)linux.dev> Cc: Catalin Marinas <catalin.marinas(a)arm.com> Cc: David Gow <davidgow(a)google.com> Cc: Frank Rowand <frowand.list(a)gmail.com> Cc: Johannes Berg <johannes(a)sipsolutions.net> Cc: Richard Weinberger <richard(a)nod.at> Cc: Rob Herring <robh+dt(a)kernel.org> Cc: Will Deacon <will(a)kernel.org> Cc: Thomas Gleixner <tglx(a)linutronix.de> Cc: Ingo Molnar <mingo(a)redhat.com> Cc: Borislav Petkov <bp(a)alien8.de> Cc: Dave Hansen <dave.hansen(a)linux.intel.com> Cc: <x86(a)kernel.org> Cc: "H. Peter Anvin" <hpa(a)zytor.com> Cc: Saurabh Sengar <ssengar(a)linux.microsoft.com> [1] https://lore.kernel.org/r/20230317053415.2254616-1-frowand.list@gmail.com base-commit: 0dd3ee31125508cd67f7e7172247f05b7fd1753a -- https://git.kernel.org/pub/scm/linux/kernel/git/clk/linux.git/ https://git.kernel.org/pub/scm/linux/kernel/git/sboyd/spmi.git

10 months, 3 weeks

3
11
0 0

[PATCH net-next] selftests: openvswitch: Test ICMP related matches work with SNAT

by Brad Cowie

Add a test case for regression in openvswitch nat that was fixed by commit e6345d2824a3 ("netfilter: nf_nat: fix action not being set for all ct states"). Link: https://lore.kernel.org/netdev/20231221224311.130319-1-brad@faucet.nz/ Link: https://mail.openvswitch.org/pipermail/ovs-dev/2024-January/410476.html Suggested-by: Aaron Conole <aconole(a)redhat.com> Signed-off-by: Brad Cowie <brad(a)faucet.nz> --- .../selftests/net/openvswitch/openvswitch.sh | 62 +++++++++++++++++++ 1 file changed, 62 insertions(+) diff --git a/tools/testing/selftests/net/openvswitch/openvswitch.sh b/tools/testing/selftests/net/openvswitch/openvswitch.sh index f8499d4c87f3..87b80bee6df4 100755 --- a/tools/testing/selftests/net/openvswitch/openvswitch.sh +++ b/tools/testing/selftests/net/openvswitch/openvswitch.sh @@ -17,6 +17,7 @@ tests=" ct_connect_v4 ip4-ct-xon: Basic ipv4 tcp connection using ct connect_v4 ip4-xon: Basic ipv4 ping between two NS nat_connect_v4 ip4-nat-xon: Basic ipv4 tcp connection via NAT + nat_related_v4 ip4-nat-related: ICMP related matches work with SNAT netlink_checks ovsnl: validate netlink attrs and settings upcall_interfaces ovs: test the upcall interfaces drop_reason drop: test drop reasons are emitted" @@ -473,6 +474,67 @@ test_nat_connect_v4 () { return 0 } +# nat_related_v4 test +# - client->server ip packets go via SNAT +# - client solicits ICMP destination unreachable packet from server +# - undo NAT for ICMP reply and test dst ip has been updated +test_nat_related_v4 () { + which nc >/dev/null 2>/dev/null || return $ksft_skip + + sbx_add "test_nat_related_v4" || return $? + + ovs_add_dp "test_nat_related_v4" natrelated4 || return 1 + info "create namespaces" + for ns in client server; do + ovs_add_netns_and_veths "test_nat_related_v4" "natrelated4" "$ns" \ + "${ns:0:1}0" "${ns:0:1}1" || return 1 + done + + ip netns exec client ip addr add 172.31.110.10/24 dev c1 + ip netns exec client ip link set c1 up + ip netns exec server ip addr add 172.31.110.20/24 dev s1 + ip netns exec server ip link set s1 up + + ip netns exec server ip route add 192.168.0.20/32 via 172.31.110.10 + + # Allow ARP + ovs_add_flow "test_nat_related_v4" natrelated4 \ + "in_port(1),eth(),eth_type(0x0806),arp()" "2" || return 1 + ovs_add_flow "test_nat_related_v4" natrelated4 \ + "in_port(2),eth(),eth_type(0x0806),arp()" "1" || return 1 + + # Allow IP traffic from client->server, rewrite source IP with SNAT to 192.168.0.20 + ovs_add_flow "test_nat_related_v4" natrelated4 \ + "ct_state(-trk),in_port(1),eth(),eth_type(0x0800),ipv4(dst=172.31.110.20)" \ + "ct(commit,nat(src=192.168.0.20)),recirc(0x1)" || return 1 + ovs_add_flow "test_nat_related_v4" natrelated4 \ + "recirc_id(0x1),ct_state(+trk-inv),in_port(1),eth(),eth_type(0x0800),ipv4()" \ + "2" || return 1 + + # Allow related ICMP responses back from server and undo NAT to restore original IP + # Drop any ICMP related packets where dst ip hasn't been restored back to original IP + ovs_add_flow "test_nat_related_v4" natrelated4 \ + "ct_state(-trk),in_port(2),eth(),eth_type(0x0800),ipv4()" \ + "ct(commit,nat),recirc(0x2)" || return 1 + ovs_add_flow "test_nat_related_v4" natrelated4 \ + "recirc_id(0x2),ct_state(+rel+trk),in_port(2),eth(),eth_type(0x0800),ipv4(src=172.31.110.20,dst=172.31.110.10,proto=1),icmp()" \ + "1" || return 1 + ovs_add_flow "test_nat_related_v4" natrelated4 \ + "recirc_id(0x2),ct_state(+rel+trk),in_port(2),eth(),eth_type(0x0800),ipv4(dst=192.168.0.20,proto=1),icmp()" \ + "drop" || return 1 + + # Solicit destination unreachable response from server + ovs_sbx "test_nat_related_v4" ip netns exec client \ + bash -c "echo a | nc -u -w 1 172.31.110.20 10000" + + # Check to make sure no packets matched the drop rule with incorrect dst ip + python3 "$ovs_base/ovs-dpctl.py" dump-flows natrelated4 \ + | grep "drop" | grep "packets:0" >/dev/null || return 1 + + info "done..." + return 0 +} + # netlink_validation # - Create a dp # - check no warning with "old version" simulation -- 2.34.1

10 months, 3 weeks

3
2
0 0

[PATCH v2 0/3] selftests/net: A couple of typos fixes in key-management/rst tests

by Dmitry Safonov

Changes in v2: - Dropped "selftests/net: Clean-up double assignment", going to send it to net-next with other changes (Simon) - Added a patch to rectify RST selftests. - Link to v1: https://lore.kernel.org/r/20240118-tcp-ao-test-key-mgmt-v1-0-3583ca147113@a… Two typo fixes, noticed by Mohammad's review. And a fix for an issue that got uncovered. Signed-off-by: Dmitry Safonov <dima(a)arista.com> --- Dmitry Safonov (2): selftests/net: Rectify key counters checks selftests/net: Repair RST passive reset selftest Mohammad Nassiri (1): selftests/net: Argument value mismatch when calling verify_counters() .../testing/selftests/net/tcp_ao/key-management.c | 46 ++++--- tools/testing/selftests/net/tcp_ao/lib/sock.c | 12 +- tools/testing/selftests/net/tcp_ao/rst.c | 138 ++++++++++++++------- 3 files changed, 124 insertions(+), 72 deletions(-) --- base-commit: ecb1b8288dc7ccbdcb3b9df005fa1c0e0c0388a7 change-id: 20240118-tcp-ao-test-key-mgmt-bb51a5fe15a2 Best regards, -- Dmitry Safonov <dima(a)arista.com>

10 months, 3 weeks

3
11
0 0

[PATCH v2] KVM: selftests: Fix the dirty_log_test semaphore imbalance

by Shaoqin Huang

When execute the dirty_log_test on some aarch64 machine, it sometimes trigger the ASSERT: ==== Test Assertion Failure ==== dirty_log_test.c:384: dirty_ring_vcpu_ring_full pid=14854 tid=14854 errno=22 - Invalid argument 1 0x00000000004033eb: dirty_ring_collect_dirty_pages at dirty_log_test.c:384 2 0x0000000000402d27: log_mode_collect_dirty_pages at dirty_log_test.c:505 3 (inlined by) run_test at dirty_log_test.c:802 4 0x0000000000403dc7: for_each_guest_mode at guest_modes.c:100 5 0x0000000000401dff: main at dirty_log_test.c:941 (discriminator 3) 6 0x0000ffff9be173c7: ?? ??:0 7 0x0000ffff9be1749f: ?? ??:0 8 0x000000000040206f: _start at ??:? Didn't continue vcpu even without ring full The dirty_log_test fails when execute the dirty-ring test, this is because the sem_vcpu_cont and the sem_vcpu_stop is non-zero value when execute the dirty_ring_collect_dirty_pages() function. When those two sem_t variables are non-zero, the dirty_ring_wait_vcpu() at the beginning of the dirty_ring_collect_dirty_pages() will not wait for the vcpu to stop, but continue to execute the following code. In this case, before vcpu stop, if the dirty_ring_vcpu_ring_full is true, and the dirty_ring_collect_dirty_pages() has passed the check for the dirty_ring_vcpu_ring_full but hasn't execute the check for the continued_vcpu, the vcpu stop, and set the dirty_ring_vcpu_ring_full to false. Then dirty_ring_collect_dirty_pages() will trigger the ASSERT. Why sem_vcpu_cont and sem_vcpu_stop can be non-zero value? It's because the dirty_ring_before_vcpu_join() execute the sem_post(&sem_vcpu_cont) at the end of each dirty-ring test. It can cause two cases: 1. sem_vcpu_cont be non-zero. When we set the host_quit to be true, the vcpu_worker directly see the host_quit to be true, it quit. So the log_mode_before_vcpu_join() function will set the sem_vcpu_cont to 1, since the vcpu_worker has quit, it won't consume it. 2. sem_vcpu_stop be non-zero. When we set the host_quit to be true, the vcpu_worker has entered the guest state, the next time it exit from guest state, it will set the sem_vcpu_stop to 1, and then see the host_quit, no one will consume the sem_vcpu_stop. When execute more and more dirty-ring tests, the sem_vcpu_cont and sem_vcpu_stop can be larger and larger, which makes many code paths don't wait for the sem_t. Thus finally cause the problem. To fix this problem, we can wait a while before set the host_quit to true, which gives the vcpu time to enter the guest state, so it will exit again. Then we can wait the vcpu to exit, and let it continue again, then the vcpu will see the host_quit. Thus the sem_vcpu_cont and sem_vcpu_stop will be both zero when test finished. Signed-off-by: Shaoqin Huang <shahuang(a)redhat.com> --- v1->v2: - Fix the real logic bug, not just fresh the context. v1: https://lore.kernel.org/all/20231116093536.22256-1-shahuang@redhat.com/ --- tools/testing/selftests/kvm/dirty_log_test.c | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/kvm/dirty_log_test.c b/tools/testing/selftests/kvm/dirty_log_test.c index 936f3a8d1b83..a6e0ff46a07c 100644 --- a/tools/testing/selftests/kvm/dirty_log_test.c +++ b/tools/testing/selftests/kvm/dirty_log_test.c @@ -417,7 +417,8 @@ static void dirty_ring_after_vcpu_run(struct kvm_vcpu *vcpu, int ret, int err) static void dirty_ring_before_vcpu_join(void) { - /* Kick another round of vcpu just to make sure it will quit */ + /* Wait vcpu exit, and let it continue to see the host_quit. */ + dirty_ring_wait_vcpu(); sem_post(&sem_vcpu_cont); } @@ -719,6 +720,7 @@ static void run_test(enum vm_guest_mode mode, void *arg) struct kvm_vm *vm; unsigned long *bmap; uint32_t ring_buf_idx = 0; + int sem_val; if (!log_mode_supported()) { print_skip("Log mode '%s' not supported", @@ -726,6 +728,11 @@ static void run_test(enum vm_guest_mode mode, void *arg) return; } + sem_getvalue(&sem_vcpu_stop, &sem_val); + assert(sem_val == 0); + sem_getvalue(&sem_vcpu_cont, &sem_val); + assert(sem_val == 0); + /* * We reserve page table for 2 times of extra dirty mem which * will definitely cover the original (1G+) test range. Here @@ -825,6 +832,13 @@ static void run_test(enum vm_guest_mode mode, void *arg) sync_global_to_guest(vm, iteration); } + /* + * + * Before we set the host_quit, let the vcpu has time to run, to make + * sure we consume the sem_vcpu_stop and the vcpu consume the + * sem_vcpu_cont, to keep the semaphore balance. + */ + usleep(p->interval * 1000); /* Tell the vcpu thread to quit */ host_quit = true; log_mode_before_vcpu_join(); -- 2.40.1

10 months, 3 weeks

2
4
0 0

[PATCH net 0/9] mptcp: fixes for recent issues reported by CI's

by Matthieu Baerts (NGI0)

This series of 9 patches fixes issues mostly identified by CI's not managed by the MPTCP maintainers. Thank you Linero (LKFT) and Netdev maintainers (NIPA) for running our kunit and selftests tests! For the first patch, it took a bit of time to identify the root cause. Some MPTCP Join selftest subtests have been "flaky", mostly in slow environments. It appears to be due to the use of a TCP-specific helper on an MPTCP socket. A fix for kernels >= v5.15. Patches 2 to 4 add missing kernel config to support NetFilter tables needed for IPTables commands. These kconfigs are usually enabled in default configurations, but apparently not for all architectures. Patches 2 and 3 can be backported up to v5.11 and the 4th one up to v5.19. Patch 5 increases the time limit for MPTCP selftests. It appears that many CI's execute tests in a VM without acceleration supports, e.g. QEmu without KVM. As a result, the tests take longer. Plus, there are more and more tests. This patch modifies the timeout added in v5.18. Patch 6 reduces the maximum rate and delay of the different links in some Simult Flows selftest subtests. The goal is to let slow VMs reach the maximum speed. The original rate was introduced in v5.11. Patch 7 lets CI changing the prefix of the subtests titles, to be able to run the same selftest multiple times with different parameters. With different titles, tests will be considered as different and not override previous results as it is the case with some CI envs. Subtests have been introduced in v6.6. Patch 8 and 9 make some MPTCP Join selftest subtests quicker by stopping the transfer when the expected events have been seen. Patch 8 can be backported up to v6.5. Signed-off-by: Matthieu Baerts (NGI0) <matttbe(a)kernel.org> --- Matthieu Baerts (NGI0) (8): selftests: mptcp: add missing kconfig for NF Filter selftests: mptcp: add missing kconfig for NF Filter in v6 selftests: mptcp: add missing kconfig for NF Mangle selftests: mptcp: increase timeout to 30 min selftests: mptcp: decrease BW in simult flows selftests: mptcp: allow changing subtests prefix selftests: mptcp: join: stop transfer when check is done (part 1) selftests: mptcp: join: stop transfer when check is done (part 2) Paolo Abeni (1): mptcp: fix data re-injection from stale subflow net/mptcp/protocol.c | 3 --- tools/testing/selftests/net/mptcp/config | 3 +++ tools/testing/selftests/net/mptcp/mptcp_join.sh | 27 +++++++++-------------- tools/testing/selftests/net/mptcp/mptcp_lib.sh | 2 +- tools/testing/selftests/net/mptcp/settings | 2 +- tools/testing/selftests/net/mptcp/simult_flows.sh | 8 +++---- 6 files changed, 20 insertions(+), 25 deletions(-) --- base-commit: c9ec85153fea6873c52ed4f5055c87263f1b54f9 change-id: 20240131-upstream-net-20240131-mptcp-ci-issues-9d68b5601e74 Best regards, -- Matthieu Baerts (NGI0) <matttbe(a)kernel.org>

10 months, 3 weeks

2
10
0 0

[PATCH net 0/3] selftests: net: more fixes

by Paolo Abeni

Another small bunch of fixes, addressing issues outlined by the netdev CI. Paolo Abeni (3): selftests: net: cut more slack for gro fwd tests. selftests: net: fix setup_ns usage in rtnetlink.sh selftests: net: enable some more knobs tools/testing/selftests/net/config | 3 +++ tools/testing/selftests/net/rtnetlink.sh | 6 ++---- tools/testing/selftests/net/udpgro_fwd.sh | 14 ++++++++++++-- tools/testing/selftests/net/udpgso_bench_rx.c | 2 +- 4 files changed, 18 insertions(+), 7 deletions(-) -- 2.43.0

10 months, 3 weeks

2
4
0 0

[PATCH net 0/5] selftests: net: More small fixes

by Benjamin Poirier

Some small fixes for net selftests which follow from these recent commits: dd2d40acdbb2 ("selftests: bonding: Add more missing config options") 49078c1b80b6 ("selftests: forwarding: Remove executable bits from lib.sh") Benjamin Poirier (5): selftests: team: Add missing config options selftests: bonding: Check initial state selftests: net: Remove executable bits from library scripts selftests: net: List helper scripts in TEST_FILES Makefile variable selftests: forwarding: List helper scripts in TEST_FILES Makefile variable .../testing/selftests/drivers/net/bonding/lag_lib.sh | 11 +++++++++++ tools/testing/selftests/drivers/net/team/config | 4 +++- tools/testing/selftests/net/Makefile | 5 ++--- tools/testing/selftests/net/forwarding/Makefile | 2 +- tools/testing/selftests/net/net_helper.sh | 0 tools/testing/selftests/net/setup_loopback.sh | 0 6 files changed, 17 insertions(+), 5 deletions(-) mode change 100755 => 100644 tools/testing/selftests/net/net_helper.sh mode change 100755 => 100644 tools/testing/selftests/net/setup_loopback.sh -- 2.43.0

10 months, 3 weeks

3
8
0 0

2024

2023

2022

2021

2020

2019

2018

2017

Linux-kselftest-mirror January 2024