- Linux-kselftest-mirror - lists.linaro.org

[PATCH 0/4] PCI: Add support and tests for FIXUP quirks in modules

by Brian Norris

This series primarily adds support for DECLARE_PCI_FIXUP_*() in modules. There are a few drivers that already use this, and so they are presumably broken when built as modules. While at it, I wrote some unit tests that emulate a fake PCI device, and let the PCI framework match/not-match its vendor/device IDs. This test can be built into the kernel or built as a module. I also include some infrastructure changes (patch 3 and 4), so that ARCH=um (the default for kunit.py), ARCH=arm, and ARCH=arm64 will run these tests by default. These patches have different maintainers and are independent, so they can probably be picked up separately. I included them because otherwise the tests in patch 2 aren't so easy to run. Brian Norris (4): PCI: Support FIXUP quirks in modules PCI: Add KUnit tests for FIXUP quirks um: Select PCI_DOMAINS_GENERIC kunit: qemu_configs: Add PCI to arm, arm64 arch/um/Kconfig | 1 + drivers/pci/Kconfig | 11 ++ drivers/pci/Makefile | 1 + drivers/pci/fixup-test.c | 197 ++++++++++++++++++++++ drivers/pci/quirks.c | 62 +++++++ include/linux/module.h | 18 ++ kernel/module/main.c | 26 +++ tools/testing/kunit/qemu_configs/arm.py | 1 + tools/testing/kunit/qemu_configs/arm64.py | 1 + 9 files changed, 318 insertions(+) create mode 100644 drivers/pci/fixup-test.c -- 2.51.0.384.g4c02a37b29-goog

1 month, 2 weeks

6
19
0 0

[PATCH v3 0/5] mm: VMA count limit fixes and improvements

by Kalesh Singh

Hi all, This series addresses an off-by-one bug in the VMA count limit check and introduces several improvements for clarity, test coverage, and observability around the VMA limit mechanism. The VMA count limit, controlled by sysctl_max_map_count, is a critical safeguard that prevents a single process from consuming excessive kernel memory by creating too many memory mappings. However, the checks in do_mmap() and do_brk_flags() used a strict inequality, allowing a process to exceed this limit by one VMA. This series begins by fixing this long-standing bug. The subsequent patches build on this by improving the surrounding code. A comprehensive selftest is added to validate VMA operations near the limit, preventing future regressions. The open-coded limit checks are replaced with a centralized helper, vma_count_remaining(), to improve readability. For better code clarity, mm_struct->map_count is renamed to the more apt vma_count. Finally, a trace event is added to provide observability for processes that fail allocations due to VMA exhaustion, which is valuable for debugging and profiling on production systems. The major changes in this version are: 1. Rebased on mm-new to resolve prior conflicts. 2. The patches to harden and add assertions for the VMA count have been dropped. David pointed out that these could be racy if sysctl_max_map_count is changed from userspace at just the wrong time. 3. The selftest has been completely rewritten per Lorenzo's feedback to make use of the kselftest harness and vm_util.h helpers. 4. The trace event has also been updated to contain more useful information and has been given a more fitting name, per feedback from Steve and Lorenzo. Tested on x86_64 and arm64: 1. Build test: allyesconfig for rename 2. Selftests: cd tools/testing/selftests/mm && \ make && \ ./run_vmtests.sh -t max_vma_count 3. vma tests: cd tools/testing/vma && \ make && \ ./vma Link to v2: https://lore.kernel.org/r/20250915163838.631445-1-kaleshsingh@google.com/ Thanks to everyone for their comments and feedback on the previous versions. --Kalesh Kalesh Singh (5): mm: fix off-by-one error in VMA count limit checks mm/selftests: add max_vma_count tests mm: introduce vma_count_remaining() mm: rename mm_struct::map_count to vma_count mm/tracing: introduce trace_mm_insufficient_vma_slots event MAINTAINERS | 2 + fs/binfmt_elf.c | 2 +- fs/coredump.c | 2 +- include/linux/mm.h | 2 - include/linux/mm_types.h | 2 +- include/trace/events/vma.h | 32 + kernel/fork.c | 2 +- mm/debug.c | 2 +- mm/internal.h | 3 + mm/mmap.c | 31 +- mm/mremap.c | 13 +- mm/nommu.c | 8 +- mm/util.c | 1 - mm/vma.c | 39 +- mm/vma_internal.h | 2 + tools/testing/selftests/mm/.gitignore | 1 + tools/testing/selftests/mm/Makefile | 1 + .../selftests/mm/max_vma_count_tests.c | 672 ++++++++++++++++++ tools/testing/selftests/mm/run_vmtests.sh | 5 + tools/testing/vma/vma.c | 32 +- tools/testing/vma/vma_internal.h | 16 +- 21 files changed, 818 insertions(+), 52 deletions(-) create mode 100644 include/trace/events/vma.h create mode 100644 tools/testing/selftests/mm/max_vma_count_tests.c base-commit: 4c4142c93fc19cd75a024e5c81b0532578a9e187 -- 2.51.0.760.g7b8bcc2412-goog

1 month, 2 weeks

5
16
0 0

[PATCH bpf-next 0/5] selftests/bpf: convert test_tc_tunnel.sh to test_progs

by Alexis Lothoré (eBPF Foundation)

Hello, this series aims to convert another test to the test_progs framework to make sure that it is executed in CI for series sent on the mailing list. test_tc_tunnel.sh tests a variety of tunnels based on BPF: packets are encapsulated by a BPF program on the client egress. We then check that those packets can be decapsulated on server ingress side, either thanks to kernel-based or BPF-based decapsulation. Those tests are run thanks to two veths in two dedicated namespaces. - patches 1 to 3 are preparatory patches - patch 4 introduce tc_tunnel test into test_progs - patch 5 gets rid of the test_tc_tunnel.sh script The new test has been executed both in some x86 local qemu machine, as well as in CI: # ./test_progs -a tc_tunnel #454/1 tc_tunnel/ipip_none:OK #454/2 tc_tunnel/ipip6_none:OK #454/3 tc_tunnel/ip6tnl_none:OK #454/4 tc_tunnel/sit_none:OK #454/5 tc_tunnel/vxlan_eth:OK #454/6 tc_tunnel/ip6vxlan_eth:OK #454/7 tc_tunnel/gre_none:OK #454/8 tc_tunnel/gre_eth:OK #454/9 tc_tunnel/gre_mpls:OK #454/10 tc_tunnel/ip6gre_none:OK #454/11 tc_tunnel/ip6gre_eth:OK #454/12 tc_tunnel/ip6gre_mpls:OK #454/13 tc_tunnel/udp_none:OK #454/14 tc_tunnel/udp_eth:OK #454/15 tc_tunnel/udp_mpls:OK #454/16 tc_tunnel/ip6udp_none:OK #454/17 tc_tunnel/ip6udp_eth:OK #454/18 tc_tunnel/ip6udp_mpls:OK #454 tc_tunnel:OK Summary: 1/18 PASSED, 0 SKIPPED, 0 FAILED Signed-off-by: Alexis Lothoré (eBPF Foundation) <alexis.lothore(a)bootlin.com> --- Alexis Lothoré (eBPF Foundation) (5): testing/selftests: rename tc_helpers.h to tcx_helpers.h selftests/bpf: add tc helpers selftests/bpf: make test_tc_tunnel.bpf.c compatible with big endian platforms selftests/bpf: integrate test_tc_tunnel.sh tests into test_progs selftests/bpf: remove test_tc_tunnel.sh tools/testing/selftests/bpf/Makefile | 2 +- tools/testing/selftests/bpf/prog_tests/tc_links.c | 46 +- tools/testing/selftests/bpf/prog_tests/tc_netkit.c | 22 +- tools/testing/selftests/bpf/prog_tests/tc_opts.c | 40 +- .../bpf/prog_tests/{tc_helpers.h => tcx_helpers.h} | 6 +- .../selftests/bpf/prog_tests/test_tc_tunnel.c | 684 +++++++++++++++++++++ .../testing/selftests/bpf/prog_tests/test_tunnel.c | 80 +-- tools/testing/selftests/bpf/progs/test_tc_tunnel.c | 99 ++- tools/testing/selftests/bpf/tc_helpers.c | 87 +++ tools/testing/selftests/bpf/tc_helpers.h | 9 + tools/testing/selftests/bpf/test_tc_tunnel.sh | 320 ---------- 11 files changed, 884 insertions(+), 511 deletions(-) --- base-commit: 22267893b8c7f2773896e814800bbe693f206e0c change-id: 20250811-tc_tunnel-c61342683f18 Best regards, -- Alexis Lothoré, Bootlin Embedded Linux and Kernel engineering https://bootlin.com

1 month, 2 weeks

3
10
0 0

[PATCH v4 10/10] RISC-V: KVM: selftests: Add Zalasr extensions to get-reg-list test

by Xu Lu

The KVM RISC-V allows Zalasr extensions for Guest/VM so add these extensions to get-reg-list test. Signed-off-by: Xu Lu <luxu.kernel(a)bytedance.com> --- tools/testing/selftests/kvm/riscv/get-reg-list.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/tools/testing/selftests/kvm/riscv/get-reg-list.c b/tools/testing/selftests/kvm/riscv/get-reg-list.c index a0b7dabb50406..3020e37f621ba 100644 --- a/tools/testing/selftests/kvm/riscv/get-reg-list.c +++ b/tools/testing/selftests/kvm/riscv/get-reg-list.c @@ -65,6 +65,7 @@ bool filter_reg(__u64 reg) case KVM_REG_RISCV_ISA_EXT | KVM_REG_RISCV_ISA_SINGLE | KVM_RISCV_ISA_EXT_ZAAMO: case KVM_REG_RISCV_ISA_EXT | KVM_REG_RISCV_ISA_SINGLE | KVM_RISCV_ISA_EXT_ZABHA: case KVM_REG_RISCV_ISA_EXT | KVM_REG_RISCV_ISA_SINGLE | KVM_RISCV_ISA_EXT_ZACAS: + case KVM_REG_RISCV_ISA_EXT | KVM_REG_RISCV_ISA_SINGLE | KVM_RISCV_ISA_EXT_ZALASR: case KVM_REG_RISCV_ISA_EXT | KVM_REG_RISCV_ISA_SINGLE | KVM_RISCV_ISA_EXT_ZALRSC: case KVM_REG_RISCV_ISA_EXT | KVM_REG_RISCV_ISA_SINGLE | KVM_RISCV_ISA_EXT_ZAWRS: case KVM_REG_RISCV_ISA_EXT | KVM_REG_RISCV_ISA_SINGLE | KVM_RISCV_ISA_EXT_ZBA: @@ -517,6 +518,7 @@ static const char *isa_ext_single_id_to_str(__u64 reg_off) KVM_ISA_EXT_ARR(ZAAMO), KVM_ISA_EXT_ARR(ZABHA), KVM_ISA_EXT_ARR(ZACAS), + KVM_ISA_EXT_ARR(ZALASR), KVM_ISA_EXT_ARR(ZALRSC), KVM_ISA_EXT_ARR(ZAWRS), KVM_ISA_EXT_ARR(ZBA), @@ -1112,6 +1114,7 @@ KVM_ISA_EXT_SIMPLE_CONFIG(svvptc, SVVPTC); KVM_ISA_EXT_SIMPLE_CONFIG(zaamo, ZAAMO); KVM_ISA_EXT_SIMPLE_CONFIG(zabha, ZABHA); KVM_ISA_EXT_SIMPLE_CONFIG(zacas, ZACAS); +KVM_ISA_EXT_SIMPLE_CONFIG(zalasr, ZALASR); KVM_ISA_EXT_SIMPLE_CONFIG(zalrsc, ZALRSC); KVM_ISA_EXT_SIMPLE_CONFIG(zawrs, ZAWRS); KVM_ISA_EXT_SIMPLE_CONFIG(zba, ZBA); @@ -1187,6 +1190,7 @@ struct vcpu_reg_list *vcpu_configs[] = { &config_zabha, &config_zacas, &config_zalrsc, + &config_zalasr, &config_zawrs, &config_zba, &config_zbb, -- 2.20.1

1 month, 2 weeks

1
0
0 0

[PATCH v4 05/10] riscv: Apply Zalasr to smp_load_acquire/smp_store_release

by Xu Lu

Replace fence instructions with Zalasr instructions during smp_load_acquire() and smp_store_release() operations. |----------------------------------| | | __smp_store_release | | |-----------------------------| | | zalasr | !zalasr | | rl |-----------------------------| | | s{b|h|w|d}.rl | fence rw, w | | | | s{b|h|w|d} | |----------------------------------| | | __smp_load_acquire | | |-----------------------------| | | zalasr | !zalasr | | aq |-----------------------------| | | l{b|h|w|d}.aq | l{b|h|w|d} | | | | fence r, rw | |----------------------------------| Signed-off-by: Xu Lu <luxu.kernel(a)bytedance.com> --- arch/riscv/include/asm/barrier.h | 91 ++++++++++++++++++++++++++++---- 1 file changed, 80 insertions(+), 11 deletions(-) diff --git a/arch/riscv/include/asm/barrier.h b/arch/riscv/include/asm/barrier.h index b8c5726d86acb..9eaf94a028096 100644 --- a/arch/riscv/include/asm/barrier.h +++ b/arch/riscv/include/asm/barrier.h @@ -51,19 +51,88 @@ */ #define smp_mb__after_spinlock() RISCV_FENCE(iorw, iorw) -#define __smp_store_release(p, v) \ -do { \ - compiletime_assert_atomic_type(*p); \ - RISCV_FENCE(rw, w); \ - WRITE_ONCE(*p, v); \ +extern void __bad_size_call_parameter(void); + +#define __smp_store_release(p, v) \ +do { \ + typeof(p) __p = (p); \ + union { typeof(*p) __val; char __c[1]; } __u = \ + { .__val = (__force typeof(*p)) (v) }; \ + compiletime_assert_atomic_type(*p); \ + switch (sizeof(*p)) { \ + case 1: \ + asm volatile(ALTERNATIVE("fence rw, w;\t\nsb %0, 0(%1)\t\n", \ + SB_RL(%0, %1) "\t\nnop\t\n", \ + 0, RISCV_ISA_EXT_ZALASR, 1) \ + : : "r" (*(__u8 *)__u.__c), "r" (__p) \ + : "memory"); \ + break; \ + case 2: \ + asm volatile(ALTERNATIVE("fence rw, w;\t\nsh %0, 0(%1)\t\n", \ + SH_RL(%0, %1) "\t\nnop\t\n", \ + 0, RISCV_ISA_EXT_ZALASR, 1) \ + : : "r" (*(__u16 *)__u.__c), "r" (__p) \ + : "memory"); \ + break; \ + case 4: \ + asm volatile(ALTERNATIVE("fence rw, w;\t\nsw %0, 0(%1)\t\n", \ + SW_RL(%0, %1) "\t\nnop\t\n", \ + 0, RISCV_ISA_EXT_ZALASR, 1) \ + : : "r" (*(__u32 *)__u.__c), "r" (__p) \ + : "memory"); \ + break; \ + case 8: \ + asm volatile(ALTERNATIVE("fence rw, w;\t\nsd %0, 0(%1)\t\n", \ + SD_RL(%0, %1) "\t\nnop\t\n", \ + 0, RISCV_ISA_EXT_ZALASR, 1) \ + : : "r" (*(__u64 *)__u.__c), "r" (__p) \ + : "memory"); \ + break; \ + default: \ + __bad_size_call_parameter(); \ + break; \ + } \ } while (0) -#define __smp_load_acquire(p) \ -({ \ - typeof(*p) ___p1 = READ_ONCE(*p); \ - compiletime_assert_atomic_type(*p); \ - RISCV_FENCE(r, rw); \ - ___p1; \ +#define __smp_load_acquire(p) \ +({ \ + union { typeof(*p) __val; char __c[1]; } __u; \ + typeof(p) __p = (p); \ + compiletime_assert_atomic_type(*p); \ + switch (sizeof(*p)) { \ + case 1: \ + asm volatile(ALTERNATIVE("lb %0, 0(%1)\t\nfence r, rw\t\n", \ + LB_AQ(%0, %1) "\t\nnop\t\n", \ + 0, RISCV_ISA_EXT_ZALASR, 1) \ + : "=r" (*(__u8 *)__u.__c) : "r" (__p) \ + : "memory"); \ + break; \ + case 2: \ + asm volatile(ALTERNATIVE("lh %0, 0(%1)\t\nfence r, rw\t\n", \ + LH_AQ(%0, %1) "\t\nnop\t\n", \ + 0, RISCV_ISA_EXT_ZALASR, 1) \ + : "=r" (*(__u16 *)__u.__c) : "r" (__p) \ + : "memory"); \ + break; \ + case 4: \ + asm volatile(ALTERNATIVE("lw %0, 0(%1)\t\nfence r, rw\t\n", \ + LW_AQ(%0, %1) "\t\nnop\t\n", \ + 0, RISCV_ISA_EXT_ZALASR, 1) \ + : "=r" (*(__u32 *)__u.__c) : "r" (__p) \ + : "memory"); \ + break; \ + case 8: \ + asm volatile(ALTERNATIVE("ld %0, 0(%1)\t\nfence r, rw\t\n", \ + LD_AQ(%0, %1) "\t\nnop\t\n", \ + 0, RISCV_ISA_EXT_ZALASR, 1) \ + : "=r" (*(__u64 *)__u.__c) : "r" (__p) \ + : "memory"); \ + break; \ + default: \ + __bad_size_call_parameter(); \ + break; \ + } \ + __u.__val; \ }) #ifdef CONFIG_RISCV_ISA_ZAWRS -- 2.20.1

1 month, 2 weeks

1
4
0 0

[PATCH net-next v6 1/2] net/tls: support setting the maximum payload size

by Wilfred Mallawa

From: Wilfred Mallawa <wilfred.mallawa(a)wdc.com> During a handshake, an endpoint may specify a maximum record size limit. Currently, the kernel defaults to TLS_MAX_PAYLOAD_SIZE (16KB) for the maximum record size. Meaning that, the outgoing records from the kernel can exceed a lower size negotiated during the handshake. In such a case, the TLS endpoint must send a fatal "record_overflow" alert [1], and thus the record is discarded. Upcoming Western Digital NVMe-TCP hardware controllers implement TLS support. For these devices, supporting TLS record size negotiation is necessary because the maximum TLS record size supported by the controller is less than the default 16KB currently used by the kernel. Currently, there is no way to inform the kernel of such a limit. This patch adds support to a new setsockopt() option `TLS_TX_MAX_PAYLOAD_LEN` that allows for setting the maximum plaintext fragment size. Once set, outgoing records are no larger than the size specified. This option can be used to specify the record size limit. [1] https://www.rfc-editor.org/rfc/rfc8449 Tested-by: syzbot(a)syzkaller.appspotmail.com Signed-off-by: Wilfred Mallawa <wilfred.mallawa(a)wdc.com> --- Changes V5 -> V6: - Add NULL check for sw_ctx. Reported by syzbot. V5: https://lore.kernel.org/netdev/20251014051825.1084403-2-wilfred.opensource@… --- Documentation/networking/tls.rst | 11 ++++++ include/net/tls.h | 3 ++ include/uapi/linux/tls.h | 2 ++ net/tls/tls_device.c | 2 +- net/tls/tls_main.c | 62 ++++++++++++++++++++++++++++++++ net/tls/tls_sw.c | 2 +- 6 files changed, 80 insertions(+), 2 deletions(-) diff --git a/Documentation/networking/tls.rst b/Documentation/networking/tls.rst index 36cc7afc2527..dabab17ab84a 100644 --- a/Documentation/networking/tls.rst +++ b/Documentation/networking/tls.rst @@ -280,6 +280,17 @@ If the record decrypted turns out to had been padded or is not a data record it will be decrypted again into a kernel buffer without zero copy. Such events are counted in the ``TlsDecryptRetry`` statistic. +TLS_TX_MAX_PAYLOAD_LEN +~~~~~~~~~~~~~~~~~~~~~~ + +Sets the maximum size for the plaintext of a protected record. + +When this option is set, the kernel enforces this limit on all transmitted TLS +records, ensuring no plaintext fragment exceeds the specified size. This can be +used to specify the TLS Record Size Limit [1]. + +[1] https://datatracker.ietf.org/doc/html/rfc8449 + Statistics ========== diff --git a/include/net/tls.h b/include/net/tls.h index 857340338b69..f2af113728aa 100644 --- a/include/net/tls.h +++ b/include/net/tls.h @@ -53,6 +53,8 @@ struct tls_rec; /* Maximum data size carried in a TLS record */ #define TLS_MAX_PAYLOAD_SIZE ((size_t)1 << 14) +/* Minimum record size limit as per RFC8449 */ +#define TLS_MIN_RECORD_SIZE_LIM ((size_t)1 << 6) #define TLS_HEADER_SIZE 5 #define TLS_NONCE_OFFSET TLS_HEADER_SIZE @@ -226,6 +228,7 @@ struct tls_context { u8 rx_conf:3; u8 zerocopy_sendfile:1; u8 rx_no_pad:1; + u16 tx_max_payload_len; int (*push_pending_record)(struct sock *sk, int flags); void (*sk_write_space)(struct sock *sk); diff --git a/include/uapi/linux/tls.h b/include/uapi/linux/tls.h index b66a800389cc..b8b9c42f848c 100644 --- a/include/uapi/linux/tls.h +++ b/include/uapi/linux/tls.h @@ -41,6 +41,7 @@ #define TLS_RX 2 /* Set receive parameters */ #define TLS_TX_ZEROCOPY_RO 3 /* TX zerocopy (only sendfile now) */ #define TLS_RX_EXPECT_NO_PAD 4 /* Attempt opportunistic zero-copy */ +#define TLS_TX_MAX_PAYLOAD_LEN 5 /* Maximum plaintext size */ /* Supported versions */ #define TLS_VERSION_MINOR(ver) ((ver) & 0xFF) @@ -194,6 +195,7 @@ enum { TLS_INFO_RXCONF, TLS_INFO_ZC_RO_TX, TLS_INFO_RX_NO_PAD, + TLS_INFO_TX_MAX_PAYLOAD_LEN, __TLS_INFO_MAX, }; #define TLS_INFO_MAX (__TLS_INFO_MAX - 1) diff --git a/net/tls/tls_device.c b/net/tls/tls_device.c index a64ae15b1a60..c6289c73cffc 100644 --- a/net/tls/tls_device.c +++ b/net/tls/tls_device.c @@ -461,7 +461,7 @@ static int tls_push_data(struct sock *sk, /* TLS_HEADER_SIZE is not counted as part of the TLS record, and * we need to leave room for an authentication tag. */ - max_open_record_len = TLS_MAX_PAYLOAD_SIZE + + max_open_record_len = tls_ctx->tx_max_payload_len + prot->prepend_size; do { rc = tls_do_allocation(sk, ctx, pfrag, prot->prepend_size); diff --git a/net/tls/tls_main.c b/net/tls/tls_main.c index a3ccb3135e51..b96c825b90e9 100644 --- a/net/tls/tls_main.c +++ b/net/tls/tls_main.c @@ -544,6 +544,28 @@ static int do_tls_getsockopt_no_pad(struct sock *sk, char __user *optval, return 0; } +static int do_tls_getsockopt_tx_payload_len(struct sock *sk, char __user *optval, + int __user *optlen) +{ + struct tls_context *ctx = tls_get_ctx(sk); + u16 payload_len = ctx->tx_max_payload_len; + int len; + + if (get_user(len, optlen)) + return -EFAULT; + + if (len < sizeof(payload_len)) + return -EINVAL; + + if (put_user(sizeof(payload_len), optlen)) + return -EFAULT; + + if (copy_to_user(optval, &payload_len, sizeof(payload_len))) + return -EFAULT; + + return 0; +} + static int do_tls_getsockopt(struct sock *sk, int optname, char __user *optval, int __user *optlen) { @@ -563,6 +585,9 @@ static int do_tls_getsockopt(struct sock *sk, int optname, case TLS_RX_EXPECT_NO_PAD: rc = do_tls_getsockopt_no_pad(sk, optval, optlen); break; + case TLS_TX_MAX_PAYLOAD_LEN: + rc = do_tls_getsockopt_tx_payload_len(sk, optval, optlen); + break; default: rc = -ENOPROTOOPT; break; @@ -812,6 +837,30 @@ static int do_tls_setsockopt_no_pad(struct sock *sk, sockptr_t optval, return rc; } +static int do_tls_setsockopt_tx_payload_len(struct sock *sk, sockptr_t optval, + unsigned int optlen) +{ + struct tls_context *ctx = tls_get_ctx(sk); + struct tls_sw_context_tx *sw_ctx = tls_sw_ctx_tx(ctx); + u16 value; + + if (sw_ctx && sw_ctx->open_rec) + return -EBUSY; + + if (sockptr_is_null(optval) || optlen != sizeof(value)) + return -EINVAL; + + if (copy_from_sockptr(&value, optval, sizeof(value))) + return -EFAULT; + + if (value < TLS_MIN_RECORD_SIZE_LIM || value > TLS_MAX_PAYLOAD_SIZE) + return -EINVAL; + + ctx->tx_max_payload_len = value; + + return 0; +} + static int do_tls_setsockopt(struct sock *sk, int optname, sockptr_t optval, unsigned int optlen) { @@ -833,6 +882,11 @@ static int do_tls_setsockopt(struct sock *sk, int optname, sockptr_t optval, case TLS_RX_EXPECT_NO_PAD: rc = do_tls_setsockopt_no_pad(sk, optval, optlen); break; + case TLS_TX_MAX_PAYLOAD_LEN: + lock_sock(sk); + rc = do_tls_setsockopt_tx_payload_len(sk, optval, optlen); + release_sock(sk); + break; default: rc = -ENOPROTOOPT; break; @@ -1022,6 +1076,7 @@ static int tls_init(struct sock *sk) ctx->tx_conf = TLS_BASE; ctx->rx_conf = TLS_BASE; + ctx->tx_max_payload_len = TLS_MAX_PAYLOAD_SIZE; update_sk_prot(sk, ctx); out: write_unlock_bh(&sk->sk_callback_lock); @@ -1111,6 +1166,12 @@ static int tls_get_info(struct sock *sk, struct sk_buff *skb, bool net_admin) goto nla_failure; } + err = nla_put_u16(skb, TLS_INFO_TX_MAX_PAYLOAD_LEN, + ctx->tx_max_payload_len); + + if (err) + goto nla_failure; + rcu_read_unlock(); nla_nest_end(skb, start); return 0; @@ -1132,6 +1193,7 @@ static size_t tls_get_info_size(const struct sock *sk, bool net_admin) nla_total_size(sizeof(u16)) + /* TLS_INFO_TXCONF */ nla_total_size(0) + /* TLS_INFO_ZC_RO_TX */ nla_total_size(0) + /* TLS_INFO_RX_NO_PAD */ + nla_total_size(sizeof(u16)) + /* TLS_INFO_TX_MAX_PAYLOAD_LEN */ 0; return size; diff --git a/net/tls/tls_sw.c b/net/tls/tls_sw.c index daac9fd4be7e..e76ea38b712a 100644 --- a/net/tls/tls_sw.c +++ b/net/tls/tls_sw.c @@ -1079,7 +1079,7 @@ static int tls_sw_sendmsg_locked(struct sock *sk, struct msghdr *msg, orig_size = msg_pl->sg.size; full_record = false; try_to_copy = msg_data_left(msg); - record_room = TLS_MAX_PAYLOAD_SIZE - msg_pl->sg.size; + record_room = tls_ctx->tx_max_payload_len - msg_pl->sg.size; if (try_to_copy >= record_room) { try_to_copy = record_room; full_record = true; -- 2.51.0

1 month, 2 weeks

2
3
0 0

Family December trip

by Pina Alvarez

Hello, I was wondering if you received the email I sent last week regarding the December trip, I would hope you can plan for myself and my family of 14 (10 Adults & 4 Children) am attaching an itinerary also for you to take a look thank you Pina Alvarez

1 month, 2 weeks

1
0
0 0

[RFC PATCH 0/4] selftests/damon: add a test for obsolete_target sysfs file

by SeongJae Park

An RFC patch series [1] that add a new DAMON sysfs file for arbitrary targets removal is under review. Add a selftest for the feature. The new test uses the feature using the python wrapper of DAMON sysfs interface, and confirm the expected internal data structure change is made using drgn. So this patch series may better to be a part of the other one [1] that introduces the obsolete_target file. But, because no significant change is requested on the series so far, I'm posting this as an individual RFC. In the next version, I may merge the two series into one, to add all related changes at one step. [1] https://lore.kernel.org/20251016214736.84286-1-sj@kernel.org SeongJae Park (4): selftests/damon/_damon_sysfs: support obsolete_target file drgn_dump_damon_status: dump damon_target->obsolete sysfs.py: extend assert_ctx_committed() for monitoring targets selftests/damon/sysfs: add obsolete_target test tools/testing/selftests/damon/_damon_sysfs.py | 11 ++++- .../selftests/damon/drgn_dump_damon_status.py | 1 + tools/testing/selftests/damon/sysfs.py | 48 +++++++++++++++++++ 3 files changed, 58 insertions(+), 2 deletions(-) base-commit: 1aba8bd57e6aaa1c9e699c8de66bcc931d4b1116 -- 2.47.3

1 month, 2 weeks

1
4
0 0

[PATCH net-next] bonding: show master index when dumping slave info

by Hangbin Liu

Currently, there is no straightforward way to obtain the master/slave relationship via netlink. Users have to retrieve all slaves through sysfs to determine these relationships. To address this, we can either list all slaves under the bond interface or display the master index in each slave. Since the number of slaves could be quite large (e.g., 100+), it is more efficient to show the master information in the slave entry. Signed-off-by: Hangbin Liu <liuhangbin(a)gmail.com> --- drivers/net/bonding/bond_netlink.c | 4 ++++ include/uapi/linux/if_link.h | 1 + 2 files changed, 5 insertions(+) diff --git a/drivers/net/bonding/bond_netlink.c b/drivers/net/bonding/bond_netlink.c index 286f11c517f7..ff3f11674a8b 100644 --- a/drivers/net/bonding/bond_netlink.c +++ b/drivers/net/bonding/bond_netlink.c @@ -29,6 +29,7 @@ static size_t bond_get_slave_size(const struct net_device *bond_dev, nla_total_size(sizeof(u16)) + /* IFLA_BOND_SLAVE_AD_PARTNER_OPER_PORT_STATE */ nla_total_size(sizeof(s32)) + /* IFLA_BOND_SLAVE_PRIO */ nla_total_size(sizeof(u16)) + /* IFLA_BOND_SLAVE_ACTOR_PORT_PRIO */ + nla_total_size(sizeof(u32)) + /* IFLA_BOND_SLAVE_MASTER */ 0; } @@ -38,6 +39,9 @@ static int bond_fill_slave_info(struct sk_buff *skb, { struct slave *slave = bond_slave_get_rtnl(slave_dev); + if (nla_put_u32(skb, IFLA_BOND_SLAVE_MASTER, bond_dev->ifindex)) + goto nla_put_failure; + if (nla_put_u8(skb, IFLA_BOND_SLAVE_STATE, bond_slave_state(slave))) goto nla_put_failure; diff --git a/include/uapi/linux/if_link.h b/include/uapi/linux/if_link.h index 3b491d96e52e..bad41a1807f7 100644 --- a/include/uapi/linux/if_link.h +++ b/include/uapi/linux/if_link.h @@ -1567,6 +1567,7 @@ enum { IFLA_BOND_SLAVE_AD_PARTNER_OPER_PORT_STATE, IFLA_BOND_SLAVE_PRIO, IFLA_BOND_SLAVE_ACTOR_PORT_PRIO, + IFLA_BOND_SLAVE_MASTER, __IFLA_BOND_SLAVE_MAX, }; -- 2.50.1

1 month, 2 weeks

4
6
0 0

Re: [PATCH 1/1] Use new `--output-format=doctest` rustdoc command line flag to improve doctest handling

by Miguel Ojeda

On Tue, 17 Jun 2025 16:00:33 +0200 Guillaume Gomez <guillaume1.gomez(a)gmail.com> wrote: > > The goal of this patch is to remove the use of 2 unstable > rustdoc features (`--no-run` and `--test-builder`) and replace it with a > stable feature: `--output-format=doctest`, which was added in > https://github.com/rust-lang/rust/pull/134531. > > Before this patch, the code was using very hacky methods in order to retrieve > doctests, modify them as needed and then concatenate all of them in one file. > > Now, with this new flag, it instead asks rustdoc to provide the doctests > code with their associated information such as file path and line number. > > Signed-off-by: Guillaume Gomez <guillaume1.gomez(a)gmail.com> > --- (Procedural bit: normally we provide a changelog between versions after this `---` line so that reviewers now what changed so far.) I finally took a look at this again, so I rebased it and got: thread 'main' panicked at scripts/rustdoc_test_gen.rs:92:15: No path candidates found for `rust_kernel_alloc_allocator.rs`.This is likely a bug in the build system, or some files went away while compiling. which brings me to the bigger point: the main reason to have the new output format is to avoid all these hacks, including the "find the real path back to the original file" hack here. More generally, to avoid the 2 scripts approach. So now we can finally get rid of all that and simplify. That is, we can just merge it all in a single script that reads the JSON and builds the result directly, since now we have everything we need (originally I needed the 2 scripts approach since `rustdoc` executed the test builder once per test so I had to somehow collect the results). i.e. no more hundreds of generated files/processes, just a simple pipe. Anyway, just to check we had everything we needed, I did a quick try -- please see the draft patch below. I gave it a go -- please see the draft patch below. The diff w.r.t. your patch would be something like +217 -341, i.e. we get rid of quite a lot of lines. I added as well some more context in the commit message, and put the right docs in the unified script. This also improves the sorting of the tests (it now follows the line number better). We still have to preserve the support for the old compilers, so what I think I will do is just have the new script separately, keeping the old ones as-is until we can remove them when we upgrade the minimum for e.g. the next Debian Stable. Cc'ing David and KUnit, since this is closer to getting ready -- please let me know if this raises alarms for anyone. Thanks! Cheers, Miguel From 4aa4581e9004cb95534805f73fdae56c454b3d1d Mon Sep 17 00:00:00 2001 From: Guillaume Gomez <guillaume1.gomez(a)gmail.com> Date: Tue, 17 Jun 2025 16:00:33 +0200 Subject: [PATCH] [TODO] rust: use new `rustdoc`'s `--output-format=doctest` The goal of this patch is to remove the use of 2 unstable `rustdoc` features (`--no-run` and `--test-builder`) and replace it with a future stable feature: `--output-format=doctest` [1]. Before this patch, the KUnit Rust doctests generation needed to employ several hacks in order to retrieve doctests, modify them as needed and then concatenate all of them in one file. In particular, it required using two scripts: one that got run as a test builder by `rustdoc` in order to extract the data and another that collected the results of all those processes. We requested upstream `rustdoc` a feature to get `rustdoc` to generate the information directly -- one that would also be designed to eventually be made stable. This resulted in the `--output-format=doctest` flag, which makes all the information neatly available as a JSON output, including filenames, line numbers, doctest test bodies and so on. Thus take advantage of the new flag, which in turn allows to just use a single script that gets piped that JSON output from the compiler and uses it to directly build the generated files to be run by KUnit. Link: https://github.com/rust-lang/rust/issues/134529 [1] Signed-off-by: Guillaume Gomez <guillaume1.gomez(a)gmail.com> Co-developed-by: Miguel Ojeda <ojeda(a)kernel.org> Signed-off-by: Miguel Ojeda <ojeda(a)kernel.org> --- rust/Makefile | 12 +- scripts/.gitignore | 1 - scripts/Makefile | 2 - scripts/json.rs | 235 +++++++++++++++++++++++++ scripts/remove-stale-files | 2 + scripts/rustdoc_test_builder.rs | 300 ++++++++++++++++++++++++++------ scripts/rustdoc_test_gen.rs | 265 ---------------------------- 7 files changed, 485 insertions(+), 332 deletions(-) create mode 100644 scripts/json.rs delete mode 100644 scripts/rustdoc_test_gen.rs diff --git a/rust/Makefile b/rust/Makefile index 23c7ae905bd2..93bc456e3576 100644 --- a/rust/Makefile +++ b/rust/Makefile @@ -57,7 +57,6 @@ RUST_LIB_SRC ?= $(rustc_sysroot)/lib/rustlib/src/rust/library ifneq ($(quiet),) rust_test_quiet=-q rustdoc_test_quiet=--test-args -q -rustdoc_test_kernel_quiet=>/dev/null endif core-cfgs = \ @@ -224,21 +223,20 @@ quiet_cmd_rustdoc_test_kernel = RUSTDOC TK $< rm -rf $(objtree)/$(obj)/test/doctests/kernel; \ mkdir -p $(objtree)/$(obj)/test/doctests/kernel; \ OBJTREE=$(abspath $(objtree)) \ - $(RUSTDOC) --test $(filter-out --remap-path-prefix=%,$(rust_flags)) \ + $(RUSTDOC) $(filter-out --remap-path-prefix=%,$(rust_flags)) \ -L$(objtree)/$(obj) --extern ffi --extern pin_init \ --extern kernel --extern build_error --extern macros \ --extern bindings --extern uapi \ - --no-run --crate-name kernel -Zunstable-options \ + --crate-name kernel -Zunstable-options \ --sysroot=/dev/null \ + --output-format=doctest \ $(rustdoc_modifiers_workaround) \ - --test-builder $(objtree)/scripts/rustdoc_test_builder \ - $< $(rustdoc_test_kernel_quiet); \ - $(objtree)/scripts/rustdoc_test_gen + $< | $(objtree)/scripts/rustdoc_test_builder %/doctests_kernel_generated.rs %/doctests_kernel_generated_kunit.c: \ $(src)/kernel/lib.rs $(obj)/kernel.o \ $(objtree)/scripts/rustdoc_test_builder \ - $(objtree)/scripts/rustdoc_test_gen FORCE + FORCE +$(call if_changed,rustdoc_test_kernel) # We cannot use `-Zpanic-abort-tests` because some tests are dynamic, diff --git a/scripts/.gitignore b/scripts/.gitignore index c2ef68848da5..6e6ab7b8f496 100644 --- a/scripts/.gitignore +++ b/scripts/.gitignore @@ -7,7 +7,6 @@ /module.lds /recordmcount /rustdoc_test_builder -/rustdoc_test_gen /sign-file /sorttable /target.json diff --git a/scripts/Makefile b/scripts/Makefile index 46f860529df5..71c7d9dcd95b 100644 --- a/scripts/Makefile +++ b/scripts/Makefile @@ -10,7 +10,6 @@ hostprogs-always-$(CONFIG_ASN1) += asn1_compiler hostprogs-always-$(CONFIG_MODULE_SIG_FORMAT) += sign-file hostprogs-always-$(CONFIG_SYSTEM_EXTRA_CERTIFICATE) += insert-sys-cert hostprogs-always-$(CONFIG_RUST_KERNEL_DOCTESTS) += rustdoc_test_builder -hostprogs-always-$(CONFIG_RUST_KERNEL_DOCTESTS) += rustdoc_test_gen ifneq ($(or $(CONFIG_X86_64),$(CONFIG_X86_32)),) always-$(CONFIG_RUST) += target.json @@ -23,7 +22,6 @@ endif hostprogs += generate_rust_target generate_rust_target-rust := y rustdoc_test_builder-rust := y -rustdoc_test_gen-rust := y HOSTCFLAGS_sorttable.o = -I$(srctree)/tools/include HOSTLDLIBS_sorttable = -lpthread diff --git a/scripts/json.rs b/scripts/json.rs new file mode 100644 index 000000000000..aff24bfd9213 --- /dev/null +++ b/scripts/json.rs @@ -0,0 +1,235 @@ +// SPDX-License-Identifier: GPL-2.0 + +//! JSON parser used to parse rustdoc output when retrieving doctests. + +use std::collections::HashMap; +use std::iter::Peekable; +use std::str::FromStr; + +#[derive(Debug, PartialEq, Eq)] +pub(crate) enum JsonValue { + Object(HashMap<String, JsonValue>), + String(String), + Number(i32), + Bool(bool), + Array(Vec<JsonValue>), + Null, +} + +fn parse_ident<I: Iterator<Item = char>>( + iter: &mut I, + output: JsonValue, + ident: &str, +) -> Result<JsonValue, String> { + let mut ident_iter = ident.chars().skip(1); + + loop { + let i = ident_iter.next(); + if i.is_none() { + return Ok(output); + } + let c = iter.next(); + if i != c { + if let Some(c) = c { + return Err(format!("Unexpected character `{c}` when parsing `{ident}`")); + } + return Err(format!("Missing character when parsing `{ident}`")); + } + } +} + +fn parse_string<I: Iterator<Item = char>>(iter: &mut I) -> Result<JsonValue, String> { + let mut out = String::new(); + + while let Some(c) = iter.next() { + match c { + '\\' => { + let Some(c) = iter.next() else { break }; + match c { + '"' | '\\' | '/' => out.push(c), + 'b' => out.push(char::from(0x8u8)), + 'f' => out.push(char::from(0xCu8)), + 't' => out.push('\t'), + 'r' => out.push('\r'), + 'n' => out.push('\n'), + _ => { + // This code doesn't handle codepoints so we put the string content as is. + out.push('\\'); + out.push(c); + } + } + } + '"' => { + return Ok(JsonValue::String(out)); + } + _ => out.push(c), + } + } + Err(format!("Unclosed JSON string `{out}`")) +} + +fn parse_number<I: Iterator<Item = char>>( + iter: &mut Peekable<I>, + digit: char, +) -> Result<JsonValue, String> { + let mut nb = String::new(); + + nb.push(digit); + loop { + // We peek next character to prevent taking it from the iterator in case it's a comma. + if matches!(iter.peek(), Some(',' | '}' | ']')) { + break; + } + let Some(c) = iter.next() else { break }; + if c.is_whitespace() { + break; + } else if !c.is_ascii_digit() { + return Err(format!("Error when parsing number `{nb}`: found `{c}`")); + } + nb.push(c); + } + i32::from_str(&nb) + .map(|nb| JsonValue::Number(nb)) + .map_err(|error| format!("Invalid number: `{error}`")) +} + +fn parse_array<I: Iterator<Item = char>>(iter: &mut Peekable<I>) -> Result<JsonValue, String> { + let mut values = Vec::new(); + + 'main: loop { + let Some(c) = iter.next() else { + return Err("Unclosed array".to_string()); + }; + if c.is_whitespace() { + continue; + } else if c == ']' { + break; + } + values.push(parse(iter, c)?); + while let Some(c) = iter.next() { + if c.is_whitespace() { + continue; + } else if c == ',' { + break; + } else if c == ']' { + break 'main; + } else { + return Err(format!("Unexpected `{c}` when parsing array")); + } + } + } + Ok(JsonValue::Array(values)) +} + +fn parse_object<I: Iterator<Item = char>>(iter: &mut Peekable<I>) -> Result<JsonValue, String> { + let mut values = HashMap::new(); + + 'main: loop { + let Some(c) = iter.next() else { + return Err("Unclosed object".to_string()); + }; + let key; + if c.is_whitespace() { + continue; + } else if c == '"' { + let JsonValue::String(k) = parse_string(iter)? else { + unreachable!() + }; + key = k; + } else if c == '}' { + break; + } else { + return Err(format!("Expected `\"` when parsing Object, found `{c}`")); + } + + // We then get the `:` separator. + loop { + let Some(c) = iter.next() else { + return Err(format!("Missing value after key `{key}`")); + }; + if c.is_whitespace() { + continue; + } else if c == ':' { + break; + } else { + return Err(format!( + "Expected `:` after key, found `{c}` when parsing object" + )); + } + } + // Then the value. + let value = loop { + let Some(c) = iter.next() else { + return Err(format!("Missing value after key `{key}`")); + }; + if c.is_whitespace() { + continue; + } else { + break parse(iter, c)?; + } + }; + + if values.contains_key(&key) { + return Err(format!("Duplicated key `{key}`")); + } + values.insert(key, value); + + while let Some(c) = iter.next() { + if c.is_whitespace() { + continue; + } else if c == ',' { + break; + } else if c == '}' { + break 'main; + } else { + return Err(format!("Unexpected `{c}` when parsing array")); + } + } + } + Ok(JsonValue::Object(values)) +} + +fn parse<I: Iterator<Item = char>>(iter: &mut Peekable<I>, c: char) -> Result<JsonValue, String> { + match c { + '{' => parse_object(iter), + '"' => parse_string(iter), + '[' => parse_array(iter), + 't' => parse_ident(iter, JsonValue::Bool(true), "true"), + 'f' => parse_ident(iter, JsonValue::Bool(false), "false"), + 'n' => parse_ident(iter, JsonValue::Null, "null"), + c => { + if c.is_ascii_digit() || c == '-' { + parse_number(iter, c) + } else { + Err(format!("Unexpected `{c}` character")) + } + } + } +} + +impl JsonValue { + pub(crate) fn parse(input: &str) -> Result<Self, String> { + let mut iter = input.chars().peekable(); + let mut value = None; + + while let Some(c) = iter.next() { + if c.is_whitespace() { + continue; + } + value = Some(parse(&mut iter, c)?); + break; + } + while let Some(c) = iter.next() { + if c.is_whitespace() { + continue; + } else { + return Err(format!("Unexpected character `{c}` after content")); + } + } + if let Some(value) = value { + Ok(value) + } else { + Err("Empty content".to_string()) + } + } +} diff --git a/scripts/remove-stale-files b/scripts/remove-stale-files index 6e39fa8540df..190dee6b50e8 100755 --- a/scripts/remove-stale-files +++ b/scripts/remove-stale-files @@ -26,3 +26,5 @@ rm -f scripts/selinux/genheaders/genheaders rm -f *.spec rm -f lib/test_fortify.log + +rm -f scripts/rustdoc_test_gen diff --git a/scripts/rustdoc_test_builder.rs b/scripts/rustdoc_test_builder.rs index f7540bcf595a..dd65bb670d25 100644 --- a/scripts/rustdoc_test_builder.rs +++ b/scripts/rustdoc_test_builder.rs @@ -1,74 +1,260 @@ // SPDX-License-Identifier: GPL-2.0 -//! Test builder for `rustdoc`-generated tests. +//! Generates KUnit tests from `rustdoc`-generated doctests. //! -//! This script is a hack to extract the test from `rustdoc`'s output. Ideally, `rustdoc` would -//! have an option to generate this information instead, e.g. as JSON output. +//! KUnit passes a context (`struct kunit *`) to each test, which should be forwarded to the other +//! KUnit functions and macros. //! -//! The `rustdoc`-generated test names look like `{file}_{line}_{number}`, e.g. -//! `...path_rust_kernel_sync_arc_rs_42_0`. `number` is the "test number", needed in cases like -//! a macro that expands into items with doctests is invoked several times within the same line. +//! However, we want to keep this as an implementation detail because: //! -//! However, since these names are used for bisection in CI, the line number makes it not stable -//! at all. In the future, we would like `rustdoc` to give us the Rust item path associated with -//! the test, plus a "test number" (for cases with several examples per item) and generate a name -//! from that. For the moment, we generate ourselves a new name, `{file}_{number}` instead, in -//! the `gen` script (done there since we need to be aware of all the tests in a given file). +//! - Test code should not care about the implementation. +//! +//! - Documentation looks worse if it needs to carry extra details unrelated to the piece +//! being described. +//! +//! - Test code should be able to define functions and call them, without having to carry +//! the context. +//! +//! - Later on, we may want to be able to test non-kernel code (e.g. `core` or third-party +//! crates) which likely use the standard library `assert*!` macros. +//! +//! For this reason, instead of the passed context, `kunit_get_current_test()` is used instead +//! (i.e. `current->kunit_test`). +//! +//! Note that this means other threads/tasks potentially spawned by a given test, if failing, will +//! report the failure in the kernel log but will not fail the actual test. Saving the pointer in +//! e.g. a `static` per test does not fully solve the issue either, because currently KUnit does +//! not support assertions (only expectations) from other tasks. Thus leave that feature for +//! the future, which simplifies the code here too. We could also simply not allow `assert`s in +//! other tasks, but that seems overly constraining, and we do want to support them, eventually. -use std::io::Read; +use std::{ + fs::File, + io::{BufWriter, Read, Write}, +}; + +use json::JsonValue; + +mod json; fn main() { let mut stdin = std::io::stdin().lock(); - let mut body = String::new(); - stdin.read_to_string(&mut body).unwrap(); + let mut rustdoc_json = String::new(); + stdin.read_to_string(&mut rustdoc_json).unwrap(); - // Find the generated function name looking for the inner function inside `main()`. - // - // The line we are looking for looks like one of the following: - // - // ``` - // fn main() { #[allow(non_snake_case)] fn _doctest_main_rust_kernel_file_rs_28_0() { - // fn main() { #[allow(non_snake_case)] fn _doctest_main_rust_kernel_file_rs_37_0() -> Result<(), impl ::core::fmt::Debug> { - // ``` - // - // It should be unlikely that doctest code matches such lines (when code is formatted properly). - let rustdoc_function_name = body - .lines() - .find_map(|line| { - Some( - line.split_once("fn main() {")? - .1 - .split_once("fn ")? - .1 - .split_once("()")? - .0, - ) - .filter(|x| x.chars().all(|c| c.is_alphanumeric() || c == '_')) - }) - .expect("No test function found in `rustdoc`'s output."); - - // Qualify `Result` to avoid the collision with our own `Result` coming from the prelude. - let body = body.replace( - &format!("{rustdoc_function_name}() -> Result<(), impl ::core::fmt::Debug> {{"), - &format!( - "{rustdoc_function_name}() -> ::core::result::Result<(), impl ::core::fmt::Debug> {{" - ), + let JsonValue::Object(rustdoc) = JsonValue::parse(&rustdoc_json).unwrap() else { + panic!("Expected an object") + }; + + let Some(JsonValue::Number(format_version)) = rustdoc.get("format_version") else { + panic!("missing `format_version` field"); + }; + assert!( + *format_version == 2, + "unsupported rustdoc format version: {format_version}" ); - // For tests that get generated with `Result`, like above, `rustdoc` generates an `unwrap()` on - // the return value to check there were no returned errors. Instead, we use our assert macro - // since we want to just fail the test, not panic the kernel. + let Some(JsonValue::Array(doctests)) = rustdoc.get("doctests") else { + panic!("`doctests` field is missing or has the wrong type"); + }; + + let mut nb_generated = 0; + let mut number = 0; + let mut last_file = ""; + let mut rust_tests = String::new(); + let mut c_test_declarations = String::new(); + let mut c_test_cases = String::new(); + for doctest in doctests { + let JsonValue::Object(doctest) = doctest else { + unreachable!() + }; + + // We check if we need to skip this test by checking it's a rust code and it's not ignored. + if let Some(JsonValue::Object(attributes)) = doctest.get("doctest_attributes") { + if attributes.get("rust") != Some(&JsonValue::Bool(true)) { + continue; + } + if let Some(JsonValue::String(ignore)) = attributes.get("ignore") { + if ignore != "None" { + continue; + } + } + } + + let ( + Some(JsonValue::String(file)), + Some(JsonValue::Number(line)), + Some(JsonValue::String(name)), + Some(JsonValue::Object(doctest_code)), + ) = ( + doctest.get("file"), + doctest.get("line"), + doctest.get("name"), + doctest.get("doctest_code"), + ) + else { + continue; + }; + + let ( + Some(JsonValue::String(code)), + Some(JsonValue::String(crate_level_code)), + Some(JsonValue::Object(wrapper)), + ) = ( + doctest_code.get("code"), + doctest_code.get("crate_level"), + doctest_code.get("wrapper"), + ) + else { + continue; + }; + + let (Some(JsonValue::String(before)), Some(JsonValue::String(after))) = + (wrapper.get("before"), wrapper.get("after")) + else { + continue; + }; + + // For tests that get generated with `Result`, `rustdoc` generates an `unwrap()` on + // the return value to check there were no returned errors. Instead, we use our assert macro + // since we want to just fail the test, not panic the kernel. + // + // We save the result in a variable so that the failed assertion message looks nicer. + let after = if let Some(JsonValue::Bool(true)) = wrapper.get("returns_result") { + "\n} let test_return_value = _inner(); assert!(test_return_value.is_ok()); }" + } else { + after.as_str() + }; + + let body = format!("{crate_level_code}\n{before}\n{code}{after}\n"); + nb_generated += 1; + + // Generate an ID sequence ("test number") for each one in the file. + if file == last_file { + number += 1; + } else { + number = 0; + last_file = file; + } + + // Generate a KUnit name (i.e. test name and C symbol) for this test. + // + // We avoid the line number, like `rustdoc` does, to make things slightly more stable for + // bisection purposes. However, to aid developers in mapping back what test failed, we will + // print a diagnostics line in the KTAP report. + let kunit_name = format!( + "rust_doctest_{}_{number}", + file.replace('/', "_").replace('.', "_") + ); + + // Calculate how many lines before `main` function (including the `main` function line). + let body_offset = body + .lines() + .take_while(|line| !line.contains("fn main() {")) + .count() + + 1; + + use std::fmt::Write; + write!( + rust_tests, + r#"/// Generated `{name}` KUnit test case from a Rust documentation test. +#[no_mangle] +pub extern "C" fn {kunit_name}(__kunit_test: *mut ::kernel::bindings::kunit) {{ + /// Overrides the usual [`assert!`] macro with one that calls KUnit instead. + #[allow(unused)] + macro_rules! assert {{ + ($cond:expr $(,)?) => {{{{ + ::kernel::kunit_assert!( + "{kunit_name}", "{file}", __DOCTEST_ANCHOR - {line}, $cond + ); + }}}} + }} + + /// Overrides the usual [`assert_eq!`] macro with one that calls KUnit instead. + #[allow(unused)] + macro_rules! assert_eq {{ + ($left:expr, $right:expr $(,)?) => {{{{ + ::kernel::kunit_assert_eq!( + "{kunit_name}", "{file}", __DOCTEST_ANCHOR - {line}, $left, $right + ); + }}}} + }} + + // Many tests need the prelude, so provide it by default. + #[allow(unused)] + use ::kernel::prelude::*; + + // Unconditionally print the location of the original doctest (i.e. rather than the location in + // the generated file) so that developers can easily map the test back to the source code. // - // We save the result in a variable so that the failed assertion message looks nicer. - let body = body.replace( - &format!("}} {rustdoc_function_name}().unwrap() }}"), - &format!("}} let test_return_value = {rustdoc_function_name}(); assert!(test_return_value.is_ok()); }}"), - ); + // This information is also printed when assertions fail, but this helps in the successful cases + // when the user is running KUnit manually, or when passing `--raw_output` to `kunit.py`. + // + // This follows the syntax for declaring test metadata in the proposed KTAP v2 spec, which may + // be used for the proposed KUnit test attributes API. Thus hopefully this will make migration + // easier later on. + ::kernel::kunit::info(fmt!(" # {kunit_name}.location: {file}:{line}\n")); + + /// The anchor where the test code body starts. + #[allow(unused)] + static __DOCTEST_ANCHOR: i32 = ::core::line!() as i32 + {body_offset} + 1; + {{ + {body} + main(); + }} +}} + +"# + ) + .unwrap(); + + write!(c_test_declarations, "void {kunit_name}(struct kunit *);\n").unwrap(); + write!(c_test_cases, " KUNIT_CASE({kunit_name}),\n").unwrap(); + } + + if nb_generated == 0 { + panic!("No test function found in `rustdoc`'s output."); + } + + let rust_tests = rust_tests.trim(); + let c_test_declarations = c_test_declarations.trim(); + let c_test_cases = c_test_cases.trim(); + + write!( + BufWriter::new(File::create("rust/doctests_kernel_generated.rs").unwrap()), + r#"//! `kernel` crate documentation tests. + +const __LOG_PREFIX: &[u8] = b"rust_doctests_kernel\0"; + +{rust_tests} +"# + ) + .unwrap(); + + write!( + BufWriter::new(File::create("rust/doctests_kernel_generated_kunit.c").unwrap()), + r#"/* + * `kernel` crate documentation tests. + */ + +#include <kunit/test.h> + +{c_test_declarations} + +static struct kunit_case test_cases[] = {{ + {c_test_cases} + {{ }} +}}; - // Figure out a smaller test name based on the generated function name. - let name = rustdoc_function_name.split_once("_rust_kernel_").unwrap().1; +static struct kunit_suite test_suite = {{ + .name = "rust_doctests_kernel", + .test_cases = test_cases, +}}; - let path = format!("rust/test/doctests/kernel/{name}"); +kunit_test_suite(test_suite); - std::fs::write(path, body.as_bytes()).unwrap(); +MODULE_LICENSE("GPL"); +"# + ) + .unwrap(); } diff --git a/scripts/rustdoc_test_gen.rs b/scripts/rustdoc_test_gen.rs deleted file mode 100644 index c8f9dc2ab976..000000000000 --- a/scripts/rustdoc_test_gen.rs +++ /dev/null @@ -1,265 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 - -//! Generates KUnit tests from saved `rustdoc`-generated tests. -//! -//! KUnit passes a context (`struct kunit *`) to each test, which should be forwarded to the other -//! KUnit functions and macros. -//! -//! However, we want to keep this as an implementation detail because: -//! -//! - Test code should not care about the implementation. -//! -//! - Documentation looks worse if it needs to carry extra details unrelated to the piece -//! being described. -//! -//! - Test code should be able to define functions and call them, without having to carry -//! the context. -//! -//! - Later on, we may want to be able to test non-kernel code (e.g. `core` or third-party -//! crates) which likely use the standard library `assert*!` macros. -//! -//! For this reason, instead of the passed context, `kunit_get_current_test()` is used instead -//! (i.e. `current->kunit_test`). -//! -//! Note that this means other threads/tasks potentially spawned by a given test, if failing, will -//! report the failure in the kernel log but will not fail the actual test. Saving the pointer in -//! e.g. a `static` per test does not fully solve the issue either, because currently KUnit does -//! not support assertions (only expectations) from other tasks. Thus leave that feature for -//! the future, which simplifies the code here too. We could also simply not allow `assert`s in -//! other tasks, but that seems overly constraining, and we do want to support them, eventually. - -use std::{ - fs, - fs::File, - io::{BufWriter, Read, Write}, - path::{Path, PathBuf}, -}; - -/// Find the real path to the original file based on the `file` portion of the test name. -/// -/// `rustdoc` generated `file`s look like `sync_locked_by_rs`. Underscores (except the last one) -/// may represent an actual underscore in a directory/file, or a path separator. Thus the actual -/// file might be `sync_locked_by.rs`, `sync/locked_by.rs`, `sync_locked/by.rs` or -/// `sync/locked/by.rs`. This function walks the file system to determine which is the real one. -/// -/// This does require that ambiguities do not exist, but that seems fair, especially since this is -/// all supposed to be temporary until `rustdoc` gives us proper metadata to build this. If such -/// ambiguities are detected, they are diagnosed and the script panics. -fn find_real_path<'a>(srctree: &Path, valid_paths: &'a mut Vec<PathBuf>, file: &str) -> &'a str { - valid_paths.clear(); - - let potential_components: Vec<&str> = file.strip_suffix("_rs").unwrap().split('_').collect(); - - find_candidates(srctree, valid_paths, Path::new(""), &potential_components); - fn find_candidates( - srctree: &Path, - valid_paths: &mut Vec<PathBuf>, - prefix: &Path, - potential_components: &[&str], - ) { - // The base case: check whether all the potential components left, joined by underscores, - // is a file. - let joined_potential_components = potential_components.join("_") + ".rs"; - if srctree - .join("rust/kernel") - .join(prefix) - .join(&joined_potential_components) - .is_file() - { - // Avoid `srctree` here in order to keep paths relative to it in the KTAP output. - valid_paths.push( - Path::new("rust/kernel") - .join(prefix) - .join(joined_potential_components), - ); - } - - // In addition, check whether each component prefix, joined by underscores, is a directory. - // If not, there is no need to check for combinations with that prefix. - for i in 1..potential_components.len() { - let (components_prefix, components_rest) = potential_components.split_at(i); - let prefix = prefix.join(components_prefix.join("_")); - if srctree.join("rust/kernel").join(&prefix).is_dir() { - find_candidates(srctree, valid_paths, &prefix, components_rest); - } - } - } - - match valid_paths.as_slice() { - [] => panic!( - "No path candidates found for `{file}`. This is likely a bug in the build system, or \ - some files went away while compiling." - ), - [valid_path] => valid_path.to_str().unwrap(), - valid_paths => { - use std::fmt::Write; - - let mut candidates = String::new(); - for path in valid_paths { - writeln!(&mut candidates, " {path:?}").unwrap(); - } - panic!( - "Several path candidates found for `{file}`, please resolve the ambiguity by \ - renaming a file or folder. Candidates:\n{candidates}", - ); - } - } -} - -fn main() { - let srctree = std::env::var("srctree").unwrap(); - let srctree = Path::new(&srctree); - - let mut paths = fs::read_dir("rust/test/doctests/kernel") - .unwrap() - .map(|entry| entry.unwrap().path()) - .collect::<Vec<_>>(); - - // Sort paths. - paths.sort(); - - let mut rust_tests = String::new(); - let mut c_test_declarations = String::new(); - let mut c_test_cases = String::new(); - let mut body = String::new(); - let mut last_file = String::new(); - let mut number = 0; - let mut valid_paths: Vec<PathBuf> = Vec::new(); - let mut real_path: &str = ""; - for path in paths { - // The `name` follows the `{file}_{line}_{number}` pattern (see description in - // `scripts/rustdoc_test_builder.rs`). Discard the `number`. - let name = path.file_name().unwrap().to_str().unwrap().to_string(); - - // Extract the `file` and the `line`, discarding the `number`. - let (file, line) = name.rsplit_once('_').unwrap().0.rsplit_once('_').unwrap(); - - // Generate an ID sequence ("test number") for each one in the file. - if file == last_file { - number += 1; - } else { - number = 0; - last_file = file.to_string(); - - // Figure out the real path, only once per file. - real_path = find_real_path(srctree, &mut valid_paths, file); - } - - // Generate a KUnit name (i.e. test name and C symbol) for this test. - // - // We avoid the line number, like `rustdoc` does, to make things slightly more stable for - // bisection purposes. However, to aid developers in mapping back what test failed, we will - // print a diagnostics line in the KTAP report. - let kunit_name = format!("rust_doctest_kernel_{file}_{number}"); - - // Read the test's text contents to dump it below. - body.clear(); - File::open(path).unwrap().read_to_string(&mut body).unwrap(); - - // Calculate how many lines before `main` function (including the `main` function line). - let body_offset = body - .lines() - .take_while(|line| !line.contains("fn main() {")) - .count() - + 1; - - use std::fmt::Write; - write!( - rust_tests, - r#"/// Generated `{name}` KUnit test case from a Rust documentation test. -#[no_mangle] -pub extern "C" fn {kunit_name}(__kunit_test: *mut ::kernel::bindings::kunit) {{ - /// Overrides the usual [`assert!`] macro with one that calls KUnit instead. - #[allow(unused)] - macro_rules! assert {{ - ($cond:expr $(,)?) => {{{{ - ::kernel::kunit_assert!( - "{kunit_name}", "{real_path}", __DOCTEST_ANCHOR - {line}, $cond - ); - }}}} - }} - - /// Overrides the usual [`assert_eq!`] macro with one that calls KUnit instead. - #[allow(unused)] - macro_rules! assert_eq {{ - ($left:expr, $right:expr $(,)?) => {{{{ - ::kernel::kunit_assert_eq!( - "{kunit_name}", "{real_path}", __DOCTEST_ANCHOR - {line}, $left, $right - ); - }}}} - }} - - // Many tests need the prelude, so provide it by default. - #[allow(unused)] - use ::kernel::prelude::*; - - // Unconditionally print the location of the original doctest (i.e. rather than the location in - // the generated file) so that developers can easily map the test back to the source code. - // - // This information is also printed when assertions fail, but this helps in the successful cases - // when the user is running KUnit manually, or when passing `--raw_output` to `kunit.py`. - // - // This follows the syntax for declaring test metadata in the proposed KTAP v2 spec, which may - // be used for the proposed KUnit test attributes API. Thus hopefully this will make migration - // easier later on. - ::kernel::kunit::info(fmt!(" # {kunit_name}.location: {real_path}:{line}\n")); - - /// The anchor where the test code body starts. - #[allow(unused)] - static __DOCTEST_ANCHOR: i32 = ::core::line!() as i32 + {body_offset} + 1; - {{ - {body} - main(); - }} -}} - -"# - ) - .unwrap(); - - write!(c_test_declarations, "void {kunit_name}(struct kunit *);\n").unwrap(); - write!(c_test_cases, " KUNIT_CASE({kunit_name}),\n").unwrap(); - } - - let rust_tests = rust_tests.trim(); - let c_test_declarations = c_test_declarations.trim(); - let c_test_cases = c_test_cases.trim(); - - write!( - BufWriter::new(File::create("rust/doctests_kernel_generated.rs").unwrap()), - r#"//! `kernel` crate documentation tests. - -const __LOG_PREFIX: &[u8] = b"rust_doctests_kernel\0"; - -{rust_tests} -"# - ) - .unwrap(); - - write!( - BufWriter::new(File::create("rust/doctests_kernel_generated_kunit.c").unwrap()), - r#"/* - * `kernel` crate documentation tests. - */ - -#include <kunit/test.h> - -{c_test_declarations} - -static struct kunit_case test_cases[] = {{ - {c_test_cases} - {{ }} -}}; - -static struct kunit_suite test_suite = {{ - .name = "rust_doctests_kernel", - .test_cases = test_cases, -}}; - -kunit_test_suite(test_suite); - -MODULE_LICENSE("GPL"); -"# - ) - .unwrap(); -} base-commit: 0d97f2067c166eb495771fede9f7b73999c67f66 -- 2.51.0

1 month, 2 weeks

2
1
0 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

Linux-kselftest-mirror