- Linux-kselftest-mirror - lists.linaro.org

[PATCH v2 bpf-next] selftests/bpf: Corrected two typos

by Lu Hongfei

When wrapping code, use ';' better than using ',' which is more in line with the coding habits of most engineers. Signed-off-by: Lu Hongfei <luhongfei(a)vivo.com> --- Compared to the previous version, the modifications made are: 1. Modified the subject to make it clearer and more accurate 2. Newly optimized typo in tcp_hdr_options.c tools/testing/selftests/bpf/benchs/bench_ringbufs.c | 2 +- tools/testing/selftests/bpf/prog_tests/tcp_hdr_options.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/tools/testing/selftests/bpf/benchs/bench_ringbufs.c b/tools/testing/selftests/bpf/benchs/bench_ringbufs.c index 3ca14ad36607..e1ee979e6acc 100644 --- a/tools/testing/selftests/bpf/benchs/bench_ringbufs.c +++ b/tools/testing/selftests/bpf/benchs/bench_ringbufs.c @@ -399,7 +399,7 @@ static void perfbuf_libbpf_setup(void) ctx->skel = perfbuf_setup_skeleton(); memset(&attr, 0, sizeof(attr)); - attr.config = PERF_COUNT_SW_BPF_OUTPUT, + attr.config = PERF_COUNT_SW_BPF_OUTPUT; attr.type = PERF_TYPE_SOFTWARE; attr.sample_type = PERF_SAMPLE_RAW; /* notify only every Nth sample */ diff --git a/tools/testing/selftests/bpf/prog_tests/tcp_hdr_options.c b/tools/testing/selftests/bpf/prog_tests/tcp_hdr_options.c index 13bcaeb028b8..56685fc03c7e 100644 --- a/tools/testing/selftests/bpf/prog_tests/tcp_hdr_options.c +++ b/tools/testing/selftests/bpf/prog_tests/tcp_hdr_options.c @@ -347,7 +347,7 @@ static void syncookie_estab(void) exp_active_estab_in.max_delack_ms = 22; exp_passive_hdr_stg.syncookie = true; - exp_active_hdr_stg.resend_syn = true, + exp_active_hdr_stg.resend_syn = true; prepare_out(); -- 2.39.0

1 year, 12 months

4
3
0 0

[linux-next:master] BUILD REGRESSION 123212f53f3e394c1ae69a58c05dfdda56fec8c6

by kernel test robot

tree/branch: https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git master branch HEAD: 123212f53f3e394c1ae69a58c05dfdda56fec8c6 Add linux-next specific files for 20230707 Error/Warning reports: https://lore.kernel.org/oe-kbuild-all/202306122223.HHER4zOo-lkp@intel.com https://lore.kernel.org/oe-kbuild-all/202306240021.qNRc4iHW-lkp@intel.com Error/Warning: (recently discovered and may have been fixed) arch/arm64/kernel/entry-ftrace.S:59: Error: undefined symbol FTRACE_OPS_DIRECT_CALL used as an immediate value arch/parisc/kernel/pdt.c:67:6: warning: no previous prototype for 'arch_report_meminfo' [-Wmissing-prototypes] drivers/net/arcnet/arc-rimi.c:107:4: error: incompatible integer to pointer conversion assigning to 'void *' from 'int' [-Wint-conversion] drivers/net/arcnet/arc-rimi.c:107:6: error: call to undeclared function 'ioremap'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration] drivers/net/arcnet/arc-rimi.c:113:3: error: call to undeclared function 'iounmap'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration] drivers/pcmcia/cistpl.c:103:15: error: incompatible integer to pointer conversion assigning to 'void *' from 'int' [-Wint-conversion] drivers/pcmcia/cistpl.c:103:17: error: call to undeclared function 'ioremap'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration] drivers/pcmcia/cistpl.c:72:3: error: call to undeclared function 'iounmap'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration] drivers/tty/ipwireless/main.c:115:21: error: incompatible integer to pointer conversion assigning to 'void *' from 'int' [-Wint-conversion] drivers/tty/ipwireless/main.c:115:23: error: call to undeclared function 'ioremap'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration] drivers/tty/ipwireless/main.c:155:2: error: call to undeclared function 'iounmap'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration] lib/kunit/executor_test.c:138:4: warning: cast from 'void (*)(const void *)' to 'kunit_action_t *' (aka 'void (*)(void *)') converts to incompatible function type [-Wcast-function-type-strict] lib/kunit/test.c:775:38: warning: cast from 'void (*)(const void *)' to 'kunit_action_t *' (aka 'void (*)(void *)') converts to incompatible function type [-Wcast-function-type-strict] Unverified Error/Warning (likely false positive, please contact us if interested): {standard input}: Error: local label `"2" (instance number 9 of a fb label)' is not defined Error/Warning ids grouped by kconfigs: gcc_recent_errors |-- arm64-randconfig-r034-20230707 | `-- arch-arm64-kernel-entry-ftrace.S:Error:undefined-symbol-FTRACE_OPS_DIRECT_CALL-used-as-an-immediate-value |-- parisc-randconfig-r034-20230707 | `-- arch-parisc-kernel-pdt.c:warning:no-previous-prototype-for-arch_report_meminfo |-- parisc-randconfig-r081-20230703 | `-- arch-parisc-kernel-pdt.c:warning:no-previous-prototype-for-arch_report_meminfo `-- sh-allmodconfig `-- standard-input:Error:local-label-(instance-number-of-a-fb-label)-is-not-defined clang_recent_errors |-- arm-randconfig-r036-20230707 | `-- lib-kunit-test.c:warning:cast-from-void-(-)(const-void-)-to-kunit_action_t-(aka-void-(-)(void-)-)-converts-to-incompatible-function-type |-- hexagon-randconfig-r041-20230707 | |-- lib-kunit-executor_test.c:warning:cast-from-void-(-)(const-void-)-to-kunit_action_t-(aka-void-(-)(void-)-)-converts-to-incompatible-function-type | `-- lib-kunit-test.c:warning:cast-from-void-(-)(const-void-)-to-kunit_action_t-(aka-void-(-)(void-)-)-converts-to-incompatible-function-type `-- s390-randconfig-r013-20230707 |-- drivers-net-arcnet-arc-rimi.c:error:call-to-undeclared-function-ioremap-ISO-C99-and-later-do-not-support-implicit-function-declarations |-- drivers-net-arcnet-arc-rimi.c:error:call-to-undeclared-function-iounmap-ISO-C99-and-later-do-not-support-implicit-function-declarations |-- drivers-net-arcnet-arc-rimi.c:error:incompatible-integer-to-pointer-conversion-assigning-to-void-from-int |-- drivers-pcmcia-cistpl.c:error:call-to-undeclared-function-ioremap-ISO-C99-and-later-do-not-support-implicit-function-declarations |-- drivers-pcmcia-cistpl.c:error:call-to-undeclared-function-iounmap-ISO-C99-and-later-do-not-support-implicit-function-declarations |-- drivers-pcmcia-cistpl.c:error:incompatible-integer-to-pointer-conversion-assigning-to-void-from-int |-- drivers-tty-ipwireless-main.c:error:call-to-undeclared-function-ioremap-ISO-C99-and-later-do-not-support-implicit-function-declarations |-- drivers-tty-ipwireless-main.c:error:call-to-undeclared-function-iounmap-ISO-C99-and-later-do-not-support-implicit-function-declarations `-- drivers-tty-ipwireless-main.c:error:incompatible-integer-to-pointer-conversion-assigning-to-void-from-int elapsed time: 737m configs tested: 137 configs skipped: 5 tested configs: alpha alldefconfig gcc alpha allyesconfig gcc alpha defconfig gcc alpha randconfig-r036-20230707 gcc arc allyesconfig gcc arc axs103_defconfig gcc arc defconfig gcc arc randconfig-r035-20230707 gcc arc randconfig-r043-20230707 gcc arm allmodconfig gcc arm allyesconfig gcc arm assabet_defconfig gcc arm defconfig gcc arm h3600_defconfig gcc arm integrator_defconfig gcc arm randconfig-r021-20230707 gcc arm randconfig-r036-20230707 clang arm randconfig-r046-20230707 gcc arm64 allyesconfig gcc arm64 defconfig gcc arm64 randconfig-r023-20230707 clang arm64 randconfig-r034-20230707 gcc csky defconfig gcc csky randconfig-r002-20230707 gcc csky randconfig-r003-20230707 gcc csky randconfig-r032-20230707 gcc csky randconfig-r033-20230707 gcc hexagon randconfig-r041-20230707 clang hexagon randconfig-r045-20230707 clang i386 allyesconfig gcc i386 buildonly-randconfig-r004-20230707 gcc i386 buildonly-randconfig-r005-20230707 gcc i386 buildonly-randconfig-r006-20230707 gcc i386 debian-10.3 gcc i386 defconfig gcc i386 randconfig-i001-20230707 gcc i386 randconfig-i002-20230707 gcc i386 randconfig-i003-20230707 gcc i386 randconfig-i004-20230707 gcc i386 randconfig-i005-20230707 gcc i386 randconfig-i006-20230707 gcc i386 randconfig-i011-20230707 clang i386 randconfig-i012-20230707 clang i386 randconfig-i013-20230707 clang i386 randconfig-i014-20230707 clang i386 randconfig-i015-20230707 clang i386 randconfig-i016-20230707 clang loongarch allmodconfig gcc loongarch allnoconfig gcc loongarch defconfig gcc loongarch loongson3_defconfig gcc m68k allmodconfig gcc m68k allyesconfig gcc m68k amcore_defconfig gcc m68k defconfig gcc m68k randconfig-r015-20230707 gcc microblaze randconfig-r024-20230707 gcc mips allmodconfig gcc mips allyesconfig gcc mips ar7_defconfig gcc mips ip22_defconfig clang mips ip28_defconfig clang mips malta_defconfig clang mips rbtx49xx_defconfig clang nios2 defconfig gcc openrisc defconfig gcc openrisc randconfig-r011-20230707 gcc openrisc randconfig-r035-20230707 gcc parisc allyesconfig gcc parisc defconfig gcc parisc randconfig-r034-20230707 gcc parisc64 defconfig gcc powerpc allmodconfig gcc powerpc allnoconfig gcc powerpc chrp32_defconfig gcc powerpc ep88xc_defconfig gcc powerpc fsp2_defconfig clang powerpc motionpro_defconfig gcc powerpc ppa8548_defconfig clang powerpc randconfig-r031-20230707 gcc powerpc storcenter_defconfig gcc powerpc warp_defconfig gcc riscv allmodconfig gcc riscv allnoconfig gcc riscv allyesconfig gcc riscv defconfig gcc riscv randconfig-r005-20230707 gcc riscv randconfig-r042-20230707 clang riscv rv32_defconfig gcc s390 allmodconfig gcc s390 allyesconfig gcc s390 defconfig gcc s390 randconfig-r013-20230707 clang s390 randconfig-r014-20230707 clang s390 randconfig-r022-20230707 clang s390 randconfig-r033-20230707 gcc s390 randconfig-r044-20230707 clang sh allmodconfig gcc sh defconfig gcc sh randconfig-r006-20230707 gcc sh randconfig-r026-20230707 gcc sh randconfig-r031-20230707 gcc sparc allyesconfig gcc sparc defconfig gcc sparc randconfig-r001-20230707 gcc sparc sparc32_defconfig gcc sparc64 randconfig-r025-20230707 gcc um allmodconfig clang um allnoconfig clang um allyesconfig clang um defconfig gcc um i386_defconfig gcc um randconfig-r032-20230707 clang um x86_64_defconfig gcc x86_64 allyesconfig gcc x86_64 buildonly-randconfig-r001-20230707 gcc x86_64 buildonly-randconfig-r002-20230707 gcc x86_64 buildonly-randconfig-r003-20230707 gcc x86_64 defconfig gcc x86_64 kexec gcc x86_64 randconfig-x001-20230707 clang x86_64 randconfig-x002-20230707 clang x86_64 randconfig-x003-20230707 clang x86_64 randconfig-x004-20230707 clang x86_64 randconfig-x005-20230707 clang x86_64 randconfig-x006-20230707 clang x86_64 randconfig-x011-20230707 gcc x86_64 randconfig-x012-20230707 gcc x86_64 randconfig-x013-20230707 gcc x86_64 randconfig-x014-20230707 gcc x86_64 randconfig-x015-20230707 gcc x86_64 randconfig-x016-20230707 gcc x86_64 rhel-8.3-rust clang x86_64 rhel-8.3 gcc xtensa alldefconfig gcc xtensa randconfig-r016-20230707 gcc xtensa smp_lx200_defconfig gcc -- 0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki

1 year, 12 months

1
0
0 0

[PATCH] tools: testing: Corrected a clerical error

by Lu Hongfei

When wrapping code, use ';' better than using ',' which is more in line with the coding habits of most engineers. Signed-off-by: Lu Hongfei <luhongfei(a)vivo.com> --- tools/testing/selftests/bpf/benchs/bench_ringbufs.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/testing/selftests/bpf/benchs/bench_ringbufs.c b/tools/testing/selftests/bpf/benchs/bench_ringbufs.c index 3ca14ad36607..e1ee979e6acc 100644 --- a/tools/testing/selftests/bpf/benchs/bench_ringbufs.c +++ b/tools/testing/selftests/bpf/benchs/bench_ringbufs.c @@ -399,7 +399,7 @@ static void perfbuf_libbpf_setup(void) ctx->skel = perfbuf_setup_skeleton(); memset(&attr, 0, sizeof(attr)); - attr.config = PERF_COUNT_SW_BPF_OUTPUT, + attr.config = PERF_COUNT_SW_BPF_OUTPUT; attr.type = PERF_TYPE_SOFTWARE; attr.sample_type = PERF_SAMPLE_RAW; /* notify only every Nth sample */ -- 2.39.0

1 year, 12 months

2
1
0 0

[RFC][PATCH 00/10] KEYS: Introduce user asymmetric keys and signatures

by Roberto Sassu

From: Roberto Sassu <roberto.sassu(a)huawei.com> Define a new TLV-based format for keys and signatures, aiming to store and use in the kernel the crypto material from other unsupported formats (e.g. PGP). TLV fields have been defined to fill the corresponding kernel structures public_key, public_key_signature and key_preparsed_payload. Keys: struct public_key { struct key_preparsed_payload { KEY_PUB --> void *key; u32 keylen; --> prep->payload.data[asym_crypto] KEY_ALGO --> const char *pkey_algo; KEY_KID0 KEY_KID1 --> prep->payload.data[asym_key_ids] KEY_KID2 KEY_DESC --> prep->description Signatures: struct public_key_signature { SIG_S --> u8 *s; u32 s_size; SIG_KEY_ALGO --> const char *pkey_algo; SIG_HASH_ALGO --> const char *hash_algo; u32 digest_size; SIG_ENC --> const char *encoding; SIG_KID0 SIG_KID1 --> struct asymmetric_key_id *auth_ids[3]; SIG_KID2 For keys, since the format conversion has to be done in user space, user space is assumed to be trusted, in this proposal. Without this assumption, a malicious conversion tool could make a user load to the kernel a different key than the one expected. That should not be a particular problem for keys that are embedded in the kernel image and loaded at boot, since the conversion happens in a trusted environment such as the building infrastructure of the Linux distribution vendor. In the other cases, such as enrolling a key through the Machine Owner Key (MOK) mechanism, the user is responsible to ensure that the crypto material carried in the original format remains the same after the conversion. For signatures, assuming the strength of the crypto algorithms, altering the crypto material is simply a Denial-of-Service (DoS), as data can be validated only with the right signature. This patch set also offers the following contributions: - An API similar to the PKCS#7 one, to verify the authenticity of system data through user asymmetric keys and signatures - A mechanism to store a keyring blob in the kernel image and to extract and load the keys at system boot - eBPF binding, so that data authenticity verification with user asymmetric keys and signatures can be carried out also with eBPF programs - A new command for gnupg (in user space), to convert keys and signatures from PGP to the new kernel format The primary use case for this patch set is to verify the authenticity of RPM package headers with the PGP keys of the Linux distribution. Once their authenticity is verified, file digests can be extracted from those RPM headers and used as reference values for IMA Appraisal. Compared to the previous patch set, the main difference is not relying on User Mode Drivers (UMDs) for the conversion from the original format to the kernel format, due to the concern that full isolation of the UMD process cannot be achieved against a fully privileged system user (root). The discussion is still ongoing here: https://lore.kernel.org/linux-integrity/eb31920bd00e2c921b0aa6ebed8745cb013… This however does not prevent the goal mentioned above of verifying the authenticity of RPM headers to be achieved. The fact that Linux distribution vendors do the conversion in their infrastructure is a good enough guarantee. A very quick way to test the patch set is to execute: # gpg --conv-kernel /etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-rawhide-primary | keyctl padd asymmetric "" @u # keyctl show @u Keyring 762357580 --alswrv 0 65534 keyring: _uid.0 567216072 --als--v 0 0 \_ asymmetric: PGP: 18b8e74c Patches 1-2 preliminarly export some definitions to user space so that conversion tools can specify the right public key algorithms and signature encodings (digest algorithms are already exported). Patches 3-5 introduce the user asymmetric keys and signatures. Patches 6 introduces a system API for verifying the authenticity of system data through user asymmetric keys and signatures. Patch 7-8 introduce a mechanism to store a keyring blob with user asymmetric keys in the kernel image, and load them at system boot. Patches 9-10 introduce the eBPF binding and corresponding test (which can be enabled only after the gnupg patches are upstreamed). Patches 1-2 [GNUPG] introduce the new gpg command --conv-kernel to convert PGP keys and signatures to the new kernel format. Changelog v1: - Remove useless check in validate_key() (suggested by Yonghong) - Don't rely on User Mode Drivers for the conversion from the original format to the kernel format - Use the more extensible TLV format, instead of a fixed structure Roberto Sassu (10): crypto: Export public key algorithm information crypto: Export signature encoding information KEYS: asymmetric: Introduce a parser for user asymmetric keys and sigs KEYS: asymmetric: Introduce the user asymmetric key parser KEYS: asymmetric: Introduce the user asymmetric key signature parser verification: Add verify_uasym_signature() and verify_uasym_sig_message() KEYS: asymmetric: Preload user asymmetric keys from a keyring blob KEYS: Introduce load_uasym_keyring() bpf: Introduce bpf_verify_uasym_signature() kfunc selftests/bpf: Prepare a test for user asymmetric key signatures MAINTAINERS | 1 + certs/Kconfig | 11 + certs/Makefile | 7 + certs/system_certificates.S | 18 + certs/system_keyring.c | 166 +++++- crypto/Kconfig | 6 + crypto/Makefile | 2 + crypto/asymmetric_keys/Kconfig | 14 + crypto/asymmetric_keys/Makefile | 10 + crypto/asymmetric_keys/asymmetric_type.c | 3 +- crypto/asymmetric_keys/uasym_key_parser.c | 229 ++++++++ crypto/asymmetric_keys/uasym_key_preload.c | 99 ++++ crypto/asymmetric_keys/uasym_parser.c | 201 +++++++ crypto/asymmetric_keys/uasym_parser.h | 43 ++ crypto/asymmetric_keys/uasym_sig_parser.c | 491 ++++++++++++++++++ crypto/pub_key_info.c | 20 + crypto/sig_enc_info.c | 16 + include/crypto/pub_key_info.h | 15 + include/crypto/sig_enc_info.h | 15 + include/crypto/uasym_keys_sigs.h | 82 +++ include/keys/asymmetric-type.h | 1 + include/linux/verification.h | 50 ++ include/uapi/linux/pub_key_info.h | 22 + include/uapi/linux/sig_enc_info.h | 18 + include/uapi/linux/uasym_parser.h | 107 ++++ kernel/trace/bpf_trace.c | 68 ++- ...y_pkcs7_sig.c => verify_pkcs7_uasym_sig.c} | 159 +++++- ...s7_sig.c => test_verify_pkcs7_uasym_sig.c} | 18 +- .../testing/selftests/bpf/verify_sig_setup.sh | 82 ++- 29 files changed, 1924 insertions(+), 50 deletions(-) create mode 100644 crypto/asymmetric_keys/uasym_key_parser.c create mode 100644 crypto/asymmetric_keys/uasym_key_preload.c create mode 100644 crypto/asymmetric_keys/uasym_parser.c create mode 100644 crypto/asymmetric_keys/uasym_parser.h create mode 100644 crypto/asymmetric_keys/uasym_sig_parser.c create mode 100644 crypto/pub_key_info.c create mode 100644 crypto/sig_enc_info.c create mode 100644 include/crypto/pub_key_info.h create mode 100644 include/crypto/sig_enc_info.h create mode 100644 include/crypto/uasym_keys_sigs.h create mode 100644 include/uapi/linux/pub_key_info.h create mode 100644 include/uapi/linux/sig_enc_info.h create mode 100644 include/uapi/linux/uasym_parser.h rename tools/testing/selftests/bpf/prog_tests/{verify_pkcs7_sig.c => verify_pkcs7_uasym_sig.c} (69%) rename tools/testing/selftests/bpf/progs/{test_verify_pkcs7_sig.c => test_verify_pkcs7_uasym_sig.c} (82%) -- 2.34.1

1 year, 12 months

2
14
0 0

[RESEND PATCH v3 0/2] RISC-V: mm: Make SV48 the default address space

by Charlie Jenkins

Make sv48 the default address space for mmap as some applications currently depend on this assumption. Also enable users to select desired address space using a non-zero hint address to mmap. Previous kernel changes caused Java and other applications to be broken on sv57 which this patch fixes. Documentation is also added to the RISC-V virtual memory section to explain these changes. Charlie Jenkins (2): RISC-V: mm: Restrict address space for sv39,sv48,sv57 RISC-V: mm: Update documentation and include test Documentation/riscv/vm-layout.rst | 22 +++++++++ arch/riscv/include/asm/elf.h | 2 +- arch/riscv/include/asm/pgtable.h | 21 ++++++-- arch/riscv/include/asm/processor.h | 34 ++++++++++--- tools/testing/selftests/riscv/Makefile | 2 +- tools/testing/selftests/riscv/mm/.gitignore | 1 + tools/testing/selftests/riscv/mm/Makefile | 21 ++++++++ .../selftests/riscv/mm/testcases/mmap.c | 49 +++++++++++++++++++ 8 files changed, 139 insertions(+), 13 deletions(-) create mode 100644 tools/testing/selftests/riscv/mm/.gitignore create mode 100644 tools/testing/selftests/riscv/mm/Makefile create mode 100644 tools/testing/selftests/riscv/mm/testcases/mmap.c -- 2.41.0

1 year, 12 months

5
10
0 0

[PATCH bpf-next v2 0/6] Support defragmenting IPv(4|6) packets in BPF

by Daniel Xu

=== Context === In the context of a middlebox, fragmented packets are tricky to handle. The full 5-tuple of a packet is often only available in the first fragment which makes enforcing consistent policy difficult. There are really only two stateless options, neither of which are very nice: 1. Enforce policy on first fragment and accept all subsequent fragments. This works but may let in certain attacks or allow data exfiltration. 2. Enforce policy on first fragment and drop all subsequent fragments. This does not really work b/c some protocols may rely on fragmentation. For example, DNS may rely on oversized UDP packets for large responses. So stateful tracking is the only sane option. RFC 8900 [0] calls this out as well in section 6.3: Middleboxes [...] should process IP fragments in a manner that is consistent with [RFC0791] and [RFC8200]. In many cases, middleboxes must maintain state in order to achieve this goal. === BPF related bits === Policy has traditionally been enforced from XDP/TC hooks. Both hooks run before kernel reassembly facilities. However, with the new BPF_PROG_TYPE_NETFILTER, we can rather easily hook into existing netfilter reassembly infra. The basic idea is we bump a refcnt on the netfilter defrag module and then run the bpf prog after the defrag module runs. This allows bpf progs to transparently see full, reassembled packets. The nice thing about this is that progs don't have to carry around logic to detect fragments. === Changelog === Changes from v1: * Drop bpf_program__attach_netfilter() patches * static -> static const where appropriate * Fix callback assignment order during registration * Only request_module() if callbacks are missing * Fix retval when modprobe fails in userspace * Fix v6 defrag module name (nf_defrag_ipv6_hooks -> nf_defrag_ipv6) * Simplify priority checking code * Add warning if module doesn't assign callbacks in the future * Take refcnt on module while defrag link is active [0]: https://datatracker.ietf.org/doc/html/rfc8900 Daniel Xu (6): netfilter: defrag: Add glue hooks for enabling/disabling defrag netfilter: bpf: Support BPF_F_NETFILTER_IP_DEFRAG in netfilter link netfilter: bpf: Prevent defrag module unload while link active bpf: selftests: Support not connecting client socket bpf: selftests: Support custom type and proto for client sockets bpf: selftests: Add defrag selftests include/linux/netfilter.h | 15 + include/uapi/linux/bpf.h | 5 + net/ipv4/netfilter/nf_defrag_ipv4.c | 17 +- net/ipv6/netfilter/nf_defrag_ipv6_hooks.c | 11 + net/netfilter/core.c | 6 + net/netfilter/nf_bpf_link.c | 149 ++++++++- tools/include/uapi/linux/bpf.h | 5 + tools/testing/selftests/bpf/Makefile | 4 +- .../selftests/bpf/generate_udp_fragments.py | 90 ++++++ .../selftests/bpf/ip_check_defrag_frags.h | 57 ++++ tools/testing/selftests/bpf/network_helpers.c | 26 +- tools/testing/selftests/bpf/network_helpers.h | 3 + .../bpf/prog_tests/ip_check_defrag.c | 282 ++++++++++++++++++ .../selftests/bpf/progs/ip_check_defrag.c | 104 +++++++ 14 files changed, 752 insertions(+), 22 deletions(-) create mode 100755 tools/testing/selftests/bpf/generate_udp_fragments.py create mode 100644 tools/testing/selftests/bpf/ip_check_defrag_frags.h create mode 100644 tools/testing/selftests/bpf/prog_tests/ip_check_defrag.c create mode 100644 tools/testing/selftests/bpf/progs/ip_check_defrag.c -- 2.41.0

1 year, 12 months

2
4
0 0

[PATCH bpf-next] selftests/bpf: Bump and validate MAX_SYMS

by Björn Töpel

From: Björn Töpel <bjorn(a)rivosinc.com> BPF tests that load /proc/kallsyms, e.g. bpf_cookie, will perform a buffer overrun if the number of syms on the system is larger than MAX_SYMS. Bump the MAX_SYMS to 400000, and add a runtime check that bails out if the maximum is reached. Signed-off-by: Björn Töpel <bjorn(a)rivosinc.com> --- tools/testing/selftests/bpf/trace_helpers.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/bpf/trace_helpers.c b/tools/testing/selftests/bpf/trace_helpers.c index 9b070cdf44ac..f83d9f65c65b 100644 --- a/tools/testing/selftests/bpf/trace_helpers.c +++ b/tools/testing/selftests/bpf/trace_helpers.c @@ -18,7 +18,7 @@ #define TRACEFS_PIPE "/sys/kernel/tracing/trace_pipe" #define DEBUGFS_PIPE "/sys/kernel/debug/tracing/trace_pipe" -#define MAX_SYMS 300000 +#define MAX_SYMS 400000 static struct ksym syms[MAX_SYMS]; static int sym_cnt; @@ -46,6 +46,9 @@ int load_kallsyms_refresh(void) break; if (!addr) continue; + if (i >= MAX_SYMS) + return -EFBIG; + syms[i].addr = (long) addr; syms[i].name = strdup(func); i++; base-commit: fd283ab196a867f8f65f36913e0fadd031fcb823 -- 2.39.2

1 year, 12 months

3
2
0 0

[linux-next:master] BUILD REGRESSION c36ac601a98fb148147640bae219108ee81566f8

by kernel test robot

tree/branch: https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git master branch HEAD: c36ac601a98fb148147640bae219108ee81566f8 Add linux-next specific files for 20230706 Error/Warning reports: https://lore.kernel.org/oe-kbuild-all/202306122223.HHER4zOo-lkp@intel.com https://lore.kernel.org/oe-kbuild-all/202307050034.tAJSN9qy-lkp@intel.com Error/Warning: (recently discovered and may have been fixed) arch/parisc/kernel/pdt.c:67:6: warning: no previous prototype for 'arch_report_meminfo' [-Wmissing-prototypes] arch/riscv/kernel/crash_core.c:14:64: error: 'VMEMMAP_START' undeclared (first use in this function) arch/riscv/kernel/crash_core.c:15:62: error: 'VMEMMAP_END' undeclared (first use in this function); did you mean 'MEMREMAP_ENC'? arch/riscv/kernel/crash_core.c:8:27: error: 'VA_BITS' undeclared (first use in this function) lib/kunit/executor_test.c:138:4: warning: cast from 'void (*)(const void *)' to 'kunit_action_t *' (aka 'void (*)(void *)') converts to incompatible function type [-Wcast-function-type-strict] lib/kunit/test.c:775:38: warning: cast from 'void (*)(const void *)' to 'kunit_action_t *' (aka 'void (*)(void *)') converts to incompatible function type [-Wcast-function-type-strict] Unverified Error/Warning (likely false positive, please contact us if interested): drivers/net/ethernet/mellanox/mlx5/core/lib/devcom.c:98 mlx5_devcom_register_device() error: uninitialized symbol 'tmp_dev'. kernel/trace/trace_functions_graph.c:1012 print_graph_return() warn: bitwise AND condition is false here kernel/trace/trace_functions_graph.c:726 print_graph_entry_leaf() warn: bitwise AND condition is false here {standard input}: Error: local label `"2" (instance number 9 of a fb label)' is not defined Error/Warning ids grouped by kconfigs: gcc_recent_errors |-- i386-randconfig-m021-20230705 | |-- kernel-trace-trace_functions_graph.c-print_graph_entry_leaf()-warn:bitwise-AND-condition-is-false-here | `-- kernel-trace-trace_functions_graph.c-print_graph_return()-warn:bitwise-AND-condition-is-false-here |-- parisc-randconfig-r003-20230706 | `-- arch-parisc-kernel-pdt.c:warning:no-previous-prototype-for-arch_report_meminfo |-- parisc-randconfig-r081-20230703 | `-- arch-parisc-kernel-pdt.c:warning:no-previous-prototype-for-arch_report_meminfo |-- riscv-randconfig-r042-20230706 | |-- arch-riscv-kernel-crash_core.c:error:VA_BITS-undeclared-(first-use-in-this-function) | |-- arch-riscv-kernel-crash_core.c:error:VMEMMAP_END-undeclared-(first-use-in-this-function) | `-- arch-riscv-kernel-crash_core.c:error:VMEMMAP_START-undeclared-(first-use-in-this-function) |-- s390-randconfig-m041-20230705 | `-- drivers-net-ethernet-mellanox-mlx5-core-lib-devcom.c-mlx5_devcom_register_device()-error:uninitialized-symbol-tmp_dev-. `-- sh-allmodconfig `-- standard-input:Error:local-label-(instance-number-of-a-fb-label)-is-not-defined clang_recent_errors |-- arm64-randconfig-r004-20230706 | `-- lib-kunit-test.c:warning:cast-from-void-(-)(const-void-)-to-kunit_action_t-(aka-void-(-)(void-)-)-converts-to-incompatible-function-type |-- hexagon-randconfig-r041-20230706 | |-- lib-kunit-executor_test.c:warning:cast-from-void-(-)(const-void-)-to-kunit_action_t-(aka-void-(-)(void-)-)-converts-to-incompatible-function-type | `-- lib-kunit-test.c:warning:cast-from-void-(-)(const-void-)-to-kunit_action_t-(aka-void-(-)(void-)-)-converts-to-incompatible-function-type `-- powerpc-allyesconfig `-- clang:error:unsupported-option-fsanitize-thread-for-target-powerpc-unknown-linux-gnu elapsed time: 735m configs tested: 144 configs skipped: 8 tested configs: alpha allyesconfig gcc alpha defconfig gcc alpha randconfig-r006-20230706 gcc arc allyesconfig gcc arc defconfig gcc arc randconfig-r043-20230706 gcc arm allmodconfig gcc arm allyesconfig gcc arm defconfig gcc arm gemini_defconfig gcc arm imx_v4_v5_defconfig clang arm jornada720_defconfig gcc arm milbeaut_m10v_defconfig clang arm mps2_defconfig gcc arm mv78xx0_defconfig clang arm pxa910_defconfig gcc arm randconfig-r046-20230706 clang arm s5pv210_defconfig clang arm spear3xx_defconfig clang arm64 allyesconfig gcc arm64 defconfig gcc arm64 randconfig-r004-20230706 clang arm64 randconfig-r024-20230706 gcc csky defconfig gcc hexagon alldefconfig clang hexagon randconfig-r041-20230706 clang hexagon randconfig-r045-20230706 clang i386 allyesconfig gcc i386 buildonly-randconfig-r004-20230706 clang i386 buildonly-randconfig-r005-20230706 clang i386 buildonly-randconfig-r006-20230706 clang i386 debian-10.3 gcc i386 defconfig gcc i386 randconfig-i001-20230706 clang i386 randconfig-i002-20230706 clang i386 randconfig-i003-20230706 clang i386 randconfig-i004-20230706 clang i386 randconfig-i005-20230706 clang i386 randconfig-i006-20230706 clang i386 randconfig-i011-20230706 gcc i386 randconfig-i012-20230706 gcc i386 randconfig-i013-20230706 gcc i386 randconfig-i014-20230706 gcc i386 randconfig-i015-20230706 gcc i386 randconfig-i016-20230706 gcc i386 randconfig-r035-20230706 clang loongarch allmodconfig gcc loongarch allnoconfig gcc loongarch defconfig gcc loongarch randconfig-r001-20230706 gcc loongarch randconfig-r025-20230706 gcc loongarch randconfig-r031-20230706 gcc m68k allmodconfig gcc m68k allyesconfig gcc m68k defconfig gcc m68k sun3_defconfig gcc m68k sun3x_defconfig gcc microblaze randconfig-r005-20230706 gcc mips allmodconfig gcc mips allyesconfig gcc mips ci20_defconfig gcc mips db1xxx_defconfig gcc mips rs90_defconfig clang nios2 defconfig gcc openrisc or1klitex_defconfig gcc openrisc randconfig-r015-20230706 gcc parisc allyesconfig gcc parisc defconfig gcc parisc randconfig-r003-20230706 gcc parisc randconfig-r005-20230706 gcc parisc randconfig-r032-20230705 gcc parisc randconfig-r036-20230706 gcc parisc64 defconfig gcc powerpc allmodconfig gcc powerpc allnoconfig gcc powerpc g5_defconfig clang powerpc mpc5200_defconfig clang powerpc mpc834x_itx_defconfig gcc powerpc pcm030_defconfig gcc powerpc randconfig-r013-20230706 gcc powerpc randconfig-r036-20230705 gcc powerpc skiroot_defconfig clang powerpc walnut_defconfig clang powerpc xes_mpc85xx_defconfig clang riscv allmodconfig gcc riscv allnoconfig gcc riscv allyesconfig gcc riscv defconfig gcc riscv randconfig-r003-20230706 clang riscv randconfig-r021-20230706 gcc riscv randconfig-r023-20230706 gcc riscv randconfig-r042-20230706 gcc riscv rv32_defconfig gcc s390 allmodconfig gcc s390 allyesconfig gcc s390 defconfig gcc s390 randconfig-r031-20230705 gcc s390 randconfig-r044-20230706 gcc sh allmodconfig gcc sh ecovec24_defconfig gcc sh rsk7264_defconfig gcc sh titan_defconfig gcc sparc allyesconfig gcc sparc defconfig gcc sparc randconfig-r016-20230706 gcc sparc sparc64_defconfig gcc sparc64 randconfig-r002-20230706 gcc sparc64 randconfig-r035-20230705 gcc um allmodconfig clang um allnoconfig clang um allyesconfig clang um defconfig gcc um i386_defconfig gcc um randconfig-r011-20230706 clang um randconfig-r034-20230706 gcc um x86_64_defconfig gcc x86_64 allyesconfig gcc x86_64 buildonly-randconfig-r001-20230706 clang x86_64 buildonly-randconfig-r002-20230706 clang x86_64 buildonly-randconfig-r003-20230706 clang x86_64 defconfig gcc x86_64 kexec gcc x86_64 randconfig-r026-20230706 gcc x86_64 randconfig-r033-20230706 clang x86_64 randconfig-x001-20230706 gcc x86_64 randconfig-x002-20230706 gcc x86_64 randconfig-x003-20230706 gcc x86_64 randconfig-x004-20230706 gcc x86_64 randconfig-x005-20230706 gcc x86_64 randconfig-x006-20230706 gcc x86_64 randconfig-x011-20230706 clang x86_64 randconfig-x012-20230706 clang x86_64 randconfig-x013-20230706 clang x86_64 randconfig-x014-20230706 clang x86_64 randconfig-x015-20230706 clang x86_64 randconfig-x016-20230706 clang x86_64 rhel-8.3-rust clang x86_64 rhel-8.3 gcc xtensa audio_kc705_defconfig gcc xtensa cadence_csp_defconfig gcc xtensa randconfig-r002-20230706 gcc xtensa randconfig-r004-20230706 gcc xtensa randconfig-r022-20230706 gcc xtensa randconfig-r034-20230705 gcc -- 0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki

1 year, 12 months

1
0
0 0

[PATCH v23 0/5] Implement IOCTL to get and optionally clear info about PTEs

by Muhammad Usama Anjum

*Changes in v23*: - Set vec_buf_index in loop only when vec_buf_index is set - Return -EFAULT instead of -EINVAL if vec is NULL - Correctly return the walk ending address to the page granularity *Changes in v22*: - Interface change: - Replace [start start + len) with [start, end) - Return the ending address of the address walk in start *Changes in v21*: - Abort walk instead of returning error if WP is to be performed on partial hugetlb *Changes in v20* - Correct PAGE_IS_FILE and add PAGE_IS_PFNZERO *Changes in v19* - Minor changes and interface updates *Changes in v18* - Rebase on top of next-20230613 - Minor updates *Changes in v17* - Rebase on top of next-20230606 - Minor improvements in PAGEMAP_SCAN IOCTL patch *Changes in v16* - Fix a corner case - Add exclusive PM_SCAN_OP_WP back *Changes in v15* - Build fix (Add missed build fix in RESEND) *Changes in v14* - Fix build error caused by #ifdef added at last minute in some configs *Changes in v13* - Rebase on top of next-20230414 - Give-up on using uffd_wp_range() and write new helpers, flush tlb only once *Changes in v12* - Update and other memory types to UFFD_FEATURE_WP_ASYNC - Rebaase on top of next-20230406 - Review updates *Changes in v11* - Rebase on top of next-20230307 - Base patches on UFFD_FEATURE_WP_UNPOPULATED - Do a lot of cosmetic changes and review updates - Remove ENGAGE_WP + !GET operation as it can be performed with UFFDIO_WRITEPROTECT *Changes in v10* - Add specific condition to return error if hugetlb is used with wp async - Move changes in tools/include/uapi/linux/fs.h to separate patch - Add documentation *Changes in v9:* - Correct fault resolution for userfaultfd wp async - Fix build warnings and errors which were happening on some configs - Simplify pagemap ioctl's code *Changes in v8:* - Update uffd async wp implementation - Improve PAGEMAP_IOCTL implementation *Changes in v7:* - Add uffd wp async - Update the IOCTL to use uffd under the hood instead of soft-dirty flags *Motivation* The real motivation for adding PAGEMAP_SCAN IOCTL is to emulate Windows GetWriteWatch() syscall [1]. The GetWriteWatch{} retrieves the addresses of the pages that are written to in a region of virtual memory. This syscall is used in Windows applications and games etc. This syscall is being emulated in pretty slow manner in userspace. Our purpose is to enhance the kernel such that we translate it efficiently in a better way. Currently some out of tree hack patches are being used to efficiently emulate it in some kernels. We intend to replace those with these patches. So the whole gaming on Linux can effectively get benefit from this. It means there would be tons of users of this code. CRIU use case [2] was mentioned by Andrei and Danylo: > Use cases for migrating sparse VMAs are binaries sanitized with ASAN, > MSAN or TSAN [3]. All of these sanitizers produce sparse mappings of > shadow memory [4]. Being able to migrate such binaries allows to highly > reduce the amount of work needed to identify and fix post-migration > crashes, which happen constantly. Andrei's defines the following uses of this code: * it is more granular and allows us to track changed pages more effectively. The current interface can clear dirty bits for the entire process only. In addition, reading info about pages is a separate operation. It means we must freeze the process to read information about all its pages, reset dirty bits, only then we can start dumping pages. The information about pages becomes more and more outdated, while we are processing pages. The new interface solves both these downsides. First, it allows us to read pte bits and clear the soft-dirty bit atomically. It means that CRIU will not need to freeze processes to pre-dump their memory. Second, it clears soft-dirty bits for a specified region of memory. It means CRIU will have actual info about pages to the moment of dumping them. * The new interface has to be much faster because basic page filtering is happening in the kernel. With the old interface, we have to read pagemap for each page. *Implementation Evolution (Short Summary)* From the definition of GetWriteWatch(), we feel like kernel's soft-dirty feature can be used under the hood with some additions like: * reset soft-dirty flag for only a specific region of memory instead of clearing the flag for the entire process * get and clear soft-dirty flag for a specific region atomically So we decided to use ioctl on pagemap file to read or/and reset soft-dirty flag. But using soft-dirty flag, sometimes we get extra pages which weren't even written. They had become soft-dirty because of VMA merging and VM_SOFTDIRTY flag. This breaks the definition of GetWriteWatch(). We were able to by-pass this short coming by ignoring VM_SOFTDIRTY until David reported that mprotect etc messes up the soft-dirty flag while ignoring VM_SOFTDIRTY [5]. This wasn't happening until [6] got introduced. We discussed if we can revert these patches. But we could not reach to any conclusion. So at this point, I made couple of tries to solve this whole VM_SOFTDIRTY issue by correcting the soft-dirty implementation: * [7] Correct the bug fixed wrongly back in 2014. It had potential to cause regression. We left it behind. * [8] Keep a list of soft-dirty part of a VMA across splits and merges. I got the reply don't increase the size of the VMA by 8 bytes. At this point, we left soft-dirty considering it is too much delicate and userfaultfd [9] seemed like the only way forward. From there onward, we have been basing soft-dirty emulation on userfaultfd wp feature where kernel resolves the faults itself when WP_ASYNC feature is used. It was straight forward to add WP_ASYNC feature in userfautlfd. Now we get only those pages dirty or written-to which are really written in reality. (PS There is another WP_UNPOPULATED userfautfd feature is required which is needed to avoid pre-faulting memory before write-protecting [9].) All the different masks were added on the request of CRIU devs to create interface more generic and better. [1] https://learn.microsoft.com/en-us/windows/win32/api/memoryapi/nf-memoryapi-… [2] https://lore.kernel.org/all/20221014134802.1361436-1-mdanylo@google.com [3] https://github.com/google/sanitizers [4] https://github.com/google/sanitizers/wiki/AddressSanitizerAlgorithm#64-bit [5] https://lore.kernel.org/all/bfcae708-db21-04b4-0bbe-712badd03071@redhat.com [6] https://lore.kernel.org/all/20220725142048.30450-1-peterx@redhat.com/ [7] https://lore.kernel.org/all/20221122115007.2787017-1-usama.anjum@collabora.… [8] https://lore.kernel.org/all/20221220162606.1595355-1-usama.anjum@collabora.… [9] https://lore.kernel.org/all/20230306213925.617814-1-peterx@redhat.com [10] https://lore.kernel.org/all/20230125144529.1630917-1-mdanylo@google.com * Original Cover letter from v8* Hello, Note: Soft-dirty pages and pages which have been written-to are synonyms. As kernel already has soft-dirty feature inside which we have given up to use, we are using written-to terminology while using UFFD async WP under the hood. This IOCTL, PAGEMAP_SCAN on pagemap file can be used to get and/or clear the info about page table entries. The following operations are supported in this ioctl: - Get the information if the pages have been written-to (PAGE_IS_WRITTEN), file mapped (PAGE_IS_FILE), present (PAGE_IS_PRESENT) or swapped (PAGE_IS_SWAPPED). - Write-protect the pages (PAGEMAP_WP_ENGAGE) to start finding which pages have been written-to. - Find pages which have been written-to and write protect the pages (atomic PAGE_IS_WRITTEN + PAGEMAP_WP_ENGAGE) It is possible to find and clear soft-dirty pages entirely in userspace. But it isn't efficient: - The mprotect and SIGSEGV handler for bookkeeping - The userfaultfd wp (synchronous) with the handler for bookkeeping Some benchmarks can be seen here[1]. This series adds features that weren't present earlier: - There is no atomic get soft-dirty/Written-to status and clear present in the kernel. - The pages which have been written-to can not be found in accurate way. (Kernel's soft-dirty PTE bit + sof_dirty VMA bit shows more soft-dirty pages than there actually are.) Historically, soft-dirty PTE bit tracking has been used in the CRIU project. The procfs interface is enough for finding the soft-dirty bit status and clearing the soft-dirty bit of all the pages of a process. We have the use case where we need to track the soft-dirty PTE bit for only specific pages on-demand. We need this tracking and clear mechanism of a region of memory while the process is running to emulate the getWriteWatch() syscall of Windows. *(Moved to using UFFD instead of soft-dirtyi feature to find pages which have been written-to from v7 patch series)*: Stop using the soft-dirty flags for finding which pages have been written to. It is too delicate and wrong as it shows more soft-dirty pages than the actual soft-dirty pages. There is no interest in correcting it [2][3] as this is how the feature was written years ago. It shouldn't be updated to changed behaviour. Peter Xu has suggested using the async version of the UFFD WP [4] as it is based inherently on the PTEs. So in this patch series, I've added a new mode to the UFFD which is asynchronous version of the write protect. When this variant of the UFFD WP is used, the page faults are resolved automatically by the kernel. The pages which have been written-to can be found by reading pagemap file (!PM_UFFD_WP). This feature can be used successfully to find which pages have been written to from the time the pages were write protected. This works just like the soft-dirty flag without showing any extra pages which aren't soft-dirty in reality. The information related to pages if the page is file mapped, present and swapped is required for the CRIU project [5][6]. The addition of the required mask, any mask, excluded mask and return masks are also required for the CRIU project [5]. The IOCTL returns the addresses of the pages which match the specific masks. The page addresses are returned in struct page_region in a compact form. The max_pages is needed to support a use case where user only wants to get a specific number of pages. So there is no need to find all the pages of interest in the range when max_pages is specified. The IOCTL returns when the maximum number of the pages are found. The max_pages is optional. If max_pages is specified, it must be equal or greater than the vec_size. This restriction is needed to handle worse case when one page_region only contains info of one page and it cannot be compacted. This is needed to emulate the Windows getWriteWatch() syscall. The patch series include the detailed selftest which can be used as an example for the uffd async wp test and PAGEMAP_IOCTL. It shows the interface usages as well. [1] https://lore.kernel.org/lkml/54d4c322-cd6e-eefd-b161-2af2b56aae24@collabora… [2] https://lore.kernel.org/all/20221220162606.1595355-1-usama.anjum@collabora.… [3] https://lore.kernel.org/all/20221122115007.2787017-1-usama.anjum@collabora.… [4] https://lore.kernel.org/all/Y6Hc2d+7eTKs7AiH@x1n [5] https://lore.kernel.org/all/YyiDg79flhWoMDZB@gmail.com/ [6] https://lore.kernel.org/all/20221014134802.1361436-1-mdanylo@google.com/ Regards, Muhammad Usama Anjum Muhammad Usama Anjum (4): fs/proc/task_mmu: Implement IOCTL to get and optionally clear info about PTEs tools headers UAPI: Update linux/fs.h with the kernel sources mm/pagemap: add documentation of PAGEMAP_SCAN IOCTL selftests: mm: add pagemap ioctl tests Peter Xu (1): userfaultfd: UFFD_FEATURE_WP_ASYNC Documentation/admin-guide/mm/pagemap.rst | 58 + Documentation/admin-guide/mm/userfaultfd.rst | 35 + fs/proc/task_mmu.c | 577 +++++++ fs/userfaultfd.c | 26 +- include/linux/hugetlb.h | 1 + include/linux/userfaultfd_k.h | 21 +- include/uapi/linux/fs.h | 55 + include/uapi/linux/userfaultfd.h | 9 +- mm/hugetlb.c | 34 +- mm/memory.c | 27 +- tools/include/uapi/linux/fs.h | 55 + tools/testing/selftests/mm/.gitignore | 2 + tools/testing/selftests/mm/Makefile | 3 +- tools/testing/selftests/mm/config | 1 + tools/testing/selftests/mm/pagemap_ioctl.c | 1464 ++++++++++++++++++ tools/testing/selftests/mm/run_vmtests.sh | 4 + 16 files changed, 2348 insertions(+), 24 deletions(-) create mode 100644 tools/testing/selftests/mm/pagemap_ioctl.c mode change 100644 => 100755 tools/testing/selftests/mm/run_vmtests.sh -- 2.39.2

1 year, 12 months

1
5
0 0

[PATCH v22 0/5] Implement IOCTL to get and optionally clear info about PTEs

by Muhammad Usama Anjum

Changes in v22: - Interface change: - Replace [start start + len) with [start, end) - Return the ending address of the address walk in start Changes in v21: - Abort walk instead of returning error if WP is to be performed on partial hugetlb *Changes in v20* - Correct PAGE_IS_FILE and add PAGE_IS_PFNZERO *Changes in v19* - Minor changes and interface updates *Changes in v18* - Rebase on top of next-20230613 - Minor updates *Changes in v17* - Rebase on top of next-20230606 - Minor improvements in PAGEMAP_SCAN IOCTL patch *Changes in v16* - Fix a corner case - Add exclusive PM_SCAN_OP_WP back *Changes in v15* - Build fix (Add missed build fix in RESEND) *Changes in v14* - Fix build error caused by #ifdef added at last minute in some configs *Changes in v13* - Rebase on top of next-20230414 - Give-up on using uffd_wp_range() and write new helpers, flush tlb only once *Changes in v12* - Update and other memory types to UFFD_FEATURE_WP_ASYNC - Rebaase on top of next-20230406 - Review updates *Changes in v11* - Rebase on top of next-20230307 - Base patches on UFFD_FEATURE_WP_UNPOPULATED - Do a lot of cosmetic changes and review updates - Remove ENGAGE_WP + !GET operation as it can be performed with UFFDIO_WRITEPROTECT *Changes in v10* - Add specific condition to return error if hugetlb is used with wp async - Move changes in tools/include/uapi/linux/fs.h to separate patch - Add documentation *Changes in v9:* - Correct fault resolution for userfaultfd wp async - Fix build warnings and errors which were happening on some configs - Simplify pagemap ioctl's code *Changes in v8:* - Update uffd async wp implementation - Improve PAGEMAP_IOCTL implementation *Changes in v7:* - Add uffd wp async - Update the IOCTL to use uffd under the hood instead of soft-dirty flags *Motivation* The real motivation for adding PAGEMAP_SCAN IOCTL is to emulate Windows GetWriteWatch() syscall [1]. The GetWriteWatch{} retrieves the addresses of the pages that are written to in a region of virtual memory. This syscall is used in Windows applications and games etc. This syscall is being emulated in pretty slow manner in userspace. Our purpose is to enhance the kernel such that we translate it efficiently in a better way. Currently some out of tree hack patches are being used to efficiently emulate it in some kernels. We intend to replace those with these patches. So the whole gaming on Linux can effectively get benefit from this. It means there would be tons of users of this code. CRIU use case [2] was mentioned by Andrei and Danylo: > Use cases for migrating sparse VMAs are binaries sanitized with ASAN, > MSAN or TSAN [3]. All of these sanitizers produce sparse mappings of > shadow memory [4]. Being able to migrate such binaries allows to highly > reduce the amount of work needed to identify and fix post-migration > crashes, which happen constantly. Andrei's defines the following uses of this code: * it is more granular and allows us to track changed pages more effectively. The current interface can clear dirty bits for the entire process only. In addition, reading info about pages is a separate operation. It means we must freeze the process to read information about all its pages, reset dirty bits, only then we can start dumping pages. The information about pages becomes more and more outdated, while we are processing pages. The new interface solves both these downsides. First, it allows us to read pte bits and clear the soft-dirty bit atomically. It means that CRIU will not need to freeze processes to pre-dump their memory. Second, it clears soft-dirty bits for a specified region of memory. It means CRIU will have actual info about pages to the moment of dumping them. * The new interface has to be much faster because basic page filtering is happening in the kernel. With the old interface, we have to read pagemap for each page. *Implementation Evolution (Short Summary)* From the definition of GetWriteWatch(), we feel like kernel's soft-dirty feature can be used under the hood with some additions like: * reset soft-dirty flag for only a specific region of memory instead of clearing the flag for the entire process * get and clear soft-dirty flag for a specific region atomically So we decided to use ioctl on pagemap file to read or/and reset soft-dirty flag. But using soft-dirty flag, sometimes we get extra pages which weren't even written. They had become soft-dirty because of VMA merging and VM_SOFTDIRTY flag. This breaks the definition of GetWriteWatch(). We were able to by-pass this short coming by ignoring VM_SOFTDIRTY until David reported that mprotect etc messes up the soft-dirty flag while ignoring VM_SOFTDIRTY [5]. This wasn't happening until [6] got introduced. We discussed if we can revert these patches. But we could not reach to any conclusion. So at this point, I made couple of tries to solve this whole VM_SOFTDIRTY issue by correcting the soft-dirty implementation: * [7] Correct the bug fixed wrongly back in 2014. It had potential to cause regression. We left it behind. * [8] Keep a list of soft-dirty part of a VMA across splits and merges. I got the reply don't increase the size of the VMA by 8 bytes. At this point, we left soft-dirty considering it is too much delicate and userfaultfd [9] seemed like the only way forward. From there onward, we have been basing soft-dirty emulation on userfaultfd wp feature where kernel resolves the faults itself when WP_ASYNC feature is used. It was straight forward to add WP_ASYNC feature in userfautlfd. Now we get only those pages dirty or written-to which are really written in reality. (PS There is another WP_UNPOPULATED userfautfd feature is required which is needed to avoid pre-faulting memory before write-protecting [9].) All the different masks were added on the request of CRIU devs to create interface more generic and better. [1] https://learn.microsoft.com/en-us/windows/win32/api/memoryapi/nf-memoryapi-… [2] https://lore.kernel.org/all/20221014134802.1361436-1-mdanylo@google.com [3] https://github.com/google/sanitizers [4] https://github.com/google/sanitizers/wiki/AddressSanitizerAlgorithm#64-bit [5] https://lore.kernel.org/all/bfcae708-db21-04b4-0bbe-712badd03071@redhat.com [6] https://lore.kernel.org/all/20220725142048.30450-1-peterx@redhat.com/ [7] https://lore.kernel.org/all/20221122115007.2787017-1-usama.anjum@collabora.… [8] https://lore.kernel.org/all/20221220162606.1595355-1-usama.anjum@collabora.… [9] https://lore.kernel.org/all/20230306213925.617814-1-peterx@redhat.com [10] https://lore.kernel.org/all/20230125144529.1630917-1-mdanylo@google.com * Original Cover letter from v8* Hello, Note: Soft-dirty pages and pages which have been written-to are synonyms. As kernel already has soft-dirty feature inside which we have given up to use, we are using written-to terminology while using UFFD async WP under the hood. This IOCTL, PAGEMAP_SCAN on pagemap file can be used to get and/or clear the info about page table entries. The following operations are supported in this ioctl: - Get the information if the pages have been written-to (PAGE_IS_WRITTEN), file mapped (PAGE_IS_FILE), present (PAGE_IS_PRESENT) or swapped (PAGE_IS_SWAPPED). - Write-protect the pages (PAGEMAP_WP_ENGAGE) to start finding which pages have been written-to. - Find pages which have been written-to and write protect the pages (atomic PAGE_IS_WRITTEN + PAGEMAP_WP_ENGAGE) It is possible to find and clear soft-dirty pages entirely in userspace. But it isn't efficient: - The mprotect and SIGSEGV handler for bookkeeping - The userfaultfd wp (synchronous) with the handler for bookkeeping Some benchmarks can be seen here[1]. This series adds features that weren't present earlier: - There is no atomic get soft-dirty/Written-to status and clear present in the kernel. - The pages which have been written-to can not be found in accurate way. (Kernel's soft-dirty PTE bit + sof_dirty VMA bit shows more soft-dirty pages than there actually are.) Historically, soft-dirty PTE bit tracking has been used in the CRIU project. The procfs interface is enough for finding the soft-dirty bit status and clearing the soft-dirty bit of all the pages of a process. We have the use case where we need to track the soft-dirty PTE bit for only specific pages on-demand. We need this tracking and clear mechanism of a region of memory while the process is running to emulate the getWriteWatch() syscall of Windows. *(Moved to using UFFD instead of soft-dirtyi feature to find pages which have been written-to from v7 patch series)*: Stop using the soft-dirty flags for finding which pages have been written to. It is too delicate and wrong as it shows more soft-dirty pages than the actual soft-dirty pages. There is no interest in correcting it [2][3] as this is how the feature was written years ago. It shouldn't be updated to changed behaviour. Peter Xu has suggested using the async version of the UFFD WP [4] as it is based inherently on the PTEs. So in this patch series, I've added a new mode to the UFFD which is asynchronous version of the write protect. When this variant of the UFFD WP is used, the page faults are resolved automatically by the kernel. The pages which have been written-to can be found by reading pagemap file (!PM_UFFD_WP). This feature can be used successfully to find which pages have been written to from the time the pages were write protected. This works just like the soft-dirty flag without showing any extra pages which aren't soft-dirty in reality. The information related to pages if the page is file mapped, present and swapped is required for the CRIU project [5][6]. The addition of the required mask, any mask, excluded mask and return masks are also required for the CRIU project [5]. The IOCTL returns the addresses of the pages which match the specific masks. The page addresses are returned in struct page_region in a compact form. The max_pages is needed to support a use case where user only wants to get a specific number of pages. So there is no need to find all the pages of interest in the range when max_pages is specified. The IOCTL returns when the maximum number of the pages are found. The max_pages is optional. If max_pages is specified, it must be equal or greater than the vec_size. This restriction is needed to handle worse case when one page_region only contains info of one page and it cannot be compacted. This is needed to emulate the Windows getWriteWatch() syscall. The patch series include the detailed selftest which can be used as an example for the uffd async wp test and PAGEMAP_IOCTL. It shows the interface usages as well. [1] https://lore.kernel.org/lkml/54d4c322-cd6e-eefd-b161-2af2b56aae24@collabora… [2] https://lore.kernel.org/all/20221220162606.1595355-1-usama.anjum@collabora.… [3] https://lore.kernel.org/all/20221122115007.2787017-1-usama.anjum@collabora.… [4] https://lore.kernel.org/all/Y6Hc2d+7eTKs7AiH@x1n [5] https://lore.kernel.org/all/YyiDg79flhWoMDZB@gmail.com/ [6] https://lore.kernel.org/all/20221014134802.1361436-1-mdanylo@google.com/ Regards, Muhammad Usama Anjum Muhammad Usama Anjum (4): fs/proc/task_mmu: Implement IOCTL to get and optionally clear info about PTEs tools headers UAPI: Update linux/fs.h with the kernel sources mm/pagemap: add documentation of PAGEMAP_SCAN IOCTL selftests: mm: add pagemap ioctl tests Peter Xu (1): userfaultfd: UFFD_FEATURE_WP_ASYNC Documentation/admin-guide/mm/pagemap.rst | 58 + Documentation/admin-guide/mm/userfaultfd.rst | 35 + fs/proc/task_mmu.c | 565 +++++++ fs/userfaultfd.c | 26 +- include/linux/hugetlb.h | 1 + include/linux/userfaultfd_k.h | 21 +- include/uapi/linux/fs.h | 55 + include/uapi/linux/userfaultfd.h | 9 +- mm/hugetlb.c | 34 +- mm/memory.c | 27 +- tools/include/uapi/linux/fs.h | 55 + tools/testing/selftests/mm/.gitignore | 2 + tools/testing/selftests/mm/Makefile | 3 +- tools/testing/selftests/mm/config | 1 + tools/testing/selftests/mm/pagemap_ioctl.c | 1464 ++++++++++++++++++ tools/testing/selftests/mm/run_vmtests.sh | 4 + 16 files changed, 2336 insertions(+), 24 deletions(-) create mode 100644 tools/testing/selftests/mm/pagemap_ioctl.c mode change 100644 => 100755 tools/testing/selftests/mm/run_vmtests.sh -- 2.39.2

1 year, 12 months

4
13
0 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

Linux-kselftest-mirror