- Linux-kselftest-mirror - lists.linaro.org

[PATCH RFC] kselftest: devices: Add test to detect missing devices

by Nícolas F. R. A. Prado

Introduce a new test to identify regressions causing devices to go missing on the system. For each bus and class on the system the test checks the number of devices present against a reference file, which needs to have been generated by the program at a previous point on a known-good kernel, and if there are missing devices they are reported. Signed-off-by: Nícolas F. R. A. Prado <nfraprado(a)collabora.com> --- Hi, Key points about this test: * Goal: Identify regressions causing devices to go missing on the system * Focus: * Ease of maintenance: the reference file is generated programatically * Minimum of false-positives: the script makes as few assumptions as possible about the stability of device identifiers to ensure renames/refactors don't trigger false-positives * How it works: For each bus and class on the system the test checks the number of devices present against a reference file, which needs to have been generated by the program at a previous point on a known-good kernel, and if there are missing devices they are reported. * Comparison to other tests: It might be possible(*) to replace the discoverable devices test [1] with this. The benefits of this test is that it's easier to setup and maintain and has wider coverage of devices. Additional detail: * Having more devices on the running system than the reference does not cause a failure, but a warning is printed in that case to suggest that the reference be updated. * Missing devices are detected per bus/class based on the number of devices. When the test fails, the known metadata for each of the expected and detected devices is printed and some simple similitarity comparison is done to suggest the devices that are the most likely to be missing. * The proposed place to store the generated reference files is the 'platform-test-parameters' repository in KernelCI [2]. Example output: This is an example of a failing test case when one of the two devices in the nvmem bus went missing: # Missing devices for subsystem 'nvmem': 1 (Expected 2, found 1) # ================= # Devices expected: # # uevent: # OF_NAME=efuse # OF_FULLNAME=/soc/efuse@11c10000 # OF_COMPATIBLE_0=mediatek,mt8195-efuse # OF_COMPATIBLE_1=mediatek,efuse # OF_COMPATIBLE_N=2 # # uevent: # OF_NAME=flash # OF_FULLNAME=/soc/spi@1132c000/flash@0 # OF_COMPATIBLE_0=jedec,spi-nor # OF_COMPATIBLE_N=1 # # ----------------- # Devices found: # # uevent: # OF_NAME=efuse # OF_FULLNAME=/soc/efuse@11c10000 # OF_COMPATIBLE_0=mediatek,mt8195-efuse # OF_COMPATIBLE_1=mediatek,efuse # OF_COMPATIBLE_N=2 # # ----------------- # Devices missing (best guess): # # uevent: # OF_NAME=flash # OF_FULLNAME=/soc/spi@1132c000/flash@0 # OF_COMPATIBLE_0=jedec,spi-nor # OF_COMPATIBLE_N=1 # # ================= not ok 19 bus.nvmem Example of how the data for these devices is encoded in the reference file: bus: ... nvmem: count: 2 devices: - info: uevent: 'OF_NAME=efuse OF_FULLNAME=/soc/efuse@11c10000 OF_COMPATIBLE_0=mediatek,mt8195-efuse OF_COMPATIBLE_1=mediatek,efuse OF_COMPATIBLE_N=2 ' - info: uevent: 'OF_NAME=flash OF_FULLNAME=/soc/spi@1132c000/flash@0 OF_COMPATIBLE_0=jedec,spi-nor OF_COMPATIBLE_N=1 ' (Full reference file: http://0x0.st/Xp60.yaml;) Caveat: Relying only on the count of devices in a subsystem makes the test susceptible to false-negatives eg. if a device goes missing and another in the same subsystem is added the count will be the same so this regression won't be reported. In order to avoid this we may include properties that must match individual devices, but we must be very careful (and it's why I haven't done it) since matching against properties that aren't guaranteed to be stable will introduce false-positives (ie. detecting false regressions) due to eventual renames. Some things to improve in the near future / gather feedback on: * (*): Currently this test only checks for the existence of devices. We could extend it to also encode into the reference which devices are bound to drivers to be able to completely replace the discoverable devices probe kselftest [1]. * Expanding identifying properties: Currently the properties that are stored (when present) in the reference for each device to be used for identification in the result output are uevent, device/uevent, firmware_node/uevent and name. Suggestions of others properties to add are welcome. * Adding more filtering to reduce noise: * Ignoring buses/classes: Currently the devlink class is ignored by the test since it seems like a kernel internal detail that userspace doesn't actually care about. We should add others that are similar. * Ignoring non-devices: There can be entries in /sys/class/ that aren't devices. For now we're filtering down to only symlinks, but there might be a better way. * As mentioned in the caveat section above we may want to add actual matching of devices based on properties to avoid false-negatives if we identify suitable properties. * It would be nice to have an option in the program to compare a newer reference to an older one to make it easier for the user to see the differences and decide if the new reference is ok. * Since the reference file is not supposed to be manually edited, JSON might be a better choice than YAML since it is included in the python standard library. Let me know your thoughts. Thanks, Nícolas [1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/too… [2] https://github.com/kernelci/platform-test-parameters --- tools/testing/selftests/Makefile | 1 + tools/testing/selftests/devices/exist/Makefile | 3 + tools/testing/selftests/devices/exist/exist.py | 268 +++++++++++++++++++++++++ 3 files changed, 272 insertions(+) diff --git a/tools/testing/selftests/Makefile b/tools/testing/selftests/Makefile index bc8fe9e8f7f2..9c49b5ec5bef 100644 --- a/tools/testing/selftests/Makefile +++ b/tools/testing/selftests/Makefile @@ -14,6 +14,7 @@ TARGETS += cpufreq TARGETS += cpu-hotplug TARGETS += damon TARGETS += devices/error_logs +TARGETS += devices/exist TARGETS += devices/probe TARGETS += dmabuf-heaps TARGETS += drivers/dma-buf diff --git a/tools/testing/selftests/devices/exist/Makefile b/tools/testing/selftests/devices/exist/Makefile new file mode 100644 index 000000000000..3075cac32092 --- /dev/null +++ b/tools/testing/selftests/devices/exist/Makefile @@ -0,0 +1,3 @@ +TEST_PROGS := exist.py + +include ../../lib.mk diff --git a/tools/testing/selftests/devices/exist/exist.py b/tools/testing/selftests/devices/exist/exist.py new file mode 100755 index 000000000000..8241b2fabc8e --- /dev/null +++ b/tools/testing/selftests/devices/exist/exist.py @@ -0,0 +1,268 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2024 Collabora Ltd + +# * Goal: Identify regressions causing devices to go missing on the system +# * Focus: +# * Ease of maintenance: the reference file is generated programatically +# * Minimum of false-positives: the script makes as few assumptions as +# possible about the stability of device identifiers to ensure +# renames/refactors don't trigger false-positives +# * How it works: For each bus and class on the system the test checks the +# number of devices present against a reference file, which needs to have been +# generated by the program at a previous point on a known-good kernel, and if +# there are missing devices they are reported. + +import os +import sys +import argparse + +import yaml + +# Allow ksft module to be imported from different directory +this_dir = os.path.dirname(os.path.realpath(__file__)) +sys.path.append(os.path.join(this_dir, "../../kselftest/")) + +import ksft + + +def generate_devs_obj(): + obj = {} + + device_sources = [ + { + "base_dir": "/sys/class", + "add_path": "", + "key_name": "class", + "ignored": ["devlink"], + }, + { + "base_dir": "/sys/bus", + "add_path": "devices", + "key_name": "bus", + "ignored": [], + }, + ] + + properties = sorted(["uevent", "device/uevent", "firmware_node/uevent", "name"]) + + for source in device_sources: + source_subsystems = {} + for subsystem in sorted(os.listdir(source["base_dir"])): + if subsystem in source["ignored"]: + continue + + devs_path = os.path.join(source["base_dir"], subsystem, source["add_path"]) + dev_dirs = [dev for dev in os.scandir(devs_path) if dev.is_symlink()] + devs_data = [] + for dev_dir in dev_dirs: + dev_path = os.path.join(devs_path, dev_dir) + dev_data = {"info": {}} + for prop in properties: + if os.path.isfile(os.path.join(dev_path, prop)): + with open(os.path.join(dev_path, prop)) as f: + dev_data["info"][prop] = f.read() + devs_data.append(dev_data) + if len(dev_dirs): + source_subsystems[subsystem] = { + "count": len(dev_dirs), + "devices": devs_data, + } + obj[source["key_name"]] = source_subsystems + + return obj + + +def commented(s): + return s.replace("\n", "\n# ") + + +def indented(s, n): + return " " * n + s.replace("\n", "\n" + " " * n) + + +def stripped(s): + return s.strip("\n") + + +def devices_difference(dev1, dev2): + difference = 0 + + for prop in dev1["info"].keys(): + for l1, l2 in zip( + dev1["info"].get(prop, "").split("\n"), + dev2["info"].get(prop, "").split("\n"), + ): + if l1 != l2: + difference += 1 + return difference + + +def guess_missing_devices(cur_devs_subsystem, ref_devs_subsystem): + # Detect what devices on the current system are the most similar to devices + # on the reference one by one until the leftovers are the most dissimilar + # devices and therefore most likely the missing ones. + found_count = cur_devs_subsystem["count"] + expected_count = ref_devs_subsystem["count"] + missing_count = found_count - expected_count + + diffs = [] + for cur_d in cur_devs_subsystem["devices"]: + for ref_d in ref_devs_subsystem["devices"]: + diffs.append((devices_difference(cur_d, ref_d), cur_d, ref_d)) + + diffs.sort(key=lambda x: x[0]) + + assigned_ref_devs = [] + assigned_cur_devs = [] + for diff in diffs: + if len(assigned_ref_devs) >= expected_count - missing_count: + break + if diff[1] in assigned_cur_devs or diff[2] in assigned_ref_devs: + continue + assigned_cur_devs.append(diff[1]) + assigned_ref_devs.append(diff[2]) + + missing_devices = [] + for d in ref_devs_subsystem["devices"]: + if d not in assigned_ref_devs: + missing_devices.append(d) + + return missing_devices + + +def dump_devices_info(cur_devs_subsystem, ref_devs_subsystem): + def dump_device_info(dev): + for name, val in dev["info"].items(): + ksft.print_msg(indented(name + ":", 2)) + val = stripped(val) + if val: + ksft.print_msg(commented(indented(val, 4))) + ksft.print_msg("") + + ksft.print_msg("=================") + ksft.print_msg("Devices expected:") + ksft.print_msg("") + for d in ref_devs_subsystem["devices"]: + dump_device_info(d) + ksft.print_msg("-----------------") + ksft.print_msg("Devices found:") + ksft.print_msg("") + for d in cur_devs_subsystem["devices"]: + dump_device_info(d) + ksft.print_msg("-----------------") + ksft.print_msg("Devices missing (best guess):") + ksft.print_msg("") + missing_devices = guess_missing_devices(cur_devs_subsystem, ref_devs_subsystem) + for d in missing_devices: + dump_device_info(d) + ksft.print_msg("=================") + + +def run_test(ref_filename): + ksft.print_msg(f"Using reference file: {ref_filename}") + + with open(ref_filename) as f: + ref_devs_obj = yaml.safe_load(f) + + num_tests = 0 + for dev_source in ref_devs_obj.values(): + num_tests += len(dev_source) + ksft.set_plan(num_tests) + + cur_devs_obj = generate_devs_obj() + + reference_outdated = False + + for source, ref_devs_source_obj in ref_devs_obj.items(): + for subsystem, ref_devs_subsystem_obj in ref_devs_source_obj.items(): + test_name = f"{source}.{subsystem}" + if not ( + cur_devs_obj.get(source) and cur_devs_obj.get(source).get(subsystem) + ): + ksft.print_msg(f"Device subsystem '{subsystem}' missing") + ksft.test_result_fail(test_name) + continue + cur_devs_subsystem_obj = cur_devs_obj[source][subsystem] + + found_count = cur_devs_subsystem_obj["count"] + expected_count = ref_devs_subsystem_obj["count"] + if found_count < expected_count: + ksft.print_msg( + f"Missing devices for subsystem '{subsystem}': {expected_count - found_count} (Expected {expected_count}, found {found_count})" + ) + dump_devices_info(cur_devs_subsystem_obj, ref_devs_subsystem_obj) + ksft.test_result_fail(test_name) + else: + ksft.test_result_pass(test_name) + if found_count > expected_count: + reference_outdated = True + + if len(cur_devs_obj[source]) > len(ref_devs_source_obj): + reference_outdated = True + + if reference_outdated: + ksft.print_msg( + "Warning: The current system contains more devices and/or subsystems than the reference. Updating the reference is recommended." + ) + + +def get_possible_ref_filenames(): + filenames = [] + + dt_board_compatible_file = "/proc/device-tree/compatible" + if os.path.exists(dt_board_compatible_file): + with open(dt_board_compatible_file) as f: + for line in f: + compatibles = [compat for compat in line.split("\0") if compat] + filenames.extend(compatibles) + else: + dmi_id_dir = "/sys/devices/virtual/dmi/id" + vendor_dmi_file = os.path.join(dmi_id_dir, "sys_vendor") + product_dmi_file = os.path.join(dmi_id_dir, "product_name") + + with open(vendor_dmi_file) as f: + vendor = f.read().replace("\n", "") + with open(product_dmi_file) as f: + product = f.read().replace("\n", "") + + filenames = [vendor + "," + product] + + return filenames + + +def get_ref_filename(ref_dir): + chosen_ref_filename = "" + full_ref_paths = [os.path.join(ref_dir, f + ".yaml") for f in get_possible_ref_filenames()] + for path in full_ref_paths: + if os.path.exists(path): + chosen_ref_filename = path + break + + if not chosen_ref_filename: + tried_paths = ",".join(["'" + p + "'" for p in full_ref_paths]) + ksft.print_msg(f"No matching reference file found (tried {tried_paths})") + ksft.exit_fail() + + return chosen_ref_filename + + +parser = argparse.ArgumentParser() +parser.add_argument( + "--reference-dir", default=".", help="Directory containing the reference files" +) +parser.add_argument("--generate-reference", action="store_true", help="Generate a reference file with the devices on the running system") +args = parser.parse_args() + +if args.generate_reference: + print(f"# Kernel version: {os.uname().release}") + print(yaml.dump(generate_devs_obj())) + sys.exit(0) + +ksft.print_header() + +ref_filename = get_ref_filename(args.reference_dir) + +run_test(ref_filename) + +ksft.finished() --- base-commit: 73399b58e5e5a1b28a04baf42e321cfcfc663c2f change-id: 20240724-kselftest-dev-exist-bb1bcf884654 Best regards, -- Nícolas F. R. A. Prado <nfraprado(a)collabora.com>

8 months, 4 weeks

3
7
0 0

Questions regarding "act_mirred: use the backlog for nested calls to mirred ingress"

by Khannanov Lenar

Hello Davide, In the following commit: commit ca22da2fbd693b54dc8e3b7b54ccc9f7e9ba3640 Author: Davide Caratti <dcaratti(a)redhat.com> Date: Fri Jan 20 18:01:40 2023 +0100 act_mirred: use the backlog for nested calls to mirred ingress you added the mirred_egress_to_ingress_tcp_test kselftest that, I noticed, hangs with "Ncat: TIMEOUT." if openvswitch module is loaded. Is this the right behaviour or was such configuration not intended? Regards, Lenar Khannanov

8 months, 4 weeks

1
0
0 0

[PATCH net-next 1/6] net: netconsole: selftests: Create a new netconsole selftest

by Breno Leitao

Adds a selftest that creates two virtual interfaces, assigns one to a new namespace, and assigns IP addresses to both. It listens on the destination interface using netcat and configures a dynamic target on netconsole, pointing to the destination IP address. The test then checks if the message was received properly on the destination interface. Signed-off-by: Breno Leitao <leitao(a)debian.org> --- MAINTAINERS | 1 + .../net/netconsole/basic_integration_test.sh | 153 ++++++++++++++++++ 2 files changed, 154 insertions(+) create mode 100755 tools/testing/selftests/net/netconsole/basic_integration_test.sh diff --git a/MAINTAINERS b/MAINTAINERS index c0a3d9e93689..59207365c9f5 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -15768,6 +15768,7 @@ M: Breno Leitao <leitao(a)debian.org> S: Maintained F: Documentation/networking/netconsole.rst F: drivers/net/netconsole.c +F: tools/testing/selftests/net/netconsole/ NETDEVSIM M: Jakub Kicinski <kuba(a)kernel.org> diff --git a/tools/testing/selftests/net/netconsole/basic_integration_test.sh b/tools/testing/selftests/net/netconsole/basic_integration_test.sh new file mode 100755 index 000000000000..fbabbc633451 --- /dev/null +++ b/tools/testing/selftests/net/netconsole/basic_integration_test.sh @@ -0,0 +1,153 @@ +#!/usr/bin/env bash +# SPDX-License-Identifier: GPL-2.0 + +# This test creates two virtual interfaces, assigns one of them (the "destination +# interface") to a new namespace, and assigns IP addresses to both interfaces. +# +# It listens on the destination interface using netcat (nc) and configures a +# dynamic target on netconsole, pointing to the destination IP address. +# +# Finally, it checks whether the message was received properly on the +# destination interface. Note that this test may pollute the kernel log buffer +# (dmesg) and relies on dynamic configuration and namespaces being configured." +# +# Author: Breno Leitao <leitao(a)debian.org> + +SCRIPTDIR=$(dirname "$(readlink -e "${BASH_SOURCE[0]}")") + +# Simple script to test dynamic targets in netconsole +SRCIF="veth0" +SRCIP=192.168.1.1 + +DSTIF="veth1" +DSTIP=192.168.1.2 + +PORT="6666" +MSG="netconsole selftest" +TARGET=$(mktemp -u netcons_XXXXX) +NETCONS_CONFIGFS="/sys/kernel/config/netconsole" +FULLPATH="${NETCONS_CONFIGFS}"/"${TARGET}" +# This will be have some tmp values appened to it in set_network() +NAMESPACE="netconsns" + +# Used to create and delete namespaces +source "${SCRIPTDIR}"/../lib.sh + +function set_network() { + # this is coming from lib.sh + setup_ns "${NAMESPACE}" + NAMESPACE=${NS_LIST[0]} + ip link add "${SRCIF}" type veth peer name "${DSTIF}" + + # "${DSTIF}" + ip link set "${DSTIF}" netns "${NAMESPACE}" + ip netns exec "${NAMESPACE}" ip addr add "${DSTIP}"/24 dev "${DSTIF}" + ip netns exec "${NAMESPACE}" ip link set "${DSTIF}" up + + # later, configure "${SRCIF}" + ip addr add "${SRCIP}"/24 dev "${SRCIF}" + ip link set "${SRCIF}" up +} + +function create_dynamic_target() { + DSTMAC=$(ip netns exec "${NAMESPACE}" ip link show "${DSTIF}" | awk '/ether/ {print $2}') + + # Create a dynamic target + mkdir ${FULLPATH} + + echo ${DSTIP} > ${FULLPATH}/remote_ip + echo ${SRCIP} > ${FULLPATH}/local_ip + echo ${DSTMAC} > ${FULLPATH}/remote_mac + echo "${SRCIF}" > ${FULLPATH}/dev_name + + echo 1 > ${FULLPATH}/enabled +} + +function cleanup() { + echo 0 > "${FULLPATH}"/enabled + rmdir "${FULLPATH}" + # This will delete DSTIF also + ip link del "${SRCIF}" + # this is coming from lib.sh + cleanup_all_ns +} + +function listen_port() { + OUTPUT=${1} + echo "Saving content in ${OUTPUT}" + timeout 2 ip netns exec "${NAMESPACE}" nc -u -l "${PORT}" | sed '/^$/q' > ${OUTPUT} +} + +function validate_result() { + TMPFILENAME=/tmp/"${TARGET}" + + # sleep until the file isc reated + sleep 1 + # Check if the file exists + if [ ! -f "$TMPFILENAME" ]; then + echo "FAIL: File was not generate." >&2 + return ${ksft_fail} + fi + + if ! grep -q "${MSG}" "${TMPFILENAME}"; then + echo "FAIL: ${MSG} not found in ${TMPFILENAME}" >&2 + cat ${TMPFILENAME} >&2 + return ${ksft_fail} + fi + + rm ${TMPFILENAME} + return ${ksft_pass} +} + +function check_for_dependencies() { + if [ "$(id -u)" -ne 0 ]; then + echo "This script must be run as root" >&2 + exit 1 + fi + + if ! which nc > /dev/null ; then + echo "SKIP: nc(1) is not available" >&2 + exit ${ksft_skip} + fi + + if ! which ip > /dev/null ; then + echo "SKIP: ip(1) is not available" >&2 + exit ${ksft_skip} + fi + + if [ ! -d "${NETCONS_CONFIGFS}" ]; then + echo "SKIP: directory ${NETCONS_CONFIGFS} does not exist. Check if NETCONSOLE_DYNAMIC is enabled" >&2 + exit ${ksft_skip} + fi + + if ip link show veth0 2> /dev/null; then + echo "SKIP: interface veth0 exists in the system. Not overwriting it." + exit ${ksft_skip} + fi +} + +# ========== # +# Start here # +# ========== # + +# Check for basic system dependency and exit if not found +check_for_dependencies +# Create one namespace and two interfaces +set_network +# Create a dynamic target for netconsole +create_dynamic_target +# Listed for netconsole port inside the namespace and destination interface +listen_port /tmp/"${TARGET}" & +# Wait for nc to start and listen to the port. +sleep 1 + +# Send the message +echo "${MSG}: ${TARGET}" > /dev/kmsg + +# Make sure the message was received in the dst part +validate_result +ret=$? +# Remove the namespace, interfaces and netconsole target +cleanup + +exit ${ret} -- 2.43.0

8 months, 4 weeks

2
2
0 0

[PATCH net 0/7] mptcp: fix endpoints with 'signal' and 'subflow' flags

by Matthieu Baerts (NGI0)

When looking at improving the user experience around the MPTCP endpoints setup, I noticed that setting an endpoint with both the 'signal' and the 'subflow' flags -- as it has been done in the past by users according to bug reports we got -- was resulting on only announcing the endpoint, but not using it to create subflows: the 'subflow' flag was then ignored. My initial thought was to modify IPRoute2 to warn the user when the two flags were set, but it doesn't sound normal to ignore one of them. I then looked at modifying the kernel not to allow having the two flags set, but when discussing about that with Mat, we thought it was maybe not ideal to do that, as there might be use-cases, we might break some configs. Then I saw it was working before v5.17. So instead, I fixed the support on the kernel side (patch 5) using Paolo's suggestion. This also includes a fix on the options side (patch 1: for v5.11+), an explicit deny of some options combinations (patch 2: for v5.18+), and some refactoring (patches 3 and 4) to ease the inclusion of the patch 5. While at it, I added a new selftest (patch 7) to validate this case -- including a modification of the chk_add_nr helper to inverse the sides were the counters are checked (patch 6) -- and allowed ADD_ADDR echo just after the MP_JOIN 3WHS. The selftests modification have the same Fixes tag as the previous commit, but no 'Cc: Stable': if the backport can work, that's good -- but it still need to be verified by running the selftests -- if not, no need to worry, many CIs will use the selftests from the last stable version to validate previous stable releases. Signed-off-by: Matthieu Baerts (NGI0) <matttbe(a)kernel.org> --- Matthieu Baerts (NGI0) (7): mptcp: fully established after ADD_ADDR echo on MPJ mptcp: pm: deny endp with signal + subflow + port mptcp: pm: reduce indentation blocks mptcp: pm: don't try to create sf if alloc failed mptcp: pm: do not ignore 'subflow' if 'signal' flag is also set selftests: mptcp: join: ability to invert ADD_ADDR check selftests: mptcp: join: test both signal & subflow net/mptcp/options.c | 3 +- net/mptcp/pm_netlink.c | 47 +++++++++++++-------- tools/testing/selftests/net/mptcp/mptcp_join.sh | 55 ++++++++++++++++++------- 3 files changed, 73 insertions(+), 32 deletions(-) --- base-commit: 0bf50cead4c4710d9f704778c32ab8af47ddf070 change-id: 20240731-upstream-net-20240731-mptcp-endp-subflow-signal-181d640cf5e8 Best regards, -- Matthieu Baerts (NGI0) <matttbe(a)kernel.org>

8 months, 4 weeks

2
8
0 0

[PATCH net-next v2 1/2] selftests: net-drv: exercise queue stats when the device is down

by Stanislav Fomichev

Verify that total device stats don't decrease after it has been turned down. Also make sure the device doesn't crash when we access per-queue stats when it's down (in case it tries to access some pointers that are NULL). KTAP version 1 1..5 ok 1 stats.check_pause ok 2 stats.check_fec ok 3 stats.pkt_byte_sum ok 4 stats.qstat_by_ifindex ok 5 stats.check_down # Totals: pass:5 fail:0 xfail:0 xpass:0 skip:0 error:0 v2: - KTAP output formatting (Jakub) - defer instead of try/finally (Jakub) - disappearing stats is an error (Jakub) - ksft_ge instead of open coding (Jakub) Signed-off-by: Stanislav Fomichev <sdf(a)fomichev.me> -- Cc: Shuah Khan <shuah(a)kernel.org> Cc: Joe Damato <jdamato(a)fastly.com> Cc: Petr Machata <petrm(a)nvidia.com> Cc: linux-kselftest(a)vger.kernel.org --- tools/testing/selftests/drivers/net/stats.py | 25 +++++++++++++++++++- 1 file changed, 24 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/drivers/net/stats.py b/tools/testing/selftests/drivers/net/stats.py index 820b8e0a22c6..93f9204f51c4 100755 --- a/tools/testing/selftests/drivers/net/stats.py +++ b/tools/testing/selftests/drivers/net/stats.py @@ -5,6 +5,7 @@ from lib.py import ksft_run, ksft_exit, ksft_pr from lib.py import ksft_ge, ksft_eq, ksft_in, ksft_true, ksft_raises, KsftSkipEx, KsftXfailEx from lib.py import EthtoolFamily, NetdevFamily, RtnlFamily, NlError from lib.py import NetDrvEnv +from lib.py import ip, defer ethnl = EthtoolFamily() netfam = NetdevFamily() @@ -133,9 +134,31 @@ rtnl = RtnlFamily() ksft_eq(cm.exception.nl_msg.extack['bad-attr'], '.ifindex') +def check_down(cfg) -> None: + try: + qstat = netfam.qstats_get({"ifindex": cfg.ifindex}, dump=True) + except NlError as e: + if e.error == 95: + raise KsftSkipEx("qstats not supported by the device") + raise + + ip(f"link set dev {cfg.dev['ifname']} down") + defer(ip, f"link set dev {cfg.dev['ifname']} up") + + qstat = qstat[0] + qstat2 = netfam.qstats_get({"ifindex": cfg.ifindex}, dump=True)[0] + for k, v in qstat.items(): + ksft_ge(qstat2[k], qstat[k], comment=f"{k} went backwards on device down") + + # exercise per-queue API to make sure that "device down" state + # is handled correctly and doesn't crash + netfam.qstats_get({"ifindex": cfg.ifindex, "scope": "queue"}, dump=True) + + def main() -> None: with NetDrvEnv(__file__) as cfg: - ksft_run([check_pause, check_fec, pkt_byte_sum, qstat_by_ifindex], + ksft_run([check_pause, check_fec, pkt_byte_sum, qstat_by_ifindex, + check_down], args=(cfg, )) ksft_exit() -- 2.45.2

8 months, 4 weeks

3
12
0 0

[PATCH] selftests: tc-testing: Fixed Typo error

by Karan Sanghavi

Corrected the typographical of the word "different" in the "name" field of the JSON object with ID "4319". Signed-off-by: Karan Sanghavi <karansanghvi98(a)gmail.com> --- tools/testing/selftests/tc-testing/tc-tests/filters/cgroup.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tools/testing/selftests/tc-testing/tc-tests/filters/cgroup.json b/tools/testing/selftests/tc-testing/tc-tests/filters/cgroup.json index 03723cf84..6897ff5ad 100644 --- a/tools/testing/selftests/tc-testing/tc-tests/filters/cgroup.json +++ b/tools/testing/selftests/tc-testing/tc-tests/filters/cgroup.json @@ -1189,7 +1189,7 @@ }, { "id": "4319", - "name": "Replace cgroup filter with diffferent match", + "name": "Replace cgroup filter with different match", "category": [ "filter", "cgroup" -- 2.43.0

8 months, 4 weeks

2
1
0 0

[PATCH net-next v8 0/3] net: A lightweight zero-copy notification mechanism for MSG_ZEROCOPY

by zijianzhang＠bytedance.com

From: Zijian Zhang <zijianzhang(a)bytedance.com> Original notification mechanism needs poll + recvmmsg which is not easy for applcations to accommodate. And, it also incurs unignorable overhead including extra system calls. While making maximum reuse of the existing MSG_ZEROCOPY related code, this patch set introduces a new zerocopy socket notification mechanism. Users of sendmsg pass a control message as a placeholder for the incoming notifications. Upon returning, kernel embeds notifications directly into user arguments passed in. By doing so, we can reduce the complexity and the overhead for managing notifications. We also have the logic related to copying cmsg to the userspace in sendmsg generic for any possible uses cases in the future. However, it introduces ABI change of sendmsg. Changelog: v1 -> v2: - Reuse errormsg queue in the new notification mechanism, users can actually use these two mechanisms in hybrid way if they want to do so. - Update case SCM_ZC_NOTIFICATION in __sock_cmsg_send 1. Regardless of 32-bit, 64-bit program, we will always handle u64 type user address. 2. The size of data to copy_to_user is precisely calculated in case of kernel stack leak. - fix (kbuild-bot) 1. Add SCM_ZC_NOTIFICATION to arch-specific header files. 2. header file types.h in include/uapi/linux/socket.h v2 -> v3: - 1. Users can now pass in the address of the zc_info_elem directly with appropriate cmsg_len instead of the ugly user interface. Plus, the handler is now compatible with MSG_CMSG_COMPAT and 32-bit pointer. - 2. Suggested by Willem, another strategy of getting zc info is briefly taking the lock of sk_error_queue and move to a private list, like net_rx_action. I thought sk_error_queue is protected by sock_lock, so that it's impossible for the handling of zc info and users recvmsg from the sk_error_queue at the same time. However, sk_error_queue is protected by its own lock. I am afraid that during the time it is handling the private list, users may fail to get other error messages in the queue via recvmsg. Thus, I don't implement the splice logic in this version. Any comments? v3 -> v4: - 1. Change SOCK_ZC_INFO_MAX to 64 to avoid large stack frame size. - 2. Fix minor typos. - 3. Change cfg_zerocopy from int to enum in msg_zerocopy.c Initially, we expect users to pass the user address of the user array as a data in cmsg, so that the kernel can copy_to_user to this address directly. As Willem commented, > The main design issue with this series is this indirection, rather > than passing the array of notifications as cmsg. > This trick circumvents having to deal with compat issues and having to > figure out copy_to_user in ____sys_sendmsg (as msg_control is an > in-kernel copy). > This is quite hacky, from an API design PoV. > As is passing a pointer, but expecting msg_controllen to hold the > length not of the pointer, but of the pointed to user buffer. > I had also hoped for more significant savings. Especially with the > higher syscall overhead due to meltdown and spectre mitigations vs > when MSG_ZEROCOPY was introduced and I last tried this optimization. We solve it by supporting put_cmsg to userspace in sendmsg path starting from v5. v4 -> v5: - 1. Passing user address directly to kernel raises concerns about ABI. In this version, we support put_cmsg to userspace in TX path to solve this problem. v5 -> v6: - 1. Cleanly copy cmsg to user upon returning of ___sys_sendmsg v6 -> v7: - 1. Remove flag MSG_CMSG_COPY_TO_USER, use a member in msghdr instead - 2. Pass msg to __sock_cmsg_send. - 3. sendmsg_copy_cmsg_to_user should be put at the end of ____sys_sendmsg to make sure msg_sys->msg_control is a valid pointer. - 4. Add struct zc_info to contain the array of zc_info_elem, so that the kernel can update the zc_info->size. Another possible solution is updating the cmsg_len directly, but it will break for_each_cmsghdr. - 5. Update selftest to make cfg_notification_limit have the same semantics in both methods for better comparison. v7 -> v8: - 1. Add a static_branch in ____sys_sendmsg to avoid overhead in the hot path. - 2. Add ZC_NOTIFICATION_MAX to limit the max size of zc_info->arr. - 3. Minimize the code in SCM_ZC_NOTIFICATION handler by adding a local sk_buff_head. * Performance We update selftests/net/msg_zerocopy.c to accommodate the new mechanism, cfg_notification_limit has the same semantics for both methods. Test results are as follows, we update skb_orphan_frags_rx to the same as skb_orphan_frags to support zerocopy in the localhost test. cfg_notification_limit = 1, both method get notifications after 1 calling of sendmsg. In this case, the new method has around 17% cpu savings in TCP and 23% cpu savings in UDP. +----------------------+---------+---------+---------+---------+ | Test Type / Protocol | TCP v4 | TCP v6 | UDP v4 | UDP v6 | +----------------------+---------+---------+---------+---------+ | ZCopy (MB) | 7523 | 7706 | 7489 | 7304 | +----------------------+---------+---------+---------+---------+ | New ZCopy (MB) | 8834 | 8993 | 9053 | 9228 | +----------------------+---------+---------+---------+---------+ | New ZCopy / ZCopy | 117.42% | 116.70% | 120.88% | 126.34% | +----------------------+---------+---------+---------+---------+ cfg_notification_limit = 32, both get notifications after 32 calling of sendmsg, which means more chances to coalesce notifications, and less overhead of poll + recvmsg for the original method. In this case, the new method has around 7% cpu savings in TCP and slightly better cpu usage in UDP. In the env of selftest, notifications of TCP are more likely to be out of order than UDP, it's easier to coalesce more notifications in UDP. The original method can get one notification with range of 32 in a recvmsg most of the time. In TCP, most notifications' range is around 2, so the original method needs around 16 recvmsgs to get notified in one round. That's the reason for the "New ZCopy / ZCopy" diff in TCP and UDP here. +----------------------+---------+---------+---------+---------+ | Test Type / Protocol | TCP v4 | TCP v6 | UDP v4 | UDP v6 | +----------------------+---------+---------+---------+---------+ | ZCopy (MB) | 8842 | 8735 | 10072 | 9380 | +----------------------+---------+---------+---------+---------+ | New ZCopy (MB) | 9366 | 9477 | 10108 | 9385 | +----------------------+---------+---------+---------+---------+ | New ZCopy / ZCopy | 106.00% | 108.28% | 100.31% | 100.01% | +----------------------+---------+---------+---------+---------+ In conclusion, when notification interval is small or notifications are hard to be coalesced, the new mechanism is highly recommended. Otherwise, the performance gain from the new mechanism is very limited. Zijian Zhang (3): sock: support copying cmsgs to the user space in sendmsg sock: add MSG_ZEROCOPY notification mechanism based on msg_control selftests: add MSG_ZEROCOPY msg_control notification test arch/alpha/include/uapi/asm/socket.h | 2 + arch/mips/include/uapi/asm/socket.h | 2 + arch/parisc/include/uapi/asm/socket.h | 2 + arch/sparc/include/uapi/asm/socket.h | 2 + include/linux/socket.h | 8 ++ include/net/sock.h | 2 +- include/uapi/asm-generic/socket.h | 2 + include/uapi/linux/socket.h | 23 +++++ net/core/sock.c | 72 +++++++++++++- net/ipv4/ip_sockglue.c | 2 +- net/ipv6/datagram.c | 2 +- net/socket.c | 63 +++++++++++- tools/testing/selftests/net/msg_zerocopy.c | 101 ++++++++++++++++++-- tools/testing/selftests/net/msg_zerocopy.sh | 1 + 14 files changed, 265 insertions(+), 19 deletions(-) -- 2.20.1

8 months, 4 weeks

4
10
0 0

Re: [syzbot] [mm?] WARNING in hpage_collapse_scan_pmd (2)

by syzbot

syzbot has bisected this issue to: commit ef5b6a542b1dbb718226a5f8208be09ef405983d Author: Paolo Bonzini <pbonzini(a)redhat.com> Date: Wed Nov 8 09:40:35 2023 +0000 selftests: kvm/s390x: use vm_create_barebones() bisection log: https://syzkaller.appspot.com/x/bisect.txt?x=14071cd3980000 start commit: e67572cd2204 Linux 6.9-rc6 git tree: upstream final oops: https://syzkaller.appspot.com/x/report.txt?x=16071cd3980000 console output: https://syzkaller.appspot.com/x/log.txt?x=12071cd3980000 kernel config: https://syzkaller.appspot.com/x/.config?x=3310e643b6ef5d69 dashboard link: https://syzkaller.appspot.com/bug?extid=5ea2845f44caa77f5543 syz repro: https://syzkaller.appspot.com/x/repro.syz?x=10874a40980000 Reported-by: syzbot+5ea2845f44caa77f5543(a)syzkaller.appspotmail.com Fixes: ef5b6a542b1d ("selftests: kvm/s390x: use vm_create_barebones()") For information about bisection process see: https://goo.gl/tpsmEJ#bisection

8 months, 4 weeks

1
0
0 0

[PATCH v4] selftest: acct: Add selftest for the acct() syscall

by Abdulrasaq Lawani

The acct() system call enables or disables process accounting. If accounting is turned on, records for each terminating process are appended to a specified filename as it terminates. An argument of NULL causes accounting to be turned off. This patch will add a test for the acct() syscall. Signed-off-by: Abdulrasaq Lawani <abdulrasaqolawani(a)gmail.com> --- Changes in v4: - Add SPDX License identifier to test Makefile - Link to v3: https://lore.kernel.org/r/20240723-kselftest-acct-syscall-v3-1-16f332498a9e… Changes in v3: - Add geteuid to check if test is ran as root. - Simplify error conditions for acct function. - Remove unecessary messages. - Add more informative messages. - Update commit message. - Add error test case for file creation failure. - Link to v2: https://lore.kernel.org/r/20240630-kselftest-acct-syscall-v2-1-b30bbe2a69cd… Changes in v2: Add testcases to test error conditions. Add kselftest function for reporting results. - Link to v1: https://lore.kernel.org/r/20240622-kselftest-acct-syscall-v1-1-d270b5be8d37… --- tools/testing/selftests/Makefile | 1 + tools/testing/selftests/acct/.gitignore | 3 ++ tools/testing/selftests/acct/Makefile | 5 ++ tools/testing/selftests/acct/acct_syscall.c | 78 +++++++++++++++++++++++++++++ 4 files changed, 87 insertions(+) diff --git a/tools/testing/selftests/Makefile b/tools/testing/selftests/Makefile index 9039f3709aff..45a58ef5ad92 100644 --- a/tools/testing/selftests/Makefile +++ b/tools/testing/selftests/Makefile @@ -1,4 +1,5 @@ # SPDX-License-Identifier: GPL-2.0 +TARGETS += acct TARGETS += alsa TARGETS += amd-pstate TARGETS += arm64 diff --git a/tools/testing/selftests/acct/.gitignore b/tools/testing/selftests/acct/.gitignore new file mode 100644 index 000000000000..7e78aac19038 --- /dev/null +++ b/tools/testing/selftests/acct/.gitignore @@ -0,0 +1,3 @@ +acct_syscall +config +process_log \ No newline at end of file diff --git a/tools/testing/selftests/acct/Makefile b/tools/testing/selftests/acct/Makefile new file mode 100644 index 000000000000..7e025099cf65 --- /dev/null +++ b/tools/testing/selftests/acct/Makefile @@ -0,0 +1,5 @@ +# SPDX-License-Identifier: GPL-2.0 +TEST_GEN_PROGS := acct_syscall +CFLAGS += -Wall + +include ../lib.mk \ No newline at end of file diff --git a/tools/testing/selftests/acct/acct_syscall.c b/tools/testing/selftests/acct/acct_syscall.c new file mode 100644 index 000000000000..e44e8fe1f4a3 --- /dev/null +++ b/tools/testing/selftests/acct/acct_syscall.c @@ -0,0 +1,78 @@ +// SPDX-License-Identifier: GPL-2.0 + +/* kselftest for acct() system call + * The acct() system call enables or disables process accounting. + */ + +#include <stdio.h> +#include <errno.h> +#include <string.h> +#include <sys/wait.h> + +#include "../kselftest.h" + +int main(void) +{ + char filename[] = "process_log"; + FILE *fp; + pid_t child_pid; + int sz; + + // Setting up kselftest framework + ksft_print_header(); + ksft_set_plan(1); + + // Check if test is run a root + if (geteuid()) { + ksft_test_result_skip("This test needs root to run!\n"); + return 1; + } + + // Create file to log closed processes + fp = fopen(filename, "w"); + + if (!fp) { + ksft_test_result_error("%s.\n", strerror(errno)); + ksft_finished(); + return 1; + } + + acct(filename); + + // Handle error conditions + if (errno) { + ksft_test_result_error("%s.\n", strerror(errno)); + fclose(fp); + ksft_finished(); + return 1; + } + + // Create child process and wait for it to terminate. + + child_pid = fork(); + + if (child_pid < 0) { + ksft_test_result_error("Creating a child process to log failed\n"); + acct(NULL); + return 1; + } else if (child_pid > 0) { + wait(NULL); + fseek(fp, 0L, SEEK_END); + sz = ftell(fp); + + acct(NULL); + + if (sz <= 0) { + ksft_test_result_fail("Terminated child process not logged\n"); + ksft_exit_fail(); + return 1; + } + + ksft_test_result_pass("Successfully logged terminated process.\n"); + fclose(fp); + ksft_exit_pass(); + return 0; + } + + return 1; +} --- base-commit: 50736169ecc8387247fe6a00932852ce7b057083 change-id: 20240622-kselftest-acct-syscall-2d90f7666b1e Best regards, -- Abdulrasaq Lawani <abdulrasaqolawani(a)gmail.com>

8 months, 4 weeks

2
1
0 0

[PATCH net-next 6/6] selftests: fib_nexthops: Test 16-bit next hop weights

by Petr Machata

Add tests that attempt to create NH groups that use full 16 bits of NH weight. Signed-off-by: Petr Machata <petrm(a)nvidia.com> Reviewed-by: Ido Schimmel <idosch(a)nvidia.com> --- tools/testing/selftests/net/fib_nexthops.sh | 55 ++++++++++++++++++++- 1 file changed, 54 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/net/fib_nexthops.sh b/tools/testing/selftests/net/fib_nexthops.sh index ac0b2c6a5761..77c83d9508d3 100755 --- a/tools/testing/selftests/net/fib_nexthops.sh +++ b/tools/testing/selftests/net/fib_nexthops.sh @@ -78,7 +78,12 @@ log_test() else ret=1 nfail=$((nfail+1)) - printf "TEST: %-60s [FAIL]\n" "${msg}" + if [[ $rc -eq $ksft_skip ]]; then + printf "TEST: %-60s [SKIP]\n" "${msg}" + else + printf "TEST: %-60s [FAIL]\n" "${msg}" + fi + if [ "$VERBOSE" = "1" ]; then echo " rc=$rc, expected $expected" fi @@ -923,6 +928,29 @@ ipv6_grp_fcnal() ipv6_grp_refs log_test $? 0 "Nexthop group replace refcounts" + + # + # 16-bit weights. + # + run_cmd "$IP nexthop add id 62 via 2001:db8:91::2 dev veth1" + run_cmd "$IP nexthop add id 63 via 2001:db8:91::3 dev veth1" + run_cmd "$IP nexthop add id 64 via 2001:db8:91::4 dev veth1" + run_cmd "$IP nexthop add id 65 via 2001:db8:91::5 dev veth1" + run_cmd "$IP nexthop add id 66 dev veth1" + + run_cmd "$IP nexthop add id 103 group 62,1000" + if [[ $? == 0 ]]; then + local GRP="id 103 group 62,254/63,255/64,256/65,257/66,65535" + run_cmd "$IP nexthop replace $GRP" + check_nexthop "id 103" "$GRP" + rc=$? + else + rc=$ksft_skip + fi + + $IP nexthop flush >/dev/null 2>&1 + + log_test $rc 0 "16-bit weights" } ipv6_res_grp_fcnal() @@ -987,6 +1015,31 @@ ipv6_res_grp_fcnal() check_nexthop_bucket "list id 102" \ "id 102 index 0 nhid 63 id 102 index 1 nhid 62 id 102 index 2 nhid 62 id 102 index 3 nhid 62" log_test $? 0 "Nexthop buckets updated after replace - nECMP" + + # + # 16-bit weights. + # + run_cmd "$IP nexthop add id 62 via 2001:db8:91::2 dev veth1" + run_cmd "$IP nexthop add id 63 via 2001:db8:91::3 dev veth1" + run_cmd "$IP nexthop add id 64 via 2001:db8:91::4 dev veth1" + run_cmd "$IP nexthop add id 65 via 2001:db8:91::5 dev veth1" + run_cmd "$IP nexthop add id 66 dev veth1" + + run_cmd "$IP nexthop add id 103 group 62,1000 type resilient buckets 32" + if [[ $? == 0 ]]; then + local GRP="id 103 group 62,254/63,255/64,256/65,257/66,65535 $(: + )type resilient buckets 32 idle_timer 0 $(: + )unbalanced_timer 0" + run_cmd "$IP nexthop replace $GRP" + check_nexthop "id 103" "$GRP unbalanced_time 0" + rc=$? + else + rc=$ksft_skip + fi + + $IP nexthop flush >/dev/null 2>&1 + + log_test $rc 0 "16-bit weights" } ipv6_fcnal_runtime() -- 2.45.0

8 months, 4 weeks

1
0
0 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

Linux-kselftest-mirror