- Linux-kselftest-mirror - lists.linaro.org

[PATCH net-next v2 00/12] selftests/vsock: refactor and improve vmtest infrastructure

by Bobby Eshleman

Hey all, This patch series refactors the vsock selftest VM infrastructure to improve test run times, improve logging, and prepare for future tests which make heavy usage of these refactored functions and have new requirements such as simultaneous QEMU processes. These patches were broken off from this prior series: https://lore.kernel.org/all/20251021-vsock-vmtest-v7-0-0661b7b6f081@meta.co… --- Changes in v2: - remove "Fixes" for some patches because they do not fix bugs in kselftest runs (some fix bugs only when using bash args that kselftest does not use or otherwise prepare functions for new usage) - broke out one fixes patch for "net" - per-patch changes - add patch for shellcheck declaration to disable false positives - Link to v1: https://lore.kernel.org/r/20251022-vsock-selftests-fixes-and-improvements-v… --- Bobby Eshleman (12): selftests/vsock: improve logging in vmtest.sh selftests/vsock: make wait_for_listener() work even if pipefail is on selftests/vsock: reuse logic for vsock_test through wrapper functions selftests/vsock: avoid multi-VM pidfile collisions with QEMU selftests/vsock: do not unconditionally die if qemu fails selftests/vsock: speed up tests by reducing the QEMU pidfile timeout selftests/vsock: add check_result() for pass/fail counting selftests/vsock: identify and execute tests that can re-use VM selftests/vsock: add BUILD=0 definition selftests/vsock: add 1.37 to tested virtme-ng versions selftests/vsock: add vsock_loopback module loading selftests/vsock: disable shellcheck SC2317 and SC2119 tools/testing/selftests/vsock/vmtest.sh | 332 +++++++++++++++++++++----------- 1 file changed, 216 insertions(+), 116 deletions(-) --- base-commit: 255d75ef029f33f75fcf5015052b7302486f7ad2 change-id: 20251021-vsock-selftests-fixes-and-improvements-057440ffb2fa Best regards, -- Bobby Eshleman <bobbyeshleman(a)meta.com>

1 week, 6 days

2
20
0 0

[PATCH net-next v2 0/5] psp: track stats from core and provide a driver stats api

by Daniel Zahka

This series introduces stats counters for psp. Device key rotations, and so called 'stale-events' are common to all drivers and are tracked by the core. A driver facing api is provided for reporting stats required by the "Implementation Requirements" section of the PSP Architecture Specification. Drivers must implement these stats. Lastly, implementations of the driver stats api for mlx5 and netdevsim are included. Here is the output of running the psp selftest suite and then printing out stats with the ynl cli on system with a psp-capable CX7: $ ./ksft-psp-stats/drivers/net/psp.py TAP version 13 1..28 ok 1 psp.test_case # SKIP Test requires IPv4 connectivity ok 2 psp.data_basic_send_v0_ip6 ok 3 psp.test_case # SKIP Test requires IPv4 connectivity ok 4 psp.data_basic_send_v1_ip6 ok 5 psp.test_case # SKIP Test requires IPv4 connectivity ok 6 psp.data_basic_send_v2_ip6 # SKIP ('PSP version not supported', 'hdr0-aes-gmac-128') ok 7 psp.test_case # SKIP Test requires IPv4 connectivity ok 8 psp.data_basic_send_v3_ip6 # SKIP ('PSP version not supported', 'hdr0-aes-gmac-256') ok 9 psp.test_case # SKIP Test requires IPv4 connectivity ok 10 psp.data_mss_adjust_ip6 ok 11 psp.dev_list_devices ok 12 psp.dev_get_device ok 13 psp.dev_get_device_bad ok 14 psp.dev_rotate ok 15 psp.dev_rotate_spi ok 16 psp.assoc_basic ok 17 psp.assoc_bad_dev ok 18 psp.assoc_sk_only_conn ok 19 psp.assoc_sk_only_mismatch ok 20 psp.assoc_sk_only_mismatch_tx ok 21 psp.assoc_sk_only_unconn ok 22 psp.assoc_version_mismatch ok 23 psp.assoc_twice ok 24 psp.data_send_bad_key ok 25 psp.data_send_disconnect ok 26 psp.data_stale_key ok 27 psp.removal_device_rx # XFAIL Test only works on netdevsim ok 28 psp.removal_device_bi # XFAIL Test only works on netdevsim # Totals: pass:19 fail:0 xfail:2 xpass:0 skip:7 error:0 # # Responder logs (0): # STDERR: # Set PSP enable on device 1 to 0x3 # Set PSP enable on device 1 to 0x0 $ cd ynl/ $ ./pyynl/cli.py --spec netlink/specs/psp.yaml --dump get-stats [{'dev-id': 1, 'key-rotations': 5, 'rx-auth-fail': 21, 'rx-bad': 0, 'rx-bytes': 11844, 'rx-error': 0, 'rx-packets': 94, 'stale-events': 6, 'tx-bytes': 1128456, 'tx-error': 0, 'tx-packets': 780}] CHANGES: v2: - don't return skb->len from psp_nl_get_stats_dumpit() on success and EMSGSIZE - use %pe to print PTR_ERR() v1: https://lore.kernel.org/netdev/20251022193739.1376320-1-daniel.zahka@gmail.… Daniel Zahka (2): selftests: drv-net: psp: add assertions on core-tracked psp dev stats netdevsim: implement psp device stats Jakub Kicinski (3): psp: report basic stats from the core psp: add stats from psp spec to driver facing api net/mlx5e: Add PSP stats support for Rx/Tx flows Documentation/netlink/specs/psp.yaml | 95 +++++++ .../mellanox/mlx5/core/en_accel/psp.c | 239 ++++++++++++++++-- .../mellanox/mlx5/core/en_accel/psp.h | 18 ++ .../mellanox/mlx5/core/en_accel/psp_rxtx.c | 1 + .../net/ethernet/mellanox/mlx5/core/en_main.c | 5 + drivers/net/netdevsim/netdevsim.h | 5 + drivers/net/netdevsim/psp.c | 27 ++ include/net/psp/types.h | 35 +++ include/uapi/linux/psp.h | 18 ++ net/psp/psp-nl-gen.c | 19 ++ net/psp/psp-nl-gen.h | 2 + net/psp/psp_main.c | 3 +- net/psp/psp_nl.c | 94 +++++++ net/psp/psp_sock.c | 4 +- tools/testing/selftests/drivers/net/psp.py | 13 + 15 files changed, 561 insertions(+), 17 deletions(-) -- 2.47.3

1 week, 6 days

4
10
0 0

[PATCH 0/8] Initial DMABUF support for iommufd

by Jason Gunthorpe

This series is the start of adding full DMABUF support to iommufd. Currently it is limited to only work with VFIO's DMABUF exporter. It sits on top of Leon's series to add a DMABUF exporter to VFIO: https://lore.kernel.org/all/cover.1760368250.git.leon@kernel.org/ The existing IOMMU_IOAS_MAP_FILE is enhanced to detect DMABUF fd's, but otherwise works the same as it does today for a memfd. The user can select a slice of the FD to map into the ioas and if the underliyng alignment requirements are met it will be placed in the iommu_domain. Though limited, it is enough to allow a VMM like QEMU to connect MMIO BAR memory from VFIO to an iommu_domain controlled by iommufd. This is used for PCI Peer to Peer support in VMs, and is the last feature that the VFIO type 1 container has that iommufd couldn't do. The VFIO type1 version extracts raw PFNs from VMAs, which has no lifetime control and is a use-after-free security problem. Instead iommufd relies on revokable DMABUFs. Whenever VFIO thinks there should be no access to the MMIO it can shoot down the mapping in iommufd which will unmap it from the iommu_domain. There is no automatic remap, this is a safety protocol so the kernel doesn't get stuck. Userspace is expected to know it is doing something that will revoke the dmabuf and map/unmap it around the activity. Eg when QEMU goes to issue FLR it should do the map/unmap to iommufd. Since DMABUF is missing some key general features for this use case it relies on a "private interconnect" between VFIO and iommufd via the vfio_pci_dma_buf_iommufd_map() call. The call confirms the DMABUF has revoke semantics and delivers a phys_addr for the memory suitable for use with iommu_map(). Medium term there is a desire to expand the supported DMABUFs to include GPU drivers to support DPDK/SPDK type use cases so future series will work to add a general concept of revoke and a general negotiation of interconnect to remove vfio_pci_dma_buf_iommufd_map(). I also plan another series to modify iommufd's vfio_compat to transparently pull a dmabuf out of a VFIO VMA to emulate more of the uAPI of type1. The latest series for interconnect negotation to exchange a phys_addr is: https://lore.kernel.org/r/20251027044712.1676175-1-vivek.kasireddy@intel.com And the discussion for design of revoke is here: https://lore.kernel.org/dri-devel/20250114173103.GE5556@nvidia.com/ This is on github: https://github.com/jgunthorpe/linux/commits/iommufd_dmabuf The branch has various modifications to Leon's series I've suggested. Jason Gunthorpe (8): iommufd: Add DMABUF to iopt_pages iommufd: Do not map/unmap revoked DMABUFs iommufd: Allow a DMABUF to be revoked iommufd: Allow MMIO pages in a batch iommufd: Have pfn_reader process DMABUF iopt_pages iommufd: Have iopt_map_file_pages convert the fd to a file iommufd: Accept a DMABUF through IOMMU_IOAS_MAP_FILE iommufd/selftest: Add some tests for the dmabuf flow drivers/iommu/iommufd/io_pagetable.c | 74 +++- drivers/iommu/iommufd/io_pagetable.h | 53 ++- drivers/iommu/iommufd/ioas.c | 8 +- drivers/iommu/iommufd/iommufd_private.h | 13 +- drivers/iommu/iommufd/iommufd_test.h | 10 + drivers/iommu/iommufd/main.c | 10 + drivers/iommu/iommufd/pages.c | 407 ++++++++++++++++-- drivers/iommu/iommufd/selftest.c | 142 ++++++ tools/testing/selftests/iommu/iommufd.c | 43 ++ tools/testing/selftests/iommu/iommufd_utils.h | 44 ++ 10 files changed, 741 insertions(+), 63 deletions(-) base-commit: fc882154e421f82677925d33577226e776bb07a4 -- 2.43.0

1 week, 6 days

3
11
0 0

[PATCH nf-next v8 0/3] Add IPIP flowtable SW acceleration

by Lorenzo Bianconi

Introduce SW acceleration for IPIP tunnels in the netfilter flowtable infrastructure. This series introduces basic infrastructure to accelerate other tunnel types (e.g. IP6IP6). --- Changes in v8: - Rebase on top of the following series (not yet applied) https://patchwork.ozlabs.org/project/netfilter-devel/list/?series=477081 - Link to v7: https://lore.kernel.org/r/20251021-nf-flowtable-ipip-v7-0-a45214896106@kern… Changes in v7: - Introduce sw acceleration for tx path of IPIP tunnels - Rely on exact match during flowtable entry lookup - Fix typos - Link to v6: https://lore.kernel.org/r/20250818-nf-flowtable-ipip-v6-0-eda90442739c@kern… Changes in v6: - Rebase on top of nf-next main branch - Link to v5: https://lore.kernel.org/r/20250721-nf-flowtable-ipip-v5-0-0865af9e58c6@kern… Changes in v5: - Rely on __ipv4_addr_hash() to compute the hash used as encap ID - Remove unnecessary pskb_may_pull() in nf_flow_tuple_encap() - Add nf_flow_ip4_ecanp_pop utility routine - Link to v4: https://lore.kernel.org/r/20250718-nf-flowtable-ipip-v4-0-f8bb1c18b986@kern… Changes in v4: - Use the hash value of the saddr, daddr and protocol of outer IP header as encapsulation id. - Link to v3: https://lore.kernel.org/r/20250703-nf-flowtable-ipip-v3-0-880afd319b9f@kern… Changes in v3: - Add outer IP header sanity checks - target nf-next tree instead of net-next - Link to v2: https://lore.kernel.org/r/20250627-nf-flowtable-ipip-v2-0-c713003ce75b@kern… Changes in v2: - Introduce IPIP flowtable selftest - Link to v1: https://lore.kernel.org/r/20250623-nf-flowtable-ipip-v1-1-2853596e3941@kern… --- Lorenzo Bianconi (3): net: netfilter: Add IPIP flowtable rx sw acceleration net: netfilter: Add IPIP flowtable tx sw acceleration selftests: netfilter: nft_flowtable.sh: Add IPIP flowtable selftest include/linux/netdevice.h | 16 +++ include/net/netfilter/nf_flow_table.h | 22 ++++ net/ipv4/ipip.c | 29 +++++ net/netfilter/nf_flow_table_core.c | 3 + net/netfilter/nf_flow_table_ip.c | 117 ++++++++++++++++++++- net/netfilter/nf_flow_table_path.c | 86 +++++++++++++-- .../selftests/net/netfilter/nft_flowtable.sh | 40 +++++++ 7 files changed, 298 insertions(+), 15 deletions(-) --- base-commit: 32e4b1bf1bbfe63e52e2fff7ade0aaeb805defe3 change-id: 20250623-nf-flowtable-ipip-1b3d7b08d067 Best regards, -- Lorenzo Bianconi <lorenzo(a)kernel.org>

1 week, 6 days

3
7
0 0

[PATCH net] selftests/vsock: avoid false-positives when checking dmesg

by Bobby Eshleman

From: Bobby Eshleman <bobbyeshleman(a)meta.com> Sometimes VMs will have some intermittent dmesg warnings that are unrelated to vsock. Change the dmesg parsing to filter on strings containing 'vsock' to avoid false positive failures that are unrelated to vsock. The downside is that it is possible for some vsock related warnings to not contain the substring 'vsock', so those will be missed. Fixes: a4a65c6fe08b ("selftests/vsock: add initial vmtest.sh for vsock") Reviewed-by: Simon Horman <horms(a)kernel.org> Signed-off-by: Bobby Eshleman <bobbyeshleman(a)meta.com> --- Previously was part of the series: https://lore.kernel.org/all/20251022-vsock-selftests-fixes-and-improvements… --- tools/testing/selftests/vsock/vmtest.sh | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/tools/testing/selftests/vsock/vmtest.sh b/tools/testing/selftests/vsock/vmtest.sh index edacebfc1632..e1732f236d14 100755 --- a/tools/testing/selftests/vsock/vmtest.sh +++ b/tools/testing/selftests/vsock/vmtest.sh @@ -389,9 +389,9 @@ run_test() { local rc host_oops_cnt_before=$(dmesg | grep -c -i 'Oops') - host_warn_cnt_before=$(dmesg --level=warn | wc -l) + host_warn_cnt_before=$(dmesg --level=warn | grep -c -i 'vsock') vm_oops_cnt_before=$(vm_ssh -- dmesg | grep -c -i 'Oops') - vm_warn_cnt_before=$(vm_ssh -- dmesg --level=warn | wc -l) + vm_warn_cnt_before=$(vm_ssh -- dmesg --level=warn | grep -c -i 'vsock') name=$(echo "${1}" | awk '{ print $1 }') eval test_"${name}" @@ -403,7 +403,7 @@ run_test() { rc=$KSFT_FAIL fi - host_warn_cnt_after=$(dmesg --level=warn | wc -l) + host_warn_cnt_after=$(dmesg --level=warn | grep -c -i vsock) if [[ ${host_warn_cnt_after} -gt ${host_warn_cnt_before} ]]; then echo "FAIL: kernel warning detected on host" | log_host "${name}" rc=$KSFT_FAIL @@ -415,7 +415,7 @@ run_test() { rc=$KSFT_FAIL fi - vm_warn_cnt_after=$(vm_ssh -- dmesg --level=warn | wc -l) + vm_warn_cnt_after=$(vm_ssh -- dmesg --level=warn | grep -c -i vsock) if [[ ${vm_warn_cnt_after} -gt ${vm_warn_cnt_before} ]]; then echo "FAIL: kernel warning detected on vm" | log_host "${name}" rc=$KSFT_FAIL --- base-commit: 255d75ef029f33f75fcf5015052b7302486f7ad2 change-id: 20251104-vsock-vmtest-dmesg-fix-b2c59e1d9c38 Best regards, -- Bobby Eshleman <bobbyeshleman(a)meta.com>

1 week, 6 days

2
2
0 0

[PATCH net v8 0/4] net: netpoll: fix memory leak and add comprehensive selftests

by Breno Leitao

Fix a memory leak in netpoll and introduce netconsole selftests that expose the issue when running with kmemleak detection enabled. This patchset includes a selftest for netpoll with multiple concurrent users (netconsole + bonding), which simulates the scenario from test[1] that originally demonstrated the issue allegedly fixed by commit efa95b01da18 ("netpoll: fix use after free") - a commit that is now being reverted. Sending this to "net" branch because this is a fix, and the selftest might help with the backports validation. Link: https://lore.kernel.org/lkml/96b940137a50e5c387687bb4f57de8b0435a653f.14048… [1] Signed-off-by: Breno Leitao <leitao(a)debian.org> --- Changes in v8: - Sending it again, now that commit 1a8fed52f7be1 ("netdevsim: set the carrier when the device goes up") has landed in net - Created one namespace for TX and one for RX (Paolo) - Used additional helpers to create and delete netdevsim (Paolo) - Link to v7: https://lore.kernel.org/r/20251003-netconsole_torture-v7-0-aa92fcce62a9@deb… Changes in v7: - Rebased on top of `net` - Link to v6: https://lore.kernel.org/r/20251002-netconsole_torture-v6-0-543bf52f6b46@deb… Changes in v6: - Expand the tests even more and some small fixups - Moved the test to bonding selftests - Link to v5: https://lore.kernel.org/r/20250918-netconsole_torture-v5-0-77e25e0a4eb6@deb… Changes in v5: - Set CONFIG_BONDING=m in selftests/drivers/net/config. - Link to v4: https://lore.kernel.org/r/20250917-netconsole_torture-v4-0-0a5b3b8f81ce@deb… Changes in v4: - Added an additional selftest to test multiple netpoll users in parallel - Link to v3: https://lore.kernel.org/r/20250905-netconsole_torture-v3-0-875c7febd316@deb… Changes in v3: - This patchset is a merge of the fix and the selftest together as recommended by Jakub. Changes in v2: - Reuse the netconsole creation from lib_netcons.sh. Thus, refactoring the create_dynamic_target() (Jakub) - Move the "wait" to after all the messages has been sent. - Link to v1: https://lore.kernel.org/r/20250902-netconsole_torture-v1-1-03c6066598e9@deb… --- Breno Leitao (4): net: netpoll: fix incorrect refcount handling causing incorrect cleanup selftest: netcons: refactor target creation selftest: netcons: create a torture test selftest: netcons: add test for netconsole over bonded interfaces net/core/netpoll.c | 7 +- tools/testing/selftests/drivers/net/Makefile | 1 + .../testing/selftests/drivers/net/bonding/Makefile | 2 + tools/testing/selftests/drivers/net/bonding/config | 4 + .../drivers/net/bonding/netcons_over_bonding.sh | 361 +++++++++++++++++++++ .../selftests/drivers/net/lib/sh/lib_netcons.sh | 82 ++++- .../selftests/drivers/net/netcons_torture.sh | 130 ++++++++ 7 files changed, 569 insertions(+), 18 deletions(-) --- base-commit: e120f46768d98151ece8756ebd688b0e43dc8b29 change-id: 20250902-netconsole_torture-8fc23f0aca99 Best regards, -- Breno Leitao <leitao(a)debian.org>

1 week, 6 days

1
5
0 0

[PATCH net-next 0/4] mptcp: pm: in-kernel: fullmesh endp nb + bind cases

by Matthieu Baerts (NGI0)

Here is a small optimisation for the in-kernel PM, joined by a small behavioural change to avoid confusions, and followed by a few more tests. - Patch 1: record fullmesh endpoints numbers, not to iterate over all endpoints to check if one is marked as fullmesh. - Patch 2: when at least one endpoint is marked as fullmesh, only use these endpoints when reacting to an ADD_ADDR, even if there are no endpoints for this IP family: this is less confusing. - Patch 3: reduce duplicated code to prepare the next patch. - Patch 4: extra "bind" cases: the listen socket restrict the bind to one IP address, not allowing MP_JOIN to extra IP addresses, except if another listening socket accepts them. Signed-off-by: Matthieu Baerts (NGI0) <matttbe(a)kernel.org> --- Matthieu Baerts (NGI0) (4): mptcp: pm: in-kernel: record fullmesh endp nb mptcp: pm: in kernel: only use fullmesh endp if any selftests: mptcp: join: do_transfer: reduce code dup selftests: mptcp: join: validate extra bind cases include/uapi/linux/mptcp.h | 3 +- net/mptcp/pm_kernel.c | 36 ++++- net/mptcp/protocol.h | 1 + net/mptcp/sockopt.c | 2 + tools/testing/selftests/net/mptcp/mptcp_connect.c | 10 +- tools/testing/selftests/net/mptcp/mptcp_join.sh | 187 +++++++++++++++++++--- 6 files changed, 213 insertions(+), 26 deletions(-) --- base-commit: 01cc760632b875c4ad0d8fec0b0c01896b8a36d4 change-id: 20251101-net-next-mptcp-fm-endp-nb-bind-cf7ab688d9f1 Best regards, -- Matthieu Baerts (NGI0) <matttbe(a)kernel.org>

2 weeks

2
5
0 0

[PATCH net] selftests: netdevsim: Fix ethtool-features.sh fail

by Wang Liang

The test 'ethtool-features.sh' failed with the below output: TAP version 13 1..1 # timeout set to 600 # selftests: drivers/net/netdevsim: ethtool-features.sh # Warning: file ethtool-features.sh is not executable # ethtool: bad command line argument(s) # For more information run ethtool -h # ethtool: bad command line argument(s) # For more information run ethtool -h # ethtool: bad command line argument(s) # For more information run ethtool -h # ethtool: bad command line argument(s) # For more information run ethtool -h # ethtool: bad command line argument(s) # For more information run ethtool -h # ethtool: bad command line argument(s) # For more information run ethtool -h # ethtool: bad command line argument(s) # For more information run ethtool -h # ethtool: bad command line argument(s) # For more information run ethtool -h # ethtool: bad command line argument(s) # For more information run ethtool -h # ethtool: bad command line argument(s) # For more information run ethtool -h # FAILED 10/10 checks not ok 1 selftests: drivers/net/netdevsim: ethtool-features.sh # exit=1 Similar to commit 18378b0e49d9 ("selftests/damon: Add executable permission to test scripts"), the script 'ethtool-features.sh' has no executable permission, which leads to the warning 'file ethtool-features.sh is not executable'. Old version ethtool (my ethtool version is 5.16) does not support command 'ethtool --json -k enp1s0', which leads to the output 'ethtool: bad command line argument(s)'. This patch adds executable permission to script 'ethtool-features.sh', and check 'ethtool --json -k' support. After this patch: TAP version 13 1..1 # timeout set to 600 # selftests: drivers/net/netdevsim: ethtool-features.sh # SKIP: No --json -k support in ethtool ok 1 selftests: drivers/net/netdevsim: ethtool-features.sh Fixes: 0189270117c3 ("selftests: netdevsim: add a test checking ethtool features") Signed-off-by: Wang Liang <wangliang74(a)huawei.com> --- .../selftests/drivers/net/netdevsim/ethtool-features.sh | 5 +++++ 1 file changed, 5 insertions(+) mode change 100644 => 100755 tools/testing/selftests/drivers/net/netdevsim/ethtool-features.sh diff --git a/tools/testing/selftests/drivers/net/netdevsim/ethtool-features.sh b/tools/testing/selftests/drivers/net/netdevsim/ethtool-features.sh old mode 100644 new mode 100755 index bc210dc6ad2d..f771dc6839ea --- a/tools/testing/selftests/drivers/net/netdevsim/ethtool-features.sh +++ b/tools/testing/selftests/drivers/net/netdevsim/ethtool-features.sh @@ -7,6 +7,11 @@ NSIM_NETDEV=$(make_netdev) set -o pipefail +if ! ethtool --json -k $NSIM_NETDEV > /dev/null 2>&1; then + echo "SKIP: No --json -k support in ethtool" + exit $ksft_skip +fi + FEATS=" tx-checksum-ip-generic tx-scatter-gather -- 2.34.1

2 weeks

4
11
0 0

[PATCH v4 nf-next] selftests: netfilter: Add bridge_fastpath.sh

by Eric Woudstra

Add a script to test various scenarios where a bridge is involved in the fastpath. It runs tests in the forward path, and also in a bridged path. The setup is similar to a basic home router with multiple lan ports. It uses 3 pairs of veth-devices. Each or all pairs can be replaced by a pair of real interfaces, interconnected by wire. This is necessary to test the behavior when dealing with dsa ports, foreign (dsa) ports and switchdev userports that support SWITCHDEV_OBJ_ID_PORT_VLAN. See the head of the script for a detailed description. Run without arguments to perform all tests on veth-devices. Signed-off-by: Eric Woudstra <ericwouds(a)gmail.com> --- This test script is written first for the proposed bridge-fastpath patch-sets, but it's use is more general and can easily be expanded. Changes in v4: - Also only match ct state in rule without fastpath. - Dropped RFC - Cosmetics Changes in v3: - Removed all warnings reported by shellcheck -x -e SC2317 - Improved del_pppoe(), check if interfaces are removed - Added is_known_issue() to warn instead of error for known issues - Link down and (hardware) interfaces to default netns at end of script - Removed matching ip(v6) address Changes in v2: - Moved test-series to functions - Moved code to set_pair_link() up/down - Added conntrack zone to bridged traffic - Test bridge chain prerouting in test without fastpath and bridge chain forward in tests with fastpath Some example outputs of this last version of patches from different hardware, without and with patches: ALL VETH: ========= ./bridge_fastpath.sh -t Setup: CLIENT 0 veth0cl | veth0rt WAN ROUTER LAN1 LAN2 veth1rt veth2rt | | veth1cl veth2cl CLIENT 1 CLIENT 2 Without patches: PASS: unaware bridge, without encaps, without fastpath PASS: unaware bridge, with single vlan encap, without fastpath WARN: unaware bridge, with double q vlan encaps, without fastpath: ipv4/6: established bytes 0 < 4194304 WARN: unaware bridge, with 802.1ad vlan encaps, without fastpath: ipv4/6: established bytes 0 < 4194304 WARN: unaware bridge, with pppoe encap, without fastpath: ipv4/6: established bytes 0 < 4194304 WARN: unaware bridge, with pppoe-in-q encaps, without fastpath: ipv4/6: established bytes 0 < 4194304 PASS: aware bridge, without/without vlan encap, without fastpath PASS: aware bridge, with/without vlan encap, without fastpath PASS: aware bridge, with/with vlan encap, without fastpath PASS: aware bridge, without/with vlan encap, without fastpath PASS: forward, without vlan-device, without vlan encap, client1, without fastpath PASS: forward, without vlan-device, without vlan encap, client1, with fastpath PASS: forward, without vlan-device, with vlan encap, client1, without fastpath WARN: forward, without vlan-device, with vlan encap, client1, with fastpath: ipv4/6: tcp broken PASS: forward, with vlan-device, without vlan encap, client1, without fastpath PASS: forward, with vlan-device, without vlan encap, client1, with fastpath PASS: forward, with vlan-device, with vlan encap, client1, without fastpath PASS: forward, with vlan-device, with vlan encap, client1, with fastpath PASS: all tests passed With patches: PASS: unaware bridge, without encaps, without fastpath PASS: unaware bridge, without encaps, with fastpath PASS: unaware bridge, with single vlan encap, without fastpath PASS: unaware bridge, with single vlan encap, with fastpath PASS: unaware bridge, with double q vlan encaps, without fastpath PASS: unaware bridge, with double q vlan encaps, with fastpath PASS: unaware bridge, with 802.1ad vlan encaps, without fastpath PASS: unaware bridge, with 802.1ad vlan encaps, with fastpath PASS: unaware bridge, with pppoe encap, without fastpath PASS: unaware bridge, with pppoe encap, with fastpath PASS: unaware bridge, with pppoe-in-q encaps, without fastpath PASS: unaware bridge, with pppoe-in-q encaps, with fastpath PASS: aware bridge, without/without vlan encap, without fastpath PASS: aware bridge, without/without vlan encap, with fastpath PASS: aware bridge, with/without vlan encap, without fastpath PASS: aware bridge, with/without vlan encap, with fastpath PASS: aware bridge, with/with vlan encap, without fastpath PASS: aware bridge, with/with vlan encap, with fastpath PASS: aware bridge, without/with vlan encap, without fastpath PASS: aware bridge, without/with vlan encap, with fastpath PASS: forward, without vlan-device, without vlan encap, client1, without fastpath PASS: forward, without vlan-device, without vlan encap, client1, with fastpath PASS: forward, without vlan-device, with vlan encap, client1, without fastpath PASS: forward, without vlan-device, with vlan encap, client1, with fastpath PASS: forward, with vlan-device, without vlan encap, client1, without fastpath PASS: forward, with vlan-device, without vlan encap, client1, with fastpath PASS: forward, with vlan-device, with vlan encap, client1, without fastpath PASS: forward, with vlan-device, with vlan encap, client1, with fastpath PASS: all tests passed BANANAPI-R3 (lan1 & lan2 are dsa): ============ Without patches: ./bridge_fastpath.sh -t -0 enu1u2,lan2 -1 enu1u1,lan1 -2 lan4,eth1 Setup: CLIENT 0 enu1u2 | lan2 WAN ROUTER LAN1 LAN2 lan1 eth1 | | enu1u1 lan4 CLIENT 1 CLIENT 2 PASS: unaware bridge, without encaps, without fastpath PASS: unaware bridge, with single vlan encap, without fastpath WARN: unaware bridge, with pppoe encap, without fastpath: ipv4/6: established bytes 0 < 4194304 WARN: unaware bridge, with pppoe-in-q encaps, without fastpath: ipv4/6: established bytes 0 < 4194304 PASS: aware bridge, without/without vlan encap, without fastpath PASS: aware bridge, with/without vlan encap, without fastpath PASS: aware bridge, with/with vlan encap, without fastpath PASS: aware bridge, without/with vlan encap, without fastpath PASS: forward, without vlan-device, without vlan encap, client1, without fastpath WARN: forward, without vlan-device, without vlan encap, client1, with fastpath: ipv4: counted bytes 2110480 > 2097152 WARN: forward, without vlan-device, without vlan encap, client1, with fastpath: ipv6: counted bytes 2116104 > 2097152 PASS: forward, without vlan-device, without vlan encap, client1, with hw_fastpath PASS: forward, without vlan-device, without vlan encap, client2, without fastpath PASS: forward, without vlan-device, without vlan encap, client2, with fastpath PASS: forward, without vlan-device, without vlan encap, client2, with hw_fastpath PASS: forward, without vlan-device, with vlan encap, client1, without fastpath WARN: forward, without vlan-device, with vlan encap, client1, with fastpath: ipv4/6: tcp broken WARN: forward, without vlan-device, with vlan encap, client1, with hw_fastpath: ipv4/6: tcp broken PASS: forward, without vlan-device, with vlan encap, client2, without fastpath WARN: forward, without vlan-device, with vlan encap, client2, with fastpath: ipv4/6: tcp broken WARN: forward, without vlan-device, with vlan encap, client2, with hw_fastpath: ipv4/6: tcp broken PASS: forward, with vlan-device, without vlan encap, client1, without fastpath PASS: forward, with vlan-device, without vlan encap, client1, with fastpath PASS: forward, with vlan-device, without vlan encap, client1, with hw_fastpath PASS: forward, with vlan-device, without vlan encap, client2, without fastpath WARN: forward, with vlan-device, without vlan encap, client2, with fastpath: ipv4: counted bytes 2122388 > 2097152 WARN: forward, with vlan-device, without vlan encap, client2, with fastpath: ipv6: counted bytes 2129280 > 2097152 WARN: forward, with vlan-device, without vlan encap, client2, with hw_fastpath: ipv4: counted bytes 2110428 > 2097152 WARN: forward, with vlan-device, without vlan encap, client2, with hw_fastpath: ipv6: counted bytes 2140144 > 2097152 PASS: forward, with vlan-device, with vlan encap, client1, without fastpath PASS: forward, with vlan-device, with vlan encap, client1, with fastpath PASS: forward, with vlan-device, with vlan encap, client1, with hw_fastpath PASS: forward, with vlan-device, with vlan encap, client2, without fastpath PASS: forward, with vlan-device, with vlan encap, client2, with fastpath PASS: forward, with vlan-device, with vlan encap, client2, with hw_fastpath PASS: all tests passed With patches: PASS: unaware bridge, without encaps, without fastpath PASS: unaware bridge, without encaps, with fastpath PASS: unaware bridge, without encaps, with hw_fastpath PASS: unaware bridge, with single vlan encap, without fastpath PASS: unaware bridge, with single vlan encap, with fastpath PASS: unaware bridge, with single vlan encap, with hw_fastpath PASS: unaware bridge, with pppoe encap, without fastpath PASS: unaware bridge, with pppoe encap, with fastpath PASS: unaware bridge, with pppoe encap, with hw_fastpath PASS: unaware bridge, with pppoe-in-q encaps, without fastpath PASS: unaware bridge, with pppoe-in-q encaps, with fastpath PASS: unaware bridge, with pppoe-in-q encaps, with hw_fastpath PASS: aware bridge, without/without vlan encap, without fastpath PASS: aware bridge, without/without vlan encap, with fastpath PASS: aware bridge, without/without vlan encap, with hw_fastpath PASS: aware bridge, with/without vlan encap, without fastpath PASS: aware bridge, with/without vlan encap, with fastpath PASS: aware bridge, with/without vlan encap, with hw_fastpath PASS: aware bridge, with/with vlan encap, without fastpath PASS: aware bridge, with/with vlan encap, with fastpath PASS: aware bridge, with/with vlan encap, with hw_fastpath PASS: aware bridge, without/with vlan encap, without fastpath PASS: aware bridge, without/with vlan encap, with fastpath PASS: aware bridge, without/with vlan encap, with hw_fastpath PASS: forward, without vlan-device, without vlan encap, client1, without fastpath PASS: forward, without vlan-device, without vlan encap, client1, with fastpath PASS: forward, without vlan-device, without vlan encap, client1, with hw_fastpath PASS: forward, without vlan-device, without vlan encap, client2, without fastpath PASS: forward, without vlan-device, without vlan encap, client2, with fastpath PASS: forward, without vlan-device, without vlan encap, client2, with hw_fastpath PASS: forward, without vlan-device, with vlan encap, client1, without fastpath PASS: forward, without vlan-device, with vlan encap, client1, with fastpath PASS: forward, without vlan-device, with vlan encap, client1, with hw_fastpath PASS: forward, without vlan-device, with vlan encap, client2, without fastpath PASS: forward, without vlan-device, with vlan encap, client2, with fastpath PASS: forward, without vlan-device, with vlan encap, client2, with hw_fastpath PASS: forward, with vlan-device, without vlan encap, client1, without fastpath PASS: forward, with vlan-device, without vlan encap, client1, with fastpath PASS: forward, with vlan-device, without vlan encap, client1, with hw_fastpath PASS: forward, with vlan-device, without vlan encap, client2, without fastpath PASS: forward, with vlan-device, without vlan encap, client2, with fastpath PASS: forward, with vlan-device, without vlan encap, client2, with hw_fastpath PASS: forward, with vlan-device, with vlan encap, client1, without fastpath PASS: forward, with vlan-device, with vlan encap, client1, with fastpath PASS: forward, with vlan-device, with vlan encap, client1, with hw_fastpath PASS: forward, with vlan-device, with vlan encap, client2, without fastpath PASS: forward, with vlan-device, with vlan encap, client2, with fastpath PASS: forward, with vlan-device, with vlan encap, client2, with hw_fastpath PASS: all tests passed .../testing/selftests/net/netfilter/Makefile | 1 + .../net/netfilter/bridge_fastpath.sh | 1050 +++++++++++++++++ 2 files changed, 1051 insertions(+) create mode 100755 tools/testing/selftests/net/netfilter/bridge_fastpath.sh diff --git a/tools/testing/selftests/net/netfilter/Makefile b/tools/testing/selftests/net/netfilter/Makefile index ee2d1a5254f8..a7edc6654040 100644 --- a/tools/testing/selftests/net/netfilter/Makefile +++ b/tools/testing/selftests/net/netfilter/Makefile @@ -10,6 +10,7 @@ TEST_PROGS := \ br_netfilter.sh \ br_netfilter_queue.sh \ bridge_brouter.sh \ + bridge_fastpath.sh \ conntrack_clash.sh \ conntrack_dump_flush.sh \ conntrack_icmp_related.sh \ diff --git a/tools/testing/selftests/net/netfilter/bridge_fastpath.sh b/tools/testing/selftests/net/netfilter/bridge_fastpath.sh new file mode 100755 index 000000000000..d09b704d7bc6 --- /dev/null +++ b/tools/testing/selftests/net/netfilter/bridge_fastpath.sh @@ -0,0 +1,1050 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +# +# Check if conntrack, nft chain and fastpath is functional in setups +# where a bridge is in the fastpath. +# +# Commandline options make it possible to use real ethernet pairs +# instead of veth-device pairs. Any, or all, pairs can be tested using +# real hardware pairs. This is can be useful to test dsa-ports, +# switchdev (dsa) foreign ports and switchdev ports supporting +# SWITCHDEV_OBJ_ID_PORT_VLAN. +# +# First tcp is tested. Conntrack and nft chain are tested using a counter. +# When there is a fastpath possible between the interfaces then the +# fastpath is also tested. +# When there is a hardware offloaded fastpath possible between the +# interfaces then the hardware offloaded path is also tested. +# +# Setup is as a typical router: +# +# nsclientwan +# | +# nsrt +# | | +# nsclient1 nsclient2 +# +# Masquerading for ipv4 only. +# +# First check if a bridge table forward chain can be setup, skip +# these tests if this is not possible. +# Then check if a inet table forward chain can be setup, skip +# these tests if this is not possible. +# +# Different setups of paths are tested that involve a bridge in the +# fastpath. This can be in the forward-fastpath or in the bridge-fastpath. +# +# The first series, in the bridge-fastpath, using a vlan-unaware bridge. +# Traffic with the following vlan-tags is checked: +# a. without vlan +# b. single vlan +# c. double q vlan (only on veth-devices) +# d. 802.1ad vlan (only on veth-devices) +# e. pppoe (when available) +# f. pppoe-in-q (when available) +# +# (for items c to f fastpath can only work when a conntrack zone is set) +# (double tag testing results in broken tcp traffic on most hardware, +# in this test setup, use '-a' argument to test it anyway) +# (pppoe testing takes place if pppd and pppoe-server are installed) +# +# The second series, in the bridge-fastpath, using a vlan-aware bridge. +# Here we test all combinations of ingress/egress with or without single +# vlan encaps. +# +# The third series, in the forward-fastpath, using a vlan-aware bridge, +# without a vlan-device linked to the master port. We test the same combinations +# of ingress/egress with or without single vlan encaps. +# +# The fourth series, in the forward-fastpath, using a vlan-aware bridge, +# with a vlan-device linked to the master port. We test the same combinations +# of ingress/egress with or without single vlan encaps. +# +# Note 1: Using dsa userports on both sides of eth-pairs client1 or client2 +# gives erratic and unpredictable results. Use, for example, an usb-eth device +# on the client side to test a dsa-userport. +# +# Note 2: Testing the hardware offloaded fastpath, it is not checked if the +# packets do not follow the software fastpath instead. A universal way to +# check this should be added at some point. +# +# Note 3: Some interfaces to test on the router side, are netns immutable. +# Use the -d or --defaultnsrouter option so that the interfaces of the router +# do not have to change netns. The router is build up in the default netns. +# + +source lib.sh + +checktool "nft --version" "run test without nft" +checktool "socat -h" "run test without socat" +checktool "bridge -V" "run test without bridge" + +NR_OF_TESTS=4 +VID1=100 +VID2=101 +BRWAN=brwan +BRLAN=brlan +BRCL=brcl +LINKUP_TIMEOUT=10 +PING_TIMEOUT=10 +SOCAT_TIMEOUT=10 +filesize=$((2 * 1024 * 1024)) + +filein=$(mktemp) +file1out=$(mktemp) +file2out=$(mktemp) +pppoeserveroptions=$(mktemp) +pppoeserverpid=$(mktemp) + +setup_ns nsclientwan nsclientlan1 nsclientlan2 + + WAN=0 ; LAN1=1 ; LAN2=2 ; ADWAN=3 ; ADLAN=4 +nsa=( "$nsclientwan" "$nsclientlan1" "$nsclientlan2" ) # $nsrt $nsrt +AD4=( '192.168.1.1' '192.168.2.101' '192.168.2.102' '192.168.1.2' '192.168.2.1' ) +AD6=( 'dead:1::1' 'dead:2::101' 'dead:2::102' 'dead:1::2' 'dead:2::1' ) + +tests_string=$(seq 1 $NR_OF_TESTS) + +while [ "${1:-}" != '' ]; do + case "$1" in + '-0' | '--pairwan') + shift + vethcl[WAN]="${1%,*}" + vethrt[WAN]="${1#*,}" + ;; + '-1' | '--pairlan1') + shift + vethcl[LAN1]="${1%,*}" + vethrt[LAN1]="${1#*,}" + ;; + '-2' | '--pairlan2') + shift + vethcl[LAN2]="${1%,*}" + vethrt[LAN2]="${1#*,}" + ;; + '-s' | '--filesize') + shift + filesize=$1 + ;; + '-p' | '--parts') + shift + tests_string=$1 + ;; + '-4' | '--ipv4') + do_ipv4=1 + ;; + '-6' | '--ipv6') + do_ipv6=1 + ;; + '-n' | '--noskip') + noskip=1 + ;; + '-d' | '--defaultnsrouter') + defaultnsrouter=1 + ;; + '-f' | '--fixmac') + fixmac=1 + ;; + '-t' | '--showtree') + showtree=1 + ;; + *) + cat <<-EOF + Usage: $(basename "$0") [OPTION]... + -0 --pairwan eth0cl,eth0rt pair of real interfaces to use on wan side + -1 --pairlan1 eth1cl,eth1rt pair of real interfaces to use on lan1 side + -2 --pairlan2 eth2cl,eth2rt pair of real interfaces to use on lan2 side + -s --filesize filesize to use for testing in bytes + -p --parts partnumbers of tests to run, comma separated + -4|-6 --ipv4|--ipv6 test ipv4/6 only + -d --defaultnsrouter router in default network namespace, caution! + -f --fixmac change mac address when conflict found + -n --noskip also perform the normally skipped tests + -t --showtree show the tree of used interfaces + EOF + exit "$ksft_skip" + ;; + esac + shift +done + +for i in ${tests_string//','/' '}; do + tests[i]="yes" +done + +if [ -n "$defaultnsrouter" ]; then + nsrt="nsrt-$(mktemp -u XXXXXX)" + touch "/var/run/netns/$nsrt" + mount --bind /proc/1/ns/net "/var/run/netns/$nsrt" +else + setup_ns nsrt +fi +nsa+=("$nsrt" "$nsrt") + +cleanup() { + if [ -n "$defaultnsrouter" ]; then + umount "/var/run/netns/$nsrt" + rm -f "/var/run/netns/$nsrt" + fi + cleanup_all_ns + rm -f "$filein" "$file1out" "$file2out" "$pppoeserveroptions" "$pppoeserverpid" +} + +trap cleanup EXIT + +head -c "$filesize" < /dev/urandom > "$filein" + +check_mac() +{ + local ns=$1 + local dev=$2 + local othermacs=$3 + local mac + + mac=$(ip -net "$ns" -br link show dev "$dev" | \ + grep -o -E '([[:xdigit:]]{1,2}:){5}[[:xdigit:]]{1,2}') + + if [[ ! "$othermacs" =~ $mac ]]; then + echo "$mac" + return 0 + fi + echo "WARN: Conflicting mac address $dev $mac" 1>&2 + + [ -z "$fixmac" ] && return 1 + + for (( j = 0 ; j < 10 ; j++ )); do + mac="${mac::6}$(printf %02x:%02x:%02x:%02x $((RANDOM%256)) \ + $((RANDOM%256)) $((RANDOM%256)) $((RANDOM%256)))" + [[ "$othermacs" =~ $mac ]] && continue + echo "$mac" + ip -net "$ns" link set dev "$dev" address "$mac" 1>&2 + return $? + done + return 1 +} + +is_link() +{ + local updown=$1 + local ns=$2 + local dev=$3 + + if ip -net "$ns" link show dev "$dev" "${updown,,}" 2>/dev/null | \ + grep -q "state ${updown^^}" + then + return 0 + fi + return 1 +} + +set_pair_link() +{ + local updown=$1 + local all="${*:2}" + local lret=0 + local i j + + for i in $all; do + ns="${nsa[$i]}" + ip -net "$ns" link set "${vethcl[$i]}" "$updown" + lret=$((lret | $?)) + ip -net "$nsrt" link set "${vethrt[$i]}" "$updown" + lret=$((lret | $?)) + done + [ $lret -ne 0 ] && return 1 + + for j in $(seq 1 $((LINKUP_TIMEOUT * 5 ))); do + lret=0 + for i in $all; do + ns="${nsa[$i]}" + is_link "$updown" "$ns" "${vethcl[$i]}" + lret=$((lret | $?)) + is_link "$updown" "$nsrt" "${vethrt[$i]}" + lret=$((lret | $?)) + done + [ $lret -eq 0 ] && break + sleep 0.2 + done + return $lret +} + +wait_ping() +{ + local i1=$1 + local i2=$2 + local ns1=${nsa[$i1]} + local j + local lret + + for j in $(seq 1 $((PING_TIMEOUT * 5 ))); do + ip netns exec "$ns1" ping -c 1 -w $PING_TIMEOUT -i 0.2 \ + -q "${AD4[$i2]}" >/dev/null 2>&1 + lret=$? + [ $lret -le 1 ] && return $lret + sleep 0.2 + done + return 1 +} + +add_addr() +{ + local i=$1 + local dev=$2 + local ns=${nsa[$i]} + local ad4=${AD4[$i]} + local ad6=${AD6[$i]} + + ip -net "$ns" addr add "${ad4}/24" dev "$dev" + ip -net "$ns" addr add "${ad6}/64" dev "$dev" nodad + if [[ "$ns" == "nsclientlan"* ]]; then + ip -net "$ns" route add default via "${AD4[$ADLAN]}" + ip -net "$ns" route add default via "${AD6[$ADLAN]}" + elif [[ "$ns" == "nsclientwan"* ]]; then + ip -net "$ns" route add default via "${AD6[$ADWAN]}" + fi + +} + +del_addr() +{ + local i=$1 + local dev=$2 + local ns=${nsa[$i]} + local ad4=${AD4[$i]} + local ad6=${AD6[$i]} + + if [[ "$ns" == "nsclientlan"* ]]; then + ip -net "$ns" route del default via "${AD6[$ADLAN]}" + ip -net "$ns" route del default via "${AD4[$ADLAN]}" + elif [[ "$ns" == "nsclientwan"* ]]; then + ip -net "$ns" route del default via "${AD6[$ADWAN]}" + fi + ip -net "$ns" addr del "${ad6}/64" dev "$dev" nodad + ip -net "$ns" addr del "${ad4}/24" dev "$dev" +} + +set_client() +{ + local i=$1 + local vlan=$2 + local arg=$3 + local ns=${nsa[$i]} + local vdev="${vethcl[$i]}" + local brdev="$BRCL" + local proto="" + local pvidslave="" + + unset_client "$i" + + if [[ "$vlan" == "qq" ]]; then + ip -net "$ns" link add link "$vdev" name "$vdev.$VID1" type vlan id $VID1 + ip -net "$ns" link add link "$vdev.$VID1" name "$vdev.$VID1.$VID2" \ + type vlan id $VID2 + ip -net "$ns" link set "$vdev.$VID1" up + ip -net "$ns" link set "$vdev.$VID1.$VID2" up + add_addr "$i" "$vdev.$VID1.$VID2" + return + fi + + [[ "$vlan" == "none" ]] && pvidslave="pvid untagged" + [[ "$vlan" == "ad" ]] && proto="vlan_protocol 802.1ad" + + # shellcheck disable=SC2086 + ip -net "$ns" link add "$brdev" type bridge vlan_filtering 1 vlan_default_pvid 0 $proto + ip -net "$ns" link set "$vdev" master "$brdev" + ip -net "$ns" link set "$brdev" up + + # shellcheck disable=SC2086 + bridge -net "$ns" vlan add dev "$vdev" vid $VID1 $pvidslave + bridge -net "$ns" vlan add dev "$brdev" vid $VID1 pvid untagged self + + if [[ "$vlan" == "ad" ]]; then + ip -net "$ns" link add link "$brdev" name "$brdev.$VID2" type vlan id $VID2 + brdev="$brdev.$VID2" + ip -net "$ns" link set "$brdev" up + fi + + if [[ "$arg" != "noaddress" ]]; then + add_addr "$i" "$brdev" + fi +} + +unset_client() +{ + local i=$1 + local ns=${nsa[$i]} + local vdev="${vethcl[$i]}" + local brdev="$BRCL" + + ip -net "$ns" link del "$brdev" type bridge 2>/dev/null + ip -net "$ns" link del "$vdev.$VID1" 2>/dev/null +} + +add_pppoe() +{ + local i1=$1 + local i2=$2 + local dev1=$3 + local dev2=$4 + local desc=$5 + local ns1=${nsa[$i1]} + local ns2=${nsa[$i2]} + + ppp1=0 + while [ -n "$(ip -net "$ns1" link show ppp$ppp1 2>/dev/null)" ] + do ((ppp1++)); done + echo "noauth defaultroute noipdefault unit $ppp1" >"$pppoeserveroptions" + ppp1="ppp$ppp1" + + if ! ip netns exec "$ns1" pppoe-server -k -L "${AD4[$i1]}" -R "${AD4[$i2]}" \ + -I "$dev1" -X "$pppoeserverpid" -O "$pppoeserveroptions" >/dev/null; then + echo "ERROR: $desc: failed to setup pppoe server" 1>&2 + return 1 + fi + + if ! ip netns exec "$ns2" pppd plugin pppoe.so nic-"$dev2" persist holdoff 0 noauth \ + defaultroute noipdefault noaccomp nodeflate noproxyarp nopcomp \ + novj novjccomp linkname "selftest-$$" >/dev/null; then + echo "ERROR: $desc: failed to setup pppoe client" 1>&2 + return 1 + fi + + if ! wait_ping "$i1" "$i2"; then + echo "ERROR: $desc: failed to setup functional pppoe connection" 1>&2 + return 1 + fi + + ppp2=$(tail -n 1 < "/run/pppd/ppp-selftest-$$.pid") + + ip -net "$ns1" addr add "${AD6[$i1]}/64" dev "$ppp1" nodad + ip -net "$ns2" addr add "${AD6[$i2]}/64" dev "$ppp2" nodad + + return 0 +} + +del_pppoe() +{ + local i1=$1 + local i2=$2 + local dev1=$3 + local dev2=$4 + local ns1=${nsa[$i1]} + local ns2=${nsa[$i2]} + local i serverpid clientpid + + serverpid="$(head -n 1 < "$pppoeserverpid")" + clientpid="$(head -n 1 < "/run/pppd/ppp-selftest-$$.pid")" + + [[ -n "$ppp1" ]] && ip -net "$ns1" addr del "${AD6[$i1]}/64" dev "$ppp1" + [[ -n "$ppp2" ]] && ip -net "$ns2" addr del "${AD6[$i2]}/64" dev "$ppp2" + + for i in $(seq 1 $((PING_TIMEOUT * 5 ))); do + if ip -net "$ns2" link show dev "$ppp2" 1>/dev/null 2>/dev/null; then + kill -9 "$clientpid" 2>/dev/null + elif ip -net "$ns1" link show dev "$ppp1" 1>/dev/null 2>/dev/null; then + kill -SIGTERM "$serverpid" 2>/dev/null + else return 0 + fi + sleep 0.2 + done + echo "ERROR: failed to remove pppoe connection" 1>&2 + return 1 +} + +listener_ready() +{ + local ns=$1 + local ipv=$2 + + ss -N "$ns" --ipv"$ipv" -lnt -o "sport = :8080" | grep -q 8080 +} + +test_tcp() { + local i1=$1 + local i2=$2 + local dofast=$3 + local desc=$4 + local ns1=${nsa[$i1]} + local ns2=${nsa[$i2]} + local i=-1 + local lret=0 + local ads="" + local ipv ad a lpid bytes error + + if [ -n "$do_ipv4" ]; then ads="${AD4[$i2]}" + elif [ -n "$do_ipv6" ]; then ads="${AD6[$i2]}" + else ads="${AD4[$i2]} ${AD6[$i2]}" + fi + for ad in $ads; do + ((i++)) + if [[ "$ad" =~ ":" ]] + then ipv="6"; a="[${ad}]" + else ipv="4"; a="${ad}" + fi + + rm -f "$file1out" "$file2out" + + # ip netns exec "$nsrt" nft reset counters >/dev/null + # But on some systems this results in 4GB values in packet and byte count, so: + (echo "flush ruleset"; ip netns exec "$nsrt" nft --stateless list ruleset) | \ + ip netns exec "$nsrt" nft -f - + + timeout "$SOCAT_TIMEOUT" ip netns exec "$ns2" socat TCP$ipv-LISTEN:8080,reuseaddr \ + STDIO <"$filein" >"$file2out" 2>/dev/null & + lpid=$! + busywait 1000 listener_ready "$ns2" "$ipv" + + timeout "$SOCAT_TIMEOUT" ip netns exec "$ns1" socat TCP$ipv:"$a":8080 \ + STDIO <"$filein" >"$file1out" 2>/dev/null + + if ! wait $lpid; then + error[i]="tcp broken" + continue + fi + if ! cmp "$filein" "$file1out" >/dev/null 2>&1; then + error[i]="file mismatch to ${ad}" + continue + fi + if ! cmp "$filein" "$file2out" >/dev/null 2>&1; then + error[i]="file mismatch from ${ad}" + continue + fi + + bytes=$(ip netns exec "$nsrt" nft list counter $family filter "check" | \ + grep "packets" | cut -d' ' -f4) + if [ -z "$dofast" ] && [ "$bytes" -lt "$((2 * filesize))" ]; then + + error[i]="established bytes $bytes < $((2 * filesize))" + continue + fi + if [ -n "$dofast" ] && [ "$bytes" -gt "$filesize" ]; then + # Significant reduction of bytes expected + error[i]="counted bytes $bytes > $filesize" + continue + fi + + done + + if [ -n "${error[0]}" ]; then + if [[ "${error[0]}" == "${error[1]}" ]]; then + error[0]="$desc: ipv4/6: ${error[0]}" + error[1]="" + else + error[0]="$desc: ipv4: ${error[0]}" + fi + fi + if [ -n "${error[1]}" ]; then + error[1]="$desc: ipv6: ${error[1]}" + fi + + for i in 0 1; do + if [ -n "${error[i]}" ]; then + if is_known_issue "$desc: ${error[i]}"; then + echo "WARN: ${error[i]}" 1>&2 + lret=$((lret | 1)) + else + echo "ERROR: ${error[i]}" 1>&2 + lret=$((lret | 2)) + fi + fi + done + if [ $lret -eq 0 ]; then + echo "PASS: $desc" + fi + return $(( lret & 2 )) +} + +known_issues=( +'*unaware bridge,*with double q vlan encaps,*without fastpath*established*' # 1 +'*unaware bridge,*with 802.1ad vlan encaps,*without fastpath*established*' # 1 +'*unaware bridge,*with pppoe encap,*without fastpath*established*' # 1 +'*unaware bridge,*with pppoe-in-q encaps,*without fastpath*established*' # 1 +'*forward,*without vlan-device, without vlan encap,*with *fastpath:*counted*' # 2 +'*forward,*without vlan-device, with vlan encap,*with *fastpath:*tcp broken*' # 3 +'*forward,*with vlan-device, without vlan encap,*with *fastpath:*counted*' # 4 +) + +is_known_issue() { + local err=$1 + for issue in "${known_issues[@]}"; do + # shellcheck disable=SC2053 + [[ "$err" == $issue ]] && return 0 + done + return 1 +} + +test_paths() { + local i1=$1 + local i2=$2 + local desc=$3 + + if ! setup_nftables "$i1" "$i2"; then + echo "ERROR: $desc: cannot setup nftables" 1>&2 + return 1 + fi + if ! test_tcp "$i1" "$i2" "" "$desc without fastpath"; then + return 1 + fi + + if ! setup_fastpath "$i1" "$i2" "" 2>/dev/null; then + return 0 + fi + if ! test_tcp "$i1" "$i2" "fast" "$desc with fastpath"; then + return 1 + fi + + if ! setup_fastpath "$i1" "$i2" "hw" 2>/dev/null; then + return 0 + fi + if ! test_tcp "$i1" "$i2" "fast" "$desc with hw_fastpath"; then + return 1 + fi + + return 0 + +} + +add_masq() +{ + if [[ $family != "bridge" ]]; then + ip netns exec "$nsrt" nft -f - <<-EOF + table ip nat { + chain postrouting { + type nat hook postrouting priority 0; + oifname ${BRWAN} masquerade + } + } + EOF + else + return 0 + fi +} + +add_zone() +{ + local devs=$1 + + if [[ $family == "bridge" ]]; then + ip netns exec "$nsrt" nft -f - <<-EOF + table ${family} filter { + chain preroutingzones { + type filter hook prerouting priority -300; + iif ${devs} ct zone set 23 + } + } + EOF + fi +} + +setup_nftables() +{ + local devs="{ ${vethrt[$1]} , ${vethrt[$2]} }" + local i1=$1 + local i2=$2 + + ip netns exec "$nsrt" nft flush ruleset + + if ! add_masq; then + return 1 + fi + + add_zone "${devs}" 2>/dev/null + + ip netns exec "$nsrt" nft -f - <<-EOF + table ${family} filter { + counter check { } + chain prerouting { + type filter hook prerouting priority 0; policy accept; + ct state established counter name "check" + } + } + EOF +} + +setup_fastpath() +{ + local devs="{ ${vethrt[$1]} , ${vethrt[$2]} }" + local arg=$3 + local flags="" + + [[ "$arg" == "hw" ]] && flags="flags offload" + + ip netns exec "$nsrt" nft flush ruleset + + if ! add_masq; then + return 1 + fi + + add_zone "${devs}" 2>/dev/null + + ip netns exec "$nsrt" nft -f - <<-EOF + table ${family} filter { + counter check { } + flowtable f { + hook ingress priority filter + devices = ${devs} + ${flags} + } + chain forward { + type filter hook forward priority 0; policy accept; + counter name "check" + ct state established flow add @f + } + } + EOF +} + +test_unaware_bridge() +{ + local lret=0 + local i + + for i in $LAN1 $LAN2; do + set_client "$i" none + done + + test_paths $LAN1 $LAN2 "unaware bridge, without encaps, " + lret=$((lret | $?)) + + for i in $LAN1 $LAN2; do + set_client "$i" q + done + + test_paths $LAN1 $LAN2 "unaware bridge, with single vlan encap, " + lret=$((lret | $?)) + + for i in $LAN1 $LAN2; do + set_client "$i" qq + done + + # Skip testing double tagged packets on real hardware + if [ -n "$lan_all_veth" ] || [ -n "$noskip" ]; then + + test_paths $LAN1 $LAN2 "unaware bridge, with double q vlan encaps, " + lret=$((lret | $?)) + + for i in $LAN1 $LAN2; do + set_client "$i" ad + done + + test_paths $LAN1 $LAN2 "unaware bridge, with 802.1ad vlan encaps, " + lret=$((lret | $?)) + + fi + # End Skip testing double tagged packets + + if [ -n "$(command -v pppd 2>/dev/null)" ] && + [ -n "$(command -v pppoe-server 2>/dev/null)" ]; then + # Start pppoe + + for i in $LAN1 $LAN2; do + set_client "$i" none noaddress + done + + if add_pppoe $LAN1 $LAN2 "$BRCL" "$BRCL" "unaware bridge, with pppoe encap"; then + test_paths $LAN1 $LAN2 "unaware bridge, with pppoe encap, " + lret=$((lret | $?)) + fi + + del_pppoe $LAN1 $LAN2 "$BRCL" "$BRCL" + lret=$((lret | $?)) + + for i in $LAN1 $LAN2; do + set_client "$i" q noaddress + done + + if add_pppoe $LAN1 $LAN2 "$BRCL" "$BRCL" "unaware bridge, with pppoe-in-q encaps"; then + test_paths $LAN1 $LAN2 "unaware bridge, with pppoe-in-q encaps, " + lret=$((lret | $?)) + fi + + del_pppoe $LAN1 $LAN2 "$BRCL" "$BRCL" + lret=$((lret | $?)) + + # End pppoe + fi + + for i in $LAN1 $LAN2; do + unset_client "$i" + done + return $lret +} + +test_aware_bridge() +{ + local lret=0 + local i + + for i in $LAN1 $LAN2; do + bridge -net "$nsrt" vlan add dev "${vethrt[$i]}" vid $VID1 pvid untagged + set_client "$i" none + done + test_paths $LAN1 $LAN2 "aware bridge, without/without vlan encap," + lret=$((lret | $?)) + + i=$LAN1 + bridge -net "$nsrt" vlan del dev "${vethrt[$i]}" vid $VID1 pvid untagged + bridge -net "$nsrt" vlan add dev "${vethrt[$i]}" vid $VID1 + set_client $i q + + test_paths $LAN1 $LAN2 "aware bridge, with/without vlan encap, " + lret=$((lret | $?)) + + i=$LAN2 + bridge -net "$nsrt" vlan del dev "${vethrt[$i]}" vid $VID1 pvid untagged + bridge -net "$nsrt" vlan add dev "${vethrt[$i]}" vid $VID1 + set_client $i q + + test_paths $LAN1 $LAN2 "aware bridge, with/with vlan encap, " + lret=$((lret | $?)) + + i=$LAN1 + bridge -net "$nsrt" vlan del dev "${vethrt[$i]}" vid $VID1 + bridge -net "$nsrt" vlan add dev "${vethrt[$i]}" vid $VID1 pvid untagged + set_client $i none + + test_paths $LAN1 $LAN2 "aware bridge, without/with vlan encap, " + lret=$((lret | $?)) + + i=$LAN1 + bridge -net "$nsrt" vlan del dev "${vethrt[$i]}" vid $VID1 pvid untagged + unset_client $i + i=$LAN2 + bridge -net "$nsrt" vlan del dev "${vethrt[$i]}" vid $VID1 + unset_client $i + + return $lret +} + +test_forward_without_vlandev() +{ + local wo=$1 + local lret=0 + local i + + [[ "$wo" == "" ]] && wo="without" + + for i in $LAN1 $LAN2; do + bridge -net "$nsrt" vlan add dev "${vethrt[$i]}" vid $VID1 pvid untagged + set_client "$i" none + done + + test_paths $LAN1 $WAN "forward, $wo vlan-device, without vlan encap, client1," + lret=$((lret | $?)) + if [ -z "$lan_all_veth" ] || [ -n "$noskip" ]; then + test_paths $LAN2 $WAN "forward, $wo vlan-device, without vlan encap, client2," + lret=$((lret | $?)) + fi + + for i in $LAN1 $LAN2; do + bridge -net "$nsrt" vlan del dev "${vethrt[$i]}" vid $VID1 pvid untagged + bridge -net "$nsrt" vlan add dev "${vethrt[$i]}" vid $VID1 + set_client "$i" q + done + + test_paths $LAN1 $WAN "forward, $wo vlan-device, with vlan encap, client1," + lret=$((lret | $?)) + if [ -z "$lan_all_veth" ] || [ -n "$noskip" ]; then + test_paths $LAN2 $WAN "forward, $wo vlan-device, with vlan encap, client2," + lret=$((lret | $?)) + fi + + for i in $LAN1 $LAN2; do + bridge -net "$nsrt" vlan del dev "${vethrt[$i]}" vid $VID1 + unset_client "$i" + done + return $lret +} + +test_forward_with_vlandev() +{ + test_forward_without_vlandev "with" + return $? +} + +ret=0 +### Start Initial Setup ### + +for i in 4 6; do + ip netns exec "$nsrt" sysctl -q net.ipv$i.conf.all.forwarding=1 +done + +### Use brwan to make sure software fastpath is ### +### direct xmit in other direction also ### + +ip -net "$nsrt" link add $BRWAN type bridge +ret=$((ret | $?)) +ip -net "$nsrt" link set $BRWAN up +ret=$((ret | $?)) +if [ $ret -ne 0 ]; then + echo "SKIP: Can't create bridge" + exit "$ksft_skip" +fi + +# If both lan clients are veth-devices, only test 1 in the forward path +if [ -z "${vethcl[$LAN1]}" ] && [ -z "${vethcl[$LAN2]}" ]; then + lan_all_veth=1 +fi + +for i in $WAN $LAN1 $LAN2; do + ns="${nsa[$i]}" + if [ -z "${vethcl[$i]}" ]; then + vethcl[i]="veth${i}cl" + vethrt[i]="veth${i}rt" + ip link add "${vethcl[$i]}" netns "$ns" type veth \ + peer name "${vethrt[$i]}" netns "$nsrt" + ret=$((ret | $?)) + else # Use pair of interconnected hardware interfaces + ip link set "${vethrt[$i]}" netns "$nsrt" + ret=$((ret | $?)) + ip link set "${vethcl[$i]}" netns "$ns" + ret=$((ret | $?)) + fi +done +if [ $ret -ne 0 ]; then + echo "SKIP: (v)eth pairs cannot be used" + exit "$ksft_skip" +fi + +if [ -n "$showtree" ]; then + cat <<-EOF + Setup: + CLIENT 0 + ${vethcl[$WAN]} + | + ${vethrt[$WAN]} + WAN + ROUTER + LAN1 LAN2 + $(printf "%14.14s" "${vethrt[$LAN1]}") ${vethrt[$LAN2]} + | | + $(printf "%14.14s" "${vethcl[$LAN1]}") ${vethcl[$LAN2]} + CLIENT 1 CLIENT 2 + + EOF +fi + +for n in nsclientwan nsclientlan; do + routerside=""; clientside="" + for i in $WAN $LAN1 $LAN2; do + ns="${nsa[$i]}" + [[ "$ns" != "$n"* ]] && continue + mac=$(check_mac "$ns" "${vethcl[$i]}" "$routerside $clientside") + ret=$((ret | $?)) + clientside+=" $mac" + mac=$(check_mac "$nsrt" "${vethrt[$i]}" "$clientside") + ret=$((ret | $?)) + routerside+=" $mac" + done +done +if [ $ret -ne 0 ]; then + echo "SKIP: conflicting mac address" + exit "$ksft_skip" +fi + +set_pair_link up $WAN $LAN1 $LAN2 +ret=$((ret | $?)) +if [ $ret -ne 0 ]; then + echo "SKIP: setting (v)eth pairs link up failed" + exit "$ksft_skip" +fi + +i=$WAN +ip -net "$nsrt" link set "${vethrt[$i]}" master $BRWAN +set_client $i none +add_addr $ADWAN "$BRWAN" + +family="bridge" +if ! setup_nftables $LAN1 $LAN2 2>/dev/null; then + echo "INFO: Cannot add nftables table $family" + tests[1]=""; tests[2]="" +fi +family="inet" +if ! setup_nftables $WAN $LAN1 2>/dev/null; then + echo "INFO: Cannot add nftables table $family" + tests[3]=""; tests[4]="" +fi + +### End Initial Setup ### + +if [ -n "${tests[1]}" ]; then + # Setup brlan as vlan unaware bridge + family="bridge" + ip -net "$nsrt" link add $BRLAN type bridge + ip -net "$nsrt" link set $BRLAN up + for i in $LAN1 $LAN2; do + ip -net "$nsrt" link set "${vethrt[$i]}" master $BRLAN + done + test_unaware_bridge + ret=$((ret | $?)) + ip -net "$nsrt" link del $BRLAN type bridge +fi + +if [ -n "${tests[2]}" ] || [ -n "${tests[3]}" ] || [ -n "${tests[4]}" ]; then + # Setup brlan as vlan aware bridge + family="bridge" + + ip -net "$nsrt" link add $BRLAN type bridge vlan_filtering 1 vlan_default_pvid 0 + ip -net "$nsrt" link set $BRLAN up + bridge -net "$nsrt" vlan add dev $BRLAN vid $VID1 pvid untagged self + add_addr $ADLAN "$BRLAN" + for i in $LAN1 $LAN2; do + ip -net "$nsrt" link set "${vethrt[$i]}" master $BRLAN + done + + if [ -n "${tests[2]}" ]; then + test_aware_bridge + ret=$((ret | $?)) + fi + + family="inet" + + if [ -n "${tests[3]}" ]; then + test_forward_without_vlandev + ret=$((ret | $?)) + fi + + if [ -n "${tests[4]}" ]; then + # Setup vlan-device linked to brlan master port + del_addr $ADLAN "$BRLAN" + ip -net "$nsrt" link set $BRLAN down + bridge -net "$nsrt" vlan del dev $BRLAN vid $VID1 pvid untagged self + bridge -net "$nsrt" vlan add dev $BRLAN vid $VID1 self + ip -net "$nsrt" link add link $BRLAN name $BRLAN.$VID1 type vlan id $VID1 + ip -net "$nsrt" link set $BRLAN up + ip -net "$nsrt" link set "$BRLAN.$VID1" up + add_addr $ADLAN "$BRLAN.$VID1" + test_forward_with_vlandev + ret=$((ret | $?)) + fi + + ip -net "$nsrt" link del $BRLAN type bridge +fi + +### Finish tests ### + +ip -net "$nsrt" link del $BRWAN type bridge + +for i in $WAN $LAN1 $LAN2; do + unset_client "$i" +done + +set_pair_link down $WAN $LAN1 $LAN2 + +for i in $WAN $LAN1 $LAN2; do + ns="${nsa[$i]}" + if [[ "${vethcl[$i]:0:4}" != "veth" ]]; then + ip netns exec "$ns" ip link set "${vethcl[$i]}" netns 1 + fi + if [[ "${vethrt[$i]:0:4}" != "veth" ]]; then + ip netns exec "$nsrt" ip link set "${vethrt[$i]}" netns 1 + fi +done + +if [ $ret -eq 0 ]; then + echo "PASS: all tests passed" +else + echo "ERROR: bridge fastpath test has failed" +fi + +exit $ret -- 2.50.0

2 weeks

1
0
0 0

[PATCH v2 7/7] KVM: LoongArch: selftests: Add time counter test

by Bibo Mao

With time counter test, it is to verify that time count starts from 0 and always grows up then. Signed-off-by: Bibo Mao <maobibo(a)loongson.cn> --- .../selftests/kvm/lib/loongarch/processor.c | 9 ++++++ .../selftests/kvm/loongarch/arch_timer.c | 29 +++++++++++++++++++ 2 files changed, 38 insertions(+) diff --git a/tools/testing/selftests/kvm/lib/loongarch/processor.c b/tools/testing/selftests/kvm/lib/loongarch/processor.c index 436990258068..ac2ffd076bff 100644 --- a/tools/testing/selftests/kvm/lib/loongarch/processor.c +++ b/tools/testing/selftests/kvm/lib/loongarch/processor.c @@ -3,6 +3,7 @@ #include <assert.h> #include <linux/compiler.h> +#include <asm/kvm.h> #include "kvm_util.h" #include "processor.h" #include "ucall_common.h" @@ -256,6 +257,11 @@ static void loongarch_set_csr(struct kvm_vcpu *vcpu, uint64_t id, uint64_t val) __vcpu_set_reg(vcpu, csrid, val); } +static void loongarch_set_reg(struct kvm_vcpu *vcpu, uint64_t id, uint64_t val) +{ + __vcpu_set_reg(vcpu, id, val); +} + static void loongarch_vcpu_setup(struct kvm_vcpu *vcpu) { int width; @@ -279,6 +285,9 @@ static void loongarch_vcpu_setup(struct kvm_vcpu *vcpu) loongarch_set_csr(vcpu, LOONGARCH_CSR_ECFG, 0); loongarch_set_csr(vcpu, LOONGARCH_CSR_TCFG, 0); loongarch_set_csr(vcpu, LOONGARCH_CSR_ASID, 1); + /* time count start from 0 */ + val = 0; + loongarch_set_reg(vcpu, KVM_REG_LOONGARCH_COUNTER, val); val = 0; width = vm->page_shift - 3; diff --git a/tools/testing/selftests/kvm/loongarch/arch_timer.c b/tools/testing/selftests/kvm/loongarch/arch_timer.c index 579132a082cd..f3a25a0163fc 100644 --- a/tools/testing/selftests/kvm/loongarch/arch_timer.c +++ b/tools/testing/selftests/kvm/loongarch/arch_timer.c @@ -133,10 +133,39 @@ static void guest_test_emulate_timer(uint32_t cpu) local_irq_enable(); } +static void guest_time_count_test(uint32_t cpu) +{ + uint32_t config_iter; + unsigned long start, end, prev, us; + + /* Assuming that test case starts to run in 1 second */ + start = timer_get_cycles(); + us = msec_to_cycles(1000); + __GUEST_ASSERT(start <= us, + "start = 0x%lx, us = 0x%lx.\n", + start, us); + + us = msec_to_cycles(test_args.timer_period_ms); + for (config_iter = 0; config_iter < test_args.nr_iter; config_iter++) { + start = timer_get_cycles(); + end = start + us; + /* test time count growing up always */ + while (start < end) { + prev = start; + start = timer_get_cycles(); + __GUEST_ASSERT(prev <= start, + "prev = 0x%lx, start = 0x%lx.\n", + prev, start); + } + } +} + static void guest_code(void) { uint32_t cpu = guest_get_vcpuid(); + /* must run at first */ + guest_time_count_test(cpu); timer_irq_enable(); local_irq_enable(); guest_test_oneshot_timer(cpu); -- 2.39.3

2 weeks, 1 day

1
0
0 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

Linux-kselftest-mirror