From: Alexander Lobakin <bloodyreaper(a)yandex.ru>
commit e131a5634830047923c694b4ce0c3b31745ff01b upstream.
gro_cells lib is used by different encapsulating netdevices, such as
geneve, macsec, vxlan etc. to speed up decapsulated traffic processing.
CPU tag is a sort of "encapsulation", and we can use the same mechs to
greatly improve overall DSA performance.
skbs are passed to the GRO layer after removing CPU tags, so we don't
need any new packet offload types as it was firstly proposed by me in
the first GRO-over-DSA variant [1].
The size of struct gro_cells is sizeof(void *), so hot struct
dsa_slave_priv becomes only 4/8 bytes bigger, and all critical fields
remain in one 32-byte cacheline.
The other positive side effect is that drivers for network devices
that can be shipped as CPU ports of DSA-driven switches can now use
napi_gro_frags() to pass skbs to kernel. Packets built that way are
completely non-linear and are likely being dropped without GRO.
This was tested on to-be-mainlined-soon Ethernet driver that uses
napi_gro_frags(), and the overall performance was on par with the
variant from [1], sometimes even better due to minimal overhead.
net.core.gro_normal_batch tuning may help to push it to the limit
on particular setups and platforms.
iperf3 IPoE VLAN NAT TCP forwarding (port1.218 -> port0) setup
on 1.2 GHz MIPS board:
5.7-rc2 baseline:
[ID] Interval Transfer Bitrate Retr
[ 5] 0.00-120.01 sec 9.00 GBytes 644 Mbits/sec 413 sender
[ 5] 0.00-120.00 sec 8.99 GBytes 644 Mbits/sec receiver
Iface RX packets TX packets
eth0 7097731 7097702
port0 426050 6671829
port1 6671681 425862
port1.218 6671677 425851
With this patch:
[ID] Interval Transfer Bitrate Retr
[ 5] 0.00-120.01 sec 12.2 GBytes 870 Mbits/sec 122 sender
[ 5] 0.00-120.00 sec 12.2 GBytes 870 Mbits/sec receiver
Iface RX packets TX packets
eth0 9474792 9474777
port0 455200 353288
port1 9019592 455035
port1.218 353144 455024
v2:
- Add some performance examples in the commit message;
- No functional changes.
[1] https://lore.kernel.org/netdev/20191230143028.27313-1-alobakin@dlink.ru/
Signed-off-by: Alexander Lobakin <bloodyreaper(a)yandex.ru>
Signed-off-by: David S. Miller <davem(a)davemloft.net>
---
This patch radically increase network performance on DSA setup.
Please include this patch into stable releases.
I have done following tests:
NAT is a tested Espressobin board (ARM64 Marvell Armada 3720 SoC with
Marvell 88E6141 DSA switch) which was configured for IPv4 masquerade.
WAN and LAN are another two static boxes on which was running iperf3.
4.19.179 without e131a5634830047923c694b4ce0c3b31745ff01b
WAN --> NAT --> LAN
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.01 sec 440 MBytes 369 Mbits/sec 12 sender
[ 5] 0.00-10.00 sec 437 MBytes 367 Mbits/sec receiver
WAN <-- NAT <-- LAN
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 390 MBytes 327 Mbits/sec 90 sender
[ 5] 0.00-10.01 sec 388 MBytes 326 Mbits/sec receiver
4.19.179 with e131a5634830047923c694b4ce0c3b31745ff01b
WAN --> NAT --> LAN
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.01 sec 616 MBytes 516 Mbits/sec 18 sender
[ 5] 0.00-10.00 sec 613 MBytes 515 Mbits/sec receiver
WAN <-- NAT <-- LAN
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 573 MBytes 480 Mbits/sec 32 sender
[ 5] 0.00-10.01 sec 570 MBytes 478 Mbits/sec receiver
5.4.103 without e131a5634830047923c694b4ce0c3b31745ff01b
WAN --> NAT --> LAN
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.01 sec 454 MBytes 380 Mbits/sec 62 sender
[ 5] 0.00-10.00 sec 451 MBytes 378 Mbits/sec receiver
WAN <-- NAT <-- LAN
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 425 MBytes 356 Mbits/sec 155 sender
[ 5] 0.00-10.01 sec 422 MBytes 354 Mbits/sec receiver
5.4.103 with e131a5634830047923c694b4ce0c3b31745ff01b
WAN --> NAT --> LAN
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.01 sec 604 MBytes 506 Mbits/sec 8 sender
[ 5] 0.00-10.00 sec 601 MBytes 504 Mbits/sec receiver
WAN <-- NAT <-- LAN
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 578 MBytes 485 Mbits/sec 79 sender
[ 5] 0.00-10.01 sec 575 MBytes 482 Mbits/sec receiver
---
net/dsa/Kconfig | 1 +
net/dsa/dsa.c | 2 +-
net/dsa/dsa_priv.h | 3 +++
net/dsa/slave.c | 10 +++++++++-
4 files changed, 14 insertions(+), 2 deletions(-)
diff --git a/net/dsa/Kconfig b/net/dsa/Kconfig
index 29e2bd5cc5af..7dce11ab2806 100644
--- a/net/dsa/Kconfig
+++ b/net/dsa/Kconfig
@@ -9,6 +9,7 @@ menuconfig NET_DSA
tristate "Distributed Switch Architecture"
depends on HAVE_NET_DSA
depends on BRIDGE || BRIDGE=n
+ select GRO_CELLS
select NET_SWITCHDEV
select PHYLINK
select NET_DEVLINK
diff --git a/net/dsa/dsa.c b/net/dsa/dsa.c
index 43120a3fb06f..ca80f86995e6 100644
--- a/net/dsa/dsa.c
+++ b/net/dsa/dsa.c
@@ -238,7 +238,7 @@ static int dsa_switch_rcv(struct sk_buff *skb, struct net_device *dev,
if (dsa_skb_defer_rx_timestamp(p, skb))
return 0;
- netif_receive_skb(skb);
+ gro_cells_receive(&p->gcells, skb);
return 0;
}
diff --git a/net/dsa/dsa_priv.h b/net/dsa/dsa_priv.h
index bf9947c577b6..d8e850724d13 100644
--- a/net/dsa/dsa_priv.h
+++ b/net/dsa/dsa_priv.h
@@ -11,6 +11,7 @@
#include <linux/netdevice.h>
#include <linux/netpoll.h>
#include <net/dsa.h>
+#include <net/gro_cells.h>
enum {
DSA_NOTIFIER_AGEING_TIME,
@@ -68,6 +69,8 @@ struct dsa_slave_priv {
struct pcpu_sw_netstats *stats64;
+ struct gro_cells gcells;
+
/* DSA port data, such as switch, port index, etc. */
struct dsa_port *dp;
diff --git a/net/dsa/slave.c b/net/dsa/slave.c
index f734ce0bcb56..06f8874d53ee 100644
--- a/net/dsa/slave.c
+++ b/net/dsa/slave.c
@@ -1431,6 +1431,11 @@ int dsa_slave_create(struct dsa_port *port)
free_netdev(slave_dev);
return -ENOMEM;
}
+
+ ret = gro_cells_init(&p->gcells, slave_dev);
+ if (ret)
+ goto out_free;
+
p->dp = port;
INIT_LIST_HEAD(&p->mall_tc_list);
INIT_WORK(&port->xmit_work, dsa_port_xmit_work);
@@ -1443,7 +1448,7 @@ int dsa_slave_create(struct dsa_port *port)
ret = dsa_slave_phy_setup(slave_dev);
if (ret) {
netdev_err(master, "error %d setting up slave phy\n", ret);
- goto out_free;
+ goto out_gcells;
}
dsa_slave_notify(slave_dev, DSA_PORT_REGISTER);
@@ -1462,6 +1467,8 @@ int dsa_slave_create(struct dsa_port *port)
phylink_disconnect_phy(p->dp->pl);
rtnl_unlock();
phylink_destroy(p->dp->pl);
+out_gcells:
+ gro_cells_destroy(&p->gcells);
out_free:
free_percpu(p->stats64);
free_netdev(slave_dev);
@@ -1482,6 +1489,7 @@ void dsa_slave_destroy(struct net_device *slave_dev)
dsa_slave_notify(slave_dev, DSA_PORT_UNREGISTER);
unregister_netdev(slave_dev);
phylink_destroy(dp->pl);
+ gro_cells_destroy(&p->gcells);
free_percpu(p->stats64);
free_netdev(slave_dev);
}
--
2.20.1
From: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org>
I'm announcing the release of the 5.11.5 kernel.
All users of the 5.11 kernel series must upgrade.
The updated 5.11.y git tree can be found at:
git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git linux-5.11.y
and can be browsed at the normal kernel.org git web browser:
https://git.kernel.org/?p=linux/kernel/git/stable/linux-stable.git;a=summary
thanks,
greg k-h
------------
Makefile | 2
arch/ia64/kernel/signal.c | 3 -
drivers/base/power/runtime.c | 62 ++++++++++++---------
drivers/block/rsxx/core.c | 8 +-
drivers/char/tpm/tpm_tis_core.c | 30 ++++++++--
drivers/gpu/drm/amd/amdgpu/amdgpu_acpi.c | 3 -
drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c | 4 -
drivers/gpu/drm/amd/amdgpu/nv.c | 6 +-
drivers/gpu/drm/amd/pm/swsmu/smu11/smu_v11_0.c | 15 ++++-
drivers/infiniband/core/cm.c | 5 +
drivers/infiniband/hw/mlx5/devx.c | 4 +
drivers/infiniband/sw/rxe/Kconfig | 1
drivers/iommu/dma-iommu.c | 15 ++---
drivers/iommu/intel/pasid.h | 4 -
drivers/iommu/tegra-smmu.c | 72 ++++++++++++++++++++++++-
drivers/md/dm-bufio.c | 4 +
drivers/md/dm-verity-fec.c | 23 ++++---
drivers/net/ethernet/realtek/r8169_main.c | 2
fs/btrfs/block-group.c | 33 +++++++++++
fs/btrfs/block-group.h | 9 +++
fs/btrfs/ctree.h | 5 +
fs/btrfs/delayed-inode.c | 2
fs/btrfs/file.c | 5 +
fs/btrfs/free-space-cache.c | 14 ++--
fs/btrfs/inode.c | 40 ++++++++++++-
fs/btrfs/ioctl.c | 19 ++++++
fs/btrfs/raid56.c | 21 +++----
fs/btrfs/reflink.c | 18 ++++++
fs/btrfs/scrub.c | 9 ++-
fs/btrfs/super.c | 4 -
fs/btrfs/tree-checker.c | 16 +----
fs/btrfs/xattr.c | 31 +++++++++-
fs/btrfs/zoned.c | 4 -
fs/io_uring.c | 3 +
include/crypto/hash.h | 8 +-
include/linux/crypto.h | 9 ++-
include/sound/intel-nhlt.h | 5 +
kernel/trace/ring_buffer.c | 11 +++
scripts/recordmcount.c | 2
security/tomoyo/network.c | 2
sound/hda/intel-nhlt.c | 54 +++++++++++++++---
sound/pci/ctxfi/cthw20k2.c | 2
sound/pci/hda/patch_realtek.c | 13 ++++
sound/usb/clock.c | 8 +-
sound/usb/mixer.c | 11 +++
sound/usb/mixer_maps.c | 10 +++
sound/usb/pcm.c | 12 +++-
sound/usb/quirks.c | 2
48 files changed, 506 insertions(+), 139 deletions(-)
Alex Deucher (1):
drm/amdgpu: Only check for S0ix if AMD_PMC is configured
Andrea Fagiani (1):
ALSA: usb-audio: use Corsair Virtuoso mapping for Corsair Virtuoso SE
Ard Biesheuvel (1):
crypto - shash: reduce minimum alignment of shash_desc structure
Asher.Song (1):
drm/amdgpu:disable VCN for Navi12 SKU
Boris Burkov (1):
btrfs: fix spurious free_space_tree remount warning
Chen Jun (1):
ftrace: Have recordmcount use w8 to read relp->r_info in arm64_is_fake_mcount
Chris Chiu (1):
ALSA: hda/realtek: Enable headset mic of Acer SWIFT with ALC256
Colin Ian King (1):
ALSA: ctxfi: cthw20k2: fix mask on conf to allow 4 bits
Dan Carpenter (2):
btrfs: validate qgroup inherit for SNAP_CREATE_V2 ioctl
rsxx: Return -EFAULT if copy_to_user() fails
Evan Quan (1):
drm/amd/pm: correct Arcturus mmTHM_BACO_CNTL register address
Filipe Manana (4):
btrfs: fix race between writes to swap files and scrub
btrfs: fix race between swap file activation and snapshot creation
btrfs: fix stale data exposure after cloning a hole with NO_HOLES enabled
btrfs: fix warning when creating a directory with smack enabled
Greg Kroah-Hartman (1):
Linux 5.11.5
Heiner Kallweit (1):
r8169: fix resuming from suspend on RTL8105e if machine runs on battery
Ira Weiny (1):
btrfs: fix raid6 qstripe kmap
Jarkko Sakkinen (1):
tpm, tpm_tis: Decorate tpm_get_timeouts() with request_locality()
Jens Axboe (2):
io_uring: ignore double poll add on the same waitqueue head
ia64: don't call handle_signal() unless there's actually a signal queued
Josef Bacik (2):
btrfs: avoid double put of block group when emptying cluster
btrfs: tree-checker: do not error out if extent ref hash doesn't match
Julian Braha (1):
RDMA/rxe: Fix missing kconfig dependency on CRYPTO
Kevin Wang (1):
drm/amdgpu: fix parameter error of RREG32_PCIE() in amdgpu_regs_pcie
Lu Baolu (1):
iommu: Don't use lazy flush for untrusted device
Lukasz Majczak (1):
tpm, tpm_tis: Decorate tpm_tis_gen_interrupt() with request_locality()
Mikulas Patocka (1):
dm bufio: subtract the number of initial sectors in dm_bufio_get_device_size
Milan Broz (1):
dm verity: fix FEC for RS roots unaligned to block size
Naohiro Aota (1):
btrfs: zoned: use sector_t for zone sectors
Nicolas MURE (1):
ALSA: usb-audio: Fix Pioneer DJM devices URB_CONTROL request direction to set samplerate
Nicolin Chen (1):
iommu/tegra-smmu: Fix mc errors on tegra124-nyan
Nikolay Borisov (3):
btrfs: fix race between extent freeing/allocation when using bitmaps
btrfs: free correct amount of space in btrfs_delayed_inode_reserve_metadata
btrfs: unlock extents in btrfs_zero_range in case of quota reservation errors
Pierre-Louis Bossart (1):
ALSA: hda: intel-nhlt: verify config type
Rafael J. Wysocki (1):
PM: runtime: Update device status before letting suppliers suspend
Saeed Mahameed (1):
RDMA/cm: Fix IRQ restore in ib_send_cm_sidr_rep
Steven Rostedt (VMware) (1):
ring-buffer: Force before_stamp and write_stamp to be different on discard
Takashi Iwai (3):
ALSA: usb-audio: Don't abort even if the clock rate differs
ALSA: usb-audio: Drop bogus dB range in too low level
ALSA: usb-audio: Allow modifying parameters with succeeding hw_params calls
Tetsuo Handa (1):
tomoyo: recognize kernel threads correctly
YueHaibing (1):
IB/mlx5: Add missing error code
Zenghui Yu (1):
iommu/vt-d: Fix status code for Allocate/Free PASID command
From: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org>
I'm announcing the release of the 5.4.104 kernel.
All users of the 5.4 kernel series must upgrade.
The updated 5.4.y git tree can be found at:
git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git linux-5.4.y
and can be browsed at the normal kernel.org git web browser:
https://git.kernel.org/?p=linux/kernel/git/stable/linux-stable.git;a=summary
thanks,
greg k-h
------------
Makefile | 2
arch/arm64/kernel/ptrace.c | 2
drivers/base/power/runtime.c | 62 ++++++++++++++++------------
drivers/block/rsxx/core.c | 8 ++-
drivers/char/tpm/tpm_tis_core.c | 30 +++++++++++--
drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c | 4 -
drivers/infiniband/hw/mlx5/devx.c | 4 +
drivers/infiniband/sw/rxe/Kconfig | 1
drivers/md/dm-bufio.c | 4 +
drivers/md/dm-verity-fec.c | 23 +++++-----
drivers/net/ethernet/realtek/r8169_main.c | 2
fs/btrfs/delayed-inode.c | 2
fs/btrfs/file.c | 5 +-
fs/btrfs/ioctl.c | 19 ++++++++
fs/btrfs/raid56.c | 58 +++++++++++---------------
fs/btrfs/xattr.c | 31 ++++++++++++--
include/crypto/hash.h | 8 +--
include/linux/crypto.h | 9 ++--
include/sound/intel-nhlt.h | 5 ++
scripts/recordmcount.c | 2
sound/hda/intel-nhlt.c | 54 ++++++++++++++++++++----
sound/pci/ctxfi/cthw20k2.c | 2
tools/usb/usbip/libsrc/usbip_host_common.c | 2
23 files changed, 231 insertions(+), 108 deletions(-)
Antonio Borneo (1):
usbip: tools: fix build error for multiple definition
Ard Biesheuvel (1):
crypto - shash: reduce minimum alignment of shash_desc structure
Chen Jun (1):
ftrace: Have recordmcount use w8 to read relp->r_info in arm64_is_fake_mcount
Colin Ian King (1):
ALSA: ctxfi: cthw20k2: fix mask on conf to allow 4 bits
Dan Carpenter (2):
btrfs: validate qgroup inherit for SNAP_CREATE_V2 ioctl
rsxx: Return -EFAULT if copy_to_user() fails
David Sterba (1):
btrfs: raid56: simplify tracking of Q stripe presence
Filipe Manana (1):
btrfs: fix warning when creating a directory with smack enabled
Greg Kroah-Hartman (1):
Linux 5.4.104
Heiner Kallweit (1):
r8169: fix resuming from suspend on RTL8105e if machine runs on battery
Ira Weiny (1):
btrfs: fix raid6 qstripe kmap
Jarkko Sakkinen (1):
tpm, tpm_tis: Decorate tpm_get_timeouts() with request_locality()
Julian Braha (1):
RDMA/rxe: Fix missing kconfig dependency on CRYPTO
Kevin Wang (1):
drm/amdgpu: fix parameter error of RREG32_PCIE() in amdgpu_regs_pcie
Lukasz Majczak (1):
tpm, tpm_tis: Decorate tpm_tis_gen_interrupt() with request_locality()
Mikulas Patocka (1):
dm bufio: subtract the number of initial sectors in dm_bufio_get_device_size
Milan Broz (1):
dm verity: fix FEC for RS roots unaligned to block size
Nikolay Borisov (2):
btrfs: free correct amount of space in btrfs_delayed_inode_reserve_metadata
btrfs: unlock extents in btrfs_zero_range in case of quota reservation errors
Pierre-Louis Bossart (1):
ALSA: hda: intel-nhlt: verify config type
Rafael J. Wysocki (1):
PM: runtime: Update device status before letting suppliers suspend
Timothy E Baldwin (1):
arm64: ptrace: Fix seccomp of traced syscall -1 (NO_SYSCALL)
YueHaibing (1):
IB/mlx5: Add missing error code