[PATCH net v3 0/4] Fix isolation of broadcast traffic and unmatched unicast traffic with MACsec offload
Some device drivers support devices that enable them to annotate whether a Rx skb refers to a packet that was processed by the MACsec offloading functionality of the device. Logic in the Rx handling for MACsec offload does not utilize this information to preemptively avoid forwarding to the macsec netdev currently. Because of this, things like multicast messages or unicast messages with an unmatched destination address such as ARP requests are forwarded to the macsec netdev whether the message received was MACsec encrypted or not. The goal of this patch series is to improve the Rx handling for MACsec offload for devices capable of annotating skbs received that were decrypted by the NIC offload for MACsec.
Here is a summary of the issue that occurs with the existing logic today.
* The current design of the MACsec offload handling path tries to use "best guess" mechanisms for determining whether a packet associated with the currently handled skb in the datapath was processed via HW offload * The best guess mechanism uses the following heuristic logic (in order of precedence) - Check if header destination MAC address matches MACsec netdev MAC address -> forward to MACsec port - Check if packet is multicast traffic -> forward to MACsec port - MACsec security channel was able to be looked up from skb offload context (mlx5 only) -> forward to MACsec port * Problem: plaintext traffic can potentially solicit a MACsec encrypted response from the offload device - Core aspect of MACsec is that it identifies unauthorized LAN connections and excludes them from communication + This behavior can be seen when not enabling offload for MACsec - The offload behavior violates this principle in MACsec
I believe this behavior is a security bug since applications utilizing MACsec could be exploited using this behavior, and the correct way to resolve this is by having the hardware correctly indicate whether MACsec offload occurred for the packet or not. In the patches in this series, I leave a warning for when the problematic path occurs because I cannot figure out a secure way to fix the security issue that applies to the core MACsec offload handling in the Rx path without breaking MACsec offload for other vendors.
Shown at the bottom is an example use case where plaintext traffic sent to a physical port of a NIC configured for MACsec offload is unable to be handled correctly by the software stack when the NIC provides awareness to the kernel about whether the received packet is MACsec traffic or not. In this specific example, plaintext ARP requests are being responded with MACsec encrypted ARP replies (which leads to routing information being unable to be built for the requester).
Side 1
ip link del macsec0 ip address flush mlx5_1 ip address add 1.1.1.1/24 dev mlx5_1 ip link set dev mlx5_1 up ip link add link mlx5_1 macsec0 type macsec sci 1 encrypt on ip link set dev macsec0 address 00:11:22:33:44:66 ip macsec offload macsec0 mac ip macsec add macsec0 tx sa 0 pn 1 on key 00 dffafc8d7b9a43d5b9a3dfbbf6a30c16 ip macsec add macsec0 rx sci 2 on ip macsec add macsec0 rx sci 2 sa 0 pn 1 on key 00 ead3664f508eb06c40ac7104cdae4ce5 ip address flush macsec0 ip address add 2.2.2.1/24 dev macsec0 ip link set dev macsec0 up
# macsec0 enters promiscuous mode. # This enables all traffic received on macsec_vlan to be processed by # the macsec offload rx datapath. This however means that traffic # meant to be received by mlx5_1 will be incorrectly steered to # macsec0 as well.
ip link add link macsec0 name macsec_vlan type vlan id 1 ip link set dev macsec_vlan address 00:11:22:33:44:88 ip address flush macsec_vlan ip address add 3.3.3.1/24 dev macsec_vlan ip link set dev macsec_vlan up
Side 2
ip link del macsec0 ip address flush mlx5_1 ip address add 1.1.1.2/24 dev mlx5_1 ip link set dev mlx5_1 up ip link add link mlx5_1 macsec0 type macsec sci 2 encrypt on ip link set dev macsec0 address 00:11:22:33:44:77 ip macsec offload macsec0 mac ip macsec add macsec0 tx sa 0 pn 1 on key 00 ead3664f508eb06c40ac7104cdae4ce5 ip macsec add macsec0 rx sci 1 on ip macsec add macsec0 rx sci 1 sa 0 pn 1 on key 00 dffafc8d7b9a43d5b9a3dfbbf6a30c16 ip address flush macsec0 ip address add 2.2.2.2/24 dev macsec0 ip link set dev macsec0 up
# macsec0 enters promiscuous mode. # This enables all traffic received on macsec_vlan to be processed by # the macsec offload rx datapath. This however means that traffic # meant to be received by mlx5_1 will be incorrectly steered to # macsec0 as well.
ip link add link macsec0 name macsec_vlan type vlan id 1 ip link set dev macsec_vlan address 00:11:22:33:44:99 ip address flush macsec_vlan ip address add 3.3.3.2/24 dev macsec_vlan ip link set dev macsec_vlan up
Side 1
ping -I mlx5_1 1.1.1.2 PING 1.1.1.2 (1.1.1.2) from 1.1.1.1 mlx5_1: 56(84) bytes of data. From 1.1.1.1 icmp_seq=1 Destination Host Unreachable ping: sendmsg: No route to host From 1.1.1.1 icmp_seq=2 Destination Host Unreachable From 1.1.1.1 icmp_seq=3 Destination Host Unreachable
Changes:
v2->v3: * Made dev paramater const for eth_skb_pkt_type helper as suggested by Sabrina Dubroca sd@queasysnail.net v1->v2: * Fixed series subject to detail the issue being fixed * Removed strange characters from cover letter * Added comment in example that illustrates the impact involving promiscuous mode * Added patch for generalizing packet type detection * Added Fixes: tags and targeting net * Removed pointless warning in the heuristic Rx path for macsec offload * Applied small refactor in Rx path offload to minimize scope of rx_sc local variable
Link: https://github.com/Binary-Eater/macsec-rx-offload/blob/trunk/MACsec_violatio... Link: https://lore.kernel.org/netdev/20240419213033.400467-5-rrameshbabu@nvidia.co... Link: https://lore.kernel.org/netdev/20240419011740.333714-1-rrameshbabu@nvidia.co... Link: https://lore.kernel.org/netdev/87r0l25y1c.fsf@nvidia.com/ Link: https://lore.kernel.org/netdev/20231116182900.46052-1-rrameshbabu@nvidia.com... Cc: Sabrina Dubroca sd@queasysnail.net Cc: stable@vger.kernel.org Signed-off-by: Rahul Rameshbabu rrameshbabu@nvidia.com --- Rahul Rameshbabu (4): macsec: Enable devices to advertise whether they update sk_buff md_dst during offloads ethernet: Add helper for assigning packet type when dest address does not match device address macsec: Detect if Rx skb is macsec-related for offloading devices that update md_dst net/mlx5e: Advertise mlx5 ethernet driver updates sk_buff md_dst for MACsec
.../mellanox/mlx5/core/en_accel/macsec.c | 1 + drivers/net/macsec.c | 46 +++++++++++++++---- include/linux/etherdevice.h | 25 ++++++++++ include/net/macsec.h | 2 + net/ethernet/eth.c | 12 +---- 5 files changed, 65 insertions(+), 21 deletions(-)
Cannot know whether a Rx skb missing md_dst is intended for MACsec or not without knowing whether the device is able to update this field during an offload. Assume that an offload to a MACsec device cannot support updating md_dst by default. Capable devices can advertise that they do indicate that an skb is related to a MACsec offloaded packet using the md_dst.
Cc: Sabrina Dubroca sd@queasysnail.net Cc: stable@vger.kernel.org Fixes: 860ead89b851 ("net/macsec: Add MACsec skb_metadata_dst Rx Data path support") Signed-off-by: Rahul Rameshbabu rrameshbabu@nvidia.com Reviewed-by: Benjamin Poirier bpoirier@nvidia.com Reviewed-by: Cosmin Ratiu cratiu@nvidia.com --- include/net/macsec.h | 2 ++ 1 file changed, 2 insertions(+)
diff --git a/include/net/macsec.h b/include/net/macsec.h index dbd22180cc5c..de216cbc6b05 100644 --- a/include/net/macsec.h +++ b/include/net/macsec.h @@ -321,6 +321,7 @@ struct macsec_context { * for the TX tag * @needed_tailroom: number of bytes reserved at the end of the sk_buff for the * TX tag + * @rx_uses_md_dst: whether MACsec device offload supports sk_buff md_dst */ struct macsec_ops { /* Device wide */ @@ -352,6 +353,7 @@ struct macsec_ops { struct sk_buff *skb); unsigned int needed_headroom; unsigned int needed_tailroom; + bool rx_uses_md_dst; };
void macsec_pn_wrapped(struct macsec_secy *secy, struct macsec_tx_sa *tx_sa);
Enable reuse of logic in eth_type_trans for determining packet type.
Suggested-by: Sabrina Dubroca sd@queasysnail.net Cc: stable@vger.kernel.org Signed-off-by: Rahul Rameshbabu rrameshbabu@nvidia.com --- include/linux/etherdevice.h | 25 +++++++++++++++++++++++++ net/ethernet/eth.c | 12 +----------- 2 files changed, 26 insertions(+), 11 deletions(-)
diff --git a/include/linux/etherdevice.h b/include/linux/etherdevice.h index 224645f17c33..297231854ada 100644 --- a/include/linux/etherdevice.h +++ b/include/linux/etherdevice.h @@ -607,6 +607,31 @@ static inline void eth_hw_addr_gen(struct net_device *dev, const u8 *base_addr, eth_hw_addr_set(dev, addr); }
+/** + * eth_skb_pkt_type - Assign packet type if destination address does not match + * @skb: Assigned a packet type if address does not match @dev address + * @dev: Network device used to compare packet address against + * + * If the destination MAC address of the packet does not match the network + * device address, assign an appropriate packet type. + */ +static inline void eth_skb_pkt_type(struct sk_buff *skb, + const struct net_device *dev) +{ + const struct ethhdr *eth = eth_hdr(skb); + + if (unlikely(!ether_addr_equal_64bits(eth->h_dest, dev->dev_addr))) { + if (unlikely(is_multicast_ether_addr_64bits(eth->h_dest))) { + if (ether_addr_equal_64bits(eth->h_dest, dev->broadcast)) + skb->pkt_type = PACKET_BROADCAST; + else + skb->pkt_type = PACKET_MULTICAST; + } else { + skb->pkt_type = PACKET_OTHERHOST; + } + } +} + /** * eth_skb_pad - Pad buffer to mininum number of octets for Ethernet frame * @skb: Buffer to pad diff --git a/net/ethernet/eth.c b/net/ethernet/eth.c index 2edc8b796a4e..049c3adeb850 100644 --- a/net/ethernet/eth.c +++ b/net/ethernet/eth.c @@ -164,17 +164,7 @@ __be16 eth_type_trans(struct sk_buff *skb, struct net_device *dev) eth = (struct ethhdr *)skb->data; skb_pull_inline(skb, ETH_HLEN);
- if (unlikely(!ether_addr_equal_64bits(eth->h_dest, - dev->dev_addr))) { - if (unlikely(is_multicast_ether_addr_64bits(eth->h_dest))) { - if (ether_addr_equal_64bits(eth->h_dest, dev->broadcast)) - skb->pkt_type = PACKET_BROADCAST; - else - skb->pkt_type = PACKET_MULTICAST; - } else { - skb->pkt_type = PACKET_OTHERHOST; - } - } + eth_skb_pkt_type(skb, dev);
/* * Some variants of DSA tagging don't have an ethertype field
Can now correctly identify where the packets should be delivered by using md_dst or its absence on devices that provide it.
This detection is not possible without device drivers that update md_dst. A fallback pattern should be used for supporting such device drivers. This fallback mode causes multicast messages to be cloned to both the non-macsec and macsec ports, independent of whether the multicast message received was encrypted over MACsec or not. Other non-macsec traffic may also fail to be handled correctly for devices in promiscuous mode.
Link: https://lore.kernel.org/netdev/ZULRxX9eIbFiVi7v@hog/ Cc: Sabrina Dubroca sd@queasysnail.net Cc: stable@vger.kernel.org Fixes: 860ead89b851 ("net/macsec: Add MACsec skb_metadata_dst Rx Data path support") Signed-off-by: Rahul Rameshbabu rrameshbabu@nvidia.com Reviewed-by: Benjamin Poirier bpoirier@nvidia.com Reviewed-by: Cosmin Ratiu cratiu@nvidia.com --- drivers/net/macsec.c | 46 ++++++++++++++++++++++++++++++++++---------- 1 file changed, 36 insertions(+), 10 deletions(-)
diff --git a/drivers/net/macsec.c b/drivers/net/macsec.c index 0206b84284ab..ff016c11b4a0 100644 --- a/drivers/net/macsec.c +++ b/drivers/net/macsec.c @@ -999,10 +999,12 @@ static enum rx_handler_result handle_not_macsec(struct sk_buff *skb) struct metadata_dst *md_dst; struct macsec_rxh_data *rxd; struct macsec_dev *macsec; + bool is_macsec_md_dst;
rcu_read_lock(); rxd = macsec_data_rcu(skb->dev); md_dst = skb_metadata_dst(skb); + is_macsec_md_dst = md_dst && md_dst->type == METADATA_MACSEC;
list_for_each_entry_rcu(macsec, &rxd->secys, secys) { struct sk_buff *nskb; @@ -1013,14 +1015,42 @@ static enum rx_handler_result handle_not_macsec(struct sk_buff *skb) * the SecTAG, so we have to deduce which port to deliver to. */ if (macsec_is_offloaded(macsec) && netif_running(ndev)) { - struct macsec_rx_sc *rx_sc = NULL; + const struct macsec_ops *ops;
- if (md_dst && md_dst->type == METADATA_MACSEC) - rx_sc = find_rx_sc(&macsec->secy, md_dst->u.macsec_info.sci); + ops = macsec_get_ops(macsec, NULL);
- if (md_dst && md_dst->type == METADATA_MACSEC && !rx_sc) + if (ops->rx_uses_md_dst && !is_macsec_md_dst) continue;
+ if (is_macsec_md_dst) { + struct macsec_rx_sc *rx_sc; + + /* All drivers that implement MACsec offload + * support using skb metadata destinations must + * indicate that they do so. + */ + DEBUG_NET_WARN_ON_ONCE(!ops->rx_uses_md_dst); + rx_sc = find_rx_sc(&macsec->secy, + md_dst->u.macsec_info.sci); + if (!rx_sc) + continue; + /* device indicated macsec offload occurred */ + skb->dev = ndev; + skb->pkt_type = PACKET_HOST; + eth_skb_pkt_type(skb, ndev); + ret = RX_HANDLER_ANOTHER; + goto out; + } + + /* This datapath is insecure because it is unable to + * enforce isolation of broadcast/multicast traffic and + * unicast traffic with promiscuous mode on the macsec + * netdev. Since the core stack has no mechanism to + * check that the hardware did indeed receive MACsec + * traffic, it is possible that the response handling + * done by the MACsec port was to a plaintext packet. + * This violates the MACsec protocol standard. + */ if (ether_addr_equal_64bits(hdr->h_dest, ndev->dev_addr)) { /* exact match, divert skb to this port */ @@ -1036,14 +1066,10 @@ static enum rx_handler_result handle_not_macsec(struct sk_buff *skb) break;
nskb->dev = ndev; - if (ether_addr_equal_64bits(hdr->h_dest, - ndev->broadcast)) - nskb->pkt_type = PACKET_BROADCAST; - else - nskb->pkt_type = PACKET_MULTICAST; + eth_skb_pkt_type(nskb, ndev);
__netif_rx(nskb); - } else if (rx_sc || ndev->flags & IFF_PROMISC) { + } else if (ndev->flags & IFF_PROMISC) { skb->dev = ndev; skb->pkt_type = PACKET_HOST; ret = RX_HANDLER_ANOTHER;
mlx5 Rx flow steering and CQE handling enable the driver to be able to update an skb's md_dst attribute as MACsec when MACsec traffic arrives when a device is configured for offloading. Advertise this to the core stack to take advantage of this capability.
Cc: stable@vger.kernel.org Fixes: b7c9400cbc48 ("net/mlx5e: Implement MACsec Rx data path using MACsec skb_metadata_dst") Signed-off-by: Rahul Rameshbabu rrameshbabu@nvidia.com Reviewed-by: Benjamin Poirier bpoirier@nvidia.com Reviewed-by: Cosmin Ratiu cratiu@nvidia.com --- drivers/net/ethernet/mellanox/mlx5/core/en_accel/macsec.c | 1 + 1 file changed, 1 insertion(+)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/macsec.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/macsec.c index b2cabd6ab86c..cc9bcc420032 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/macsec.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/macsec.c @@ -1640,6 +1640,7 @@ static const struct macsec_ops macsec_offload_ops = { .mdo_add_secy = mlx5e_macsec_add_secy, .mdo_upd_secy = mlx5e_macsec_upd_secy, .mdo_del_secy = mlx5e_macsec_del_secy, + .rx_uses_md_dst = true, };
bool mlx5e_macsec_handle_tx_skb(struct mlx5e_macsec *macsec, struct sk_buff *skb)
2024-04-23, 11:13:01 -0700, Rahul Rameshbabu wrote:
Rahul Rameshbabu (4): macsec: Enable devices to advertise whether they update sk_buff md_dst during offloads ethernet: Add helper for assigning packet type when dest address does not match device address macsec: Detect if Rx skb is macsec-related for offloading devices that update md_dst net/mlx5e: Advertise mlx5 ethernet driver updates sk_buff md_dst for MACsec
.../mellanox/mlx5/core/en_accel/macsec.c | 1 + drivers/net/macsec.c | 46 +++++++++++++++---- include/linux/etherdevice.h | 25 ++++++++++ include/net/macsec.h | 2 + net/ethernet/eth.c | 12 +---- 5 files changed, 65 insertions(+), 21 deletions(-)
Thanks Rahul.
Series: Reviewed-by: Sabrina Dubroca sd@queasysnail.net
Hello:
This series was applied to netdev/net.git (main) by Jakub Kicinski kuba@kernel.org:
On Tue, 23 Apr 2024 11:13:01 -0700 you wrote:
Some device drivers support devices that enable them to annotate whether a Rx skb refers to a packet that was processed by the MACsec offloading functionality of the device. Logic in the Rx handling for MACsec offload does not utilize this information to preemptively avoid forwarding to the macsec netdev currently. Because of this, things like multicast messages or unicast messages with an unmatched destination address such as ARP requests are forwarded to the macsec netdev whether the message received was MACsec encrypted or not. The goal of this patch series is to improve the Rx handling for MACsec offload for devices capable of annotating skbs received that were decrypted by the NIC offload for MACsec.
[...]
Here is the summary with links: - [net,v3,1/4] macsec: Enable devices to advertise whether they update sk_buff md_dst during offloads https://git.kernel.org/netdev/net/c/475747a19316 - [net,v3,2/4] ethernet: Add helper for assigning packet type when dest address does not match device address https://git.kernel.org/netdev/net/c/6e159fd653d7 - [net,v3,3/4] macsec: Detect if Rx skb is macsec-related for offloading devices that update md_dst https://git.kernel.org/netdev/net/c/642c984dd0e3 - [net,v3,4/4] net/mlx5e: Advertise mlx5 ethernet driver updates sk_buff md_dst for MACsec https://git.kernel.org/netdev/net/c/39d26a8f2efc
You are awesome, thank you!
linux-stable-mirror@lists.linaro.org
-
patchwork-bot+netdevbpf＠kernel.org -
Rahul Rameshbabu -
Sabrina Dubroca