igc_configure_rx_ring() function will be called as part of XDP program setup. If Rx hardware timestamp is enabled prio to XDP program setup, this timestamp enablement will be overwritten when buffer size is written into SRRCTL register.
Thus, this commit read the register value before write to SRRCTL register. This commit is tested by using xdp_hw_metadata bpf selftest tool. The tool enables Rx hardware timestamp and then attach XDP program to igc driver. It will display hardware timestamp of UDP packet with port number 9092. Below are detail of test steps and results.
Command on DUT: sudo ./xdp_hw_metadata <interface name>
Command on Link Partner: echo -n skb | nc -u -q1 <destination IPv4 addr> 9092
Result before this patch: skb hwtstamp is not found!
Result after this patch: found skb hwtstamp = 1677762212.590696226
Fixes: fc9df2a0b520 ("igc: Enable RX via AF_XDP zero-copy") Cc: stable@vger.kernel.org # 5.14+ Signed-off-by: Song Yoong Siang yoong.siang.song@intel.com --- drivers/net/ethernet/intel/igc/igc_base.h | 7 +++++-- drivers/net/ethernet/intel/igc/igc_main.c | 5 ++++- 2 files changed, 9 insertions(+), 3 deletions(-)
diff --git a/drivers/net/ethernet/intel/igc/igc_base.h b/drivers/net/ethernet/intel/igc/igc_base.h index 7a992befca24..b95007d51d13 100644 --- a/drivers/net/ethernet/intel/igc/igc_base.h +++ b/drivers/net/ethernet/intel/igc/igc_base.h @@ -87,8 +87,11 @@ union igc_adv_rx_desc { #define IGC_RXDCTL_SWFLUSH 0x04000000 /* Receive Software Flush */
/* SRRCTL bit definitions */ -#define IGC_SRRCTL_BSIZEPKT_SHIFT 10 /* Shift _right_ */ -#define IGC_SRRCTL_BSIZEHDRSIZE_SHIFT 2 /* Shift _left_ */ +#define IGC_SRRCTL_BSIZEPKT_MASK GENMASK(6, 0) +#define IGC_SRRCTL_BSIZEPKT_SHIFT 10 /* Shift _right_ */ +#define IGC_SRRCTL_BSIZEHDRSIZE_MASK GENMASK(13, 8) +#define IGC_SRRCTL_BSIZEHDRSIZE_SHIFT 2 /* Shift _left_ */ +#define IGC_SRRCTL_DESCTYPE_MASK GENMASK(27, 25) #define IGC_SRRCTL_DESCTYPE_ADV_ONEBUF 0x02000000
#endif /* _IGC_BASE_H */ diff --git a/drivers/net/ethernet/intel/igc/igc_main.c b/drivers/net/ethernet/intel/igc/igc_main.c index 25fc6c65209b..de7b21c2ccd6 100644 --- a/drivers/net/ethernet/intel/igc/igc_main.c +++ b/drivers/net/ethernet/intel/igc/igc_main.c @@ -641,7 +641,10 @@ static void igc_configure_rx_ring(struct igc_adapter *adapter, else buf_size = IGC_RXBUFFER_2048;
- srrctl = IGC_RX_HDR_LEN << IGC_SRRCTL_BSIZEHDRSIZE_SHIFT; + srrctl = rd32(IGC_SRRCTL(reg_idx)); + srrctl &= ~(IGC_SRRCTL_BSIZEPKT_MASK | IGC_SRRCTL_BSIZEHDRSIZE_MASK | + IGC_SRRCTL_DESCTYPE_MASK); + srrctl |= IGC_RX_HDR_LEN << IGC_SRRCTL_BSIZEHDRSIZE_SHIFT; srrctl |= buf_size >> IGC_SRRCTL_BSIZEPKT_SHIFT; srrctl |= IGC_SRRCTL_DESCTYPE_ADV_ONEBUF;
On 4/13/2023 8:12 AM, Song Yoong Siang wrote:
igc_configure_rx_ring() function will be called as part of XDP program setup. If Rx hardware timestamp is enabled prio to XDP program setup, this timestamp enablement will be overwritten when buffer size is written into SRRCTL register.
Thus, this commit read the register value before write to SRRCTL register. This commit is tested by using xdp_hw_metadata bpf selftest tool. The tool enables Rx hardware timestamp and then attach XDP program to igc driver. It will display hardware timestamp of UDP packet with port number 9092. Below are detail of test steps and results.
Command on DUT: sudo ./xdp_hw_metadata <interface name>
Command on Link Partner: echo -n skb | nc -u -q1 <destination IPv4 addr> 9092
Result before this patch: skb hwtstamp is not found!
Result after this patch: found skb hwtstamp = 1677762212.590696226
Fixes: fc9df2a0b520 ("igc: Enable RX via AF_XDP zero-copy") Cc: stable@vger.kernel.org # 5.14+ Signed-off-by: Song Yoong Siang yoong.siang.song@intel.com
Reviewed-by: Jacob Keller jacob.e.keller@intel.com
On 13/04/2023 17.12, Song Yoong Siang wrote:
igc_configure_rx_ring() function will be called as part of XDP program setup. If Rx hardware timestamp is enabled prio to XDP program setup, this timestamp enablement will be overwritten when buffer size is written into SRRCTL register.
Ah, I believe I have hit this bug with my igc patches. Thanks for fixing.
Thus, this commit read the register value before write to SRRCTL register. This commit is tested by using xdp_hw_metadata bpf selftest tool. The tool enables Rx hardware timestamp and then attach XDP program to igc driver. It will display hardware timestamp of UDP packet with port number 9092. Below are detail of test steps and results.
Command on DUT: sudo ./xdp_hw_metadata <interface name>
Why port 9092 ? The ./xdp_hw_metadata prog will redirect port 9091
Command on Link Partner: echo -n skb | nc -u -q1 <destination IPv4 addr> 9092
Again port 9092 ?
Result before this patch: skb hwtstamp is not found!
Result after this patch: found skb hwtstamp = 1677762212.590696226
I usually use this cmd to see if number is sane:
$ date -d @1677762212 2023-03-02T14:03:32 CET
Fixes: fc9df2a0b520 ("igc: Enable RX via AF_XDP zero-copy") Cc: stable@vger.kernel.org # 5.14+ Signed-off-by: Song Yoong Siang yoong.siang.song@intel.com
drivers/net/ethernet/intel/igc/igc_base.h | 7 +++++-- drivers/net/ethernet/intel/igc/igc_main.c | 5 ++++- 2 files changed, 9 insertions(+), 3 deletions(-)
diff --git a/drivers/net/ethernet/intel/igc/igc_base.h b/drivers/net/ethernet/intel/igc/igc_base.h index 7a992befca24..b95007d51d13 100644 --- a/drivers/net/ethernet/intel/igc/igc_base.h +++ b/drivers/net/ethernet/intel/igc/igc_base.h @@ -87,8 +87,11 @@ union igc_adv_rx_desc { #define IGC_RXDCTL_SWFLUSH 0x04000000 /* Receive Software Flush */ /* SRRCTL bit definitions */ -#define IGC_SRRCTL_BSIZEPKT_SHIFT 10 /* Shift _right_ */ -#define IGC_SRRCTL_BSIZEHDRSIZE_SHIFT 2 /* Shift _left_ */ +#define IGC_SRRCTL_BSIZEPKT_MASK GENMASK(6, 0) +#define IGC_SRRCTL_BSIZEPKT_SHIFT 10 /* Shift _right_ */ +#define IGC_SRRCTL_BSIZEHDRSIZE_MASK GENMASK(13, 8) +#define IGC_SRRCTL_BSIZEHDRSIZE_SHIFT 2 /* Shift _left_ */ +#define IGC_SRRCTL_DESCTYPE_MASK GENMASK(27, 25) #define IGC_SRRCTL_DESCTYPE_ADV_ONEBUF 0x02000000 #endif /* _IGC_BASE_H */ diff --git a/drivers/net/ethernet/intel/igc/igc_main.c b/drivers/net/ethernet/intel/igc/igc_main.c index 25fc6c65209b..de7b21c2ccd6 100644 --- a/drivers/net/ethernet/intel/igc/igc_main.c +++ b/drivers/net/ethernet/intel/igc/igc_main.c @@ -641,7 +641,10 @@ static void igc_configure_rx_ring(struct igc_adapter *adapter, else buf_size = IGC_RXBUFFER_2048;
- srrctl = IGC_RX_HDR_LEN << IGC_SRRCTL_BSIZEHDRSIZE_SHIFT;
- srrctl = rd32(IGC_SRRCTL(reg_idx));
- srrctl &= ~(IGC_SRRCTL_BSIZEPKT_MASK | IGC_SRRCTL_BSIZEHDRSIZE_MASK |
IGC_SRRCTL_DESCTYPE_MASK);- srrctl |= IGC_RX_HDR_LEN << IGC_SRRCTL_BSIZEHDRSIZE_SHIFT; srrctl |= buf_size >> IGC_SRRCTL_BSIZEPKT_SHIFT; srrctl |= IGC_SRRCTL_DESCTYPE_ADV_ONEBUF;
On Friday, April 14, 2023 2:42 AM, Jesper Dangaard Brouer jbrouer@redhat.com wrote:
On 13/04/2023 17.12, Song Yoong Siang wrote:
igc_configure_rx_ring() function will be called as part of XDP program setup. If Rx hardware timestamp is enabled prio to XDP program setup, this timestamp enablement will be overwritten when buffer size is written into SRRCTL register.
Ah, I believe I have hit this bug with my igc patches. Thanks for fixing.
No problem. I found it when testing your patches too.
Thus, this commit read the register value before write to SRRCTL register. This commit is tested by using xdp_hw_metadata bpf selftest tool. The tool enables Rx hardware timestamp and then attach XDP program to igc driver. It will display hardware timestamp of UDP packet with port number 9092. Below are detail of test steps and results.
Command on DUT: sudo ./xdp_hw_metadata <interface name>
Why port 9092 ? The ./xdp_hw_metadata prog will redirect port 9091
Yes, you are right. But this patch is tested without your patches. So, igc Rx XDP metadata support is not there. We only can test XDP_PASS which the timestamp is put into skb. xdp_hw_metadata tool will open a SOCK_DGRAM server on port 9092 to dump out timestamp in skb.
I use xdp_hw_metadata tool to test because it can reproduce the issue well. This issue happens only when we enable hw timestamp before attach XDP prog. No issue if enable hw timestamp after attach XDP prog.
Command on Link Partner: echo -n skb | nc -u -q1 <destination IPv4 addr> 9092
Again port 9092 ?
Result before this patch: skb hwtstamp is not found!
Result after this patch: found skb hwtstamp = 1677762212.590696226
I usually use this cmd to see if number is sane:
$ date -d @1677762212 2023-03-02T14:03:32 CET
Since this patch is focusing on hw timestamp enablement. So I think checking on whether Rx HW timestamp is there when rx_filter = HWTSTAMP_FILTER_ALL is good enough.
Thanks & Regards Siang
Fixes: fc9df2a0b520 ("igc: Enable RX via AF_XDP zero-copy") Cc: stable@vger.kernel.org # 5.14+ Signed-off-by: Song Yoong Siang yoong.siang.song@intel.com
drivers/net/ethernet/intel/igc/igc_base.h | 7 +++++-- drivers/net/ethernet/intel/igc/igc_main.c | 5 ++++- 2 files changed, 9 insertions(+), 3 deletions(-)
diff --git a/drivers/net/ethernet/intel/igc/igc_base.h b/drivers/net/ethernet/intel/igc/igc_base.h index 7a992befca24..b95007d51d13 100644 --- a/drivers/net/ethernet/intel/igc/igc_base.h +++ b/drivers/net/ethernet/intel/igc/igc_base.h @@ -87,8 +87,11 @@ union igc_adv_rx_desc { #define IGC_RXDCTL_SWFLUSH 0x04000000 /* Receive
Software Flush */
/* SRRCTL bit definitions */ -#define IGC_SRRCTL_BSIZEPKT_SHIFT 10 /* Shift _right_ */ -#define IGC_SRRCTL_BSIZEHDRSIZE_SHIFT 2 /* Shift _left_ */ +#define IGC_SRRCTL_BSIZEPKT_MASK GENMASK(6, 0) +#define IGC_SRRCTL_BSIZEPKT_SHIFT 10 /* Shift _right_ */ +#define IGC_SRRCTL_BSIZEHDRSIZE_MASK GENMASK(13, 8) +#define IGC_SRRCTL_BSIZEHDRSIZE_SHIFT 2 /* Shift _left_ */ +#define IGC_SRRCTL_DESCTYPE_MASK GENMASK(27, 25) #define IGC_SRRCTL_DESCTYPE_ADV_ONEBUF 0x02000000
#endif /* _IGC_BASE_H */ diff --git a/drivers/net/ethernet/intel/igc/igc_main.c b/drivers/net/ethernet/intel/igc/igc_main.c index 25fc6c65209b..de7b21c2ccd6 100644 --- a/drivers/net/ethernet/intel/igc/igc_main.c +++ b/drivers/net/ethernet/intel/igc/igc_main.c @@ -641,7 +641,10 @@ static void igc_configure_rx_ring(struct igc_adapter
*adapter,
else buf_size = IGC_RXBUFFER_2048;
- srrctl = IGC_RX_HDR_LEN << IGC_SRRCTL_BSIZEHDRSIZE_SHIFT;
- srrctl = rd32(IGC_SRRCTL(reg_idx));
- srrctl &= ~(IGC_SRRCTL_BSIZEPKT_MASK |
IGC_SRRCTL_BSIZEHDRSIZE_MASK |
IGC_SRRCTL_DESCTYPE_MASK);- srrctl |= IGC_RX_HDR_LEN << IGC_SRRCTL_BSIZEHDRSIZE_SHIFT; srrctl |= buf_size >> IGC_SRRCTL_BSIZEPKT_SHIFT; srrctl |= IGC_SRRCTL_DESCTYPE_ADV_ONEBUF;
On 13/04/2023 17.12, Song Yoong Siang wrote:
igc_configure_rx_ring() function will be called as part of XDP program setup. If Rx hardware timestamp is enabled prio to XDP program setup, this timestamp enablement will be overwritten when buffer size is written into SRRCTL register.
Thus, this commit read the register value before write to SRRCTL register. This commit is tested by using xdp_hw_metadata bpf selftest tool. The tool enables Rx hardware timestamp and then attach XDP program to igc driver. It will display hardware timestamp of UDP packet with port number 9092. Below are detail of test steps and results.
[...]
diff --git a/drivers/net/ethernet/intel/igc/igc_base.h b/drivers/net/ethernet/intel/igc/igc_base.h index 7a992befca24..b95007d51d13 100644 --- a/drivers/net/ethernet/intel/igc/igc_base.h +++ b/drivers/net/ethernet/intel/igc/igc_base.h @@ -87,8 +87,11 @@ union igc_adv_rx_desc { #define IGC_RXDCTL_SWFLUSH 0x04000000 /* Receive Software Flush */ /* SRRCTL bit definitions */ -#define IGC_SRRCTL_BSIZEPKT_SHIFT 10 /* Shift _right_ */ -#define IGC_SRRCTL_BSIZEHDRSIZE_SHIFT 2 /* Shift _left_ */ +#define IGC_SRRCTL_BSIZEPKT_MASK GENMASK(6, 0) +#define IGC_SRRCTL_BSIZEPKT_SHIFT 10 /* Shift _right_ */ +#define IGC_SRRCTL_BSIZEHDRSIZE_MASK GENMASK(13, 8) +#define IGC_SRRCTL_BSIZEHDRSIZE_SHIFT 2 /* Shift _left_ */ +#define IGC_SRRCTL_DESCTYPE_MASK GENMASK(27, 25) #define IGC_SRRCTL_DESCTYPE_ADV_ONEBUF 0x02000000 #endif /* _IGC_BASE_H */ diff --git a/drivers/net/ethernet/intel/igc/igc_main.c b/drivers/net/ethernet/intel/igc/igc_main.c index 25fc6c65209b..de7b21c2ccd6 100644 --- a/drivers/net/ethernet/intel/igc/igc_main.c +++ b/drivers/net/ethernet/intel/igc/igc_main.c @@ -641,7 +641,10 @@ static void igc_configure_rx_ring(struct igc_adapter *adapter, else buf_size = IGC_RXBUFFER_2048;
- srrctl = IGC_RX_HDR_LEN << IGC_SRRCTL_BSIZEHDRSIZE_SHIFT;
- srrctl = rd32(IGC_SRRCTL(reg_idx));
- srrctl &= ~(IGC_SRRCTL_BSIZEPKT_MASK | IGC_SRRCTL_BSIZEHDRSIZE_MASK |
IGC_SRRCTL_DESCTYPE_MASK);
^^ Please fix indention, moving IGC_SRRCTL_DESCTYPE_MASK such that it aligns with IGC_SRRCTL_BSIZEPKT_MASK. This make is easier for the eye to spot that it is part of the negation (~).
- srrctl |= IGC_RX_HDR_LEN << IGC_SRRCTL_BSIZEHDRSIZE_SHIFT; srrctl |= buf_size >> IGC_SRRCTL_BSIZEPKT_SHIFT; srrctl |= IGC_SRRCTL_DESCTYPE_ADV_ONEBUF;
On Friday, April 14, 2023 3:06 AM , Jesper Dangaard Brouer jbrouer@redhat.com wrote:
On 13/04/2023 17.12, Song Yoong Siang wrote:
igc_configure_rx_ring() function will be called as part of XDP program setup. If Rx hardware timestamp is enabled prio to XDP program setup, this timestamp enablement will be overwritten when buffer size is written into SRRCTL register.
Thus, this commit read the register value before write to SRRCTL register. This commit is tested by using xdp_hw_metadata bpf selftest tool. The tool enables Rx hardware timestamp and then attach XDP program to igc driver. It will display hardware timestamp of UDP packet with port number 9092. Below are detail of test steps and results.
[...]
diff --git a/drivers/net/ethernet/intel/igc/igc_base.h b/drivers/net/ethernet/intel/igc/igc_base.h index 7a992befca24..b95007d51d13 100644 --- a/drivers/net/ethernet/intel/igc/igc_base.h +++ b/drivers/net/ethernet/intel/igc/igc_base.h @@ -87,8 +87,11 @@ union igc_adv_rx_desc { #define IGC_RXDCTL_SWFLUSH 0x04000000 /* Receive Software Flush */
/* SRRCTL bit definitions */ -#define IGC_SRRCTL_BSIZEPKT_SHIFT 10 /* Shift _right_ */ -#define IGC_SRRCTL_BSIZEHDRSIZE_SHIFT 2 /* Shift _left_ */ +#define IGC_SRRCTL_BSIZEPKT_MASK GENMASK(6, 0) +#define IGC_SRRCTL_BSIZEPKT_SHIFT 10 /* Shift _right_ */ +#define IGC_SRRCTL_BSIZEHDRSIZE_MASK GENMASK(13, 8) +#define IGC_SRRCTL_BSIZEHDRSIZE_SHIFT 2 /* Shift _left_ */ +#define IGC_SRRCTL_DESCTYPE_MASK GENMASK(27, 25) #define IGC_SRRCTL_DESCTYPE_ADV_ONEBUF 0x02000000
#endif /* _IGC_BASE_H */ diff --git a/drivers/net/ethernet/intel/igc/igc_main.c b/drivers/net/ethernet/intel/igc/igc_main.c index 25fc6c65209b..de7b21c2ccd6 100644 --- a/drivers/net/ethernet/intel/igc/igc_main.c +++ b/drivers/net/ethernet/intel/igc/igc_main.c @@ -641,7 +641,10 @@ static void igc_configure_rx_ring(struct igc_adapter *adapter, else buf_size = IGC_RXBUFFER_2048;
- srrctl = IGC_RX_HDR_LEN << IGC_SRRCTL_BSIZEHDRSIZE_SHIFT;
- srrctl = rd32(IGC_SRRCTL(reg_idx));
- srrctl &= ~(IGC_SRRCTL_BSIZEPKT_MASK | IGC_SRRCTL_BSIZEHDRSIZE_MASK |
IGC_SRRCTL_DESCTYPE_MASK);^^Please fix indention, moving IGC_SRRCTL_DESCTYPE_MASK such that it aligns with IGC_SRRCTL_BSIZEPKT_MASK. This make is easier for the eye to spot that it is part of the negation (~).
Sure. Thanks for your comment. I will fix it in v2.
- srrctl |= IGC_RX_HDR_LEN << IGC_SRRCTL_BSIZEHDRSIZE_SHIFT; srrctl |= buf_size >> IGC_SRRCTL_BSIZEPKT_SHIFT; srrctl |= IGC_SRRCTL_DESCTYPE_ADV_ONEBUF;
On 4/13/2023 18:12, Song Yoong Siang wrote:
igc_configure_rx_ring() function will be called as part of XDP program setup. If Rx hardware timestamp is enabled prio to XDP program setup, this timestamp enablement will be overwritten when buffer size is written into SRRCTL register.
Thus, this commit read the register value before write to SRRCTL register. This commit is tested by using xdp_hw_metadata bpf selftest tool. The tool enables Rx hardware timestamp and then attach XDP program to igc driver. It will display hardware timestamp of UDP packet with port number 9092. Below are detail of test steps and results.
Command on DUT: sudo ./xdp_hw_metadata <interface name>
Command on Link Partner: echo -n skb | nc -u -q1 <destination IPv4 addr> 9092
Result before this patch: skb hwtstamp is not found!
Result after this patch: found skb hwtstamp = 1677762212.590696226
Fixes: fc9df2a0b520 ("igc: Enable RX via AF_XDP zero-copy") Cc: stable@vger.kernel.org # 5.14+ Signed-off-by: Song Yoong Siang yoong.siang.song@intel.com
drivers/net/ethernet/intel/igc/igc_base.h | 7 +++++-- drivers/net/ethernet/intel/igc/igc_main.c | 5 ++++- 2 files changed, 9 insertions(+), 3 deletions(-)
Tested-by: Naama Meir naamax.meir@linux.intel.com
linux-stable-mirror@lists.linaro.org
-
Jacob Keller -
Jesper Dangaard Brouer -
naamax.meir -
Song Yoong Siang -
Song, Yoong Siang