From: Ping-Ke Shih pkshih@realtek.com
[ Upstream commit 45742881f9eee2a4daeb6008e648a460dd3742cd ]
Coverity reported that u8 rx_mask << 24 will become signed 32 bits, which casting to unsigned 64 bits will do sign extension. For example, putting 0x80000000 (signed 32 bits) to a u64 variable will become 0xFFFFFFFF_80000000.
The real case we meet is: rx_mask[0...3] = ff ff 00 00 ra_mask = 0xffffffff_ff0ff000
After this fix: rx_mask[0...3] = ff ff 00 00 ra_mask = 0x00000000_ff0ff000
Fortunately driver does bitwise-AND with incorrect ra_mask and supported rates (1ss and 2ss rate only) afterward, so the final rate mask of original code is still correct.
Addresses-Coverity-ID: 1504762 ("Unintended sign extension")
Signed-off-by: Ping-Ke Shih pkshih@realtek.com Link: https://patch.msgid.link/20240809072012.84152-5-pkshih@realtek.com Signed-off-by: Sasha Levin sashal@kernel.org --- drivers/net/wireless/realtek/rtw89/phy.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/net/wireless/realtek/rtw89/phy.c b/drivers/net/wireless/realtek/rtw89/phy.c index ad11d1414874a..c038e5ca3e45a 100644 --- a/drivers/net/wireless/realtek/rtw89/phy.c +++ b/drivers/net/wireless/realtek/rtw89/phy.c @@ -353,8 +353,8 @@ static void rtw89_phy_ra_sta_update(struct rtw89_dev *rtwdev, csi_mode = RTW89_RA_RPT_MODE_HT; ra_mask |= ((u64)sta->deflink.ht_cap.mcs.rx_mask[3] << 48) | ((u64)sta->deflink.ht_cap.mcs.rx_mask[2] << 36) | - (sta->deflink.ht_cap.mcs.rx_mask[1] << 24) | - (sta->deflink.ht_cap.mcs.rx_mask[0] << 12); + ((u64)sta->deflink.ht_cap.mcs.rx_mask[1] << 24) | + ((u64)sta->deflink.ht_cap.mcs.rx_mask[0] << 12); high_rate_masks = rtw89_ra_mask_ht_rates; if (sta->deflink.ht_cap.cap & IEEE80211_HT_CAP_RX_STBC) stbc_en = 1;