While this code is executed with the wait_lock held, a reader can
acquire the lock without holding wait_lock. The writer side loops
checking the value with the atomic_cond_read_acquire(), but only truly
acquires the lock when the compare-and-exchange is completed
successfully which isn’t ordered. This exposes the window between the
acquire and the cmpxchg to an A-B-A problem which allows reads following
the lock acquisition to observe values speculatively before the write
lock is truly acquired.
We've seen a problem in epoll where the reader does a xchg while
holding the read lock, but the writer can see a value change out from under it.
Writer | Reader 2
--------------------------------------------------------------------------------
ep_scan_ready_list() |
|- write_lock_irq() |
|- queued_write_lock_slowpath() |
|- atomic_cond_read_acquire() |
| read_lock_irqsave(&ep->lock, flags);
--> (observes value before unlock)| chain_epi_lockless()
| | epi->next = xchg(&ep->ovflist, epi);
| | read_unlock_irqrestore(&ep->lock, flags);
| |
| atomic_cmpxchg_relaxed() |
|-- READ_ONCE(ep->ovflist); |
A core can order the read of the ovflist ahead of the
atomic_cmpxchg_relaxed(). Switching the cmpxchg to use acquire semantics
addresses this issue at which point the atomic_cond_read can be switched
to use relaxed semantics.
Fixes: b519b56e378ee ("locking/qrwlock: Use atomic_cond_read_acquire() when spinning in qrwlock")
Signed-off-by: Ali Saidi <alisaidi(a)amazon.com>
Cc: stable(a)vger.kernel.org
Acked-by: Will Deacon <will(a)kernel.org>
Tested-by: Steve Capper <steve.capper(a)arm.com>
Reviewed-by: Steve Capper <steve.capper(a)arm.com>
---
kernel/locking/qrwlock.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/kernel/locking/qrwlock.c b/kernel/locking/qrwlock.c
index 4786dd271b45..10770f6ac4d9 100644
--- a/kernel/locking/qrwlock.c
+++ b/kernel/locking/qrwlock.c
@@ -73,8 +73,8 @@ void queued_write_lock_slowpath(struct qrwlock *lock)
/* When no more readers or writers, set the locked flag */
do {
- atomic_cond_read_acquire(&lock->cnts, VAL == _QW_WAITING);
- } while (atomic_cmpxchg_relaxed(&lock->cnts, _QW_WAITING,
+ atomic_cond_read_relaxed(&lock->cnts, VAL == _QW_WAITING);
+ } while (atomic_cmpxchg_acquire(&lock->cnts, _QW_WAITING,
_QW_LOCKED) != _QW_WAITING);
unlock:
arch_spin_unlock(&lock->wait_lock);
--
2.24.4.AMZN
Dear stable
The famous brand John Lewis Partnership, is UK's largest multi-
channel retailer with over 126 shops and multiple expansion in
Africa furnished by European/Asian/American products. We are
sourcing
new products to attract new customers and also retain our
existing ones, create new partnerships with companies dealing
with different kinds of goods globally.
Your company's products are of interest to our market as we have
an amazing market for your products.
Provide us your current catalog through email to review more. We
hope to be able to order with you and start a long-term friendly,
respectable and solid business partnership. Please we would
appreciate it if you could send us your stock availability via
email if any.
Our payment terms are 15 days net in Europe, 30 days Net in UK
and 30 days net in Asia/USA as we operate with over 5297
suppliers around the globe for the past 50 years now. For
immediate response Send your reply to
robturner.procurement(a)johnlewis-trade.com for us to be able to
treat with care and urgency.
On behalf of our entire team, we wish you a fruitful 2021.
Best Regards
Rob Turner
Head Of Procurement Operations
John Lewis Partnership.
robturner.procurement(a)johnlewis-trade.com
Tel: +44-7451-274090
WhatsApp: +447497483925
www.johnlewis.com
REGISTERED OFFICE: 171 VICTORIA STREET, LONDON SW1E 5NN