From: Kefeng Wang wangkefeng.wang@huawei.com
[ Upstream commit ed59dfd9509d172e4920994ed9cbebf93b0050cc ]
The memory barrier dma_mb() is introduced by commit a76a37777f2c ("iommu/arm-smmu-v3: Ensure queue is read after updating prod pointer"), which is used to ensure that prior (both reads and writes) accesses to memory by a CPU are ordered w.r.t. a subsequent MMIO write.
Reviewed-by: Arnd Bergmann arnd@arndb.de # for asm-generic Signed-off-by: Kefeng Wang wangkefeng.wang@huawei.com Reviewed-by: Marco Elver elver@google.com Link: https://lore.kernel.org/r/20220523113126.171714-2-wangkefeng.wang@huawei.com Signed-off-by: Will Deacon will@kernel.org Stable-dep-of: aa6956150f82 ("wifi: ath11k: fix dest ring-buffer corruption when ring is full") Signed-off-by: Sasha Levin sashal@kernel.org --- Documentation/memory-barriers.txt | 11 ++++++----- include/asm-generic/barrier.h | 8 ++++++++ 2 files changed, 14 insertions(+), 5 deletions(-)
diff --git a/Documentation/memory-barriers.txt b/Documentation/memory-barriers.txt index 7367ada13208..d69edaada266 100644 --- a/Documentation/memory-barriers.txt +++ b/Documentation/memory-barriers.txt @@ -1894,6 +1894,7 @@ There are some more advanced barrier functions:
(*) dma_wmb(); (*) dma_rmb(); + (*) dma_mb();
These are for use with consistent memory to guarantee the ordering of writes or reads of shared memory accessible to both the CPU and a @@ -1925,11 +1926,11 @@ There are some more advanced barrier functions: The dma_rmb() allows us guarantee the device has released ownership before we read the data from the descriptor, and the dma_wmb() allows us to guarantee the data is written to the descriptor before the device - can see it now has ownership. Note that, when using writel(), a prior - wmb() is not needed to guarantee that the cache coherent memory writes - have completed before writing to the MMIO region. The cheaper - writel_relaxed() does not provide this guarantee and must not be used - here. + can see it now has ownership. The dma_mb() implies both a dma_rmb() and + a dma_wmb(). Note that, when using writel(), a prior wmb() is not needed + to guarantee that the cache coherent memory writes have completed before + writing to the MMIO region. The cheaper writel_relaxed() does not provide + this guarantee and must not be used here.
See the subsection "Kernel I/O barrier effects" for more information on relaxed I/O accessors and the Documentation/core-api/dma-api.rst file for diff --git a/include/asm-generic/barrier.h b/include/asm-generic/barrier.h index 05aae2922485..ce75c53dd338 100644 --- a/include/asm-generic/barrier.h +++ b/include/asm-generic/barrier.h @@ -37,6 +37,10 @@ #define wmb() do { kcsan_wmb(); __wmb(); } while (0) #endif
+#ifdef __dma_mb +#define dma_mb() do { kcsan_mb(); __dma_mb(); } while (0) +#endif + #ifdef __dma_rmb #define dma_rmb() do { kcsan_rmb(); __dma_rmb(); } while (0) #endif @@ -64,6 +68,10 @@ #define wmb() mb() #endif
+#ifndef dma_mb +#define dma_mb() mb() +#endif + #ifndef dma_rmb #define dma_rmb() rmb() #endif