From: Chao Gao chao.gao@intel.com
commit 9e02977bfad006af328add9434c8bffa40e053bb upstream.
When we looked into FIO performance with swiotlb enabled in VM, we found swiotlb_bounce() is always called one more time than expected for each DMA read request.
It turns out that the bounce buffer is copied to original DMA buffer twice after the completion of a DMA request (one is done by in dma_direct_sync_single_for_cpu(), the other by swiotlb_tbl_unmap_single()). But the content in bounce buffer actually doesn't change between the two rounds of copy. So, one round of copy is redundant.
Pass DMA_ATTR_SKIP_CPU_SYNC flag to swiotlb_tbl_unmap_single() to skip the memory copy in it.
This fix increases FIO 64KB sequential read throughput in a guest with swiotlb=force by 5.6%.
Fixes: 55897af63091 ("dma-direct: merge swiotlb_dma_ops into the dma_direct code") Reported-by: Wang Zhaoyang1 zhaoyang1.wang@intel.com Reported-by: Gao Liang liang.gao@intel.com Signed-off-by: Chao Gao chao.gao@intel.com Reviewed-by: Kevin Tian kevin.tian@intel.com Signed-off-by: Christoph Hellwig hch@lst.de Signed-off-by: Greg Kroah-Hartman gregkh@linuxfoundation.org --- kernel/dma/direct.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
--- a/kernel/dma/direct.c +++ b/kernel/dma/direct.c @@ -306,7 +306,8 @@ void dma_direct_unmap_page(struct device dma_direct_sync_single_for_cpu(dev, addr, size, dir);
if (unlikely(is_swiotlb_buffer(phys))) - swiotlb_tbl_unmap_single(dev, phys, size, size, dir, attrs); + swiotlb_tbl_unmap_single(dev, phys, size, size, dir, + attrs | DMA_ATTR_SKIP_CPU_SYNC); } EXPORT_SYMBOL(dma_direct_unmap_page);