Linaro-mm-sig

linaro-mm-sig@lists.linaro.org

5 participants
2539 discussions

[PATCH] dma-buf/heaps: map CMA all pages for user

by Hsia-Jun Li

Page fault would raise a CPU interrupt, it is not a good idea. Signed-off-by: Hsia-Jun(Randy) Li <randy.li(a)synaptics.com> --- drivers/dma-buf/heaps/cma_heap.c | 26 +++----------------------- 1 file changed, 3 insertions(+), 23 deletions(-) diff --git a/drivers/dma-buf/heaps/cma_heap.c b/drivers/dma-buf/heaps/cma_heap.c index ee899f8e6721..7d0b15ad21a7 100644 --- a/drivers/dma-buf/heaps/cma_heap.c +++ b/drivers/dma-buf/heaps/cma_heap.c @@ -160,35 +160,15 @@ static int cma_heap_dma_buf_end_cpu_access(struct dma_buf *dmabuf, return 0; } -static vm_fault_t cma_heap_vm_fault(struct vm_fault *vmf) -{ - struct vm_area_struct *vma = vmf->vma; - struct cma_heap_buffer *buffer = vma->vm_private_data; - - if (vmf->pgoff > buffer->pagecount) - return VM_FAULT_SIGBUS; - - vmf->page = buffer->pages[vmf->pgoff]; - get_page(vmf->page); - - return 0; -} - -static const struct vm_operations_struct dma_heap_vm_ops = { - .fault = cma_heap_vm_fault, -}; - static int cma_heap_mmap(struct dma_buf *dmabuf, struct vm_area_struct *vma) { struct cma_heap_buffer *buffer = dmabuf->priv; - if ((vma->vm_flags & (VM_SHARED | VM_MAYSHARE)) == 0) return -EINVAL; - vma->vm_ops = &dma_heap_vm_ops; - vma->vm_private_data = buffer; - - return 0; + return remap_pfn_range(vma, vma->vm_start, + page_to_pfn(buffer->pages[vma->vm_pgoff]), + vma->vm_end - vma->vm_start, vma->vm_page_prot); } static void *cma_heap_do_vmap(struct cma_heap_buffer *buffer) -- 2.17.1

1 year, 4 months

[PATCH] dma-buf/sw_sync: Avoid recursive lock during fence signal

by Rob Clark

From: Rob Clark <robdclark(a)chromium.org> If a signal callback releases the sw_sync fence, that will trigger a deadlock as the timeline_fence_release recurses onto the fence->lock (used both for signaling and the the timeline tree). To avoid that, temporarily hold an extra reference to the signalled fences until after we drop the lock. (This is an alternative implementation of https://patchwork.kernel.org/patch/11664717/ which avoids some potential UAF issues with the original patch.) Reported-by: Bas Nieuwenhuizen <bas(a)basnieuwenhuizen.nl> Fixes: d3c6dd1fb30d ("dma-buf/sw_sync: Synchronize signal vs syncpt free") Signed-off-by: Rob Clark <robdclark(a)chromium.org> --- drivers/dma-buf/sw_sync.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/drivers/dma-buf/sw_sync.c b/drivers/dma-buf/sw_sync.c index 63f0aeb66db6..ceb6a0408624 100644 --- a/drivers/dma-buf/sw_sync.c +++ b/drivers/dma-buf/sw_sync.c @@ -191,6 +191,7 @@ static const struct dma_fence_ops timeline_fence_ops = { */ static void sync_timeline_signal(struct sync_timeline *obj, unsigned int inc) { + LIST_HEAD(signalled); struct sync_pt *pt, *next; trace_sync_timeline(obj); @@ -203,9 +204,13 @@ static void sync_timeline_signal(struct sync_timeline *obj, unsigned int inc) if (!timeline_fence_signaled(&pt->base)) break; + dma_fence_get(&pt->base); + list_del_init(&pt->link); rb_erase(&pt->node, &obj->pt_tree); + list_add_tail(&pt->link, &signalled); + /* * A signal callback may release the last reference to this * fence, causing it to be freed. That operation has to be @@ -218,6 +223,11 @@ static void sync_timeline_signal(struct sync_timeline *obj, unsigned int inc) } spin_unlock_irq(&obj->lock); + + list_for_each_entry_safe(pt, next, &signalled, link) { + list_del(&pt->link); + dma_fence_put(&pt->base); + } } /** -- 2.41.0

1 year, 4 months

[PATCH -next] drm/scheduler: Remove unused declarations

by Yue Haibing

Commit 06a2d7cc3f04 ("drm/amdgpu: revert "implement tdr advanced mode"") removed drm_sched_reset_karma()/drm_sched_increase_karma_ext() but leave the declarations. Commit 2cf9886e2816 ("drm/scheduler: remove drm_sched_dependency_optimized") removed drm_sched_dependency_optimized() but not its declaration. Signed-off-by: Yue Haibing <yuehaibing(a)huawei.com> --- include/drm/gpu_scheduler.h | 4 ---- 1 file changed, 4 deletions(-) diff --git a/include/drm/gpu_scheduler.h b/include/drm/gpu_scheduler.h index f9544d9b670d..cd8ac90865fc 100644 --- a/include/drm/gpu_scheduler.h +++ b/include/drm/gpu_scheduler.h @@ -554,10 +554,6 @@ void drm_sched_stop(struct drm_gpu_scheduler *sched, struct drm_sched_job *bad); void drm_sched_start(struct drm_gpu_scheduler *sched, bool full_recovery); void drm_sched_resubmit_jobs(struct drm_gpu_scheduler *sched); void drm_sched_increase_karma(struct drm_sched_job *bad); -void drm_sched_reset_karma(struct drm_sched_job *bad); -void drm_sched_increase_karma_ext(struct drm_sched_job *bad, int type); -bool drm_sched_dependency_optimized(struct dma_fence* fence, - struct drm_sched_entity *entity); void drm_sched_fault(struct drm_gpu_scheduler *sched); void drm_sched_rq_add_entity(struct drm_sched_rq *rq, -- 2.34.1

1 year, 4 months

[PATCH v3] misc: sram: Add DMA-BUF Heap exporting of SRAM areas

by Andrew Davis

This new export type exposes to userspace the SRAM area as a DMA-BUF Heap, this allows for allocations of DMA-BUFs that can be consumed by various DMA-BUF supporting devices. Signed-off-by: Andrew Davis <afd(a)ti.com> --- Changes from v2: - Make sram_dma_heap_allocate static (kernel test robot) - Rebase on v6.5-rc1 drivers/misc/Kconfig | 7 + drivers/misc/Makefile | 1 + drivers/misc/sram-dma-heap.c | 245 +++++++++++++++++++++++++++++++++++ drivers/misc/sram.c | 6 + drivers/misc/sram.h | 16 +++ 5 files changed, 275 insertions(+) create mode 100644 drivers/misc/sram-dma-heap.c diff --git a/drivers/misc/Kconfig b/drivers/misc/Kconfig index 75e427f124b28..ee34dfb61605f 100644 --- a/drivers/misc/Kconfig +++ b/drivers/misc/Kconfig @@ -448,6 +448,13 @@ config SRAM config SRAM_EXEC bool +config SRAM_DMA_HEAP + bool "Export on-chip SRAM pools using DMA-Heaps" + depends on DMABUF_HEAPS && SRAM + help + This driver allows the export of on-chip SRAM marked as both pool + and exportable to userspace using the DMA-Heaps interface. + config DW_XDATA_PCIE depends on PCI tristate "Synopsys DesignWare xData PCIe driver" diff --git a/drivers/misc/Makefile b/drivers/misc/Makefile index f2a4d1ff65d46..5e7516bfaa8de 100644 --- a/drivers/misc/Makefile +++ b/drivers/misc/Makefile @@ -47,6 +47,7 @@ obj-$(CONFIG_VMWARE_VMCI) += vmw_vmci/ obj-$(CONFIG_LATTICE_ECP3_CONFIG) += lattice-ecp3-config.o obj-$(CONFIG_SRAM) += sram.o obj-$(CONFIG_SRAM_EXEC) += sram-exec.o +obj-$(CONFIG_SRAM_DMA_HEAP) += sram-dma-heap.o obj-$(CONFIG_GENWQE) += genwqe/ obj-$(CONFIG_ECHO) += echo/ obj-$(CONFIG_CXL_BASE) += cxl/ diff --git a/drivers/misc/sram-dma-heap.c b/drivers/misc/sram-dma-heap.c new file mode 100644 index 0000000000000..c054c04dff33e --- /dev/null +++ b/drivers/misc/sram-dma-heap.c @@ -0,0 +1,245 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * SRAM DMA-Heap userspace exporter + * + * Copyright (C) 2019-2022 Texas Instruments Incorporated - https://www.ti.com/ + * Andrew Davis <afd(a)ti.com> + */ + +#include <linux/dma-mapping.h> +#include <linux/err.h> +#include <linux/genalloc.h> +#include <linux/io.h> +#include <linux/mm.h> +#include <linux/scatterlist.h> +#include <linux/slab.h> +#include <linux/dma-buf.h> +#include <linux/dma-heap.h> + +#include "sram.h" + +struct sram_dma_heap { + struct dma_heap *heap; + struct gen_pool *pool; +}; + +struct sram_dma_heap_buffer { + struct gen_pool *pool; + struct list_head attachments; + struct mutex attachments_lock; + unsigned long len; + void *vaddr; + phys_addr_t paddr; +}; + +struct dma_heap_attachment { + struct device *dev; + struct sg_table *table; + struct list_head list; +}; + +static int dma_heap_attach(struct dma_buf *dmabuf, + struct dma_buf_attachment *attachment) +{ + struct sram_dma_heap_buffer *buffer = dmabuf->priv; + struct dma_heap_attachment *a; + struct sg_table *table; + + a = kzalloc(sizeof(*a), GFP_KERNEL); + if (!a) + return -ENOMEM; + + table = kmalloc(sizeof(*table), GFP_KERNEL); + if (!table) { + kfree(a); + return -ENOMEM; + } + if (sg_alloc_table(table, 1, GFP_KERNEL)) { + kfree(table); + kfree(a); + return -ENOMEM; + } + sg_set_page(table->sgl, pfn_to_page(PFN_DOWN(buffer->paddr)), buffer->len, 0); + + a->table = table; + a->dev = attachment->dev; + INIT_LIST_HEAD(&a->list); + + attachment->priv = a; + + mutex_lock(&buffer->attachments_lock); + list_add(&a->list, &buffer->attachments); + mutex_unlock(&buffer->attachments_lock); + + return 0; +} + +static void dma_heap_detatch(struct dma_buf *dmabuf, + struct dma_buf_attachment *attachment) +{ + struct sram_dma_heap_buffer *buffer = dmabuf->priv; + struct dma_heap_attachment *a = attachment->priv; + + mutex_lock(&buffer->attachments_lock); + list_del(&a->list); + mutex_unlock(&buffer->attachments_lock); + + sg_free_table(a->table); + kfree(a->table); + kfree(a); +} + +static struct sg_table *dma_heap_map_dma_buf(struct dma_buf_attachment *attachment, + enum dma_data_direction direction) +{ + struct dma_heap_attachment *a = attachment->priv; + struct sg_table *table = a->table; + + /* + * As this heap is backed by uncached SRAM memory we do not need to + * perform any sync operations on the buffer before allowing device + * domain access. For this reason we use SKIP_CPU_SYNC and also do + * not use or provide begin/end_cpu_access() dma-buf functions. + */ + if (!dma_map_sg_attrs(attachment->dev, table->sgl, table->nents, + direction, DMA_ATTR_SKIP_CPU_SYNC)) + return ERR_PTR(-ENOMEM); + + return table; +} + +static void dma_heap_unmap_dma_buf(struct dma_buf_attachment *attachment, + struct sg_table *table, + enum dma_data_direction direction) +{ + dma_unmap_sg_attrs(attachment->dev, table->sgl, table->nents, + direction, DMA_ATTR_SKIP_CPU_SYNC); +} + +static void dma_heap_dma_buf_release(struct dma_buf *dmabuf) +{ + struct sram_dma_heap_buffer *buffer = dmabuf->priv; + + gen_pool_free(buffer->pool, (unsigned long)buffer->vaddr, buffer->len); + kfree(buffer); +} + +static int dma_heap_mmap(struct dma_buf *dmabuf, struct vm_area_struct *vma) +{ + struct sram_dma_heap_buffer *buffer = dmabuf->priv; + int ret; + + /* SRAM mappings are not cached */ + vma->vm_page_prot = pgprot_writecombine(vma->vm_page_prot); + + ret = vm_iomap_memory(vma, buffer->paddr, buffer->len); + if (ret) + pr_err("Could not map buffer to userspace\n"); + + return ret; +} + +static int dma_heap_vmap(struct dma_buf *dmabuf, struct iosys_map *map) +{ + struct sram_dma_heap_buffer *buffer = dmabuf->priv; + + iosys_map_set_vaddr(map, buffer->vaddr); + + return 0; +} + +static const struct dma_buf_ops sram_dma_heap_buf_ops = { + .attach = dma_heap_attach, + .detach = dma_heap_detatch, + .map_dma_buf = dma_heap_map_dma_buf, + .unmap_dma_buf = dma_heap_unmap_dma_buf, + .release = dma_heap_dma_buf_release, + .mmap = dma_heap_mmap, + .vmap = dma_heap_vmap, +}; + +static struct dma_buf *sram_dma_heap_allocate(struct dma_heap *heap, + unsigned long len, + unsigned long fd_flags, + unsigned long heap_flags) +{ + struct sram_dma_heap *sram_dma_heap = dma_heap_get_drvdata(heap); + struct sram_dma_heap_buffer *buffer; + + DEFINE_DMA_BUF_EXPORT_INFO(exp_info); + struct dma_buf *dmabuf; + int ret = 0; + + buffer = kzalloc(sizeof(*buffer), GFP_KERNEL); + if (!buffer) + return ERR_PTR(-ENOMEM); + buffer->pool = sram_dma_heap->pool; + INIT_LIST_HEAD(&buffer->attachments); + mutex_init(&buffer->attachments_lock); + buffer->len = len; + + buffer->vaddr = (void *)gen_pool_alloc(buffer->pool, buffer->len); + if (!buffer->vaddr) { + ret = -ENOMEM; + goto free_buffer; + } + + buffer->paddr = gen_pool_virt_to_phys(buffer->pool, (unsigned long)buffer->vaddr); + if (buffer->paddr == -1) { + ret = -ENOMEM; + goto free_pool; + } + + /* create the dmabuf */ + exp_info.exp_name = dma_heap_get_name(heap); + exp_info.ops = &sram_dma_heap_buf_ops; + exp_info.size = buffer->len; + exp_info.flags = fd_flags; + exp_info.priv = buffer; + dmabuf = dma_buf_export(&exp_info); + if (IS_ERR(dmabuf)) { + ret = PTR_ERR(dmabuf); + goto free_pool; + } + + return dmabuf; + +free_pool: + gen_pool_free(buffer->pool, (unsigned long)buffer->vaddr, buffer->len); +free_buffer: + kfree(buffer); + + return ERR_PTR(ret); +} + +static struct dma_heap_ops sram_dma_heap_ops = { + .allocate = sram_dma_heap_allocate, +}; + +int sram_add_dma_heap(struct sram_dev *sram, + struct sram_reserve *block, + phys_addr_t start, + struct sram_partition *part) +{ + struct sram_dma_heap *sram_dma_heap; + struct dma_heap_export_info exp_info; + + dev_info(sram->dev, "Exporting SRAM Heap '%s'\n", block->label); + + sram_dma_heap = kzalloc(sizeof(*sram_dma_heap), GFP_KERNEL); + if (!sram_dma_heap) + return -ENOMEM; + sram_dma_heap->pool = part->pool; + + exp_info.name = kasprintf(GFP_KERNEL, "sram_%s", block->label); + exp_info.ops = &sram_dma_heap_ops; + exp_info.priv = sram_dma_heap; + sram_dma_heap->heap = dma_heap_add(&exp_info); + if (IS_ERR(sram_dma_heap->heap)) { + int ret = PTR_ERR(sram_dma_heap->heap); + kfree(sram_dma_heap); + return ret; + } + + return 0; +} diff --git a/drivers/misc/sram.c b/drivers/misc/sram.c index 5757adf418b1d..6dd173a2fba8e 100644 --- a/drivers/misc/sram.c +++ b/drivers/misc/sram.c @@ -120,6 +120,12 @@ static int sram_add_partition(struct sram_dev *sram, struct sram_reserve *block, ret = sram_add_pool(sram, block, start, part); if (ret) return ret; + + if (block->export) { + ret = sram_add_dma_heap(sram, block, start, part); + if (ret) + return ret; + } } if (block->export) { ret = sram_add_export(sram, block, start, part); diff --git a/drivers/misc/sram.h b/drivers/misc/sram.h index 397205b8bf6ff..062bdd25fa068 100644 --- a/drivers/misc/sram.h +++ b/drivers/misc/sram.h @@ -60,4 +60,20 @@ static inline int sram_add_protect_exec(struct sram_partition *part) return -ENODEV; } #endif /* CONFIG_SRAM_EXEC */ + +#ifdef CONFIG_SRAM_DMA_HEAP +int sram_add_dma_heap(struct sram_dev *sram, + struct sram_reserve *block, + phys_addr_t start, + struct sram_partition *part); +#else +static inline int sram_add_dma_heap(struct sram_dev *sram, + struct sram_reserve *block, + phys_addr_t start, + struct sram_partition *part) +{ + return 0; +} +#endif /* CONFIG_SRAM_DMA_HEAP */ + #endif /* __SRAM_H */ -- 2.39.2

1 year, 4 months

Re: [PATCH v2] dma-contiguous: define proper name for global cma region

by Christoph Hellwig

Hi Pintu, On Sat, Jul 29, 2023 at 08:05:15AM +0530, Pintu Kumar wrote: > The current global cma region name defined as "reserved" > which is misleading, creates confusion and too generic. > > Also, the default cma allocation happens from global cma region, > so, if one has to figure out all allocations happening from > global cma region, this seems easier. > > Thus, change the name from "reserved" to "global-cma-region". I agree that reserved is not a very useful name. Unfortuately the name of the region leaks to userspace through cma_heap. So I think we need prep patches to hardcode "reserved" in add_default_cma_heap first, and then remove the cma_get_name first.

1 year, 5 months

[PATCH] drm: manager: Remove the unused variable prev

by Jiapeng Chong

Variable prev is not effectively used, so delete it. drivers/gpu/drm/drm_gpuva_mgr.c:1079:32: warning: variable ‘prev’ set but not used. Reported-by: Abaci Robot <abaci(a)linux.alibaba.com> Signed-off-by: Jiapeng Chong <jiapeng.chong(a)linux.alibaba.com> --- drivers/gpu/drm/drm_gpuva_mgr.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/drivers/gpu/drm/drm_gpuva_mgr.c b/drivers/gpu/drm/drm_gpuva_mgr.c index f86bfad74ff8..afa5330445b3 100644 --- a/drivers/gpu/drm/drm_gpuva_mgr.c +++ b/drivers/gpu/drm/drm_gpuva_mgr.c @@ -1076,7 +1076,7 @@ __drm_gpuva_sm_map(struct drm_gpuva_manager *mgr, u64 req_addr, u64 req_range, struct drm_gem_object *req_obj, u64 req_offset) { - struct drm_gpuva *va, *next, *prev = NULL; + struct drm_gpuva *va, *next = NULL; u64 req_end = req_addr + req_range; int ret; @@ -1206,7 +1206,6 @@ __drm_gpuva_sm_map(struct drm_gpuva_manager *mgr, } } next: - prev = va; } return op_map_cb(ops, priv, -- 2.20.1.7.g153144c

1 year, 5 months

[PATCH AUTOSEL 6.4 1/7] drm/sched: Make sure we wait for all dependencies in kill_jobs_cb()

by Sasha Levin

From: Boris Brezillon <boris.brezillon(a)collabora.com> [ Upstream commit e30cb0599799aac099209e3b045379613c80730e ] drm_sched_entity_kill_jobs_cb() logic is omitting the last fence popped from the dependency array that was waited upon before drm_sched_entity_kill() was called (drm_sched_entity::dependency field), so we're basically waiting for all dependencies except one. In theory, this wait shouldn't be needed because resources should have their users registered to the dma_resv object, thus guaranteeing that future jobs wanting to access these resources wait on all the previous users (depending on the access type, of course). But we want to keep these explicit waits in the kill entity path just in case. Let's make sure we keep all dependencies in the array in drm_sched_job_dependency(), so we can iterate over the array and wait in drm_sched_entity_kill_jobs_cb(). We also make sure we wait on drm_sched_fence::finished if we were originally asked to wait on drm_sched_fence::scheduled. In that case, we assume the intent was to delegate the wait to the firmware/GPU or rely on the pipelining done at the entity/scheduler level, but when killing jobs, we really want to wait for completion not just scheduling. v2: - Don't evict deps in drm_sched_job_dependency() v3: - Always wait for drm_sched_fence::finished fences in drm_sched_entity_kill_jobs_cb() when we see a sched_fence v4: - Fix commit message - Fix a use-after-free bug v5: - Flag deps on which we should only wait for the scheduled event at insertion time v6: - Back to v4 implementation - Add Christian's R-b Cc: Frank Binns <frank.binns(a)imgtec.com> Cc: Sarah Walker <sarah.walker(a)imgtec.com> Cc: Donald Robson <donald.robson(a)imgtec.com> Cc: Luben Tuikov <luben.tuikov(a)amd.com> Cc: David Airlie <airlied(a)gmail.com> Cc: Daniel Vetter <daniel(a)ffwll.ch> Cc: Sumit Semwal <sumit.semwal(a)linaro.org> Cc: "Christian König" <christian.koenig(a)amd.com> Signed-off-by: Boris Brezillon <boris.brezillon(a)collabora.com> Suggested-by: "Christian König" <christian.koenig(a)amd.com> Reviewed-by: "Christian König" <christian.koenig(a)amd.com> Acked-by: Luben Tuikov <luben.tuikov(a)amd.com> Link: https://patchwork.freedesktop.org/patch/msgid/20230619071921.3465992-1-bori… Signed-off-by: Sasha Levin <sashal(a)kernel.org> --- drivers/gpu/drm/scheduler/sched_entity.c | 41 +++++++++++++++++++----- 1 file changed, 33 insertions(+), 8 deletions(-) diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm/scheduler/sched_entity.c index e0a8890a62e23..42021d1f7e016 100644 --- a/drivers/gpu/drm/scheduler/sched_entity.c +++ b/drivers/gpu/drm/scheduler/sched_entity.c @@ -155,16 +155,32 @@ static void drm_sched_entity_kill_jobs_cb(struct dma_fence *f, { struct drm_sched_job *job = container_of(cb, struct drm_sched_job, finish_cb); - int r; + unsigned long index; dma_fence_put(f); /* Wait for all dependencies to avoid data corruptions */ - while (!xa_empty(&job->dependencies)) { - f = xa_erase(&job->dependencies, job->last_dependency++); - r = dma_fence_add_callback(f, &job->finish_cb, - drm_sched_entity_kill_jobs_cb); - if (!r) + xa_for_each(&job->dependencies, index, f) { + struct drm_sched_fence *s_fence = to_drm_sched_fence(f); + + if (s_fence && f == &s_fence->scheduled) { + /* The dependencies array had a reference on the scheduled + * fence, and the finished fence refcount might have + * dropped to zero. Use dma_fence_get_rcu() so we get + * a NULL fence in that case. + */ + f = dma_fence_get_rcu(&s_fence->finished); + + /* Now that we have a reference on the finished fence, + * we can release the reference the dependencies array + * had on the scheduled fence. + */ + dma_fence_put(&s_fence->scheduled); + } + + xa_erase(&job->dependencies, index); + if (f && !dma_fence_add_callback(f, &job->finish_cb, + drm_sched_entity_kill_jobs_cb)) return; dma_fence_put(f); @@ -394,8 +410,17 @@ static struct dma_fence * drm_sched_job_dependency(struct drm_sched_job *job, struct drm_sched_entity *entity) { - if (!xa_empty(&job->dependencies)) - return xa_erase(&job->dependencies, job->last_dependency++); + struct dma_fence *f; + + /* We keep the fence around, so we can iterate over all dependencies + * in drm_sched_entity_kill_jobs_cb() to ensure all deps are signaled + * before killing the job. + */ + f = xa_load(&job->dependencies, job->last_dependency); + if (f) { + job->last_dependency++; + return dma_fence_get(f); + } if (job->sched->ops->prepare_job) return job->sched->ops->prepare_job(job, entity); -- 2.40.1

1 year, 5 months

[RFC PATCH 00/10] Device Memory TCP

by Mina Almasry

* TL;DR: Device memory TCP (devmem TCP) is a proposal for transferring data to and/or from device memory efficiently, without bouncing the data to a host memory buffer. * Problem: A large amount of data transfers have device memory as the source and/or destination. Accelerators drastically increased the volume of such transfers. Some examples include: - ML accelerators transferring large amounts of training data from storage into GPU/TPU memory. In some cases ML training setup time can be as long as 50% of TPU compute time, improving data transfer throughput & efficiency can help improving GPU/TPU utilization. - Distributed training, where ML accelerators, such as GPUs on different hosts, exchange data among them. - Distributed raw block storage applications transfer large amounts of data with remote SSDs, much of this data does not require host processing. Today, the majority of the Device-to-Device data transfers the network are implemented as the following low level operations: Device-to-Host copy, Host-to-Host network transfer, and Host-to-Device copy. The implementation is suboptimal, especially for bulk data transfers, and can put significant strains on system resources, such as host memory bandwidth, PCIe bandwidth, etc. One important reason behind the current state is the kernel’s lack of semantics to express device to network transfers. * Proposal: In this patch series we attempt to optimize this use case by implementing socket APIs that enable the user to: 1. send device memory across the network directly, and 2. receive incoming network packets directly into device memory. Packet _payloads_ go directly from the NIC to device memory for receive and from device memory to NIC for transmit. Packet _headers_ go to/from host memory and are processed by the TCP/IP stack normally. The NIC _must_ support header split to achieve this. Advantages: - Alleviate host memory bandwidth pressure, compared to existing network-transfer + device-copy semantics. - Alleviate PCIe BW pressure, by limiting data transfer to the lowest level of the PCIe tree, compared to traditional path which sends data through the root complex. With this proposal we're able to reach ~96.6% line rate speeds with data sent and received directly from/to device memory. * Patch overview: ** Part 1: struct paged device memory Currently the standard for device memory sharing is DMABUF, which doesn't generate struct pages. On the other hand, networking stack (skbs, drivers, and page pool) operate on pages. We have 2 options: 1. Generate struct pages for dmabuf device memory, or, 2. Modify the networking stack to understand a new memory type. This proposal implements option #1. We implement a small framework to generate struct pages for an sg_table returned from dma_buf_map_attachment(). The support added here should be generic and easily extended to other use cases interested in struct paged device memory. We use this framework to generate pages that can be used in the networking stack. ** Part 2: recvmsg() & sendmsg() APIs We define user APIs for the user to send and receive these dmabuf pages. ** part 3: support for unreadable skb frags Dmabuf pages are not accessible by the host; we implement changes throughput the networking stack to correctly handle skbs with unreadable frags. ** part 4: page pool support We piggy back on Jakub's page pool memory providers idea: https://github.com/kuba-moo/linux/tree/pp-providers It allows the page pool to define a memory provider that provides the page allocation and freeing. It helps abstract most of the device memory TCP changes from the driver. This is not strictly necessary, the driver can choose to allocate dmabuf pages and use them directly without going through the page pool (if acceptable to their maintainers). Not included with this RFC is the GVE devmem TCP support, just to simplify the review. Code available here if desired: https://github.com/mina/linux/tree/tcpdevmem This RFC is built on top of v6.4-rc7 with Jakub's pp-providers changes cherry-picked. * NIC dependencies: 1. (strict) Devmem TCP require the NIC to support header split, i.e. the capability to split incoming packets into a header + payload and to put each into a separate buffer. Devmem TCP works by using dmabuf pages for the packet payload, and host memory for the packet headers. 2. (optional) Devmem TCP works better with flow steering support & RSS support, i.e. the NIC's ability to steer flows into certain rx queues. This allows the sysadmin to enable devmem TCP on a subset of the rx queues, and steer devmem TCP traffic onto these queues and non devmem TCP elsewhere. The NIC I have access to with these properties is the GVE with DQO support running in Google Cloud, but any NIC that supports these features would suffice. I may be able to help reviewers bring up devmem TCP on their NICs. * Testing: The series includes a udmabuf kselftest that show a simple use case of devmem TCP and validates the entire data path end to end without a dependency on a specific dmabuf provider. Not included in this series is our devmem TCP benchmark, which transfers data to/from GPU dmabufs directly. With this implementation & benchmark we're able to reach ~96.6% line rate speeds with 4 GPU/NIC pairs running bi-direction traffic, with all the packet payloads going straight to the GPU memory (no host buffer bounce). ** Test Setup Kernel: v6.4-rc7, with this RFC and Jakub's memory provider API cherry-picked locally. Hardware: Google Cloud A3 VMs. NIC: GVE with header split & RSS & flow steering support. Benchmark: custom devmem TCP benchmark not yet open sourced. Mina Almasry (10): dma-buf: add support for paged attachment mappings dma-buf: add support for NET_RX pages dma-buf: add support for NET_TX pages net: add support for skbs with unreadable frags tcp: implement recvmsg() RX path for devmem TCP net: add SO_DEVMEM_DONTNEED setsockopt to release RX pages tcp: implement sendmsg() TX path for for devmem tcp selftests: add ncdevmem, netcat for devmem TCP memory-provider: updates core provider API for devmem TCP memory-provider: add dmabuf devmem provider drivers/dma-buf/dma-buf.c | 444 ++++++++++++++++ include/linux/dma-buf.h | 142 +++++ include/linux/netdevice.h | 1 + include/linux/skbuff.h | 34 +- include/linux/socket.h | 1 + include/net/page_pool.h | 21 + include/net/sock.h | 4 + include/net/tcp.h | 6 +- include/uapi/asm-generic/socket.h | 6 + include/uapi/linux/dma-buf.h | 12 + include/uapi/linux/uio.h | 10 + net/core/datagram.c | 3 + net/core/page_pool.c | 111 +++- net/core/skbuff.c | 81 ++- net/core/sock.c | 47 ++ net/ipv4/tcp.c | 262 +++++++++- net/ipv4/tcp_input.c | 13 +- net/ipv4/tcp_ipv4.c | 8 + net/ipv4/tcp_output.c | 5 +- net/packet/af_packet.c | 4 +- tools/testing/selftests/net/.gitignore | 1 + tools/testing/selftests/net/Makefile | 1 + tools/testing/selftests/net/ncdevmem.c | 693 +++++++++++++++++++++++++ 23 files changed, 1868 insertions(+), 42 deletions(-) create mode 100644 tools/testing/selftests/net/ncdevmem.c -- 2.41.0.390.g38632f3daf-goog

1 year, 5 months

[PATCH v2] dma-buf: Fix the typo in DMA-BUF statistics doc

by Luc Ma

From: Luc Ma <luc(a)sietium.com> The kernel-doc for DMA-BUF statistics mentions /sys/kernel/dma-buf/buffers but the correct path is /sys/kernel/dmabuf/buffers instead. Signed-off-by: Luc Ma <luc(a)sietium.com> Reviewed-by: Javier Martinez Canillas <javierm(a)redhat.com> --- drivers/dma-buf/dma-buf-sysfs-stats.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/dma-buf/dma-buf-sysfs-stats.c b/drivers/dma-buf/dma-buf-sysfs-stats.c index 6cfbbf0720bd..b5b62e40ccc1 100644 --- a/drivers/dma-buf/dma-buf-sysfs-stats.c +++ b/drivers/dma-buf/dma-buf-sysfs-stats.c @@ -33,7 +33,7 @@ * into their address space. This necessitated the creation of the DMA-BUF sysfs * statistics interface to provide per-buffer information on production systems. * - * The interface at ``/sys/kernel/dma-buf/buffers`` exposes information about + * The interface at ``/sys/kernel/dmabuf/buffers`` exposes information about * every DMA-BUF when ``CONFIG_DMABUF_SYSFS_STATS`` is enabled. * * The following stats are exposed by the interface: -- 2.25.1

1 year, 5 months

[PATCH v2] scsi: lpfc: Fix potential deadlock on &ndlp->lock

by Chengfeng Ye

As &ndlp->lock is acquired by timer lpfc_els_retry_delay() under softirq context, process context code acquiring the lock &ndlp->lock should disable irq or bh, otherwise deadlock could happen if the timer preempt the execution while the lock is held in process context on the same CPU. The two lock acquisition inside lpfc_cleanup_pending_mbox() does not disable irq or softirq. [Deadlock Scenario] lpfc_cmpl_els_fdisc() -> lpfc_cleanup_pending_mbox() -> spin_lock(&ndlp->lock); <irq> -> lpfc_els_retry_delay() -> lpfc_nlp_get() -> spin_lock_irqsave(&ndlp->lock, flags); (deadlock here) This flaw was found by an experimental static analysis tool I am developing for irq-related deadlock. The patch fix the potential deadlock by spin_lock_irq() to disable irq. Signed-off-by: Chengfeng Ye <dg573847474(a)gmail.com> --- drivers/scsi/lpfc/lpfc_sli.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/drivers/scsi/lpfc/lpfc_sli.c b/drivers/scsi/lpfc/lpfc_sli.c index 58d10f8f75a7..8555f6bb9742 100644 --- a/drivers/scsi/lpfc/lpfc_sli.c +++ b/drivers/scsi/lpfc/lpfc_sli.c @@ -21049,9 +21049,9 @@ lpfc_cleanup_pending_mbox(struct lpfc_vport *vport) mb->mbox_flag |= LPFC_MBX_IMED_UNREG; restart_loop = 1; spin_unlock_irq(&phba->hbalock); - spin_lock(&ndlp->lock); + spin_lock_irq(&ndlp->lock); ndlp->nlp_flag &= ~NLP_IGNR_REG_CMPL; - spin_unlock(&ndlp->lock); + spin_unlock_irq(&ndlp->lock); spin_lock_irq(&phba->hbalock); break; } @@ -21067,9 +21067,9 @@ lpfc_cleanup_pending_mbox(struct lpfc_vport *vport) ndlp = (struct lpfc_nodelist *)mb->ctx_ndlp; mb->ctx_ndlp = NULL; if (ndlp) { - spin_lock(&ndlp->lock); + spin_lock_irq(&ndlp->lock); ndlp->nlp_flag &= ~NLP_IGNR_REG_CMPL; - spin_unlock(&ndlp->lock); + spin_unlock_irq(&ndlp->lock); lpfc_nlp_put(ndlp); } } -- 2.17.1

1 year, 5 months

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

Linaro-mm-sig