Linaro-mm-sig

linaro-mm-sig@lists.linaro.org

8 participants
3609 discussions

[PATCH] drm/virtio: check virtio_gpu_array_lock_resv() return in cursor update

by Deepanshu Kartikey

virtio_gpu_cursor_plane_update() calls virtio_gpu_array_lock_resv() but ignores its return value. The function can fail in two ways: - dma_resv_lock_interruptible() returns -ERESTARTSYS when a signal is delivered while waiting for the reservation lock. - dma_resv_reserve_fences() returns -ENOMEM if it fails to allocate a fence slot; in this case lock_resv unlocks before returning. In both cases the resv lock is not held on return. The cursor path proceeds to queue a fenced transfer command. The queue path then walks the object array and calls dma_resv_add_fence() on the cursor BO's reservation. dma_resv_add_fence() requires the resv lock to be held; with lockdep enabled the missing lock trips dma_resv_assert_held(): WARNING: drivers/dma-buf/dma-resv.c:296 at dma_resv_add_fence+0x71e/0x840 Call Trace: virtio_gpu_array_add_fence+0xcd/0x140 virtio_gpu_queue_ctrl_sgs virtio_gpu_queue_fenced_ctrl_buffer+0x578/0xfb0 virtio_gpu_cursor_plane_update+0x411/0xbc0 drm_atomic_helper_commit_planes+0x497/0xf10 ... drm_mode_cursor_ioctl+0xd4/0x110 drm_ioctl+0x5e6/0xc60 __x64_sys_ioctl+0x18e/0x210 Beyond the WARN, mutating the dma_resv fence list without the lock races with concurrent readers/writers and can corrupt the list. Check the return value of virtio_gpu_array_lock_resv(). On failure, drop the references taken by virtio_gpu_array_add_obj() with virtio_gpu_array_put_free() (which does not unlock, matching the not-locked state) and return without queueing the command. A skipped cursor frame is harmless; the WARN and the underlying race are not. The bug was reported by syzbot, triggered via fault injection (fail_nth) on the DRM_IOCTL_MODE_CURSOR path, which forces the -ENOMEM branch in dma_resv_reserve_fences(). Reported-by: syzbot+72bd3dd3a5d5f39a0271(a)syzkaller.appspotmail.com Closes: https://syzkaller.appspot.com/bug?extid=72bd3dd3a5d5f39a0271 Fixes: 5cfd31c5b3a3 ("drm/virtio: fix virtio_gpu_cursor_plane_update().") Cc: stable(a)vger.kernel.org Signed-off-by: Deepanshu Kartikey <kartikey406(a)gmail.com> --- drivers/gpu/drm/virtio/virtgpu_plane.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/virtio/virtgpu_plane.c b/drivers/gpu/drm/virtio/virtgpu_plane.c index a126d1b25f46..ca379b08b9ec 100644 --- a/drivers/gpu/drm/virtio/virtgpu_plane.c +++ b/drivers/gpu/drm/virtio/virtgpu_plane.c @@ -459,7 +459,10 @@ static void virtio_gpu_cursor_plane_update(struct drm_plane *plane, if (!objs) return; virtio_gpu_array_add_obj(objs, vgfb->base.obj[0]); - virtio_gpu_array_lock_resv(objs); + if (virtio_gpu_array_lock_resv(objs)) { + virtio_gpu_array_put_free(objs); + return; + } virtio_gpu_cmd_transfer_to_host_2d (vgdev, 0, plane->state->crtc_w, -- 2.43.0

4 months, 3 weeks

Re: [PATCH RFC 21/26] dma-buf: Add the Physical Address List DMA mapping type

by Jason Gunthorpe

On Sat, May 09, 2026 at 01:31:56PM +0800, Xu Yilun wrote: > > Would you be open to an in-between? The exporter and importer both > > have information that should not leak into each other's drivers. > > > > What if the dmabuf mapping type core code was the only thing that had > > access to *BOTH*? The exporter provides the address data, the importer > > provides the iommu_domain. The core code, and only the core code, has > > both and does the required operation? > > I think that may not work for KVM. On IOMMU side, IOMMUFD acts as the > address space (iova) manager and dma_api/IOMMU driver acts as the > actual page table mapper. But for KVM, it is both. KVM doesn't allow > another component to provide an unknown address space (GPA space) and > say "map it", so doesn't expose to other components about "KVM domain". > > Even if we expose "KVM domain", KVM still acts as the importer and the > mapper, is it wierd to say we trust KVM-the-mapper, but don't trust > KVM-the-as-manager? > > Is it also wierd that we trust IOMMU-the-mapper, but don't trust > IOMMUFD-the-as-manager? There are more IOMMU drivers than IOMMUFD... Yeah, it doesn't work well for kvm, and yes it is really weird and worse that phys in every way.. Jason

4 months, 3 weeks

Re: [RFC PATCH 04/12] vfio/pci: Allow MMIO regions to be exported through dma-buf

by Jason Gunthorpe

On Thu, May 07, 2026 at 05:16:56PM +1000, Alexey Kardashevskiy wrote: > true but either way dmabuf slicing will be directed by QEMU's > msix-table emulation MR and this slicing needs to match the TDISP > report so I'll have to teach QEMU these reports, right? I am worried > if I miss something obvious, again. Thanks, I don't think so.. It just needs to slice it into the MSI page blindly. When the VM goes to validate the TDISP report against the mappings it will fail to accept the device if there is a mismatch. The only thing qemu could do is fail sooner, but I don't know that is worth the complexity as we do expect all devices to have their MSI range unprotected. Jason

4 months, 3 weeks

[PATCH] accel/ivpu: Reject PRIME export of userptr BOs

by Ziyi Guo

Userptr BOs wrap pinned user pages in a private dma-buf solely for internal use by the NPU driver. Allowing userspace to re-export such a BO via DRM_IOCTL_PRIME_HANDLE_TO_FD would expose those pages to other drivers through an interface that was never intended to be shared. Override the driver's prime_handle_to_fd callback to detect dma-bufs backed by ivpu_gem_userptr_dmabuf_ops and reject the export with -EINVAL. Signed-off-by: Ziyi Guo <n7l8m4(a)u.northwestern.edu> --- drivers/accel/ivpu/ivpu_drv.c | 1 + drivers/accel/ivpu/ivpu_gem.c | 28 +++++++++++++++++++++++++++ drivers/accel/ivpu/ivpu_gem.h | 3 +++ drivers/accel/ivpu/ivpu_gem_userptr.c | 5 +++++ 4 files changed, 37 insertions(+) diff --git a/drivers/accel/ivpu/ivpu_drv.c b/drivers/accel/ivpu/ivpu_drv.c index 2801378e3e19..086d4c769b33 100644 --- a/drivers/accel/ivpu/ivpu_drv.c +++ b/drivers/accel/ivpu/ivpu_drv.c @@ -545,6 +545,7 @@ static const struct drm_driver driver = { .gem_create_object = ivpu_gem_create_object, .gem_prime_import = ivpu_gem_prime_import, + .prime_handle_to_fd = ivpu_gem_prime_handle_to_fd, .ioctls = ivpu_drm_ioctls, .num_ioctls = ARRAY_SIZE(ivpu_drm_ioctls), diff --git a/drivers/accel/ivpu/ivpu_gem.c b/drivers/accel/ivpu/ivpu_gem.c index 4f2005a8d496..82079f372b39 100644 --- a/drivers/accel/ivpu/ivpu_gem.c +++ b/drivers/accel/ivpu/ivpu_gem.c @@ -12,6 +12,7 @@ #include <drm/drm_cache.h> #include <drm/drm_debugfs.h> #include <drm/drm_file.h> +#include <drm/drm_prime.h> #include <drm/drm_utils.h> #include "ivpu_drv.h" @@ -249,6 +250,33 @@ struct drm_gem_object *ivpu_gem_prime_import(struct drm_device *dev, return ERR_PTR(ret); } +int ivpu_gem_prime_handle_to_fd(struct drm_device *dev, struct drm_file *file_priv, + u32 handle, u32 flags, int *prime_fd) +{ + struct ivpu_device *vdev = to_ivpu_device(dev); + struct dma_buf *dmabuf; + int fd; + + dmabuf = drm_gem_prime_handle_to_dmabuf(dev, file_priv, handle, flags); + if (IS_ERR(dmabuf)) + return PTR_ERR(dmabuf); + + if (ivpu_gem_is_userptr_dma_buf(dmabuf)) { + ivpu_dbg(vdev, IOCTL, "Exporting userptr BO is not allowed\n"); + dma_buf_put(dmabuf); + return -EINVAL; + } + + fd = dma_buf_fd(dmabuf, flags); + if (fd < 0) { + dma_buf_put(dmabuf); + return fd; + } + + *prime_fd = fd; + return 0; +} + static struct ivpu_bo *ivpu_bo_alloc(struct ivpu_device *vdev, u64 size, u32 flags) { struct drm_gem_shmem_object *shmem; diff --git a/drivers/accel/ivpu/ivpu_gem.h b/drivers/accel/ivpu/ivpu_gem.h index 0c3350f22b55..bfd15ce02354 100644 --- a/drivers/accel/ivpu/ivpu_gem.h +++ b/drivers/accel/ivpu/ivpu_gem.h @@ -29,6 +29,9 @@ void ivpu_bo_unbind_all_bos_from_context(struct ivpu_device *vdev, struct ivpu_m struct drm_gem_object *ivpu_gem_create_object(struct drm_device *dev, size_t size); struct drm_gem_object *ivpu_gem_prime_import(struct drm_device *dev, struct dma_buf *dma_buf); +int ivpu_gem_prime_handle_to_fd(struct drm_device *dev, struct drm_file *file_priv, + u32 handle, u32 flags, int *prime_fd); +bool ivpu_gem_is_userptr_dma_buf(struct dma_buf *dma_buf); struct ivpu_bo *ivpu_bo_create(struct ivpu_device *vdev, struct ivpu_mmu_context *ctx, struct ivpu_addr_range *range, u64 size, u32 flags); struct ivpu_bo *ivpu_bo_create_runtime(struct ivpu_device *vdev, u64 addr, u64 size, u32 flags); diff --git a/drivers/accel/ivpu/ivpu_gem_userptr.c b/drivers/accel/ivpu/ivpu_gem_userptr.c index 7cbf3a4cdc73..45eabea5961e 100644 --- a/drivers/accel/ivpu/ivpu_gem_userptr.c +++ b/drivers/accel/ivpu/ivpu_gem_userptr.c @@ -61,6 +61,11 @@ static const struct dma_buf_ops ivpu_gem_userptr_dmabuf_ops = { .release = ivpu_gem_userptr_dmabuf_release, }; +bool ivpu_gem_is_userptr_dma_buf(struct dma_buf *dma_buf) +{ + return dma_buf->ops == &ivpu_gem_userptr_dmabuf_ops; +} + static struct dma_buf * ivpu_create_userptr_dmabuf(struct ivpu_device *vdev, void __user *user_ptr, size_t size, uint32_t flags) -- 2.34.1

4 months, 3 weeks

[PATCH 0/8] drm/panthor: Protected mode support for Mali CSF GPUs

by Ketil Johnsen

Hi, This is a patch series covering the support for protected mode execution in Mali Panthor CSF kernel driver. It builds on the initial RFC posted by Florent Tomasin back in January of 2025. The initial RFC can be found here: https://lore.kernel.org/lkml/cover.1738228114.git.florent.tomasin@arm.com/ The Mali CSF GPUs come with the support for protected mode execution at the HW level. This feature requires two main changes in the kernel driver: 1) Configure the GPU with a protected buffer. The system must provide a DMA heap from which the driver can allocate a protected buffer. It can be a carved-out memory or dynamically allocated protected memory region. Some system includes a trusted FW which is in charge of the protected memory. Since this problem is integration specific, the Mali Panthor CSF kernel driver must import the protected memory from a device specific exporter. 2) Handle enter and exit of the GPU HW from normal to protected mode of execution. FW sends a request for protected mode entry to the kernel driver. The acknowledgment of that request is a scheduling decision. Effectively, protected mode execution should not overrule normal mode of execution. A fair distribution of execution time will guaranty the overall performance of the device, including the UI (usually executing in normal mode), will not regress when a protected mode job is submitted by an application. Background ---------- Current Mali Panthor CSF driver does not allow a user space application to execute protected jobs on the GPU. This use case is quite common on end-user-device. A user may want to watch a video or render content that is under a "Digital Right Management" protection, or launch an application with user private data. 1) User-space: In order for an application to execute protected jobs on a Mali CSF GPU the user space application must submit jobs to the GPU within a "protected regions" (range of commands to execute in protected mode). Find here an example of a command buffer that contains protected commands: ``` <--- Normal mode ---><--- Protected mode ---><--- Normal mode ---> +-------------------------------------------------------------------------+ | ... | CMD_0 | ... | CMD_N | PROT_REGION | CMD_N+1 | ... | CMD_N+M | ... | +-------------------------------------------------------------------------+ ``` The PROT_REGION command acts as a barrier to notify the HW of upcoming protected jobs. It also defines the number of commands to execute in protected mode. The Mesa definition of the opcode can be found here: https://gitlab.freedesktop.org/mesa/mesa/-/blob/main/src/panfrost/lib/genxm… 2) Kernel-space: When loading the FW image, the Kernel driver must also load the data section of CSF FW that comes from the protected memory, in order to allow FW to execute in protected mode. Important: this memory is not owned by any process. It is a GPU device level protected memory. In addition, when a CSG (group) is created, it must have a protected suspend buffer. This memory is allocated within the kernel but bound to a specific CSG that belongs to a process. The kernel owns this allocation and does not allow user space mapping. The format of the data in this buffer is only known by the FW and does not need to be shared with other entities. The purpose of this buffer is the same as the normal suspend buffer but for protected mode. FW will use it to suspend the execution of PROT_REGION before returning to normal mode of execution. Design decisions ---------------- The Mali Panthor CSF kernel driver will allocate protected DMA buffers using a global protected DMA heap. The name of the heap can vary on the system and is integration specific. Therefore, the kernel driver will retrieve it using the DTB entry: "protected-heap-name". The Mali Panthor CSF kernel driver will handle enter/exit of protected mode with a fair consideration of the job scheduling. If the system integrator does not provide a protected DMA heap, the driver will not allow any protected mode execution. Patch series ------------ [PATCHES 1-2]: Thees patches comes from the following patch series: https://lore.kernel.org/lkml/20240720071606.27930-1-yunfei.dong@mediatek.co… These extend the DMA-buf heap API to allow other kernel drivers to Find and allocate memory from dma heaps. Note: This patch series do not include a protected DMA heap, as this is platform specific. * dma-heap: Add proper kref handling on dma-buf heaps * dma-heap: Provide accessors so that in-kernel drivers can allocate dmabufs from specific heaps [PATCHES 3, 5 and 6]: These are refactoring to aid the implementation of the protected rendering feature itself. * drm/panthor: De-duplicate FW memory section sync * drm/panthor: Minor scheduler refactoring * drm/panthor: Explicit expansion of locked VM region [Patch 4]: This introduces allocation of protected memory inside the Panthor driver. It also ensures the protected FW sections are loaded. * drm/panthor: Add support for protected memory allocation in panthor [PATCH 7]: This patch implements the logic to handle enter/exit of the GPU protected mode in Panthor CSF driver. Note: to prevent scheduler priority inversion, only a single CSG is allowed to execute while in protected mode. It must be the top priority one. * drm/panthor: Add support for entering and exiting protected mode [PATCH 8]: The final patch exposes this feature via the uAPI. * drm/panthor: Expose protected rendering features Testing ------- 1) Platform and development environment Any platform containing a Mali CSF type of GPU and a protected memory allocator that is based on DMA Heap can be used. For example, it can be a physical platform or a simulator such as Arm Total Compute FVPs platforms. Reference to the latter: https://developer.arm.com/Tools%20and%20Software/Fixed%20Virtual%20Platform… 2) Mesa: PanVK support can be found here: https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/40044 This is still work in progress. Constraints ----------- At the time of developing the feature, Linux kernel does not have a generic way of implementing protected DMA heaps. The patch series relies on previous work to expose the DMA heap API to the kernel drivers. The Mali CSF GPU requires device level allocated protected memory, which do not belong to a process. The current Linux implementation of DMA heap only allows a user space program to allocate from such heap. Having the ability to allocate this memory at the kernel level via the DMA heap API would allow support for protected mode on Mali CSF GPUs. Florent Tomasin (3): drm/panthor: Add support for protected memory allocation in panthor drm/panthor: Minor scheduler refactoring drm/panthor: Add support for entering and exiting protected mode John Stultz (2): dma-heap: Add proper kref handling on dma-buf heaps dma-heap: Provide accessors so that in-kernel drivers can allocate dmabufs from specific heaps Ketil Johnsen (3): drm/panthor: De-duplicate FW memory section sync drm/panthor: Explicit expansion of locked VM region drm/panthor: Expose protected rendering features Documentation/gpu/panthor.rst | 47 +++ drivers/dma-buf/dma-heap.c | 109 ++++++- drivers/gpu/drm/panthor/Kconfig | 1 + drivers/gpu/drm/panthor/panthor_device.c | 29 +- drivers/gpu/drm/panthor/panthor_device.h | 15 + drivers/gpu/drm/panthor/panthor_drv.c | 21 +- drivers/gpu/drm/panthor/panthor_fw.c | 137 ++++++-- drivers/gpu/drm/panthor/panthor_fw.h | 7 + drivers/gpu/drm/panthor/panthor_gem.c | 77 ++++- drivers/gpu/drm/panthor/panthor_gem.h | 16 +- drivers/gpu/drm/panthor/panthor_gpu.c | 14 +- drivers/gpu/drm/panthor/panthor_gpu.h | 6 + drivers/gpu/drm/panthor/panthor_heap.c | 2 + drivers/gpu/drm/panthor/panthor_mmu.c | 79 +++-- drivers/gpu/drm/panthor/panthor_sched.c | 387 ++++++++++++++++++----- include/linux/dma-heap.h | 8 + include/uapi/drm/panthor_drm.h | 45 ++- 17 files changed, 864 insertions(+), 136 deletions(-) -- 2.43.0

5 months

Re: [RFC PATCH 04/12] vfio/pci: Allow MMIO regions to be exported through dma-buf

by Alexey Kardashevskiy

Hi! Let's reignite this topic. I've been using these patches + QEMU side hacks for 6+ months. And it's been fine until I got a device where MSIX BAR is in a middle of another BAR marked as TEE in the TDISP interface report. And no trusted MSIX yet. Every time QEMU mmaps a BAR - I request a dmabuf fd from VFIO in QEMU. Since mapping of an entire MSIX BAR is allowed by default, VFIORegion::nr_mmaps==1 and it is an entire BAR. Problem: KVM memslot mismatches the dmabuf fd size How: QEMU emulates MSIX table and PBA by adding emulated MemoryRegions on top of the mapped BARs. In the QEMU memory flatview this splits the BAR into 2 or 3 sections (2 if MSIX at the start/end, 3 if in a middle). QEMU tries registering 1 or 2 KVM memory slots for regions outside of MSIX which fails in kvm_vfio_dmabuf_bind() as regions are smaller than the exported dmabuf fd (which is the entire BAR == 32KB). Solution1: use QEMU x-msix-relocation hack to move MSIX elsewhere - end of some BAR (doubles the bar size so problematic for huge BARs like 512GB+), or another BAR (there may be no available as 3 of 64bit BARs is the limit). Solution2: modify logic in VFIO dmabuf to allow multiple KVM memory slots per dmabuf. Now it is kvm_memory_slot::dmabuf_attach with no offset into the dmabuf and one kvm_vfio_dmabuf per dma_buf. Solution3: hack QEMU into smashing a MSIX-containing BAR in vfio_pci_fixup_msix_region. Here is what it does: 0000380004000000-0000380004002fff (prio 0, ramd): 0000:41:04.0 BAR 4 mmaps[0] KVM 0000380004003000-0000380004006fff (prio 0, i/o): msix-table 0000380004007000-0000380004007fff (prio 0, ramd): 0000:41:04.0 BAR 4 mmaps[1] KVM The problem now is that the TDI report must have the same split of the BAR as the VM is going to validate TEE (==trusted) MMIO ranges and this has to match the QEMU view. Harder than it sounds as the size of MSIX table in bytes is not exactly specified anywhere except the report. Solution4: the above + modify QEMU to check the TDI report for not reporting MSIX+PBA where QEMU emulates them. The problem is that when QEMU adds emulated MRs - there is no report yet (the report is created later on, some TDISP witchery). So when the device is accepted - QEMU could reinitialize those emulated msix and pba MRs to match the report exactly so the flatview generates correct KVM memory regions and then KVM can work with dmabuf fine. Any thoughts? what is acceptable for everybody? Thanks, On 8/1/25 01:27, Xu Yilun wrote: > From: Vivek Kasireddy <vivek.kasireddy(a)intel.com> > > This is a reduced version of Vivek's series [1]. Removed the > dma_buf_ops.attach/map/unmap_dma_buf/mmap() as they are not necessary > in this series, also because of the WIP p2p dma mapping opens [2]. Just > focus on the private MMIO get PFN function in this early stage. > >>From Jason Gunthorpe: > "dma-buf has become a way to safely acquire a handle to non-struct page > memory that can still have lifetime controlled by the exporter. Notably > RDMA can now import dma-buf FDs and build them into MRs which allows for > PCI P2P operations. Extend this to allow vfio-pci to export MMIO memory > from PCI device BARs. > > The patch design loosely follows the pattern in commit > db1a8dd916aa ("habanalabs: add support for dma-buf exporter") except this > does not support pinning. > > Instead, this implements what, in the past, we've called a revocable > attachment using move. In normal situations the attachment is pinned, as a > BAR does not change physical address. However when the VFIO device is > closed, or a PCI reset is issued, access to the MMIO memory is revoked. > > Revoked means that move occurs, but an attempt to immediately re-map the > memory will fail. In the reset case a future move will be triggered when > MMIO access returns. As both close and reset are under userspace control > it is expected that userspace will suspend use of the dma-buf before doing > these operations, the revoke is purely for kernel self-defense against a > hostile userspace." > > [1] https://lore.kernel.org/kvm/20240624065552.1572580-4-vivek.kasireddy@intel.… > [2] https://lore.kernel.org/all/IA0PR11MB7185FDD56CFDD0A2B8D21468F83B2@IA0PR11M… > > Original-patch-by: Jason Gunthorpe <jgg(a)nvidia.com> > Signed-off-by: Vivek Kasireddy <vivek.kasireddy(a)intel.com> > Signed-off-by: Xu Yilun <yilun.xu(a)linux.intel.com> > --- > drivers/vfio/pci/Makefile | 1 + > drivers/vfio/pci/dma_buf.c | 223 +++++++++++++++++++++++++++++ > drivers/vfio/pci/vfio_pci_config.c | 22 ++- > drivers/vfio/pci/vfio_pci_core.c | 20 ++- > drivers/vfio/pci/vfio_pci_priv.h | 25 ++++ > include/linux/vfio_pci_core.h | 1 + > include/uapi/linux/vfio.h | 29 ++++ > 7 files changed, 316 insertions(+), 5 deletions(-) > create mode 100644 drivers/vfio/pci/dma_buf.c > > diff --git a/drivers/vfio/pci/Makefile b/drivers/vfio/pci/Makefile > index cf00c0a7e55c..0cfdc9ede82f 100644 > --- a/drivers/vfio/pci/Makefile > +++ b/drivers/vfio/pci/Makefile > @@ -2,6 +2,7 @@ > > vfio-pci-core-y := vfio_pci_core.o vfio_pci_intrs.o vfio_pci_rdwr.o vfio_pci_config.o > vfio-pci-core-$(CONFIG_VFIO_PCI_ZDEV_KVM) += vfio_pci_zdev.o > +vfio-pci-core-$(CONFIG_DMA_SHARED_BUFFER) += dma_buf.o > obj-$(CONFIG_VFIO_PCI_CORE) += vfio-pci-core.o > > vfio-pci-y := vfio_pci.o > diff --git a/drivers/vfio/pci/dma_buf.c b/drivers/vfio/pci/dma_buf.c > new file mode 100644 > index 000000000000..1d5f46744922 > --- /dev/null > +++ b/drivers/vfio/pci/dma_buf.c > @@ -0,0 +1,223 @@ > +// SPDX-License-Identifier: GPL-2.0-only > +/* Copyright (c) 2024, NVIDIA CORPORATION & AFFILIATES. > + */ > +#include <linux/dma-buf.h> > +#include <linux/dma-resv.h> > + > +#include "vfio_pci_priv.h" > + > +MODULE_IMPORT_NS("DMA_BUF"); > + > +struct vfio_pci_dma_buf { > + struct dma_buf *dmabuf; > + struct vfio_pci_core_device *vdev; > + struct list_head dmabufs_elm; > + unsigned int nr_ranges; > + struct vfio_region_dma_range *dma_ranges; > + bool revoked; > +}; > + > +static void vfio_pci_dma_buf_unpin(struct dma_buf_attachment *attachment) > +{ > +} > + > +static int vfio_pci_dma_buf_pin(struct dma_buf_attachment *attachment) > +{ > + /* > + * Uses the dynamic interface but must always allow for > + * dma_buf_move_notify() to do revoke > + */ > + return -EINVAL; > +} > + > +static int vfio_pci_dma_buf_get_pfn(struct dma_buf_attachment *attachment, > + pgoff_t pgoff, u64 *pfn, int *max_order) > +{ > + /* TODO */ > + return -EOPNOTSUPP; > +} > + > +static void vfio_pci_dma_buf_release(struct dma_buf *dmabuf) > +{ > + struct vfio_pci_dma_buf *priv = dmabuf->priv; > + > + /* > + * Either this or vfio_pci_dma_buf_cleanup() will remove from the list. > + * The refcount prevents both. > + */ > + if (priv->vdev) { > + down_write(&priv->vdev->memory_lock); > + list_del_init(&priv->dmabufs_elm); > + up_write(&priv->vdev->memory_lock); > + vfio_device_put_registration(&priv->vdev->vdev); > + } > + kfree(priv); > +} > + > +static const struct dma_buf_ops vfio_pci_dmabuf_ops = { > + .pin = vfio_pci_dma_buf_pin, > + .unpin = vfio_pci_dma_buf_unpin, > + .get_pfn = vfio_pci_dma_buf_get_pfn, > + .release = vfio_pci_dma_buf_release, > +}; > + > +static int check_dma_ranges(struct vfio_pci_dma_buf *priv, u64 *dmabuf_size) > +{ > + struct vfio_region_dma_range *dma_ranges = priv->dma_ranges; > + struct pci_dev *pdev = priv->vdev->pdev; > + resource_size_t bar_size; > + int i; > + > + for (i = 0; i < priv->nr_ranges; i++) { > + /* > + * For PCI the region_index is the BAR number like > + * everything else. > + */ > + if (dma_ranges[i].region_index >= VFIO_PCI_ROM_REGION_INDEX) > + return -EINVAL; > + > + bar_size = pci_resource_len(pdev, dma_ranges[i].region_index); > + if (!bar_size) > + return -EINVAL; > + > + if (!dma_ranges[i].offset && !dma_ranges[i].length) > + dma_ranges[i].length = bar_size; > + > + if (!IS_ALIGNED(dma_ranges[i].offset, PAGE_SIZE) || > + !IS_ALIGNED(dma_ranges[i].length, PAGE_SIZE) || > + dma_ranges[i].length > bar_size || > + dma_ranges[i].offset >= bar_size || > + dma_ranges[i].offset + dma_ranges[i].length > bar_size) > + return -EINVAL; > + > + *dmabuf_size += dma_ranges[i].length; > + } > + > + return 0; > +} > + > +int vfio_pci_core_feature_dma_buf(struct vfio_pci_core_device *vdev, u32 flags, > + struct vfio_device_feature_dma_buf __user *arg, > + size_t argsz) > +{ > + struct vfio_device_feature_dma_buf get_dma_buf; > + struct vfio_region_dma_range *dma_ranges; > + DEFINE_DMA_BUF_EXPORT_INFO(exp_info); > + struct vfio_pci_dma_buf *priv; > + u64 dmabuf_size = 0; > + int ret; > + > + ret = vfio_check_feature(flags, argsz, VFIO_DEVICE_FEATURE_GET, > + sizeof(get_dma_buf)); > + if (ret != 1) > + return ret; > + > + if (copy_from_user(&get_dma_buf, arg, sizeof(get_dma_buf))) > + return -EFAULT; > + > + dma_ranges = memdup_array_user(&arg->dma_ranges, > + get_dma_buf.nr_ranges, > + sizeof(*dma_ranges)); > + if (IS_ERR(dma_ranges)) > + return PTR_ERR(dma_ranges); > + > + priv = kzalloc(sizeof(*priv), GFP_KERNEL); > + if (!priv) { > + kfree(dma_ranges); > + return -ENOMEM; > + } > + > + priv->vdev = vdev; > + priv->nr_ranges = get_dma_buf.nr_ranges; > + priv->dma_ranges = dma_ranges; > + > + ret = check_dma_ranges(priv, &dmabuf_size); > + if (ret) > + goto err_free_priv; > + > + if (!vfio_device_try_get_registration(&vdev->vdev)) { > + ret = -ENODEV; > + goto err_free_priv; > + } > + > + exp_info.ops = &vfio_pci_dmabuf_ops; > + exp_info.size = dmabuf_size; > + exp_info.flags = get_dma_buf.open_flags; > + exp_info.priv = priv; > + > + priv->dmabuf = dma_buf_export(&exp_info); > + if (IS_ERR(priv->dmabuf)) { > + ret = PTR_ERR(priv->dmabuf); > + goto err_dev_put; > + } > + > + /* dma_buf_put() now frees priv */ > + INIT_LIST_HEAD(&priv->dmabufs_elm); > + down_write(&vdev->memory_lock); > + dma_resv_lock(priv->dmabuf->resv, NULL); > + priv->revoked = !__vfio_pci_memory_enabled(vdev); > + list_add_tail(&priv->dmabufs_elm, &vdev->dmabufs); > + dma_resv_unlock(priv->dmabuf->resv); > + up_write(&vdev->memory_lock); > + > + /* > + * dma_buf_fd() consumes the reference, when the file closes the dmabuf > + * will be released. > + */ > + return dma_buf_fd(priv->dmabuf, get_dma_buf.open_flags); > + > +err_dev_put: > + vfio_device_put_registration(&vdev->vdev); > +err_free_priv: > + kfree(dma_ranges); > + kfree(priv); > + return ret; > +} > + > +void vfio_pci_dma_buf_move(struct vfio_pci_core_device *vdev, bool revoked) > +{ > + struct vfio_pci_dma_buf *priv; > + struct vfio_pci_dma_buf *tmp; > + > + lockdep_assert_held_write(&vdev->memory_lock); > + > + list_for_each_entry_safe(priv, tmp, &vdev->dmabufs, dmabufs_elm) { > + /* > + * Returns true if a reference was successfully obtained. > + * The caller must interlock with the dmabuf's release > + * function in some way, such as RCU, to ensure that this > + * is not called on freed memory. > + */ > + if (!get_file_rcu(&priv->dmabuf->file)) > + continue; > + > + if (priv->revoked != revoked) { > + dma_resv_lock(priv->dmabuf->resv, NULL); > + priv->revoked = revoked; > + dma_buf_move_notify(priv->dmabuf); > + dma_resv_unlock(priv->dmabuf->resv); > + } > + dma_buf_put(priv->dmabuf); > + } > +} > + > +void vfio_pci_dma_buf_cleanup(struct vfio_pci_core_device *vdev) > +{ > + struct vfio_pci_dma_buf *priv; > + struct vfio_pci_dma_buf *tmp; > + > + down_write(&vdev->memory_lock); > + list_for_each_entry_safe(priv, tmp, &vdev->dmabufs, dmabufs_elm) { > + if (!get_file_rcu(&priv->dmabuf->file)) > + continue; > + dma_resv_lock(priv->dmabuf->resv, NULL); > + list_del_init(&priv->dmabufs_elm); > + priv->vdev = NULL; > + priv->revoked = true; > + dma_buf_move_notify(priv->dmabuf); > + dma_resv_unlock(priv->dmabuf->resv); > + vfio_device_put_registration(&vdev->vdev); > + dma_buf_put(priv->dmabuf); > + } > + up_write(&vdev->memory_lock); > +} > diff --git a/drivers/vfio/pci/vfio_pci_config.c b/drivers/vfio/pci/vfio_pci_config.c > index ea2745c1ac5e..5cc200e15edc 100644 > --- a/drivers/vfio/pci/vfio_pci_config.c > +++ b/drivers/vfio/pci/vfio_pci_config.c > @@ -589,10 +589,12 @@ static int vfio_basic_config_write(struct vfio_pci_core_device *vdev, int pos, > virt_mem = !!(le16_to_cpu(*virt_cmd) & PCI_COMMAND_MEMORY); > new_mem = !!(new_cmd & PCI_COMMAND_MEMORY); > > - if (!new_mem) > + if (!new_mem) { > vfio_pci_zap_and_down_write_memory_lock(vdev); > - else > + vfio_pci_dma_buf_move(vdev, true); > + } else { > down_write(&vdev->memory_lock); > + } > > /* > * If the user is writing mem/io enable (new_mem/io) and we > @@ -627,6 +629,8 @@ static int vfio_basic_config_write(struct vfio_pci_core_device *vdev, int pos, > *virt_cmd &= cpu_to_le16(~mask); > *virt_cmd |= cpu_to_le16(new_cmd & mask); > > + if (__vfio_pci_memory_enabled(vdev)) > + vfio_pci_dma_buf_move(vdev, false); > up_write(&vdev->memory_lock); > } > > @@ -707,12 +711,16 @@ static int __init init_pci_cap_basic_perm(struct perm_bits *perm) > static void vfio_lock_and_set_power_state(struct vfio_pci_core_device *vdev, > pci_power_t state) > { > - if (state >= PCI_D3hot) > + if (state >= PCI_D3hot) { > vfio_pci_zap_and_down_write_memory_lock(vdev); > - else > + vfio_pci_dma_buf_move(vdev, true); > + } else { > down_write(&vdev->memory_lock); > + } > > vfio_pci_set_power_state(vdev, state); > + if (__vfio_pci_memory_enabled(vdev)) > + vfio_pci_dma_buf_move(vdev, false); > up_write(&vdev->memory_lock); > } > > @@ -900,7 +908,10 @@ static int vfio_exp_config_write(struct vfio_pci_core_device *vdev, int pos, > > if (!ret && (cap & PCI_EXP_DEVCAP_FLR)) { > vfio_pci_zap_and_down_write_memory_lock(vdev); > + vfio_pci_dma_buf_move(vdev, true); > pci_try_reset_function(vdev->pdev); > + if (__vfio_pci_memory_enabled(vdev)) > + vfio_pci_dma_buf_move(vdev, true); > up_write(&vdev->memory_lock); > } > } > @@ -982,7 +993,10 @@ static int vfio_af_config_write(struct vfio_pci_core_device *vdev, int pos, > > if (!ret && (cap & PCI_AF_CAP_FLR) && (cap & PCI_AF_CAP_TP)) { > vfio_pci_zap_and_down_write_memory_lock(vdev); > + vfio_pci_dma_buf_move(vdev, true); > pci_try_reset_function(vdev->pdev); > + if (__vfio_pci_memory_enabled(vdev)) > + vfio_pci_dma_buf_move(vdev, true); > up_write(&vdev->memory_lock); > } > } > diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c > index c3269d708411..f69eda5956ad 100644 > --- a/drivers/vfio/pci/vfio_pci_core.c > +++ b/drivers/vfio/pci/vfio_pci_core.c > @@ -287,6 +287,8 @@ static int vfio_pci_runtime_pm_entry(struct vfio_pci_core_device *vdev, > * semaphore. > */ > vfio_pci_zap_and_down_write_memory_lock(vdev); > + vfio_pci_dma_buf_move(vdev, true); > + > if (vdev->pm_runtime_engaged) { > up_write(&vdev->memory_lock); > return -EINVAL; > @@ -370,6 +372,8 @@ static void vfio_pci_runtime_pm_exit(struct vfio_pci_core_device *vdev) > */ > down_write(&vdev->memory_lock); > __vfio_pci_runtime_pm_exit(vdev); > + if (__vfio_pci_memory_enabled(vdev)) > + vfio_pci_dma_buf_move(vdev, false); > up_write(&vdev->memory_lock); > } > > @@ -690,6 +694,8 @@ void vfio_pci_core_close_device(struct vfio_device *core_vdev) > #endif > vfio_pci_core_disable(vdev); > > + vfio_pci_dma_buf_cleanup(vdev); > + > mutex_lock(&vdev->igate); > if (vdev->err_trigger) { > eventfd_ctx_put(vdev->err_trigger); > @@ -1234,7 +1240,10 @@ static int vfio_pci_ioctl_reset(struct vfio_pci_core_device *vdev, > */ > vfio_pci_set_power_state(vdev, PCI_D0); > > + vfio_pci_dma_buf_move(vdev, true); > ret = pci_try_reset_function(vdev->pdev); > + if (__vfio_pci_memory_enabled(vdev)) > + vfio_pci_dma_buf_move(vdev, false); > up_write(&vdev->memory_lock); > > return ret; > @@ -1523,6 +1532,8 @@ int vfio_pci_core_ioctl_feature(struct vfio_device *device, u32 flags, > return vfio_pci_core_pm_exit(vdev, flags, arg, argsz); > case VFIO_DEVICE_FEATURE_PCI_VF_TOKEN: > return vfio_pci_core_feature_token(vdev, flags, arg, argsz); > + case VFIO_DEVICE_FEATURE_DMA_BUF: > + return vfio_pci_core_feature_dma_buf(vdev, flags, arg, argsz); > default: > return -ENOTTY; > } > @@ -2098,6 +2109,7 @@ int vfio_pci_core_init_dev(struct vfio_device *core_vdev) > INIT_LIST_HEAD(&vdev->dummy_resources_list); > INIT_LIST_HEAD(&vdev->ioeventfds_list); > INIT_LIST_HEAD(&vdev->sriov_pfs_item); > + INIT_LIST_HEAD(&vdev->dmabufs); > init_rwsem(&vdev->memory_lock); > xa_init(&vdev->ctx); > > @@ -2480,11 +2492,17 @@ static int vfio_pci_dev_set_hot_reset(struct vfio_device_set *dev_set, > * cause the PCI config space reset without restoring the original > * state (saved locally in 'vdev->pm_save'). > */ > - list_for_each_entry(vdev, &dev_set->device_list, vdev.dev_set_list) > + list_for_each_entry(vdev, &dev_set->device_list, vdev.dev_set_list) { > + vfio_pci_dma_buf_move(vdev, true); > vfio_pci_set_power_state(vdev, PCI_D0); > + } > > ret = pci_reset_bus(pdev); > > + list_for_each_entry(vdev, &dev_set->device_list, vdev.dev_set_list) > + if (__vfio_pci_memory_enabled(vdev)) > + vfio_pci_dma_buf_move(vdev, false); > + > vdev = list_last_entry(&dev_set->device_list, > struct vfio_pci_core_device, vdev.dev_set_list); > > diff --git a/drivers/vfio/pci/vfio_pci_priv.h b/drivers/vfio/pci/vfio_pci_priv.h > index 5e4fa69aee16..d27f383f3931 100644 > --- a/drivers/vfio/pci/vfio_pci_priv.h > +++ b/drivers/vfio/pci/vfio_pci_priv.h > @@ -101,4 +101,29 @@ static inline bool vfio_pci_is_vga(struct pci_dev *pdev) > return (pdev->class >> 8) == PCI_CLASS_DISPLAY_VGA; > } > > +#ifdef CONFIG_DMA_SHARED_BUFFER > +int vfio_pci_core_feature_dma_buf(struct vfio_pci_core_device *vdev, u32 flags, > + struct vfio_device_feature_dma_buf __user *arg, > + size_t argsz); > +void vfio_pci_dma_buf_cleanup(struct vfio_pci_core_device *vdev); > +void vfio_pci_dma_buf_move(struct vfio_pci_core_device *vdev, bool revoked); > +#else > +static int > +vfio_pci_core_feature_dma_buf(struct vfio_pci_core_device *vdev, u32 flags, > + struct vfio_device_feature_dma_buf __user *arg, > + size_t argsz) > +{ > + return -ENOTTY; > +} > + > +static inline void vfio_pci_dma_buf_cleanup(struct vfio_pci_core_device *vdev) > +{ > +} > + > +static inline void vfio_pci_dma_buf_move(struct vfio_pci_core_device *vdev, > + bool revoked) > +{ > +} > +#endif > + > #endif > diff --git a/include/linux/vfio_pci_core.h b/include/linux/vfio_pci_core.h > index fbb472dd99b3..da5d8955ae56 100644 > --- a/include/linux/vfio_pci_core.h > +++ b/include/linux/vfio_pci_core.h > @@ -94,6 +94,7 @@ struct vfio_pci_core_device { > struct vfio_pci_core_device *sriov_pf_core_dev; > struct notifier_block nb; > struct rw_semaphore memory_lock; > + struct list_head dmabufs; > }; > > /* Will be exported for vfio pci drivers usage */ > diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h > index c8dbf8219c4f..f43dfbde7352 100644 > --- a/include/uapi/linux/vfio.h > +++ b/include/uapi/linux/vfio.h > @@ -1458,6 +1458,35 @@ struct vfio_device_feature_bus_master { > }; > #define VFIO_DEVICE_FEATURE_BUS_MASTER 10 > > +/** > + * Upon VFIO_DEVICE_FEATURE_GET create a dma_buf fd for the > + * regions selected. > + * > + * For struct struct vfio_device_feature_dma_buf, open_flags are the typical > + * flags passed to open(2), eg O_RDWR, O_CLOEXEC, etc. nr_ranges is the total > + * number of dma_ranges that comprise the dmabuf. > + * > + * For struct vfio_region_dma_range, region_index/offset/length specify a slice > + * of the region to create the dmabuf from, if both offset & length are 0 then > + * the whole region is used. > + * > + * Return: The fd number on success, -1 and errno is set on failure. > + */ > +struct vfio_region_dma_range { > + __u32 region_index; > + __u32 __pad; > + __u64 offset; > + __u64 length; > +}; > + > +struct vfio_device_feature_dma_buf { > + __u32 open_flags; > + __u32 nr_ranges; > + struct vfio_region_dma_range dma_ranges[]; > +}; > + > +#define VFIO_DEVICE_FEATURE_DMA_BUF 11 > + > /* -------- API for Type1 VFIO IOMMU -------- */ > > /** -- Alexey

5 months

[PATCH 0/4] Let userspace explicitly trigger memory reclaims

by Nicolas Frattaroli

RAM is not, in fact, cheap. Especially on embedded systems with a low amount of memory, but known and well-defined userspace, more explicit resource management can lead to better utilisation patterns. As an example, a resource manager process on a purpose-built device may wish to launch, and then explicitly swap out, memory of processes that are kept "warm", to improve perceived startup latency of individual full-screen applications without making the kernel figure out the usage pattern from observation alone in order to swap out the right pages. To allow for this explicit control in the context of panthor's GPU memory, add two new sysfs knobs. The first, mem_reclaim, runs an explicit priv BO reclaim cycle on the TGID written to it. The second, mem_claim, does the opposite: it swaps BOs back into active memory. Signed-off-by: Nicolas Frattaroli <nicolas.frattaroli(a)collabora.com> --- Nicolas Frattaroli (4): drm/panthor: Add freed_sz parameter to reclaim_priv_bos MAINTAINERS: Add sysfs ABI docs to list of panthor files drm/panthor: Add explicit memory reclaim sysfs knob drm/panthor: Add explicit memory claim sysfs knob Documentation/ABI/testing/sysfs-driver-panthor-mem | 34 ++++++++ MAINTAINERS | 1 + drivers/gpu/drm/panthor/panthor_drv.c | 93 ++++++++++++++++++++++ drivers/gpu/drm/panthor/panthor_gem.c | 7 +- drivers/gpu/drm/panthor/panthor_gem.h | 1 + drivers/gpu/drm/panthor/panthor_mmu.c | 70 +++++++++++++++- drivers/gpu/drm/panthor/panthor_mmu.h | 4 + 7 files changed, 205 insertions(+), 5 deletions(-) --- base-commit: 2c4b906cd135bbb44855287d0d0eff0ee0b47afe change-id: 20260506-panthor-explicit-reclaim-3dffed028d8c Best regards, -- Nicolas Frattaroli <nicolas.frattaroli(a)collabora.com>

5 months

[PATCH v2 00/10] dma-bug: heaps: Add Tegra VPR support

by Thierry Reding

From: Thierry Reding <treding(a)nvidia.com> Hi, This series adds support for the video protection region (VPR) used on Tegra SoC devices. It's a special region of memory that is protected from accesses by the CPU and used to store DRM protected content (both decrypted stream data as well as decoded video frames). Patches 1 and 2 add DT binding documentation for the VPR and add the VPR to the list of memory-region items for display and host1x. Patch 3 adds bitmap_allocate(), which is like bitmap_allocate_region() but works on sizes that are not a power of two. Patch 4 introduces new APIs needed by the Tegra VPR implementation that allow CMA areas to be dynamically created at runtime rather than using the fixed, system-wide list. This is used in this driver specifically because it can use an arbitrary number of these areas (though they are currently limited to 4). Patch 5 adds some infrastructure for DMA heap implementations to provide information through debugfs. The Tegra VPR implementation is added in patch 6. See its commit message for more details about the specifics of this implementation. Finally, patches 7-10 add the VPR placeholder node on Tegra234 and hook it up to the host1x and GPU nodes so that they can make use of this region. Changes in v2: - Tegra VPR implementation is now more optimized to reduce the number of (very slow) resize operations, and allows cross-chunk allocations - dynamic CMA areas are now trackd separately from static ones, but the global number of CMA pages accounts for all areas Thierry Thierry Reding (10): dt-bindings: reserved-memory: Document Tegra VPR dt-bindings: display: tegra: Document memory regions bitmap: Add bitmap_allocate() function mm/cma: Allow dynamically creating CMA areas dma-buf: heaps: Add debugfs support dma-buf: heaps: Add support for Tegra VPR arm64: tegra: Add VPR placeholder node on Tegra234 arm64: tegra: Add GPU node on Tegra234 arm64: tegra: Hook up VPR to host1x arm64: tegra: Hook up VPR to the GPU .../display/tegra/nvidia,tegra186-dc.yaml | 10 + .../display/tegra/nvidia,tegra20-dc.yaml | 10 +- .../display/tegra/nvidia,tegra20-host1x.yaml | 7 + .../nvidia,tegra-video-protection-region.yaml | 55 + arch/arm/mm/dma-mapping.c | 2 +- arch/arm64/boot/dts/nvidia/tegra234.dtsi | 60 + arch/s390/mm/init.c | 2 +- drivers/dma-buf/dma-heap.c | 56 + drivers/dma-buf/heaps/Kconfig | 7 + drivers/dma-buf/heaps/Makefile | 1 + drivers/dma-buf/heaps/cma_heap.c | 2 +- drivers/dma-buf/heaps/tegra-vpr.c | 1265 +++++++++++++++++ include/linux/bitmap.h | 25 +- include/linux/cma.h | 7 +- include/linux/dma-heap.h | 2 + include/trace/events/tegra_vpr.h | 57 + mm/cma.c | 187 ++- mm/cma.h | 5 +- 18 files changed, 1713 insertions(+), 47 deletions(-) create mode 100644 Documentation/devicetree/bindings/reserved-memory/nvidia,tegra-video-protection-region.yaml create mode 100644 drivers/dma-buf/heaps/tegra-vpr.c create mode 100644 include/trace/events/tegra_vpr.h -- 2.52.0

5 months

[PATCH] dma-buf/dma_fence_array: optimize handling v2

by Christian König

Removing the signal on any feature allows to simplfy the dma_fence_array code a lot and saves us from the need to install a callback on all fences at the same time. This results is less memory and CPU overhead. v2: fix potential double locking pointed out by Tvrtko Signed-off-by: Christian König <christian.koenig(a)amd.com> --- drivers/dma-buf/dma-fence-array.c | 134 +++++++++++++----------------- drivers/gpu/drm/xe/xe_vm.c | 2 +- include/linux/dma-fence-array.h | 22 ++--- 3 files changed, 66 insertions(+), 92 deletions(-) diff --git a/drivers/dma-buf/dma-fence-array.c b/drivers/dma-buf/dma-fence-array.c index 5e10e8df372f..8b94c6287482 100644 --- a/drivers/dma-buf/dma-fence-array.c +++ b/drivers/dma-buf/dma-fence-array.c @@ -42,97 +42,88 @@ static void dma_fence_array_clear_pending_error(struct dma_fence_array *array) cmpxchg(&array->base.error, PENDING_ERROR, 0); } -static void irq_dma_fence_array_work(struct irq_work *wrk) +static void dma_fence_array_cb_func(struct dma_fence *f, + struct dma_fence_cb *cb) { - struct dma_fence_array *array = container_of(wrk, typeof(*array), work); - - dma_fence_array_clear_pending_error(array); + struct dma_fence_array *array = + container_of(cb, struct dma_fence_array, callback); - dma_fence_signal(&array->base); - dma_fence_put(&array->base); + irq_work_queue(&array->work); } -static void dma_fence_array_cb_func(struct dma_fence *f, - struct dma_fence_cb *cb) +static bool dma_fence_array_try_add_cb(struct dma_fence_array *array) { - struct dma_fence_array_cb *array_cb = - container_of(cb, struct dma_fence_array_cb, cb); - struct dma_fence_array *array = array_cb->array; + while (array->num_pending) { + struct dma_fence *f = array->fences[array->num_pending - 1]; - dma_fence_array_set_pending_error(array, f->error); + if (!dma_fence_add_callback(f, &array->callback, + dma_fence_array_cb_func)) + return true; - if (atomic_dec_and_test(&array->num_pending)) - irq_work_queue(&array->work); - else + dma_fence_array_set_pending_error(array, f->error); + --array->num_pending; + } + return false; +} + +static void dma_fence_array_irq_work(struct irq_work *wrk) +{ + struct dma_fence_array *array = container_of(wrk, typeof(*array), work); + + --array->num_pending; + if (!dma_fence_array_try_add_cb(array)) { + dma_fence_signal(&array->base); dma_fence_put(&array->base); + } } static bool dma_fence_array_enable_signaling(struct dma_fence *fence) { struct dma_fence_array *array = to_dma_fence_array(fence); - struct dma_fence_array_cb *cb = array->callbacks; - unsigned i; - for (i = 0; i < array->num_fences; ++i) { - cb[i].array = array; + /* + * As we may report that the fence is signaled before all + * callbacks are complete, we need to take an additional + * reference count on the array so that we do not free it too + * early. The core fence handling will only hold the reference + * until we signal the array as complete (but that is now + * insufficient). + */ + dma_fence_get(&array->base); + if (!dma_fence_array_try_add_cb(array)) { /* - * As we may report that the fence is signaled before all - * callbacks are complete, we need to take an additional - * reference count on the array so that we do not free it too - * early. The core fence handling will only hold the reference - * until we signal the array as complete (but that is now - * insufficient). + * When all fences are already signaled we can drop the reference again + * and report to the caller that the array can be signaled as well. */ - dma_fence_get(&array->base); - if (dma_fence_add_callback(array->fences[i], &cb[i].cb, - dma_fence_array_cb_func)) { - int error = array->fences[i]->error; - - dma_fence_array_set_pending_error(array, error); - dma_fence_put(&array->base); - if (atomic_dec_and_test(&array->num_pending)) { - dma_fence_array_clear_pending_error(array); - return false; - } - } + dma_fence_put(&array->base); + return false; } - return true; } static bool dma_fence_array_signaled(struct dma_fence *fence) { struct dma_fence_array *array = to_dma_fence_array(fence); - int num_pending; + int num_pending, error = 0; unsigned int i; /* - * We need to read num_pending before checking the enable_signal bit - * to avoid racing with the enable_signaling() implementation, which - * might decrement the counter, and cause a partial check. - * atomic_read_acquire() pairs with atomic_dec_and_test() in - * dma_fence_array_enable_signaling() - * - * The !--num_pending check is here to account for the any_signaled case - * if we race with enable_signaling(), that means the !num_pending check - * in the is_signalling_enabled branch might be outdated (num_pending - * might have been decremented), but that's fine. The user will get the - * right value when testing again later. + * Reading num_pending without a memory barrier here is correct since + * that is only for optimization, it is perfectly acceptable to have a + * stale value for it. In all other cases num_pending is accessed by a + * single call chain. */ - num_pending = atomic_read_acquire(&array->num_pending); - if (test_bit(DMA_FENCE_FLAG_ENABLE_SIGNAL_BIT, &array->base.flags)) { - if (num_pending <= 0) - goto signal; - return false; - } + num_pending = READ_ONCE(array->num_pending); + for (i = 0; i < num_pending; ++i) { + struct dma_fence *f = array->fences[i]; - for (i = 0; i < array->num_fences; ++i) { - if (dma_fence_is_signaled(array->fences[i]) && !--num_pending) - goto signal; - } - return false; + if (!dma_fence_is_signaled(f)) + return false; -signal: + if (!error) + error = f->error; + } + dma_fence_array_set_pending_error(array, error); dma_fence_array_clear_pending_error(array); return true; } @@ -171,15 +162,12 @@ EXPORT_SYMBOL(dma_fence_array_ops); /** * dma_fence_array_alloc - Allocate a custom fence array - * @num_fences: [in] number of fences to add in the array * * Return dma fence array on success, NULL on failure */ -struct dma_fence_array *dma_fence_array_alloc(int num_fences) +struct dma_fence_array *dma_fence_array_alloc(void) { - struct dma_fence_array *array; - - return kzalloc_flex(*array, callbacks, num_fences); + return kzalloc_obj(struct dma_fence_array); } EXPORT_SYMBOL(dma_fence_array_alloc); @@ -203,10 +191,13 @@ void dma_fence_array_init(struct dma_fence_array *array, WARN_ON(!num_fences || !fences); array->num_fences = num_fences; + array->num_pending = num_fences; + array->fences = fences; + array->base.error = PENDING_ERROR; dma_fence_init(&array->base, &dma_fence_array_ops, NULL, context, seqno); - init_irq_work(&array->work, irq_dma_fence_array_work); + init_irq_work(&array->work, dma_fence_array_irq_work); /* * dma_fence_array_enable_signaling() is invoked while holding @@ -220,11 +211,6 @@ void dma_fence_array_init(struct dma_fence_array *array, */ lockdep_set_class(&array->base.inline_lock, &dma_fence_array_lock_key); - atomic_set(&array->num_pending, num_fences); - array->fences = fences; - - array->base.error = PENDING_ERROR; - /* * dma_fence_array objects should never contain any other fence * containers or otherwise we run into recursion and potential kernel @@ -265,7 +251,7 @@ struct dma_fence_array *dma_fence_array_create(int num_fences, { struct dma_fence_array *array; - array = dma_fence_array_alloc(num_fences); + array = dma_fence_array_alloc(); if (!array) return NULL; diff --git a/drivers/gpu/drm/xe/xe_vm.c b/drivers/gpu/drm/xe/xe_vm.c index 62a87a051be7..8f472911469d 100644 --- a/drivers/gpu/drm/xe/xe_vm.c +++ b/drivers/gpu/drm/xe/xe_vm.c @@ -3370,7 +3370,7 @@ static struct dma_fence *ops_execute(struct xe_vm *vm, goto err_trace; } - cf = dma_fence_array_alloc(n_fence); + cf = dma_fence_array_alloc(); if (!cf) { fence = ERR_PTR(-ENOMEM); goto err_out; diff --git a/include/linux/dma-fence-array.h b/include/linux/dma-fence-array.h index 1b1d87579c38..3ee55c0e2fa4 100644 --- a/include/linux/dma-fence-array.h +++ b/include/linux/dma-fence-array.h @@ -15,16 +15,6 @@ #include <linux/dma-fence.h> #include <linux/irq_work.h> -/** - * struct dma_fence_array_cb - callback helper for fence array - * @cb: fence callback structure for signaling - * @array: reference to the parent fence array object - */ -struct dma_fence_array_cb { - struct dma_fence_cb cb; - struct dma_fence_array *array; -}; - /** * struct dma_fence_array - fence to represent an array of fences * @base: fence base class @@ -33,18 +23,17 @@ struct dma_fence_array_cb { * @num_pending: fences in the array still pending * @fences: array of the fences * @work: internal irq_work function - * @callbacks: array of callback helpers + * @callback: callback structure for signaling */ struct dma_fence_array { struct dma_fence base; - unsigned num_fences; - atomic_t num_pending; + unsigned int num_fences; + unsigned int num_pending; struct dma_fence **fences; struct irq_work work; - - struct dma_fence_array_cb callbacks[] __counted_by(num_fences); + struct dma_fence_cb callback; }; /** @@ -78,11 +67,10 @@ to_dma_fence_array(struct dma_fence *fence) for (index = 0, fence = dma_fence_array_first(head); fence; \ ++(index), fence = dma_fence_array_next(head, index)) -struct dma_fence_array *dma_fence_array_alloc(int num_fences); +struct dma_fence_array *dma_fence_array_alloc(void); void dma_fence_array_init(struct dma_fence_array *array, int num_fences, struct dma_fence **fences, u64 context, unsigned seqno); - struct dma_fence_array *dma_fence_array_create(int num_fences, struct dma_fence **fences, u64 context, unsigned seqno); -- 2.43.0

5 months

[PATCH] dma-buf: system_heap: Optimize sg_table-to-pages conversion in vmap

by Barry Song (Xiaomi)

From: Xueyuan Chen <Xueyuan.chen21(a)gmail.com> Replace the heavy for_each_sgtable_page() iterator in system_heap_do_vmap() with a more efficient nested loop approach. Instead of iterating page by page, we now iterate through the scatterlist entries via for_each_sgtable_sg(). Because pages within a single sg entry are physically contiguous, we can populate the page array with a in an inner loop using simple pointer math. This save a lot of time. The WARN_ON check is also pulled out of the loop to save branch instructions. Performance results mapping a 2GB buffer on Radxa O6: - Before: ~1440000 ns - After: ~232000 ns (~84% reduction in iteration time, or ~6.2x faster) Cc: Sumit Semwal <sumit.semwal(a)linaro.org> Cc: Benjamin Gaignard <benjamin.gaignard(a)collabora.com> Cc: Brian Starkey <Brian.Starkey(a)arm.com> Cc: John Stultz <jstultz(a)google.com> Cc: T.J. Mercier <tjmercier(a)google.com> Cc: Christian König <christian.koenig(a)amd.com> Signed-off-by: Xueyuan Chen <Xueyuan.chen21(a)gmail.com> Signed-off-by: Barry Song (Xiaomi) <baohua(a)kernel.org> --- drivers/dma-buf/heaps/system_heap.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/drivers/dma-buf/heaps/system_heap.c b/drivers/dma-buf/heaps/system_heap.c index b3650d8fd651..769f01f0cc96 100644 --- a/drivers/dma-buf/heaps/system_heap.c +++ b/drivers/dma-buf/heaps/system_heap.c @@ -224,16 +224,21 @@ static void *system_heap_do_vmap(struct system_heap_buffer *buffer) int npages = PAGE_ALIGN(buffer->len) / PAGE_SIZE; struct page **pages = vmalloc(sizeof(struct page *) * npages); struct page **tmp = pages; - struct sg_page_iter piter; void *vaddr; + u32 i, j, count; + struct page *base_page; + struct scatterlist *sg; if (!pages) return ERR_PTR(-ENOMEM); - for_each_sgtable_page(table, &piter, 0) { - WARN_ON(tmp - pages >= npages); - *tmp++ = sg_page_iter_page(&piter); + for_each_sgtable_sg(table, sg, i) { + base_page = sg_page(sg); + count = sg->length >> PAGE_SHIFT; + for (j = 0; j < count; j++) + *tmp++ = base_page + j; } + WARN_ON(tmp - pages != npages); vaddr = vmap(pages, npages, VM_MAP, PAGE_KERNEL); vfree(pages); -- 2.39.3 (Apple Git-146)

5 months

Jump to page:

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

Linaro-mm-sig