Hi Linus,
I would like to ask again for pulling the dma-mapping framework updates for v3.4.
The following changes since commit c16fa4f2ad19908a47c63d8fa436a1178438c7e7:
Linux 3.3 (2012-03-18 16:15:34 -0700)
with the top-most commit 64d70fe5d3640e1a89790ed21120921278f8cb86
common: DMA-mapping: add NON-CONSISTENT attribute
from the git repository at:
git://git.linaro.org/people/mszyprowski/linux-dma-mapping.git for-linus
This is an updated version of the patches which have been earlier rejected,
see the following thread:
https://lkml.org/lkml/2012/3/20/29
I hope I've fixed the issues pointed there:
- collected a few more acks from arch maintainers,
- resolved signed-off-by weirdness,
- removed dependences between the patches.
Short summary for the whole series:
A few limitations have been identified in the current dma-mapping design and
its implementations for various architectures. There exist more than one function
for allocating and freeing the buffers: currently these 3 are used dma_{alloc,
free}_coherent, dma_{alloc,free}_writecombine, dma_{alloc,free}_noncoherent.
For most of the systems these calls are almost equivalent and can be interchanged.
For others, especially the truly non-coherent ones (like ARM), the difference can
be easily noticed in overall driver performance. Sadly not all architectures
provide implementations for all of them, so the drivers might need to be adapted
and cannot be easily shared between different architectures. The provided patches
unify all these functions and hide the differences under the already existing
dma attributes concept. The thread with more references is available here:
http://www.spinics.net/lists/linux-sh/msg09777.html
These patches are also a prerequisite for unifying DMA-mapping implementation
on ARM architecture with the common one provided by dma_map_ops structure and
extending it with IOMMU support. More information is available in the following
thread: http://thread.gmane.org/gmane.linux.kernel.cross-arch/12819
More works on dma-mapping framework are planned, especially in the area of buffer
sharing and managing the shared mappings (together with the recently introduced
dma_buf interface: commit d15bd7ee445d0702ad801fdaece348fdb79e6581 "dma-buf:
Introduce dma buffer sharing mechanism").
The patches in the current set introduce a new alloc/free methods (with
support for memory attributes) in dma_map_ops structure, which will
later replace dma_alloc_coherent and dma_alloc_writecombine functions.
Thanks!
Best regards
Marek Szyprowski
Samsung Poland R&D Center
Patch summary:
Andrzej Pietrasiewicz (8):
X86 & IA64: adapt for dma_map_ops changes
MIPS: adapt for dma_map_ops changes
PowerPC: adapt for dma_map_ops changes
SPARC: adapt for dma_map_ops changes
Alpha: adapt for dma_map_ops changes
SH: adapt for dma_map_ops changes
Microblaze: adapt for dma_map_ops changes
Unicore32: adapt for dma_map_ops changes
Marek Szyprowski (6):
common: dma-mapping: introduce generic alloc() and free() methods
Hexagon: adapt for dma_map_ops changes
common: dma-mapping: remove old alloc_coherent and free_coherent methods
common: dma-mapping: introduce mmap method
common: DMA-mapping: add WRITE_COMBINE attribute
common: DMA-mapping: add NON-CONSISTENT attribute
Documentation/DMA-attributes.txt | 18 ++++++++++++++++++
arch/alpha/include/asm/dma-mapping.h | 18 ++++++++++++------
arch/alpha/kernel/pci-noop.c | 10 ++++++----
arch/alpha/kernel/pci_iommu.c | 10 ++++++----
arch/hexagon/include/asm/dma-mapping.h | 18 ++++++++++++------
arch/hexagon/kernel/dma.c | 9 +++++----
arch/ia64/hp/common/sba_iommu.c | 11 ++++++-----
arch/ia64/include/asm/dma-mapping.h | 18 ++++++++++++------
arch/ia64/kernel/pci-swiotlb.c | 14 +++++++++++---
arch/ia64/sn/pci/pci_dma.c | 9 +++++----
arch/microblaze/include/asm/dma-mapping.h | 18 ++++++++++++------
arch/microblaze/kernel/dma.c | 10 ++++++----
arch/mips/cavium-octeon/dma-octeon.c | 12 ++++++------
arch/mips/include/asm/dma-mapping.h | 18 ++++++++++++------
arch/mips/mm/dma-default.c | 8 ++++----
arch/powerpc/include/asm/dma-mapping.h | 24 ++++++++++++++++--------
arch/powerpc/kernel/dma-iommu.c | 10 ++++++----
arch/powerpc/kernel/dma-swiotlb.c | 4 ++--
arch/powerpc/kernel/dma.c | 10 ++++++----
arch/powerpc/kernel/ibmebus.c | 10 ++++++----
arch/powerpc/kernel/vio.c | 14 ++++++++------
arch/powerpc/platforms/cell/iommu.c | 16 +++++++++-------
arch/powerpc/platforms/ps3/system-bus.c | 13 +++++++------
arch/sh/include/asm/dma-mapping.h | 28 ++++++++++++++++++----------
arch/sh/kernel/dma-nommu.c | 4 ++--
arch/sh/mm/consistent.c | 6 ++++--
arch/sparc/include/asm/dma-mapping.h | 18 ++++++++++++------
arch/sparc/kernel/iommu.c | 10 ++++++----
arch/sparc/kernel/ioport.c | 18 ++++++++++--------
arch/sparc/kernel/pci_sun4v.c | 9 +++++----
arch/unicore32/include/asm/dma-mapping.h | 18 ++++++++++++------
arch/unicore32/mm/dma-swiotlb.c | 18 ++++++++++++++++--
arch/x86/include/asm/dma-mapping.h | 26 ++++++++++++++++----------
arch/x86/kernel/amd_gart_64.c | 11 ++++++-----
arch/x86/kernel/pci-calgary_64.c | 9 +++++----
arch/x86/kernel/pci-dma.c | 3 ++-
arch/x86/kernel/pci-nommu.c | 6 +++---
arch/x86/kernel/pci-swiotlb.c | 17 +++++++++++++----
arch/x86/xen/pci-swiotlb-xen.c | 4 ++--
drivers/iommu/amd_iommu.c | 10 ++++++----
drivers/iommu/intel-iommu.c | 9 +++++----
drivers/xen/swiotlb-xen.c | 5 +++--
include/linux/dma-attrs.h | 2 ++
include/linux/dma-mapping.h | 13 +++++++++----
include/xen/swiotlb-xen.h | 6 ++++--
45 files changed, 354 insertions(+), 198 deletions(-)
Hello everyone,
This patchset is an incremental patch to patchset created by Sumit Semwal [1].
The patches are dedicated to help find a better solution for support of buffer
sharing by V4L2 API. It is expected to start discussion on the final
installment for dma-buf in vb2-dma-contig allocator. Current version of the
patches contain little documentation. It is going to be fixed after achieving
consensus about design for buffer exporting. Moreover the API between vb2-core
and the allocator should be revised.
The patches were successfully tested to cooperate with EXYNOS DRM driver using
DMABUF mechanism.
Please note, that the amount of changes to vb2-dma-contig.c was significant
making the difference patch very difficult to read.
The patchset makes use of dma_get_pages extension for DMA API, which is posted
on a top of dma-mapping patches by Marek Szyprowski [4] [5].
The tree, that contains all needed patches, can be found here [6].
v2:
- extended VIDIOC_EXPBUF argument from integer memoffset to struct
v4l2_exportbuffer
- added patch that breaks DMABUF spec on (un)map_atachment callcacks but allows
to work with existing implementation of DMABUF prime in DRM
- all dma-contig code refactoring patches were squashed
- bugfixes
v1: List of changes since [1].
- support for DMA api extension dma_get_pages, the function is used to retrieve
pages used to create DMA mapping.
- small fixes/code cleanup to videobuf2
- added prepare and finish callbacks to vb2 allocators, it is used keep
consistency between dma-cpu acess to the memory (by Marek Szyprowski)
- support for exporting of DMABUF buffer in V4L2 and Videobuf2, originated from
[3].
- support for dma-buf exporting in vb2-dma-contig allocator
- support for DMABUF for s5p-tv and s5p-fimc (capture interface) drivers,
originated from [3]
- changed handling for userptr buffers (by Marek Szyprowski, Andrzej
Pietrasiewicz)
- let mmap method to use dma_mmap_writecombine call (by Marek Szyprowski)
[1] http://thread.gmane.org/gmane.linux.drivers.video-input-infrastructure/4296…
[2] https://lkml.org/lkml/2011/12/26/29
[3] http://thread.gmane.org/gmane.linux.drivers.video-input-infrastructure/3635…
[4] http://thread.gmane.org/gmane.linux.kernel.cross-arch/12819
[5] http://git.infradead.org/users/kmpark/linux-2.6-samsung/shortlog/refs/heads…
[6] http://git.infradead.org/users/kmpark/linux-2.6-samsung/shortlog/refs/heads…
Sumit Semwal (1):
v4l: vb2: Add dma-contig allocator as dma_buf user
Tomasz Stanislawski (8):
v4l: vb2: fixes for DMABUF support
v4l: vb2-dma-contig: update and code refactoring
v4l: add buffer exporting via dmabuf
v4l: vb2: add buffer exporting via dmabuf
v4l: vb2-dma-contig: add support for DMABUF exporting
v4l: vb2-dma-contig: change map/unmap behaviour
v4l: fimc: integrate capture i-face with dmabuf
v4l: s5p-tv: mixer: integrate with dmabuf
drivers/media/video/Kconfig | 1 +
drivers/media/video/s5p-fimc/fimc-capture.c | 11 +-
drivers/media/video/s5p-tv/Kconfig | 1 +
drivers/media/video/s5p-tv/mixer_video.c | 12 +-
drivers/media/video/v4l2-compat-ioctl32.c | 1 +
drivers/media/video/v4l2-ioctl.c | 11 +
drivers/media/video/videobuf2-core.c | 88 +++-
drivers/media/video/videobuf2-dma-contig.c | 717 ++++++++++++++++++++++++---
include/linux/videodev2.h | 20 +
include/media/v4l2-ioctl.h | 2 +
include/media/videobuf2-core.h | 8 +-
11 files changed, 779 insertions(+), 93 deletions(-)
--
1.7.5.4
Hello eveyone,
On Linaro Memory Management meeting in Budapest (May 2011) we have
discussed about the design of DMA mapping framework. We tried to
identify the drawbacks and limitations as well as to provide some a
solution for them. The discussion was mainly about ARM architecture, but
some of the conclusions need to be applied to cross-architecture code.
The first issue we identified is the fact that on some platform (again,
mainly ARM) there are several functions for allocating DMA buffers:
dma_alloc_coherent, dma_alloc_writecombine and dma_alloc_noncoherent
(not functional now). For each of them there is a match dma_free_*
function. This gives us quite a lot of functions in the public API and
complicates things when we need to have several different
implementations for different devices selected in runtime (if IOMMU
controller is available only for a few devices in the system). Also the
drivers which use less common variants are less portable because of the
lacks of dma_alloc_writecombine on other architectures.
The solution we found is to introduce a new public dma mapping functions
with additional attributes argument: dma_alloc_attrs and
dma_free_attrs(). This way all different kinds of architecture specific
buffer mappings can be hidden behind the attributes without the need of
creating several versions of dma_alloc_ function.
dma_alloc_coherent() can be wrapped on top of new dma_alloc_attrs() with
NULL attrs parameter. dma_alloc_writecombine and dma_alloc_noncoherent
can be implemented as a simple wrappers which sets attributes to
DMA_ATTRS_WRITECOMBINE or DMA_ATTRS_NON_CONSISTENT respectively. These
new attributes will be implemented only on the architectures that really
support them, the others will simply ignore them defaulting to the
dma_alloc_coherent equivalent.
The next step in dma mapping framework update is the introduction of
dma_mmap/dma_mmap_attrs() function. There are a number of drivers
(mainly V4L2 and ALSA) that only exports the DMA buffers to user space.
Creating a userspace mapping with correct page attributes is not an easy
task for the driver. Also the DMA-mapping framework is the only place
where the complete information about the allocated pages is available,
especially if the implementation uses IOMMU controller to provide a
contiguous buffer in DMA address space which is scattered in physical
memory space.
Usually these drivers don't touch the buffer data at all, so the mapping
in kernel virtual address space is not needed. We can introduce
DMA_ATTRIB_NO_KERNEL_MAPPING attribute which lets kernel to skip/ignore
creation of kernel virtual mapping. This way we can save previous
vmalloc area and simply some mapping operation on a few architectures.
This patch series is a preparation for the above changes in the public
dma mapping API. The main goal is to modify dma_map_ops structure and
let all users to use for implementation of the new public funtions.
The proof-of-concept patches for ARM architecture have been already
posted a few times and now they are working resonably well. They perform
conversion to dma_map_ops based implementation and add support for
generic IOMMU-based dma mapping implementation. To get them merged we
first need to get acceptance for the changes in the common,
cross-architecture structures. More information about these patches can
be found in the following threads:
http://www.spinics.net/lists/linux-mm/msg19856.htmlhttp://www.spinics.net/lists/linux-mm/msg21241.htmlhttp://lists.linaro.org/pipermail/linaro-mm-sig/2011-September/000571.htmlhttp://lists.linaro.org/pipermail/linaro-mm-sig/2011-September/000577.htmlhttp://www.spinics.net/lists/linux-mm/msg25490.html
The patches are prepared on top of Linux Kernel v3.2-rc6. I would
appreciate any comments and help with getting this patch series into
linux-next tree.
The idea apllied in this patch set have been also presented during the
Kernel Summit 2011 and ELC-E 2011 in Prague, in the presentation 'ARM
DMA-Mapping Framework Redesign and IOMMU integration'.
I'm really sorry if I missed any of the relevant architecture mailing
lists. I've did my best to include everyone. Feel free to forward this
patchset to all interested developers and maintainers. I've already feel
like a nasty spammer.
Best regards
Marek Szyprowski
Samsung Poland R&D Center
Patch summary:
Andrzej Pietrasiewicz (9):
X86: adapt for dma_map_ops changes
MIPS: adapt for dma_map_ops changes
PowerPC: adapt for dma_map_ops changes
IA64: adapt for dma_map_ops changes
SPARC: adapt for dma_map_ops changes
Alpha: adapt for dma_map_ops changes
SH: adapt for dma_map_ops changes
Microblaze: adapt for dma_map_ops changes
Unicore32: adapt for dma_map_ops changes
Marek Szyprowski (5):
common: dma-mapping: introduce alloc_attrs and free_attrs methods
common: dma-mapping: remove old alloc_coherent and free_coherent
methods
common: dma-mapping: introduce mmap method
common: DMA-mapping: add WRITE_COMBINE attribute
common: DMA-mapping: add NON-CONSISTENT attribute
Documentation/DMA-attributes.txt | 19 +++++++++++++++++++
arch/alpha/include/asm/dma-mapping.h | 18 ++++++++++++------
arch/alpha/kernel/pci-noop.c | 10 ++++++----
arch/alpha/kernel/pci_iommu.c | 10 ++++++----
arch/ia64/hp/common/sba_iommu.c | 11 ++++++-----
arch/ia64/include/asm/dma-mapping.h | 18 ++++++++++++------
arch/ia64/kernel/pci-swiotlb.c | 9 +++++----
arch/ia64/sn/pci/pci_dma.c | 9 +++++----
arch/microblaze/include/asm/dma-mapping.h | 18 ++++++++++++------
arch/microblaze/kernel/dma.c | 10 ++++++----
arch/mips/include/asm/dma-mapping.h | 18 ++++++++++++------
arch/mips/mm/dma-default.c | 8 ++++----
arch/powerpc/include/asm/dma-mapping.h | 24 ++++++++++++++++--------
arch/powerpc/kernel/dma-iommu.c | 10 ++++++----
arch/powerpc/kernel/dma-swiotlb.c | 4 ++--
arch/powerpc/kernel/dma.c | 10 ++++++----
arch/powerpc/kernel/ibmebus.c | 10 ++++++----
arch/powerpc/platforms/cell/iommu.c | 16 +++++++++-------
arch/powerpc/platforms/ps3/system-bus.c | 13 +++++++------
arch/sh/include/asm/dma-mapping.h | 28 ++++++++++++++++++----------
arch/sh/kernel/dma-nommu.c | 4 ++--
arch/sh/mm/consistent.c | 6 ++++--
arch/sparc/include/asm/dma-mapping.h | 18 ++++++++++++------
arch/sparc/kernel/iommu.c | 10 ++++++----
arch/sparc/kernel/ioport.c | 18 ++++++++++--------
arch/sparc/kernel/pci_sun4v.c | 9 +++++----
arch/unicore32/include/asm/dma-mapping.h | 18 ++++++++++++------
arch/unicore32/mm/dma-swiotlb.c | 4 ++--
arch/x86/include/asm/dma-mapping.h | 26 ++++++++++++++++----------
arch/x86/kernel/amd_gart_64.c | 11 ++++++-----
arch/x86/kernel/pci-calgary_64.c | 9 +++++----
arch/x86/kernel/pci-dma.c | 3 ++-
arch/x86/kernel/pci-nommu.c | 6 +++---
arch/x86/kernel/pci-swiotlb.c | 12 +++++++-----
arch/x86/xen/pci-swiotlb-xen.c | 4 ++--
drivers/iommu/amd_iommu.c | 10 ++++++----
drivers/iommu/intel-iommu.c | 9 +++++----
drivers/xen/swiotlb-xen.c | 5 +++--
include/linux/dma-attrs.h | 2 ++
include/linux/dma-mapping.h | 13 +++++++++----
include/linux/swiotlb.h | 6 ++++--
include/xen/swiotlb-xen.h | 6 ++++--
lib/swiotlb.c | 5 +++--
43 files changed, 305 insertions(+), 182 deletions(-)
--
1.7.1.569.g6f426
Hi Linus,
Could you please pull the dma-buf updates for 3.4? This includes the
following key items:
- kernel cpu access support,
- flag-passing to dma_buf_fd,
- relevant Documentation updates, and
- some minor cleanups and fixes.
These changes are needed for the drm prime/dma-buf interface code that
Dave Airlie plans to submit in this merge window.
Thanks, and best regards,
~Sumit.
The following changes since commit c16fa4f2ad19908a47c63d8fa436a1178438c7e7:
Linux 3.3 (2012-03-18 16:15:34 -0700)
are available in the git repository at:
git://git.linaro.org/people/sumitsemwal/linux-dma-buf.git for-linus-3.4
Daniel Vetter (3):
dma-buf: don't hold the mutex around map/unmap calls
dma-buf: add support for kernel cpu access
dma_buf: Add documentation for the new cpu access support
Dave Airlie (1):
dma-buf: pass flags into dma_buf_fd.
Laurent Pinchart (4):
dma-buf: Constify ops argument to dma_buf_export()
dma-buf: Remove unneeded sanity checks
dma-buf: Return error instead of using a goto statement when possible
dma-buf: Move code out of mutex-protected section in dma_buf_attach()
Rob Clark (2):
dma-buf: add get_dma_buf()
dma-buf: document fd flags and O_CLOEXEC requirement
Sumit Semwal (2):
dma-buf: add dma_data_direction to unmap dma_buf_op
dma-buf: correct dummy function declarations.
Documentation/dma-buf-sharing.txt | 120 ++++++++++++++++++++++++++-
drivers/base/dma-buf.c | 165 +++++++++++++++++++++++++++++++------
include/linux/dma-buf.h | 97 +++++++++++++++++++--
3 files changed, 345 insertions(+), 37 deletions(-)
Hello all,
I intend to send the pull request to Linus by Monday - the set below
looks to me to be the current content that it'd have on top of 3.3.
[You can view this in for-next branch at
git://git.linaro.org/people/sumitsemwal/linux-dma-buf.git]
I think I could take the userspace mmap patch if we reach a consensus
on which one to take (Rob's?, Daniel's?) - I guess we are most likely
to agree on Daniel's, with possibly the suggestion from Rob about
including IOCTLS but leaving them out of dmabuf ops for this release.
Daniel, Rob: would you think I can have an agreed-upon patch by
tomorrow afternoon? Then I could push it into for-next by tomorrow,
and send a pull request by Monday?
Also, if you think I've missed out something, please let me know.
Daniel Vetter (3):
dma-buf: don't hold the mutex around map/unmap calls
dma-buf: add support for kernel cpu access
dma_buf: Add documentation for the new cpu access support
Dave Airlie (1):
dma-buf: pass flags into dma_buf_fd.
Laurent Pinchart (4):
dma-buf: Constify ops argument to dma_buf_export()
dma-buf: Remove unneeded sanity checks
dma-buf: Return error instead of using a goto statement when possible
dma-buf: Move code out of mutex-protected section in dma_buf_attach()
Rob Clark (2):
dma-buf: add get_dma_buf()
dma-buf: document fd flags and O_CLOEXEC requirement
Sumit Semwal (1):
dma-buf: add dma_data_direction to unmap dma_buf_op
--
Thanks and regards,
Sumit Semwal
Linaro Kernel Engineer - Graphics working group
Linaro.org │ Open source software for ARM SoCs
From: Rob Clark <rob(a)ti.com>
Otherwise subsystems will get this wrong and end up with a second
export ioctl with the flag and O_CLOEXEC support added.
Signed-off-by: Rob Clark <rob(a)ti.com>
Reviewed-by: Daniel Vetter <daniel.vetter(a)ffwll.ch>
---
Updated version of Daniel's original documentation patch with (hopefully)
improved wording, and a better description of the motivation.
Documentation/dma-buf-sharing.txt | 18 ++++++++++++++++++
1 files changed, 18 insertions(+), 0 deletions(-)
diff --git a/Documentation/dma-buf-sharing.txt b/Documentation/dma-buf-sharing.txt
index 225f96d..3b51134 100644
--- a/Documentation/dma-buf-sharing.txt
+++ b/Documentation/dma-buf-sharing.txt
@@ -223,6 +223,24 @@ Miscellaneous notes:
- Any exporters or users of the dma-buf buffer sharing framework must have
a 'select DMA_SHARED_BUFFER' in their respective Kconfigs.
+- In order to avoid fd leaks on exec, the FD_CLOEXEC flag must be set
+ on the file descriptor. This is not just a resource leak, but a
+ potential security hole. It could give the newly exec'd application
+ access to buffers, via the leaked fd, to which it should otherwise
+ not be permitted access.
+
+ The problem with doing this via a separate fcntl() call, versus doing it
+ atomically when the fd is created, is that this is inherently racy in a
+ multi-threaded app[3]. The issue is made worse when it is library code
+ opening/creating the file descriptor, as the application may not even be
+ aware of the fd's.
+
+ To avoid this problem, userspace must have a way to request O_CLOEXEC
+ flag be set when the dma-buf fd is created. So any API provided by
+ the exporting driver to create a dmabuf fd must provide a way to let
+ userspace control setting of O_CLOEXEC flag passed in to dma_buf_fd().
+
References:
[1] struct dma_buf_ops in include/linux/dma-buf.h
[2] All interfaces mentioned above defined in include/linux/dma-buf.h
+[3] https://lwn.net/Articles/236486/
--
1.7.5.4
The mutex protects the attachment list and hence needs to be held
around the callbakc to the exporters (optional) attach/detach
functions.
Holding the mutex around the map/unmap calls doesn't protect any
dma_buf state. Exporters need to properly protect any of their own
state anyway (to protect against calls from their own interfaces).
So this only makes the locking messier (and lockdep easier to anger).
Therefore let's just drop this.
v2: Rebased on top of latest dma-buf-next git.
Signed-off-by: Daniel Vetter <daniel.vetter(a)ffwll.ch>
Reviewed-by: Rob Clark <rob.clark(a)linaro.org>
---
drivers/base/dma-buf.c | 5 -----
include/linux/dma-buf.h | 2 +-
2 files changed, 1 insertions(+), 6 deletions(-)
diff --git a/drivers/base/dma-buf.c b/drivers/base/dma-buf.c
index 3c8c023..5641b9c 100644
--- a/drivers/base/dma-buf.c
+++ b/drivers/base/dma-buf.c
@@ -258,9 +258,7 @@ struct sg_table *dma_buf_map_attachment(struct dma_buf_attachment *attach,
if (WARN_ON(!attach || !attach->dmabuf))
return ERR_PTR(-EINVAL);
- mutex_lock(&attach->dmabuf->lock);
sg_table = attach->dmabuf->ops->map_dma_buf(attach, direction);
- mutex_unlock(&attach->dmabuf->lock);
return sg_table;
}
@@ -282,10 +280,7 @@ void dma_buf_unmap_attachment(struct dma_buf_attachment *attach,
if (WARN_ON(!attach || !attach->dmabuf || !sg_table))
return;
- mutex_lock(&attach->dmabuf->lock);
attach->dmabuf->ops->unmap_dma_buf(attach, sg_table,
direction);
- mutex_unlock(&attach->dmabuf->lock);
-
}
EXPORT_SYMBOL_GPL(dma_buf_unmap_attachment);
diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h
index bc4203dc..24e0f48 100644
--- a/include/linux/dma-buf.h
+++ b/include/linux/dma-buf.h
@@ -88,7 +88,7 @@ struct dma_buf {
struct file *file;
struct list_head attachments;
const struct dma_buf_ops *ops;
- /* mutex to serialize list manipulation and other ops */
+ /* mutex to serialize list manipulation and attach/detach */
struct mutex lock;
void *priv;
};
--
1.7.7.5
Let's have some competition here for dma_buf mmap support ;-)
Compared to Rob Clarke's RFC I've ditched the prepare/finish hooks
and corresponding ioctls on the dma_buf file. The major reason for
that is that many people seem to be under the impression that this is
also for synchronization with outstanding asynchronous processsing.
I'm pretty massively opposed to this because:
- It boils down reinventing a new rather general-purpose userspace
synchronization interface. If we look at things like futexes, this
is hard to get right.
- Furthermore a lot of kernel code has to interact with this
synchronization primitive. This smells a look like the dri1 hw_lock,
a horror show I prefer not to reinvent.
- Even more fun is that multiple different subsystems would interact
here, so we have plenty of opportunities to create funny deadlock
scenarios.
I think synchronization is a wholesale different problem from data
sharing and should be tackled as an orthogonal problem.
Now we could demand that prepare/finish may only ensure cache
coherency (as Rob intended), but that runs up into the next problem:
We not only need mmap support to facilitate sw-only processing nodes
in a pipeline (without jumping through hoops by importing the dma_buf
into some sw-access only importer), which allows for a nicer
ION->dma-buf upgrade path for existing Android userspace. We also need
mmap support for existing importing subsystems to support existing
userspace libraries. And a loot of these subsystems are expected to
export coherent userspace mappings.
So prepare/finish can only ever be optional and the exporter /needs/
to support coherent mappings. Given that mmap access is always
somewhat fallback-y in nature I've decided to drop this optimization,
instead of just making it optional. If we demonstrate a clear need for
this, supported by benchmark results, we can always add it in again
later as an optional extension.
Other differences compared to Rob's RFC is the above mentioned support
for mapping a dma-buf through facilities provided by the importer.
Which results in mmap support no longer being optional.
Note taht this dma-buf mmap patch does _not_ support every possible
insanity an existing subsystem could pull of with mmap: Because it
does not allow to intercept pagefaults and shoot down ptes importing
subsystems can't add some magic of their own at these points (e.g. to
automatically synchronize with outstanding rendering or set up some
special resources). I've done a cursory read through a few mmap
implementions of various subsytems and I'm hopeful that we can avoid
this (and the complexity it'd bring with it).
Additonally I've extended the documentation a bit to explain the hows
and whys of this mmap extension.
Comments, reviews and flames highly welcome.
Cheers, Daniel
---
Documentation/dma-buf-sharing.txt | 84 +++++++++++++++++++++++++++++++++---
drivers/base/dma-buf.c | 64 +++++++++++++++++++++++++++-
include/linux/dma-buf.h | 16 +++++++
3 files changed, 156 insertions(+), 8 deletions(-)
diff --git a/Documentation/dma-buf-sharing.txt b/Documentation/dma-buf-sharing.txt
index a6d4c37..c42a4a5 100644
--- a/Documentation/dma-buf-sharing.txt
+++ b/Documentation/dma-buf-sharing.txt
@@ -29,13 +29,6 @@ The buffer-user
in memory, mapped into its own address space, so it can access the same area
of memory.
-*IMPORTANT*: [see https://lkml.org/lkml/2011/12/20/211 for more details]
-For this first version, A buffer shared using the dma_buf sharing API:
-- *may* be exported to user space using "mmap" *ONLY* by exporter, outside of
- this framework.
-- with this new iteration of the dma-buf api cpu access from the kernel has been
- enable, see below for the details.
-
dma-buf operations for device dma only
--------------------------------------
@@ -313,6 +306,83 @@ Access to a dma_buf from the kernel context involves three steps:
enum dma_data_direction dir);
+Direct Userspace Access/mmap Support
+------------------------------------
+
+Being able to mmap an export dma-buf buffer object has 2 main use-cases:
+- CPU fallback processing in a pipeline and
+- supporting existing mmap interfaces in importers.
+
+1. CPU fallback processing in a pipeline
+
+ In many processing pipelines it is sometimes required that the cpu can access
+ the data in a dma-buf (e.g. for thumbnail creation, snapshots, ...). To avoid
+ the need to handle this specially in userspace frameworks for buffer sharing
+ it's ideal if the dma_buf fd itself can be used to access the backing storage
+ from userspace using mmap.
+
+ Furthermore Android's ION framework already supports this (and is otherwise
+ rather similar to dma-buf from a userspace consumer side with using fds as
+ handles, too). So it's beneficial to support this in a similar fashion on
+ dma-buf to have a good transition path for existing Android userspace.
+
+ No special interfaces, userspace simply calls mmap on the dma-buf fd.
+
+2. Supporting existing mmap interfaces in exporters
+
+ Similar to the motivation for kernel cpu access it is again important that
+ the userspace code of a given importing subsystem can use the same interfaces
+ with a imported dma-buf buffer object as with a native buffer object. This is
+ especially important for drm where the userspace part of contemporary OpenGL,
+ X, and other drivers is huge, and reworking them to use a different way to
+ mmap a buffer rather invasive.
+
+ The assumption in the current dma-buf interfaces is that redirecting the
+ initial mmap is all that's needed. A survey of some of the existing
+ subsystems shows that no driver seems to do any nefarious thing like syncing
+ up with outstanding asynchronous processing on the device or allocating
+ special resources at fault time. So hopefully this is good enough, since
+ adding interfaces to intercept pagefaults and allow pte shootdowns would
+ increase the complexity quite a bit.
+
+ Interface:
+ int dma_buf_mmap(struct dma_buf *, struct vm_area_struct *,
+ unsigned long);
+
+ If the importing subsystem simply provides a special-purpose mmap call to set
+ up a mapping in userspace, calling do_mmap with dma_buf->file will equally
+ achieve that for a dma-buf object.
+
+3. Implementation notes for exporters
+
+ Because dma-buf buffers have invariant size over their lifetime, the dma-buf
+ core checks whether a vma is too large and rejects such mappings. The
+ exporter hence does not need to duplicate this check.
+
+ Because existing importing subsystems might presume coherent mappings for
+ userspace, the exporter needs to set up a coherent mapping. If that's not
+ possible, it needs to fake coherency by manually shooting down ptes when
+ leaving the cpu domain and flushing caches at fault time. Note that all the
+ dma_buf files share the same anon inode, hence the exporter needs to replace
+ the dma_buf file stored in vma->vm_file with it's own if pte shootdown is
+ requred. This is because the kernel uses the underlying inode's address_space
+ for vma tracking (and hence pte tracking at shootdown time with
+ unmap_mapping_range).
+
+ If the above shootdown dance turns out to be too expensive in certain
+ scenarios, we can extend dma-buf with a more explicit cache tracking scheme
+ for userspace mappings. But the current assumption is that using mmap is
+ always a slower path, so some inefficiencies should be acceptable.
+
+ Exporters that shoot down mappings (for any reasons) shall not do any
+ synchronization at fault time with outstanding device operations.
+ Synchronization is an orthogonal issue to sharing the backing storage of a
+ buffer and hence should not be handled by dma-buf itself. This is explictly
+ mentioned here because many people seem to want something like this, but if
+ different exporters handle this differently, buffer sharing can fail in
+ interesting ways depending upong the exporter (if userspace starts depending
+ upon this implicit synchronization).
+
Miscellaneous notes
-------------------
diff --git a/drivers/base/dma-buf.c b/drivers/base/dma-buf.c
index 07cbbc6..f3b923c 100644
--- a/drivers/base/dma-buf.c
+++ b/drivers/base/dma-buf.c
@@ -44,8 +44,26 @@ static int dma_buf_release(struct inode *inode, struct file *file)
return 0;
}
+static int dma_buf_mmap_internal(struct file *file, struct vm_areat_struct *vma)
+{
+ struct dma_buf *dmabuf;
+
+ if (!is_dma_buf_file(file))
+ return -EINVAL;
+
+ /* check for overflowing the buffer's size */
+ if (vma->vm_pgoff + ((vma->vm_end - vma->vm_start) >> PAGE_SHIFT) >
+ dmabuf->size >> PAGE_SHIFT)
+ return -EINVAL;
+
+ dmabuf = file->private_data;
+
+ return dmabuf->ops->mmap(dmabuf, vma);
+}
+
static const struct file_operations dma_buf_fops = {
.release = dma_buf_release,
+ .mmap = dma_buf_mmap_internal,
};
/*
@@ -82,7 +100,8 @@ struct dma_buf *dma_buf_export(void *priv, const struct dma_buf_ops *ops,
|| !ops->unmap_dma_buf
|| !ops->release
|| !ops->kmap_atomic
- || !ops->kmap)) {
+ || !ops->kmap
+ || !ops->mmap)) {
return ERR_PTR(-EINVAL);
}
@@ -406,3 +425,46 @@ void dma_buf_kunmap(struct dma_buf *dmabuf, unsigned long page_num,
dmabuf->ops->kunmap(dmabuf, page_num, vaddr);
}
EXPORT_SYMBOL_GPL(dma_buf_kunmap);
+
+
+/**
+ * dma_buf_mmap - Setup up a userspace mmap with the given vma
+ * @dma_buf: [in] buffer that should back the vma
+ * @vma: [in] vma for the mmap
+ * @pgoff: [in] offset in pages where this mmap should start within the
+ * dma-buf buffer.
+ *
+ * This function adjusts the passed in vma so that it points at the file of the
+ * dma_buf operation. It alsog adjusts the starting pgoff and does bounds
+ * checking on the size of the vma. Then it calls the exporters mmap function to
+ * set up the mapping.
+ *
+ * Can return negative error values, returns 0 on success.
+ */
+int dma_buf_mmap(struct dma_buf *dmabuf, struct vm_area_struct *vma,
+ unsigned long pgoff)
+{
+ if (WARN_ON(!dmabuf || !vma))
+ return -EINVAL;
+
+ /* check for offset overflow */
+ if (pgoff + ((vma->vm_end - vma->vm_start) >> PAGE_SHIFT) < pgoff)
+ return -EOVERFLOW;
+
+ /* check for overflowing the buffer's size */
+ if (pgoff + ((vma->vm_end - vma->vm_start) >> PAGE_SHIFT) >
+ dmabuf->size >> PAGE_SHIFT)
+ return -EINVAL;
+
+ /* readjust the vma */
+ if (vma->vm_file)
+ fput(vma->vm_file);
+
+ vma->vm_file = dmabuf->file;
+ get_file(vma->vm_file);
+
+ vma->vm_pgoff = pgoff;
+
+ return dmabuf->ops->mmap(dmabuf, vma);
+}
+EXPORT_SYMBOL_GPL(dma_buf_mmap);
diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h
index ee7ef99..d254630 100644
--- a/include/linux/dma-buf.h
+++ b/include/linux/dma-buf.h
@@ -61,6 +61,10 @@ struct dma_buf_attachment;
* This Callback must not sleep.
* @kmap: maps a page from the buffer into kernel address space.
* @kunmap: [optional] unmaps a page from the buffer.
+ * @mmap: used to expose the backing storage to userspace. Note that the
+ * mapping needs to be coherent - if the exporter doesn't directly
+ * support this, it needs to fake coherency by shooting down any ptes
+ * when transitioning away from the cpu domain.
*/
struct dma_buf_ops {
int (*attach)(struct dma_buf *, struct device *,
@@ -92,6 +96,8 @@ struct dma_buf_ops {
void (*kunmap_atomic)(struct dma_buf *, unsigned long, void *);
void *(*kmap)(struct dma_buf *, unsigned long);
void (*kunmap)(struct dma_buf *, unsigned long, void *);
+
+ int (*mmap)(struct dma_buf *, struct vm_area_struct *vma);
};
/**
@@ -167,6 +173,9 @@ void *dma_buf_kmap_atomic(struct dma_buf *, unsigned long);
void dma_buf_kunmap_atomic(struct dma_buf *, unsigned long, void *);
void *dma_buf_kmap(struct dma_buf *, unsigned long);
void dma_buf_kunmap(struct dma_buf *, unsigned long, void *);
+
+int dma_buf_mmap(struct dma_buf *, struct vm_area_struct *,
+ unsigned long);
#else
static inline struct dma_buf_attachment *dma_buf_attach(struct dma_buf *dmabuf,
@@ -247,6 +256,13 @@ static inline void dma_buf_kunmap(struct dma_buf *, unsigned long,
void *)
{
}
+
+static inline int dma_buf_mmap(struct dma_buf *,
+ struct vm_area_struct *vma,
+ unsigned long pgoff)
+{
+ return -ENODEV;
+}
#endif /* CONFIG_DMA_SHARED_BUFFER */
#endif /* __DMA_BUF_H__ */
--
1.7.7.5
Hi Sumit,
Here are 4 dma-buf patches that fix small issues.
Laurent Pinchart (4):
dma-buf: Constify ops argument to dma_buf_export()
dma-buf: Remove unneeded sanity checks
dma-buf: Return error instead of using a goto statement when possible
dma-buf: Move code out of mutex-protected section in dma_buf_attach()
drivers/base/dma-buf.c | 26 +++++++++++---------------
include/linux/dma-buf.h | 8 ++++----
2 files changed, 15 insertions(+), 19 deletions(-)
--
Regards,
Laurent Pinchart
Hello everyone,
This patchset is an incremental patch to patchset created by Sumit
Semwal [1]. The patches are dedicated to help find a better solution for
support of buffer sharing by V4L2 API. It is expected to start discussion on
final installment for dma-buf in vb2-dma-contig allocator. Current version of
the patches contain little documentation. It is going to be fixed after
achieving consensus about design for buffer exporting. Moreover the API
between vb2-core and the allocator should be revised.
The amount of changes to vb2-dma-contig.c was significant making the difference
patch very difficult to read. Therefore the patch was split into two parts.
One removes old file, the next patch creates the version of the file.
The patchset contains extension for DMA API and its implementation for ARM
architecture. Therefore the patchset should be applied on the top of:
http://git.infradead.org/users/kmpark/linux-2.6-samsung/shortlog/refs/heads…
After applying patches from [2] and [1].
v1: List of changes since [1].
- support for DMA api extension dma_get_pages, the function is used to retrieve pages
used to create DMA mapping.
- small fixes/code cleanup to videobuf2
- added prepare and finish callbacks to vb2 allocators, it is used keep consistency between dma-cpu acess to the memory (by Marek Szyprowski)
- support for exporting of DMABUF buffer in V4L2 and Videobuf2, originated from [3].
- support for dma-buf exporting in vb2-dma-contig allocator
- support for DMABUF for s5p-tv and s5p-fimc (capture interface) drivers, originated from [3]
- changed handling for userptr buffers (by Marek Szyprowski, Andrzej Pietrasiewicz)
- let mmap method to use dma_mmap_writecombine call (by Marek Szyprowski)
[1] http://thread.gmane.org/gmane.linux.drivers.video-input-infrastructure/4296…
[2] https://lkml.org/lkml/2011/12/26/29
[3] http://thread.gmane.org/gmane.linux.drivers.video-input-infrastructure/3635…
Marek Szyprowski (2):
[media] media: vb2: remove plane argument from call_memop and cleanup
mempriv usage
media: vb2: add prepare/finish callbacks to allocators
Tomasz Stanislawski (8):
arm: dma: support for dma_get_pages
v4l: vb2: fixes for DMABUF support
v4l: add buffer exporting via dmabuf
v4l: vb2: add buffer exporting via dmabuf
v4l: vb2: remove dma-contig allocator
v4l: vb2-dma-contig: code refactoring, support for DMABUF exporting
v4l: fimc: integrate capture i-face with dmabuf
v4l: s5p-tv: mixer: integrate with dmabuf
arch/arm/include/asm/dma-mapping.h | 8 +
arch/arm/mm/dma-mapping.c | 44 ++
drivers/media/video/s5p-fimc/fimc-capture.c | 11 +-
drivers/media/video/s5p-tv/mixer_video.c | 11 +-
drivers/media/video/v4l2-compat-ioctl32.c | 1 +
drivers/media/video/v4l2-ioctl.c | 11 +
drivers/media/video/videobuf2-core.c | 114 ++++-
drivers/media/video/videobuf2-dma-contig.c | 754 +++++++++++++++++++++------
include/linux/dma-mapping.h | 2 +
include/linux/videodev2.h | 1 +
include/media/v4l2-ioctl.h | 1 +
include/media/videobuf2-core.h | 10 +-
12 files changed, 789 insertions(+), 179 deletions(-)
--
1.7.5.4
From: Benjamin Gaignard <benjamin.gaignard(a)linaro.org>
The goal of those patches is to allow ION clients (drivers or userland applications)
to use Contiguous Memory Allocator (CMA).
To get more info about CMA:
http://lists.linaro.org/pipermail/linaro-mm-sig/2012-February/001328.html
patches version 4:
- add ION_HEAP_TYPE_DMA heap type in ion_heap_type enum.
- CMA heap is now a "native" ION heap.
- add ion_heap_create_full function to keep backward compatibilty.
- clean up included files in CMA heap
- ux500-ion is using ion_heap_create_full instead of ion_heap_create
patches version 3:
- add a private field in ion_heap structure instead of expose ion_device
structure to all heaps
- ion_cma_heap is no more a platform driver
- ion_cma_heap use ion_heap private field to store the device pointer and
make the link with reserved CMA regions
- provide ux500-ion driver and configuration file for snowball board to give
an example of how use CMA heaps
patches version 2:
- fix comments done by Andy Green
Benjamin Gaignard (4):
fix ion_platform_data definition
add private field in ion_heap structure
add CMA heap
add test/example driver for ux500 platform
arch/arm/mach-ux500/board-mop500.c | 77 +++++++++++++++++++
drivers/gpu/ion/Kconfig | 6 ++
drivers/gpu/ion/Makefile | 4 +-
drivers/gpu/ion/ion_cma_heap.c | 129 ++++++++++++++++++++++++++++++++
drivers/gpu/ion/ion_heap.c | 14 +++-
drivers/gpu/ion/ion_priv.h | 11 +++
drivers/gpu/ion/ux500/Makefile | 1 +
drivers/gpu/ion/ux500/ux500_ion.c | 142 ++++++++++++++++++++++++++++++++++++
include/linux/ion.h | 5 +-
9 files changed, 386 insertions(+), 3 deletions(-)
create mode 100644 drivers/gpu/ion/ion_cma_heap.c
create mode 100644 drivers/gpu/ion/ux500/Makefile
create mode 100644 drivers/gpu/ion/ux500/ux500_ion.c
> If the API was to also be used for synchronization it would have to
> include an atomic "prepare multiple" ioctl which blocked until all
> the buffers listed by the application were available. In the same
Too slow already. You are now serializing stuff while what we want to do
really is
nobody_else_gets_buffers_next([list])
on available(buffer)
dispatch_work(buffer)
so that you can maximise parallelism without allowing deadlocks. If
you've got a high memory bandwith and 8+ cores the 'stop everything'
model isn't great.
> This might be a good argument for keeping synchronization and cache
> maintenance separate, though even ignoring synchronization I would
> think being able to issue cache maintenance operations for multiple
> buffers in a single ioctl might present some small efficiency gains.
> However as Rob points out, CPU access is already in slow/legacy
> territory.
Dangerous assumption. I do think they should be separate. For one it
makes the case of synchronization needed but hardware cache management
much easier to split cleanly. Assuming CPU access is slow/legacy reflects
a certain model of relatively slow CPU and accelerators where falling off
the acceleration path is bad. On a higher end processor falling off the
acceleration path isn't a performance matter so much as a power concern.
> KDS we differentiated jobs which needed "exclusive access" to a
> buffer and jobs which needed "shared access" to a buffer. Multiple
> jobs could access a buffer at the same time if those jobs all
Makes sense as it's a reader/writer lock and it reflects MESI/MOESI
caching and cache policy in some hardware/software assists.
> display controller will be reading the front buffer, but the GPU
> might also need to read that front buffer. So perhaps adding
> "read-only" & "read-write" access flags to prepare could also be
> interpreted as shared & exclusive accesses, if we went down this
> route for synchronization that is. :-)
mmap includes read/write info so probably using that works out. It also
means that you have the stuff mapped in a way that will bus error or
segfault anyone who goofs rather than give them the usual 'deep
weirdness' behaviour you get with mishandling of caching bits.
Alan