I've been buried all day, but here's my response to a bunch of the discussion points today:
I definitely don't think it makes sense to solve part of the problem and then leave the gpu's to require their own allocator for their common case. Why solve the allocator problem twice? Also, we want to be able to support passing of these buffers between the camera, gpu, video decoder etc. It'd be much easier if they were from a common source.
The android team's graphics folks, the their counterparts at intel, imagination tech, nvidia, qualcomm and arm have all told me that they need a method for mapping buffers uncached to userspace. The common case for this is to write vertexes, textures etc to these buffers once and never touch them again. This may happen several (or even several 10s or more) of times per frame. My experience with cache flushes on ARM architectures matches Marek's. Typically write combine makes streaming writes really really fast, and on several SOC's we've found it cheaper to flush the whole cache than to flush by line. Clearly this impacts the rest of system performance, not to mention the fact that a couple of large textures and you've totally blown your caches for the rest of the system.
Once we've solved the multiple mapping problem, it becomes quite easy to support both cached AND uncached accesses from the cpu, so I think that covers the cases where it's actually desirable to have a cached mapping -- software rendering, processing frames from a camera etc.
I think the issue of contiguous memory allocation and cache attributes are totally separate, except that in cases where large contiguous regions are necessary -- the problem qualcom's pmem and friends were written to solve -- you pretty much end up needing to put aside a pool of buffers at boot anyway in order to guarantee the availability of large order allocations. Once you've done that, the attributes problem goes away since your memory is not in the direct map.
On Wed, Apr 20, 2011 at 1:34 PM, Dave Airlie
<airlied@gmail.com> wrote:
>
>
> [TC] I’m not sure I completely agree with this being a use case. From my
> understanding, the problem we’re trying to solve here isn’t a generic
> graphics memory manager but rather a memory manager to facilitate
> cross-device sharing of 2D image buffers. GPU drivers will still have their
> own allocators for textures which will probably be in a tiled or other
> proprietary format no other device can understand anyway. The use case where
> we (GPU driver vendors) might want uncached memory is for one-time texture
> upload. If we have a texture we know we’re only going to write to once (with
> the CPU), there is no benefit in using cached memory. In fact, there’s a
> potential performance drop if you used cached memory because the texture
> upload will cause useful cache lines to be evicted and replaced with useless
> lines for the texture. However, I don’t see any use case where we’d then
> want to share that CPU-uploaded texture with another device, in which case
> we would use our own (uncached) allocator, not this “cross-device”
> allocator. There’s also a school of thought (so I’m told) that for one-time
> texture upload you still want to use cached memory because more modern cores
> have smaller write buffers (so you want to use the cache for better
> combining of writes) and are good at detecting large sequential writes and
> thus don’t use the whole cache for those anyway. So, other than one-time
> texture upload, are there other graphics use cases you know of where it
> might be more optimal to use uncached memory? What about video decoder
> use-cases?
>
The memory mangaer should be used for all internal GPU memory
management as well if desired.
If we have any hope of ever making open source ARM GPU drivers get
upstream they can't all just
go reinventing the wheel. They need to be based on a common layer.
Dave.