Just a temporary holdover to make locking/unlocking the dma_resv lock much easier.

Signed-off-by: Lyude Paul lyude@redhat.com Co-authored-by: Alexandre Courbot acourbot@nvidia.com Signed-off-by: Alexandre Courbot acourbot@nvidia.com

--- V17: * Fix format of commit message title

rust/kernel/drm/gem/shmem.rs | 31 ++++++++++++++++++++++++++++++- 1 file changed, 30 insertions(+), 1 deletion(-)

diff --git a/rust/kernel/drm/gem/shmem.rs b/rust/kernel/drm/gem/shmem.rs index 084b798ce795b..650c34dd8b7a4 100644 --- a/rust/kernel/drm/gem/shmem.rs +++ b/rust/kernel/drm/gem/shmem.rs @@ -30,7 +30,10 @@ Deref, DerefMut, // }, - ptr::NonNull, // + ptr::{ + self, + NonNull, // + }, }; use gem::{ BaseObjectPrivate, @@ -244,3 +247,29 @@ impl<T: DriverObject, C: DeviceContext> driver::AllocImpl for Object<T, C> { dumb_map_offset: None, }; } + +/// Private helper-type for holding the `dma_resv` object for a GEM shmem object. +/// +/// When this is dropped, the `dma_resv` lock is dropped as well. +/// +// TODO: This should be replace with a WwMutex equivalent once we have such bindings in the kernel. +struct DmaResvGuard<'a, T: DriverObject, C: DeviceContext = Registered>(&'a Object<T, C>); + +impl<'a, T: DriverObject, C: DeviceContext> DmaResvGuard<'a, T, C> { + #[inline(always)] + #[expect(unused)] + fn new(obj: &'a Object<T, C>) -> Self { + // SAFETY: This lock is initialized throughout the lifetime of `object`. + unsafe { bindings::dma_resv_lock(obj.raw_dma_resv(), ptr::null_mut()) }; + + Self(obj) + } +} + +impl<'a, T: DriverObject, C: DeviceContext> Drop for DmaResvGuard<'a, T, C> { + #[inline(always)] + fn drop(&mut self) { + // SAFETY: We are releasing the lock grabbed during the creation of this object. + unsafe { bindings::dma_resv_unlock(self.0.raw_dma_resv()) }; + } +}

One of the more obvious use cases for gem shmem objects is the ability to create mappings into their contents. So, let's hook this up in our rust bindings.

Signed-off-by: Lyude Paul lyude@redhat.com

--- V7: * Switch over to the new iosys map bindings that use the Io trait V8: * Get rid of iosys_map bindings for now, only support non-iomem types * s/as_shmem()/as_raw_shmem() V9: * Get rid of some outdated comments I missed * Add missing SIZE check to raw_vmap() * Add a proper unit test that ensures that we actually validate SIZE at compile-time. Turns out it takes only 34 lines to make a boilerplate DRM driver for a kunit test :) * Add unit tests * Add some missing #[inline]s V10: * Correct issue with iomem error path We previously called raw_vunmap() if we got an iomem allocation, but raw_vunmap() was written such that it assumed all allocations were sysmem allocations. Fix this by just making raw_vunmap() accept a iosys_map. V11: * Use Alexandre's clever solution to remove the macros we were using for maintaining two different VMap types. * Change the order of items in Object<T> to ensure that sgt_res is always dropped before obj. * Fix typo in Object.raw_vmap() * s/raw_vmap()/make_vmap()/ Deduplicate code a bit more as well by using more generics here V15: * Add these patches back * We only have one VMap type now! * Use ObjectConfig::default() in unit tests since we unbroke it. V16: * Fix huge rebase error I made and did not notice that squashed 1.5 patches together that were definitely not supposed to be squashed * Update old commit message V17: * Rebase * Fix format of commit message title

rust/kernel/drm/gem/shmem.rs | 312 ++++++++++++++++++++++++++++++++++- 1 file changed, 311 insertions(+), 1 deletion(-)

diff --git a/rust/kernel/drm/gem/shmem.rs b/rust/kernel/drm/gem/shmem.rs index 650c34dd8b7a4..1f05a5bc5fe66 100644 --- a/rust/kernel/drm/gem/shmem.rs +++ b/rust/kernel/drm/gem/shmem.rs @@ -20,12 +20,19 @@ Registered, // }, error::to_result, + io::{ + Io, + IoCapable, + IoKnownSize, // + }, prelude::*, sync::aref::ARef, types::Opaque, // }; use core::{ + ffi::c_void, marker::PhantomData, + mem::MaybeUninit, // ops::{ Deref, DerefMut, // @@ -36,6 +43,7 @@ }, }; use gem::{ + BaseObject, BaseObjectPrivate, DriverObject, IntoGEMObject, // @@ -197,6 +205,80 @@ extern "C" fn free_callback(obj: *mut bindings::drm_gem_object) { // SAFETY: We're recovering the Kbox<> we created in gem_create_object() let _ = unsafe { KBox::from_raw(this) }; } + + /// Attempt to create a vmap from the gem object, and confirm the size of said vmap. + fn make_vmap<'a, R, const SIZE: usize>(&'a self) -> Result<VMap<T, R, C, SIZE>> + where + R: Deref<Target = Self> + From<&'a Self>, + { + // INVARIANT: We check here that the gem object is at least as large as `SIZE`. + if self.size() < SIZE { + return Err(ENOSPC); + } + + let mut map: MaybeUninitbindings::iosys_map = MaybeUninit::uninit(); + let guard = DmaResvGuard::new(self); + + // SAFETY: drm_gem_shmem_vmap can be called with the DMA reservation lock held + to_result(unsafe { + bindings::drm_gem_shmem_vmap_locked(self.as_raw_shmem(), map.as_mut_ptr()) + })?; + + // Drop the guard explicitly here, since we may need to call raw_vunmap() (which re-acquires + // the lock). + drop(guard); + + // SAFETY: The call to drm_gem_shmem_vmap_locked succeeded above, so we are guaranteed that + // map is properly initialized. + let map = unsafe { map.assume_init() }; + + // XXX: We don't currently support iomem allocations + if map.is_iomem { + // SAFETY: + // - The vmap operation above succeeded, guaranteeing that `map` points to a valid + // memory mapping. + // - We checked that this is an iomem allocation, making it safe to read vaddr_iomem + unsafe { self.raw_vunmap(map) }; + + Err(ENOTSUPP) + } else { + Ok(VMap { + // SAFETY: We checked that this is not an iomem allocation, making it safe to read + // vaddr + addr: unsafe { map.__bindgen_anon_1.vaddr }, + owner: self.into(), + }) + } + } + + /// Unmap a vmap from the gem object. + /// + /// # Safety + /// + /// - The caller promises that `map` is a valid vmap on this gem object. + /// - The caller promises that the memory pointed to by map will no longer be accesed through + /// this instance. + unsafe fn raw_vunmap(&self, mut map: bindings::iosys_map) { + let _guard = DmaResvGuard::new(self); + + // SAFETY: + // - This function is safe to call with the DMA reservation lock held. + // - Our `ARef` is proof that the underlying gem object here is initialized and thus safe to + // dereference. + unsafe { bindings::drm_gem_shmem_vunmap_locked(self.as_raw_shmem(), &mut map) }; + } + + /// Creates and returns a virtual kernel memory mapping for this object. + #[inline] + pub fn vmap<const SIZE: usize>(&self) -> Result<VMapRef<'_, T, C, SIZE>> { + self.make_vmap() + } + + /// Creates and returns an owned reference to a virtual kernel memory mapping for this object. + #[inline] + pub fn owned_vmap<const SIZE: usize>(&self) -> Result<VMapOwned<T, C, SIZE>> { + self.make_vmap() + } }

impl<T: DriverObject, C: DeviceContext> Deref for Object<T, C> { @@ -257,7 +339,6 @@ impl<T: DriverObject, C: DeviceContext> driver::AllocImpl for Object<T, C> {

impl<'a, T: DriverObject, C: DeviceContext> DmaResvGuard<'a, T, C> { #[inline(always)] - #[expect(unused)] fn new(obj: &'a Object<T, C>) -> Self { // SAFETY: This lock is initialized throughout the lifetime of `object`. unsafe { bindings::dma_resv_lock(obj.raw_dma_resv(), ptr::null_mut()) }; @@ -273,3 +354,232 @@ fn drop(&mut self) { unsafe { bindings::dma_resv_unlock(self.0.raw_dma_resv()) }; } } + +macro_rules! impl_vmap_io_capable { + ($impl:ident, $ty:ty) => { + impl<D, R, C, const SIZE: usize> IoCapable<$ty> for $impl<D, R, C, SIZE> + where + D: DriverObject, + C: DeviceContext, + R: Deref<Target = Object<D, C>>, + { + #[inline(always)] + unsafe fn io_read(&self, address: usize) -> $ty { + let ptr = address as *mut $ty; + + // SAFETY: The safety contract of `io_read` guarantees that address is a valid + // address within the bounds of `Self` of at least the size of $ty, and is properly + // aligned. + unsafe { ptr::read(ptr) } + } + + #[inline(always)] + unsafe fn io_write(&self, value: $ty, address: usize) { + let ptr = address as *mut $ty; + + // SAFETY: The safety contract of `io_write` guarantees that address is a valid + // address within the bounds of `Self` of at least the size of $ty, and is properly + // aligned. + unsafe { ptr::write(ptr, value) } + } + } + }; +} + +/// A reference to a virtual mapping for an shmem-based GEM object in kernel address space. +/// +/// # Invariants +/// +/// - The size of `owner` is >= SIZE. +/// - The memory pointed to by addr remains valid at least until this object is dropped. +pub struct VMap<D, R, C = Registered, const SIZE: usize = 0> +where + D: DriverObject, + C: DeviceContext, + R: Deref<Target = Object<D, C>>, +{ + addr: *mut c_void, + owner: R, +} + +/// An alias type for a reference to a shmem-based GEM object's VMap. +pub type VMapRef<'a, D, C, const SIZE: usize = 0> = VMap<D, &'a Object<D, C>, C, SIZE>; + +/// An alias type for an owned reference to a shmem-based GEM object's VMap. +pub type VMapOwned<D, C, const SIZE: usize = 0> = VMap<D, ARef<Object<D, C>>, C, SIZE>; + +impl<D, R, C, const SIZE: usize> VMap<D, R, C, SIZE> +where + D: DriverObject, + C: DeviceContext, + R: Deref<Target = Object<D, C>>, +{ + /// Borrows a reference to the object that owns this virtual mapping. + #[inline(always)] + pub fn owner(&self) -> &Object<D, C> { + &self.owner + } +} + +impl<D, R, C, const SIZE: usize> Drop for VMap<D, R, C, SIZE> +where + D: DriverObject, + C: DeviceContext, + R: Deref<Target = Object<D, C>>, +{ + #[inline(always)] + fn drop(&mut self) { + // SAFETY: + // - Our existence is proof that this map was previously created using self.owner. + // - Since we are in Drop, we are guaranteed that no one will access the memory + // through this mapping after calling this. + unsafe { + self.owner.raw_vunmap(bindings::iosys_map { + is_iomem: false, + __bindgen_anon_1: bindings::iosys_map__bindgen_ty_1 { vaddr: self.addr }, + }) + }; + } +} + +impl<D, R, C, const SIZE: usize> Io for VMap<D, R, C, SIZE> +where + D: DriverObject, + C: DeviceContext, + R: Deref<Target = Object<D, C>>, +{ + #[inline(always)] + fn addr(&self) -> usize { + self.addr as usize + } + + #[inline(always)] + fn maxsize(&self) -> usize { + self.owner.size() + } +} + +impl<D, R, C, const SIZE: usize> IoKnownSize for VMap<D, R, C, SIZE> +where + D: DriverObject, + C: DeviceContext, + R: Deref<Target = Object<D, C>>, +{ + const MIN_SIZE: usize = SIZE; +} + +impl_vmap_io_capable!(VMap, u8); +impl_vmap_io_capable!(VMap, u16); +impl_vmap_io_capable!(VMap, u32); +#[cfg(CONFIG_64BIT)] +impl_vmap_io_capable!(VMap, u64); + +#[kunit_tests(rust_drm_gem_shmem)] +mod tests { + use super::*; + use crate::{ + drm::{ + self, + UnregisteredDevice, // + }, + faux, + page::PAGE_SIZE, // + }; + + // The bare minimum needed to create a fake drm driver for kunit + + #[pin_data] + struct KunitData {} + struct KunitDriver; + struct KunitFile; + #[pin_data] + struct KunitObject {} + + const INFO: drm::DriverInfo = drm::DriverInfo { + major: 0, + minor: 0, + patchlevel: 0, + name: c"kunit", + desc: c"Kunit", + }; + + impl drm::file::DriverFile for KunitFile { + type Driver = KunitDriver; + + fn open(_dev: &drm::Device<KunitDriver>) -> Result<Pin<KBox<Self>>> { + Ok(KBox::new(Self, GFP_KERNEL)?.into()) + } + } + + impl gem::DriverObject for KunitObject { + type Driver = KunitDriver; + type Args = (); + + fn new<C: DeviceContext>( + _dev: &drm::Device<KunitDriver, C>, + _size: usize, + _args: Self::Args, + ) -> impl PinInit<Self, Error> { + try_pin_init!(KunitObject {}) + } + } + + #[vtable] + impl drm::Driver for KunitDriver { + type Data = KunitData; + type File = KunitFile; + type Object<Ctx: DeviceContext> = Object<KunitObject, Ctx>; + + const INFO: drm::DriverInfo = INFO; + const IOCTLS: &'static [drm::ioctl::DrmIoctlDescriptor] = &[]; + } + + fn create_drm_dev() -> Result<(faux::Registration, UnregisteredDevice<KunitDriver>)> { + // Create a faux DRM device so we can test gem object creation. + let data = try_pin_init!(KunitData {}); + let dev = faux::Registration::new(c"Kunit", None)?; + let drm = UnregisteredDevice::new(dev.as_ref(), data)?; + + Ok((dev, drm)) + } + + #[test] + fn compile_time_vmap_sizes() -> Result { + let (_dev, drm) = create_drm_dev()?; + + let obj = Object::<KunitObject, _>::new(&drm, PAGE_SIZE, ObjectConfig::default(), ())?; + + // Try creating a normal vmap + obj.vmap::<PAGE_SIZE>()?; + + // Try creating a vmap that's smaller then the size we specified + obj.vmap::<{ PAGE_SIZE - 100 }>()?; + + // Make sure creating a vmap that's too large fails + assert!(obj.vmap::<{ PAGE_SIZE + 200 }>().is_err()); + + Ok(()) + } + + #[test] + fn vmap_io() -> Result { + let (_dev, drm) = create_drm_dev()?; + + let obj = Object::<KunitObject, _>::new(&drm, PAGE_SIZE, ObjectConfig::default(), ())?; + + let vmap = obj.vmap::<PAGE_SIZE>()?; + + vmap.write8(0xDE, 0x0); + assert_eq!(vmap.read8(0x0), 0xDE); + vmap.write32(0xFFFFFFFF, 0x20); + + assert_eq!(vmap.read32(0x20), 0xFFFFFFFF); + + assert_eq!(vmap.read8(0x20), 0xFF); + assert_eq!(vmap.read8(0x21), 0xFF); + assert_eq!(vmap.read8(0x22), 0xFF); + assert_eq!(vmap.read8(0x23), 0xFF); + + Ok(()) + } +}

When writing up some rust code that used faux devices for unit testing, I noticed that we never actually added the Bound device context to faux::Registration's AsRefdevice::Device implementation. This being said: the Registration object itself is proof that a driver is bound to the device - so this should be safe.

Signed-off-by: Lyude Paul lyude@redhat.com

--- V18: - Add notes from Danilo to safety comment.

rust/kernel/faux.rs | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-)

diff --git a/rust/kernel/faux.rs b/rust/kernel/faux.rs index 43b4974f48cd2..20ab638885354 100644 --- a/rust/kernel/faux.rs +++ b/rust/kernel/faux.rs @@ -25,7 +25,8 @@ /// /// # Invariants /// -/// `self.0` always holds a valid pointer to an initialized and registered [`struct faux_device`]. +/// - `self.0` always holds a valid pointer to an initialized and registered [`struct faux_device`]. +/// - This object is proof that the object described by this `Registration` is bound to a device. /// /// [`struct faux_device`]: srctree/include/linux/device/faux.h pub struct Registration(NonNullbindings::faux_device); @@ -59,10 +60,15 @@ fn as_raw(&self) -> *mut bindings::faux_device { } }

-impl AsRefdevice::Device for Registration { - fn as_ref(&self) -> &device::Device { - // SAFETY: The underlying `device` in `faux_device` is guaranteed by the C API to be - // a valid initialized `device`. +impl AsRef<device::Devicedevice::Bound> for Registration { + fn as_ref(&self) -> &device::Devicedevice::Bound { + // SAFETY: + // - The underlying `device` in `faux_device` is guaranteed by the C API to be a valid + // initialized `device`. + // - faux_match() always returns 1, and probe runs synchronously (PROBE_FORCE_SYNCHRONOUS). + // - suppress_bind_attrs = true on faux_driver prevents userspace-triggered unbind via sysfs + // - mem::forget(Registration) is not a problem; if the Registration is leaked, the faux + // device stays bound forever. unsafe { device::Device::from_raw(addr_of_mut!((*self.as_raw()).dev)) } } }

In order to do this, we need to be careful to ensure that any interface we expose for scatterlists ensures that any mappings created from one are destroyed on driver-unbind. To do this, we introduce a Devres resource into shmem::Object that we use in order to ensure that we release any SGTable mappings on driver-unbind.

There's some other slightly unfortunate caveats of this:

* Drivers don't have explicit control at the moment over when unmapping happens (which is exactly the same as the C side atm, so it might not be a problem). * We can't just return `SGTableMap` to the user through an Arc to attempt to fix the last caveat - because that implies the gem object would need to hold a reference count to the scatterlist mapping, which just leaves us with the same problem.

Signed-off-by: Lyude Paul lyude@redhat.com

--- V3: * Rename OwnedSGTable to shmem::SGTable. Since the current version of the SGTable abstractions now has a `Owned` and `Borrowed` variant, I think renaming this to shmem::SGTable makes things less confusing. We do however, keep the name of owned_sg_table() as-is. V4: * Clarify safety comments for SGTable to explain why the object is thread-safe. * Rename from SGTableRef to SGTable V10: * Use Devres in order to ensure that SGTables are revocable, and are unmapped on driver-unbind. V11: * s/create_sg_table()/get_sg_table() * Get rid of extraneous `ret = ` in shmem::Object::get_sg_table() V12: * Actually move sgt_res in this patch and not the next one V13: * Use DmaResvGuard suggestion from Alexander * Use Alexander's (much better) solution for get_sg_table() * Use SetOnce instead of UnsafeCell * s/SGTableRef/SGTableMap * Fix typo in SGTableMap documentation * Create fallible constructor for SGTableMap * Don't reuse dma_resv lock for protecting Object contents, just use Mutex + SetOnce * Drop use of drm_gem_shmem_get_pages_sgt_locked(), since we don't need to hold the dma_resv lock ourselves for anything but this function. * Check that the device we receive in the bounds for sg_table() and owned_sg_table() that said Device is in fact, the correct device. * Remove redundant docs in owned_sg_table(), just point it back to sg_table(). * Implement Deborah's suggestion to fix double-free in free_callback() * Restore original order of Object<T> * Fix doc typo for SGTableMap V14: * Use new InitOnce container over the Mutex/SetOnce horror show we had before. * Start using LazyInit container for storing Devres for sgt unmap * Add some kunit tests for sg_table (not sure why I didn't do this before) using some of the boilerplate code leftover from the vmap bindings * Get rid of the owned SGTable variant for now, we'll add it back in a future patch if people actually need it. * Use new LazyInit container from me to get rid of the horrid Mutex<SetOnce<>> mess. * Add the best we can do for unit tests w/r/t SGTable at the moment V16: * Get rid of LazyInit, go back to SetOnce, use trick that Alice recommended that is a lot cleaner. * Fix horrid rebasing mistake V17: * Rebase * Fix missing safety comment in free_callback() (we forgot to justify why &mut is safe in `unsafe { &mut (*this).sgt_res }.reset()`) V18: * Use ManuallyDrop instead of SetOnce::reset()

rust/kernel/drm/gem/shmem.rs | 172 +++++++++++++++++++++++++++++++++-- 1 file changed, 162 insertions(+), 10 deletions(-)

diff --git a/rust/kernel/drm/gem/shmem.rs b/rust/kernel/drm/gem/shmem.rs index 1f05a5bc5fe66..a9970fca1d298 100644 --- a/rust/kernel/drm/gem/shmem.rs +++ b/rust/kernel/drm/gem/shmem.rs @@ -11,6 +11,11 @@

use crate::{ container_of, + device::{ + self, + Bound, // + }, + devres::*, drm::{ driver, gem, @@ -19,20 +24,32 @@ DeviceContext, Registered, // }, - error::to_result, + error::{ + from_err_ptr, + to_result, // + }, io::{ Io, IoCapable, IoKnownSize, // }, prelude::*, - sync::aref::ARef, + scatterlist, + sync::{ + aref::ARef, + new_mutex, + Mutex, + SetOnce, // + }, types::Opaque, // }; use core::{ ffi::c_void, marker::PhantomData, - mem::MaybeUninit, // + mem::{ + ManuallyDrop, + MaybeUninit, // + }, ops::{ Deref, DerefMut, // @@ -87,6 +104,11 @@ pub struct Object<T: DriverObject, C: DeviceContext = Registered> { obj: Opaquebindings::drm_gem_shmem_object, /// Parent object that owns this object's DMA reservation object. parent_resv_obj: Option<ARef<Object<T, C>>>, + /// Devres object for unmapping any SGTable on driver-unbind. + sgt_res: ManuallyDrop<SetOnce<Devres<SGTableMap<T, C>>>>, + #[pin] + /// Lock for protecting initialization of `sgt_res`. + sgt_lock: Mutex<()>, #[pin] inner: T, _ctx: PhantomData<C>, @@ -145,6 +167,8 @@ pub fn new( try_pin_init!(Self { obj <- Opaque::init_zeroed(), parent_resv_obj: config.parent_resv_obj.map(|p| p.into()), + sgt_res: ManuallyDrop::new(SetOnce::new()), + sgt_lock <- new_mutex!(()), inner <- T::new(dev, size, args), _ctx: PhantomData::<C>, }), @@ -189,18 +213,26 @@ extern "C" fn free_callback(obj: *mut bindings::drm_gem_object) { // - DRM always passes a valid gem object here // - We used drm_gem_shmem_create() in our create_gem_object callback, so we know that // `obj` is contained within a drm_gem_shmem_object - let this = unsafe { container_of!(obj, bindings::drm_gem_shmem_object, base) }; - - // SAFETY: - // - We're in free_callback - so this function is safe to call. - // - We won't be using the gem resources on `this` after this call. - unsafe { bindings::drm_gem_shmem_release(this) }; + let base = unsafe { container_of!(obj, bindings::drm_gem_shmem_object, base) };

// SAFETY: // - We verified above that `obj` is valid, which makes `this` valid // - This function is set in AllocOps, so we know that `this` is contained within a // `Object<T, C>` - let this = unsafe { container_of!(Opaque::cast_from(this), Self, obj) }.cast_mut(); + let this = unsafe { container_of!(Opaque::cast_from(base), Self, obj) }.cast_mut(); + + // We need to drop `sgt_res` first, since doing so requires that the GEM object is still + // alive. + // SAFETY: + // - We verified above that `this` is valid. + // - We are in free_callback, guaranteeing we have exclusive access to `this` and that + // `sgt_res` will not be used after dropping it here. + unsafe { ManuallyDrop::drop(&mut (*this).sgt_res) }; + + // SAFETY: + // - We're in free_callback - so this function is safe to call. + // - We won't be using the gem resources on `this` after this call. + unsafe { bindings::drm_gem_shmem_release(base) };

// SAFETY: We're recovering the Kbox<> we created in gem_create_object() let _ = unsafe { KBox::from_raw(this) }; @@ -279,6 +311,45 @@ pub fn vmap<const SIZE: usize>(&self) -> Result<VMapRef<'_, T, C, SIZE>> { pub fn owned_vmap<const SIZE: usize>(&self) -> Result<VMapOwned<T, C, SIZE>> { self.make_vmap() } + + /// Creates (if necessary) and returns an immutable reference to a scatter-gather table of DMA + /// pages for this object. + /// + /// This will pin the object in memory. It is expected that `dev` should be a pointer to the + /// same [`device::Device`] which `self` belongs to, otherwise this function will return + /// `Err(EINVAL)`. + pub fn sg_table<'a>( + &'a self, + dev: &'a device::Device<Bound>, + ) -> Result<&'a scatterlist::SGTable> { + if dev.as_raw() != self.dev().as_ref().as_raw() { + return Err(EINVAL); + } + + let sgt_res = 'out: { + // Fast path: sgt_res is already initialized + if let Some(sgt_res) = self.sgt_res.as_ref() { + break 'out sgt_res; + } + + // Slow path: Grab the lock and see if we need to initialize sgt_res. + let _guard = self.sgt_lock.lock(); + + // If someone initialized it while we were waiting, we can exit early. + if let Some(sgt_res) = self.sgt_res.as_ref() { + break 'out sgt_res; + } + + // If not, finish initializing and return. + self.sgt_res + .populate(Devres::new(dev, SGTableMap::new(self))?); + + // SAFETY: We just populated sgt_res above. + unsafe { self.sgt_res.as_ref().unwrap_unchecked() } + }; + + Ok(sgt_res.access(dev)?) + } }

impl<T: DriverObject, C: DeviceContext> Deref for Object<T, C> { @@ -474,6 +545,63 @@ impl<D, R, C, const SIZE: usize> IoKnownSize for VMap<D, R, C, SIZE> #[cfg(CONFIG_64BIT)] impl_vmap_io_capable!(VMap, u64);

+/// A reference to a GEM object that is known to have a mapped [`SGTable`]. +/// +/// This is used by the Rust bindings with [`Devres`] in order to ensure that mappings for SGTables +/// on GEM shmem objects are revoked on driver-unbind. +/// +/// # Invariants +/// +/// - `self.obj` always points to a valid GEM object. +/// - This object is proof that `self.obj.owner.sgt` has an initialized and valid +/// [`scatterlist::SGTable`]. +pub struct SGTableMap<T: DriverObject, C: DeviceContext> { + obj: NonNull<Object<T, C>>, +} + +impl<T: DriverObject, C: DeviceContext> Deref for SGTableMap<T, C> { + type Target = scatterlist::SGTable; + + fn deref(&self) -> &Self::Target { + // SAFETY: + // - The NonNull is guaranteed to be valid via our type invariants. + // - The sgt field is guaranteed to be initialized and valid via our type invariants. + unsafe { scatterlist::SGTable::from_raw((*self.obj.as_ref().as_raw_shmem()).sgt) } + } +} + +impl<T: DriverObject, C: DeviceContext> Drop for SGTableMap<T, C> { + fn drop(&mut self) { + // SAFETY: `obj` is always valid via our type invariants + let obj = unsafe { self.obj.as_ref() }; + let _lock = DmaResvGuard::new(obj); + + // SAFETY: We acquired the lock needed for calling this function above + unsafe { bindings::__drm_gem_shmem_free_sgt_locked(obj.as_raw_shmem()) }; + } +} + +impl<T: DriverObject, C: DeviceContext> SGTableMap<T, C> { + fn new(obj: &Object<T, C>) -> impl Init<Self, Error> { + // INVARIANT: + // - We call drm_gem_shmem_get_pages_sgt_locked below and check whether or not it + // succeeds, fulfilling the invariant of SGTableMap that the object's `sgt` field is + // initialized. + // SAFETY: + // - `obj` is fully initialized, making this function safe to call. + from_err_ptr(unsafe { bindings::drm_gem_shmem_get_pages_sgt(obj.as_raw_shmem()) })?; + + Ok(Self { obj: obj.into() }) + } +} + +// SAFETY: The NonNull in SGTableMap is guaranteed valid by our type invariants, and the GEM object +// it points to is guaranteed to be thread-safe. +unsafe impl<T: DriverObject, C: DeviceContext> Send for SGTableMap<T, C> {} +// SAFETY: The NonNull in SGTableMap is guaranteed valid by our type invariants, and the GEM object +// it points to is guaranteed to be thread-safe. +unsafe impl<T: DriverObject, C: DeviceContext> Sync for SGTableMap<T, C> {} + #[kunit_tests(rust_drm_gem_shmem)] mod tests { use super::*; @@ -582,4 +710,28 @@ fn vmap_io() -> Result {

Ok(()) } + + // TODO: I would love to actually test the success paths of sg_table(), but that would require + // also implementing dummy dma_ops so that trying to create a mapping doesn't explode. So, leave + // that for someone else. + + // Ensures that passing the wrong device to sg_table() fails as we expect, and also ensure it + // skips initializing `sgt_res` since we could otherwise create `sgt_res` with the wrong device + // bound to it. + #[test] + fn fail_sg_table_on_wrong_dev() -> Result { + let (_dev, drm) = create_drm_dev()?; + let wrong_dev = faux::Registration::new(c"EvilKunit", None)?; + + let obj = Object::<KunitObject, _>::new(&drm, PAGE_SIZE, ObjectConfig::default(), ())?; + + assert_eq!(obj.sg_table(wrong_dev.as_ref()).err().unwrap(), EINVAL); + + // If sgt_res was not initialized mistakenly with the wrong device, this should still fail. + assert_eq!(obj.sg_table(wrong_dev.as_ref()).err().unwrap(), EINVAL); + + // TODO: Someday, we should test that creating an sg_table here still succeeds. + + Ok(()) + } }