Hi guys,
The version I've send out yesterday had a rather obvious coding error and I honestly forgot the cover letter.
This one here is better tested and will now hopefully not be torn apart from the CI system immediately.
I tried to address all review and documentation comments as best as I could, so I'm hoping that we can now considering pushing this.
Cheers,
Christian.
On Wed, Sep 22, 2021 at 7:31 AM Andrey Grodzovsky
<andrey.grodzovsky(a)amd.com> wrote:
>
>
> On 2021-09-21 11:32 p.m., Rob Clark wrote:
> > On Tue, Sep 21, 2021 at 7:18 PM Andrey Grodzovsky
> > <andrey.grodzovsky(a)amd.com> wrote:
> >>
> >> On 2021-09-21 4:47 p.m., Rob Clark wrote:
> >>> On Tue, Sep 21, 2021 at 1:09 PM Andrey Grodzovsky
> >>> <andrey.grodzovsky(a)amd.com> wrote:
> >>>> On 2021-09-03 2:47 p.m., Rob Clark wrote:
> >>>>
> >>>>> From: Rob Clark <robdclark(a)chromium.org>
> >>>>>
> >>>>> As the finished fence is the one that is exposed to userspace, and
> >>>>> therefore the one that other operations, like atomic update, would
> >>>>> block on, we need to propagate the deadline from from the finished
> >>>>> fence to the actual hw fence.
> >>>>>
> >>>>> v2: Split into drm_sched_fence_set_parent() (ckoenig)
> >>>>>
> >>>>> Signed-off-by: Rob Clark <robdclark(a)chromium.org>
> >>>>> ---
> >>>>> drivers/gpu/drm/scheduler/sched_fence.c | 34 +++++++++++++++++++++++++
> >>>>> drivers/gpu/drm/scheduler/sched_main.c | 2 +-
> >>>>> include/drm/gpu_scheduler.h | 8 ++++++
> >>>>> 3 files changed, 43 insertions(+), 1 deletion(-)
> >>>>>
> >>>>> diff --git a/drivers/gpu/drm/scheduler/sched_fence.c b/drivers/gpu/drm/scheduler/sched_fence.c
> >>>>> index bcea035cf4c6..4fc41a71d1c7 100644
> >>>>> --- a/drivers/gpu/drm/scheduler/sched_fence.c
> >>>>> +++ b/drivers/gpu/drm/scheduler/sched_fence.c
> >>>>> @@ -128,6 +128,30 @@ static void drm_sched_fence_release_finished(struct dma_fence *f)
> >>>>> dma_fence_put(&fence->scheduled);
> >>>>> }
> >>>>>
> >>>>> +static void drm_sched_fence_set_deadline_finished(struct dma_fence *f,
> >>>>> + ktime_t deadline)
> >>>>> +{
> >>>>> + struct drm_sched_fence *fence = to_drm_sched_fence(f);
> >>>>> + unsigned long flags;
> >>>>> +
> >>>>> + spin_lock_irqsave(&fence->lock, flags);
> >>>>> +
> >>>>> + /* If we already have an earlier deadline, keep it: */
> >>>>> + if (test_bit(DMA_FENCE_FLAG_HAS_DEADLINE_BIT, &f->flags) &&
> >>>>> + ktime_before(fence->deadline, deadline)) {
> >>>>> + spin_unlock_irqrestore(&fence->lock, flags);
> >>>>> + return;
> >>>>> + }
> >>>>> +
> >>>>> + fence->deadline = deadline;
> >>>>> + set_bit(DMA_FENCE_FLAG_HAS_DEADLINE_BIT, &f->flags);
> >>>>> +
> >>>>> + spin_unlock_irqrestore(&fence->lock, flags);
> >>>>> +
> >>>>> + if (fence->parent)
> >>>>> + dma_fence_set_deadline(fence->parent, deadline);
> >>>>> +}
> >>>>> +
> >>>>> static const struct dma_fence_ops drm_sched_fence_ops_scheduled = {
> >>>>> .get_driver_name = drm_sched_fence_get_driver_name,
> >>>>> .get_timeline_name = drm_sched_fence_get_timeline_name,
> >>>>> @@ -138,6 +162,7 @@ static const struct dma_fence_ops drm_sched_fence_ops_finished = {
> >>>>> .get_driver_name = drm_sched_fence_get_driver_name,
> >>>>> .get_timeline_name = drm_sched_fence_get_timeline_name,
> >>>>> .release = drm_sched_fence_release_finished,
> >>>>> + .set_deadline = drm_sched_fence_set_deadline_finished,
> >>>>> };
> >>>>>
> >>>>> struct drm_sched_fence *to_drm_sched_fence(struct dma_fence *f)
> >>>>> @@ -152,6 +177,15 @@ struct drm_sched_fence *to_drm_sched_fence(struct dma_fence *f)
> >>>>> }
> >>>>> EXPORT_SYMBOL(to_drm_sched_fence);
> >>>>>
> >>>>> +void drm_sched_fence_set_parent(struct drm_sched_fence *s_fence,
> >>>>> + struct dma_fence *fence)
> >>>>> +{
> >>>>> + s_fence->parent = dma_fence_get(fence);
> >>>>> + if (test_bit(DMA_FENCE_FLAG_HAS_DEADLINE_BIT,
> >>>>> + &s_fence->finished.flags))
> >>>>> + dma_fence_set_deadline(fence, s_fence->deadline);
> >>>> I believe above you should pass be s_fence->finished to
> >>>> dma_fence_set_deadline
> >>>> instead it fence which is the HW fence itself.
> >>> Hmm, unless this has changed recently with some patches I don't have,
> >>> s_fence->parent is the one signalled by hw, so it is the one we want
> >>> to set the deadline on
> >>>
> >>> BR,
> >>> -R
> >>
> >> No it didn't change. But then when exactly will
> >> drm_sched_fence_set_deadline_finished
> >> execute such that fence->parent != NULL ? In other words, I am not clear
> >> how propagation
> >> happens otherwise - if dma_fence_set_deadline is called with the HW
> >> fence then the assumption
> >> here is that driver provided driver specific
> >> dma_fence_ops.dma_fence_set_deadline callback executes
> >> but I was under impression that drm_sched_fence_set_deadline_finished is
> >> the one that propagates
> >> the deadline to the HW fence's callback and for it to execute
> >> dma_fence_set_deadline needs to be called
> >> with s_fence->finished.
> > Assuming I didn't screw up drm/msm conversion to scheduler,
> > &s_fence->finished is the one that will be returned to userspace.. and
> > later passed back to kernel for atomic commit (or to the compositor).
> > So it is the one that fence->set_deadline() will be called on. But
> > s_fence->parent is the actual hw fence that needs to know about the
> > deadline. Depending on whether or not the job has been written into
> > hw ringbuffer or not, there are two cases:
> >
> > 1) not scheduled yet, s_fence will store the deadline and propagate it
> > later once s_fence->parent is known
>
>
> And by later you mean the call to drm_sched_fence_set_parent
> after HW fence is returned ? If yes I think i get it now.
Yup :-)
BR,
-R
> Andrey
>
>
> > 2) already scheduled, in which case s_fence->finished.set_deadline
> > will propagate it directly to the real fence
> >
> > BR,
> > -R
> >
> >> Andrey
> >>
> >>
> >>
> >>>> Andrey
> >>>>
> >>>>
> >>>>> +}
> >>>>> +
> >>>>> struct drm_sched_fence *drm_sched_fence_alloc(struct drm_sched_entity *entity,
> >>>>> void *owner)
> >>>>> {
> >>>>> diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c
> >>>>> index 595e47ff7d06..27bf0ac0625f 100644
> >>>>> --- a/drivers/gpu/drm/scheduler/sched_main.c
> >>>>> +++ b/drivers/gpu/drm/scheduler/sched_main.c
> >>>>> @@ -978,7 +978,7 @@ static int drm_sched_main(void *param)
> >>>>> drm_sched_fence_scheduled(s_fence);
> >>>>>
> >>>>> if (!IS_ERR_OR_NULL(fence)) {
> >>>>> - s_fence->parent = dma_fence_get(fence);
> >>>>> + drm_sched_fence_set_parent(s_fence, fence);
> >>>>> r = dma_fence_add_callback(fence, &sched_job->cb,
> >>>>> drm_sched_job_done_cb);
> >>>>> if (r == -ENOENT)
> >>>>> diff --git a/include/drm/gpu_scheduler.h b/include/drm/gpu_scheduler.h
> >>>>> index 7f77a455722c..158ddd662469 100644
> >>>>> --- a/include/drm/gpu_scheduler.h
> >>>>> +++ b/include/drm/gpu_scheduler.h
> >>>>> @@ -238,6 +238,12 @@ struct drm_sched_fence {
> >>>>> */
> >>>>> struct dma_fence finished;
> >>>>>
> >>>>> + /**
> >>>>> + * @deadline: deadline set on &drm_sched_fence.finished which
> >>>>> + * potentially needs to be propagated to &drm_sched_fence.parent
> >>>>> + */
> >>>>> + ktime_t deadline;
> >>>>> +
> >>>>> /**
> >>>>> * @parent: the fence returned by &drm_sched_backend_ops.run_job
> >>>>> * when scheduling the job on hardware. We signal the
> >>>>> @@ -505,6 +511,8 @@ void drm_sched_entity_set_priority(struct drm_sched_entity *entity,
> >>>>> enum drm_sched_priority priority);
> >>>>> bool drm_sched_entity_is_ready(struct drm_sched_entity *entity);
> >>>>>
> >>>>> +void drm_sched_fence_set_parent(struct drm_sched_fence *s_fence,
> >>>>> + struct dma_fence *fence);
> >>>>> struct drm_sched_fence *drm_sched_fence_alloc(
> >>>>> struct drm_sched_entity *s_entity, void *owner);
> >>>>> void drm_sched_fence_init(struct drm_sched_fence *fence,
On Tue, Sep 21, 2021 at 7:18 PM Andrey Grodzovsky
<andrey.grodzovsky(a)amd.com> wrote:
>
>
> On 2021-09-21 4:47 p.m., Rob Clark wrote:
> > On Tue, Sep 21, 2021 at 1:09 PM Andrey Grodzovsky
> > <andrey.grodzovsky(a)amd.com> wrote:
> >> On 2021-09-03 2:47 p.m., Rob Clark wrote:
> >>
> >>> From: Rob Clark <robdclark(a)chromium.org>
> >>>
> >>> As the finished fence is the one that is exposed to userspace, and
> >>> therefore the one that other operations, like atomic update, would
> >>> block on, we need to propagate the deadline from from the finished
> >>> fence to the actual hw fence.
> >>>
> >>> v2: Split into drm_sched_fence_set_parent() (ckoenig)
> >>>
> >>> Signed-off-by: Rob Clark <robdclark(a)chromium.org>
> >>> ---
> >>> drivers/gpu/drm/scheduler/sched_fence.c | 34 +++++++++++++++++++++++++
> >>> drivers/gpu/drm/scheduler/sched_main.c | 2 +-
> >>> include/drm/gpu_scheduler.h | 8 ++++++
> >>> 3 files changed, 43 insertions(+), 1 deletion(-)
> >>>
> >>> diff --git a/drivers/gpu/drm/scheduler/sched_fence.c b/drivers/gpu/drm/scheduler/sched_fence.c
> >>> index bcea035cf4c6..4fc41a71d1c7 100644
> >>> --- a/drivers/gpu/drm/scheduler/sched_fence.c
> >>> +++ b/drivers/gpu/drm/scheduler/sched_fence.c
> >>> @@ -128,6 +128,30 @@ static void drm_sched_fence_release_finished(struct dma_fence *f)
> >>> dma_fence_put(&fence->scheduled);
> >>> }
> >>>
> >>> +static void drm_sched_fence_set_deadline_finished(struct dma_fence *f,
> >>> + ktime_t deadline)
> >>> +{
> >>> + struct drm_sched_fence *fence = to_drm_sched_fence(f);
> >>> + unsigned long flags;
> >>> +
> >>> + spin_lock_irqsave(&fence->lock, flags);
> >>> +
> >>> + /* If we already have an earlier deadline, keep it: */
> >>> + if (test_bit(DMA_FENCE_FLAG_HAS_DEADLINE_BIT, &f->flags) &&
> >>> + ktime_before(fence->deadline, deadline)) {
> >>> + spin_unlock_irqrestore(&fence->lock, flags);
> >>> + return;
> >>> + }
> >>> +
> >>> + fence->deadline = deadline;
> >>> + set_bit(DMA_FENCE_FLAG_HAS_DEADLINE_BIT, &f->flags);
> >>> +
> >>> + spin_unlock_irqrestore(&fence->lock, flags);
> >>> +
> >>> + if (fence->parent)
> >>> + dma_fence_set_deadline(fence->parent, deadline);
> >>> +}
> >>> +
> >>> static const struct dma_fence_ops drm_sched_fence_ops_scheduled = {
> >>> .get_driver_name = drm_sched_fence_get_driver_name,
> >>> .get_timeline_name = drm_sched_fence_get_timeline_name,
> >>> @@ -138,6 +162,7 @@ static const struct dma_fence_ops drm_sched_fence_ops_finished = {
> >>> .get_driver_name = drm_sched_fence_get_driver_name,
> >>> .get_timeline_name = drm_sched_fence_get_timeline_name,
> >>> .release = drm_sched_fence_release_finished,
> >>> + .set_deadline = drm_sched_fence_set_deadline_finished,
> >>> };
> >>>
> >>> struct drm_sched_fence *to_drm_sched_fence(struct dma_fence *f)
> >>> @@ -152,6 +177,15 @@ struct drm_sched_fence *to_drm_sched_fence(struct dma_fence *f)
> >>> }
> >>> EXPORT_SYMBOL(to_drm_sched_fence);
> >>>
> >>> +void drm_sched_fence_set_parent(struct drm_sched_fence *s_fence,
> >>> + struct dma_fence *fence)
> >>> +{
> >>> + s_fence->parent = dma_fence_get(fence);
> >>> + if (test_bit(DMA_FENCE_FLAG_HAS_DEADLINE_BIT,
> >>> + &s_fence->finished.flags))
> >>> + dma_fence_set_deadline(fence, s_fence->deadline);
> >>
> >> I believe above you should pass be s_fence->finished to
> >> dma_fence_set_deadline
> >> instead it fence which is the HW fence itself.
> > Hmm, unless this has changed recently with some patches I don't have,
> > s_fence->parent is the one signalled by hw, so it is the one we want
> > to set the deadline on
> >
> > BR,
> > -R
>
>
> No it didn't change. But then when exactly will
> drm_sched_fence_set_deadline_finished
> execute such that fence->parent != NULL ? In other words, I am not clear
> how propagation
> happens otherwise - if dma_fence_set_deadline is called with the HW
> fence then the assumption
> here is that driver provided driver specific
> dma_fence_ops.dma_fence_set_deadline callback executes
> but I was under impression that drm_sched_fence_set_deadline_finished is
> the one that propagates
> the deadline to the HW fence's callback and for it to execute
> dma_fence_set_deadline needs to be called
> with s_fence->finished.
Assuming I didn't screw up drm/msm conversion to scheduler,
&s_fence->finished is the one that will be returned to userspace.. and
later passed back to kernel for atomic commit (or to the compositor).
So it is the one that fence->set_deadline() will be called on. But
s_fence->parent is the actual hw fence that needs to know about the
deadline. Depending on whether or not the job has been written into
hw ringbuffer or not, there are two cases:
1) not scheduled yet, s_fence will store the deadline and propagate it
later once s_fence->parent is known
2) already scheduled, in which case s_fence->finished.set_deadline
will propagate it directly to the real fence
BR,
-R
> Andrey
>
>
>
> >
> >> Andrey
> >>
> >>
> >>> +}
> >>> +
> >>> struct drm_sched_fence *drm_sched_fence_alloc(struct drm_sched_entity *entity,
> >>> void *owner)
> >>> {
> >>> diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c
> >>> index 595e47ff7d06..27bf0ac0625f 100644
> >>> --- a/drivers/gpu/drm/scheduler/sched_main.c
> >>> +++ b/drivers/gpu/drm/scheduler/sched_main.c
> >>> @@ -978,7 +978,7 @@ static int drm_sched_main(void *param)
> >>> drm_sched_fence_scheduled(s_fence);
> >>>
> >>> if (!IS_ERR_OR_NULL(fence)) {
> >>> - s_fence->parent = dma_fence_get(fence);
> >>> + drm_sched_fence_set_parent(s_fence, fence);
> >>> r = dma_fence_add_callback(fence, &sched_job->cb,
> >>> drm_sched_job_done_cb);
> >>> if (r == -ENOENT)
> >>> diff --git a/include/drm/gpu_scheduler.h b/include/drm/gpu_scheduler.h
> >>> index 7f77a455722c..158ddd662469 100644
> >>> --- a/include/drm/gpu_scheduler.h
> >>> +++ b/include/drm/gpu_scheduler.h
> >>> @@ -238,6 +238,12 @@ struct drm_sched_fence {
> >>> */
> >>> struct dma_fence finished;
> >>>
> >>> + /**
> >>> + * @deadline: deadline set on &drm_sched_fence.finished which
> >>> + * potentially needs to be propagated to &drm_sched_fence.parent
> >>> + */
> >>> + ktime_t deadline;
> >>> +
> >>> /**
> >>> * @parent: the fence returned by &drm_sched_backend_ops.run_job
> >>> * when scheduling the job on hardware. We signal the
> >>> @@ -505,6 +511,8 @@ void drm_sched_entity_set_priority(struct drm_sched_entity *entity,
> >>> enum drm_sched_priority priority);
> >>> bool drm_sched_entity_is_ready(struct drm_sched_entity *entity);
> >>>
> >>> +void drm_sched_fence_set_parent(struct drm_sched_fence *s_fence,
> >>> + struct dma_fence *fence);
> >>> struct drm_sched_fence *drm_sched_fence_alloc(
> >>> struct drm_sched_entity *s_entity, void *owner);
> >>> void drm_sched_fence_init(struct drm_sched_fence *fence,
Abstract the complexity of iterating over all the fences
in a dma_resv object.
The new loop handles the whole RCU and retry dance and
returns only fences where we can be sure we grabbed the
right one.
v2: fix accessing the shared fences while they might be freed,
improve kerneldoc, rename _cursor to _iter, add
dma_resv_iter_is_exclusive, add dma_resv_iter_begin/end
v3: restructor the code, move rcu_read_lock()/unlock() into the
iterator, add dma_resv_iter_is_restarted()
Signed-off-by: Christian König <christian.koenig(a)amd.com>
---
drivers/dma-buf/dma-resv.c | 98 ++++++++++++++++++++++++++++++++++++++
include/linux/dma-resv.h | 95 ++++++++++++++++++++++++++++++++++++
2 files changed, 193 insertions(+)
diff --git a/drivers/dma-buf/dma-resv.c b/drivers/dma-buf/dma-resv.c
index 84fbe60629e3..11b5399f4bd3 100644
--- a/drivers/dma-buf/dma-resv.c
+++ b/drivers/dma-buf/dma-resv.c
@@ -323,6 +323,104 @@ void dma_resv_add_excl_fence(struct dma_resv *obj, struct dma_fence *fence)
}
EXPORT_SYMBOL(dma_resv_add_excl_fence);
+/**
+ * dma_resv_iter_restart_unlocked - restart the unlocked iterator
+ * @cursor: The dma_resv_iter object to restart
+ *
+ * Restart the unlocked iteration by initializing the cursor object.
+ */
+static void dma_resv_iter_restart_unlocked(struct dma_resv_iter *cursor)
+{
+ cursor->seq = read_seqcount_begin(&cursor->obj->seq);
+ cursor->index = -1;
+ if (cursor->all_fences) {
+ rcu_read_lock();
+ cursor->fences = dma_resv_shared_list(cursor->obj);
+ rcu_read_unlock();
+ } else {
+ cursor->fences = NULL;
+ }
+ cursor->is_restarted = true;
+}
+
+/**
+ * dma_resv_iter_walk_unlocked - walk over fences in a dma_resv obj
+ * @cursor: cursor to record the current position
+ *
+ * Return all the fences in the dma_resv object which are not yet signaled.
+ * The returned fence has an extra local reference so will stay alive.
+ * If a concurrent modify is detected the whole iterration is started over again.
+ */
+static void dma_resv_iter_walk_unlocked(struct dma_resv_iter *cursor)
+{
+ struct dma_resv *obj = cursor->obj;
+
+ do {
+ /* Drop the reference from the previous round */
+ dma_fence_put(cursor->fence);
+
+ if (cursor->index++ == -1) {
+ cursor->fence = dma_resv_excl_fence(obj);
+ cursor->fence = dma_fence_get_rcu(cursor->fence);
+
+ } else if (!cursor->fences ||
+ cursor->index >= cursor->fences->shared_count) {
+ cursor->fence = NULL;
+
+ } else {
+ struct dma_resv_list *fences = cursor->fences;
+ unsigned int idx = cursor->index;
+
+ cursor->fence = rcu_dereference(fences->shared[idx]);
+ cursor->fence = dma_fence_get_rcu(cursor->fence);
+ }
+ } while (cursor->fence && dma_fence_is_signaled(cursor->fence));
+}
+
+/**
+ * dma_resv_iter_first_unlocked - first fence in an unlocked dma_resv obj.
+ * @cursor: the cursor with the current position
+ *
+ * Returns the first fence from an unlocked dma_resv obj.
+ */
+struct dma_fence *dma_resv_iter_first_unlocked(struct dma_resv_iter *cursor)
+{
+ rcu_read_lock();
+ do {
+ dma_resv_iter_restart_unlocked(cursor);
+ dma_resv_iter_walk_unlocked(cursor);
+ } while (read_seqcount_retry(&cursor->obj->seq, cursor->seq));
+ rcu_read_unlock();
+
+ return cursor->fence;
+}
+EXPORT_SYMBOL(dma_resv_iter_first_unlocked);
+
+/**
+ * dma_resv_iter_next_unlocked - next fence in an unlocked dma_resv obj.
+ * @cursor: the cursor with the current position
+ *
+ * Returns the next fence from an unlocked dma_resv obj.
+ */
+struct dma_fence *dma_resv_iter_next_unlocked(struct dma_resv_iter *cursor)
+{
+ bool restart;
+
+ rcu_read_lock();
+ cursor->is_restarted = false;
+ restart = read_seqcount_retry(&cursor->obj->seq, cursor->seq);
+ do {
+ if (restart)
+ dma_resv_iter_restart_unlocked(cursor);
+ dma_resv_iter_walk_unlocked(cursor);
+ restart = true;
+ } while (read_seqcount_retry(&cursor->obj->seq, cursor->seq));
+ rcu_read_unlock();
+
+ return cursor->fence;
+}
+EXPORT_SYMBOL(dma_resv_iter_next_unlocked);
+
/**
* dma_resv_copy_fences - Copy all fences from src to dst.
* @dst: the destination reservation object
diff --git a/include/linux/dma-resv.h b/include/linux/dma-resv.h
index 9100dd3dc21f..baf77a542392 100644
--- a/include/linux/dma-resv.h
+++ b/include/linux/dma-resv.h
@@ -149,6 +149,101 @@ struct dma_resv {
struct dma_resv_list __rcu *fence;
};
+/**
+ * struct dma_resv_iter - current position into the dma_resv fences
+ *
+ * Don't touch this directly in the driver, use the accessor function instead.
+ */
+struct dma_resv_iter {
+ /** @obj: The dma_resv object we iterate over */
+ struct dma_resv *obj;
+
+ /** @all_fences: If all fences should be returned */
+ bool all_fences;
+
+ /** @fence: the currently handled fence */
+ struct dma_fence *fence;
+
+ /** @seq: sequence number to check for modifications */
+ unsigned int seq;
+
+ /** @index: index into the shared fences */
+ unsigned int index;
+
+ /** @fences: the shared fences */
+ struct dma_resv_list *fences;
+
+ /** @is_restarted: true if this is the first returned fence */
+ bool is_restarted;
+};
+
+struct dma_fence *dma_resv_iter_first_unlocked(struct dma_resv_iter *cursor);
+struct dma_fence *dma_resv_iter_next_unlocked(struct dma_resv_iter *cursor);
+
+/**
+ * dma_resv_iter_begin - initialize a dma_resv_iter object
+ * @cursor: The dma_resv_iter object to initialize
+ * @obj: The dma_resv object which we want to iterator over
+ * @all_fences: If all fences should be returned or just the exclusive one
+ */
+static inline void dma_resv_iter_begin(struct dma_resv_iter *cursor,
+ struct dma_resv *obj,
+ bool all_fences)
+{
+ cursor->obj = obj;
+ cursor->all_fences = all_fences;
+ cursor->fence = NULL;
+}
+
+/**
+ * dma_resv_iter_end - cleanup a dma_resv_iter object
+ * @cursor: the dma_resv_iter object which should be cleaned up
+ *
+ * Make sure that the reference to the fence in the cursor is properly
+ * dropped.
+ */
+static inline void dma_resv_iter_end(struct dma_resv_iter *cursor)
+{
+ dma_fence_put(cursor->fence);
+}
+
+/**
+ * dma_resv_iter_is_exclusive - test if the current fence is the exclusive one
+ * @cursor: the cursor of the current position
+ *
+ * Returns true if the currently returned fence is the exclusive one.
+ */
+static inline bool dma_resv_iter_is_exclusive(struct dma_resv_iter *cursor)
+{
+ return cursor->index == -1;
+}
+
+/**
+ * dma_resv_iter_is_restarted - test if this is the first fence after a restart
+ * @cursor: the cursor with the current position
+ *
+ * Return true if this is the first fence in an interation after a restart.
+ */
+static inline bool dma_resv_iter_is_restarted(struct dma_resv_iter *cursor)
+{
+ return cursor->is_restarted;
+}
+
+/**
+ * dma_resv_for_each_fence_unlocked - unlocked fence iterator
+ * @cursor: a struct dma_resv_iter pointer
+ * @fence: the current fence
+ *
+ * Iterate over the fences in a struct dma_resv object without holding the
+ * &dma_resv.lock and using RCU instead. The cursor needs to be initialized
+ * with dma_resv_iter_begin() and cleaned up with dma_resv_iter_end(). Inside
+ * the iterator a reference to the dma_fence is hold and the RCU lock dropped.
+ * When the dma_resv is modified the iteration starts over again.
+ */
+#define dma_resv_for_each_fence_unlocked(cursor, fence) \
+ for (fence = dma_resv_iter_first_unlocked(cursor); \
+ fence; fence = dma_resv_iter_next_unlocked(cursor))
+
#define dma_resv_held(obj) lockdep_is_held(&(obj)->lock.base)
#define dma_resv_assert_held(obj) lockdep_assert_held(&(obj)->lock.base)
--
2.25.1
Hopefully the last round for this.
Added dma_resv_iter_begin/end as requested by Daniel. Fixed a bunch of
problems pointed out by the CI systems and found a few more myselve.
Please review and/or comment,
Christian.
On Fri, Sep 17, 2021 at 02:59:42PM +0200, Alexandre Bailon wrote:
> This adds the device tree bindings for the APU DRM driver.
>
> Signed-off-by: Alexandre Bailon <abailon(a)baylibre.com>
> ---
> .../devicetree/bindings/gpu/mtk,apu-drm.yaml | 38 +++++++++++++++++++
> 1 file changed, 38 insertions(+)
> create mode 100644 Documentation/devicetree/bindings/gpu/mtk,apu-drm.yaml
>
> diff --git a/Documentation/devicetree/bindings/gpu/mtk,apu-drm.yaml b/Documentation/devicetree/bindings/gpu/mtk,apu-drm.yaml
> new file mode 100644
> index 0000000000000..6f432d3ea478c
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/gpu/mtk,apu-drm.yaml
> @@ -0,0 +1,38 @@
> +# SPDX-License-Identifier: GPL-2.0-only OR BSD-2-Clause
> +%YAML 1.2
> +---
> +$id: http://devicetree.org/schemas/gpu/mediatek,apu-drm.yaml#
> +$schema: http://devicetree.org/meta-schemas/core.yaml#
> +
> +title: AI Processor Unit DRM
DRM is a linux thing, not h/w.
> +
> +properties:
> + compatible:
> + const: mediatek,apu-drm
> +
> + remoteproc:
So is remoteproc.
Why don't you have the remoteproc driver create the DRM device?
> + maxItems: 2
> + description:
> + Handle to remoteproc devices controlling the APU
> +
> + iova:
> + maxItems: 1
> + description:
> + Address and size of virtual memory that could used by the APU
Why does this need to be in DT? If you need to reserve certain VAs, then
this discussion[1] might be of interest.
Rob
[1] https://lore.kernel.org/all/YUIPCxnyRutMS47%2F@orome.fritz.box/
On Fri, 17 Sep 2021 14:59:42 +0200, Alexandre Bailon wrote:
> This adds the device tree bindings for the APU DRM driver.
>
> Signed-off-by: Alexandre Bailon <abailon(a)baylibre.com>
> ---
> .../devicetree/bindings/gpu/mtk,apu-drm.yaml | 38 +++++++++++++++++++
> 1 file changed, 38 insertions(+)
> create mode 100644 Documentation/devicetree/bindings/gpu/mtk,apu-drm.yaml
>
My bot found errors running 'make DT_CHECKER_FLAGS=-m dt_binding_check'
on your patch (DT_CHECKER_FLAGS is new in v5.13):
yamllint warnings/errors:
dtschema/dtc warnings/errors:
/builds/robherring/linux-dt-review/Documentation/devicetree/bindings/gpu/mtk,apu-drm.yaml: 'maintainers' is a required property
hint: Metaschema for devicetree binding documentation
from schema $id: http://devicetree.org/meta-schemas/base.yaml#
./Documentation/devicetree/bindings/gpu/mtk,apu-drm.yaml: $id: relative path/filename doesn't match actual path or filename
expected: http://devicetree.org/schemas/gpu/mtk,apu-drm.yaml#
/builds/robherring/linux-dt-review/Documentation/devicetree/bindings/gpu/mtk,apu-drm.yaml: ignoring, error in schema:
warning: no schema found in file: ./Documentation/devicetree/bindings/gpu/mtk,apu-drm.yaml
Documentation/devicetree/bindings/gpu/mtk,apu-drm.example.dts:19.15-23.11: Warning (unit_address_vs_reg): /example-0/apu@0: node has a unit name, but no reg or ranges property
Documentation/devicetree/bindings/gpu/mtk,apu-drm.example.dt.yaml:0:0: /example-0/apu@0: failed to match any schema with compatible: ['mediatek,apu-drm']
doc reference errors (make refcheckdocs):
See https://patchwork.ozlabs.org/patch/1529388
This check can fail if there are any dependencies. The base for a patch
series is generally the most recent rc1.
If you already ran 'make dt_binding_check' and didn't see the above
error(s), then make sure 'yamllint' is installed and dt-schema is up to
date:
pip3 install dtschema --upgrade
Please check and re-submit.
On Wed, Sep 15, 2021 at 10:45:36AM +0300, Oded Gabbay wrote:
> On Tue, Sep 14, 2021 at 7:12 PM Jason Gunthorpe <jgg(a)ziepe.ca> wrote:
> >
> > On Tue, Sep 14, 2021 at 04:18:31PM +0200, Daniel Vetter wrote:
> > > On Sun, Sep 12, 2021 at 07:53:07PM +0300, Oded Gabbay wrote:
> > > > Hi,
> > > > Re-sending this patch-set following the release of our user-space TPC
> > > > compiler and runtime library.
> > > >
> > > > I would appreciate a review on this.
> > >
> > > I think the big open we have is the entire revoke discussions. Having the
> > > option to let dma-buf hang around which map to random local memory ranges,
> > > without clear ownership link and a way to kill it sounds bad to me.
> > >
> > > I think there's a few options:
> > > - We require revoke support. But I've heard rdma really doesn't like that,
> > > I guess because taking out an MR while holding the dma_resv_lock would
> > > be an inversion, so can't be done. Jason, can you recap what exactly the
> > > hold-up was again that makes this a no-go?
> >
> > RDMA HW can't do revoke.
Like why? I'm assuming when the final open handle or whatever for that MR
is closed, you do clean up everything? Or does that MR still stick around
forever too?
> > So we have to exclude almost all the HW and several interesting use
> > cases to enable a revoke operation.
> >
> > > - For non-revokable things like these dma-buf we'd keep a drm_master
> > > reference around. This would prevent the next open to acquire
> > > ownership rights, which at least prevents all the nasty potential
> > > problems.
> >
> > This is what I generally would expect, the DMABUF FD and its DMA
> > memory just floats about until the unrevokable user releases it, which
> > happens when the FD that is driving the import eventually gets closed.
> This is exactly what we are doing in the driver. We make sure
> everything is valid until the unrevokable user releases it and that
> happens only when the dmabuf fd gets closed.
> And the user can't close it's fd of the device until he performs the
> above, so there is no leakage between users.
Maybe I got the device security model all wrong, but I thought Guadi is
single user, and the only thing it protects is the system against the
Gaudi device trhough iommu/device gart. So roughly the following can
happen:
1. User A opens gaudi device, sets up dma-buf export
2. User A registers that with RDMA, or anything else that doesn't support
revoke.
3. User A closes gaudi device
4. User B opens gaudi device, assumes that it has full control over the
device and uploads some secrets, which happen to end up in the dma-buf
region user A set up
5. User B extracts secrets.
> > I still don't think any of the complexity is needed, pinnable memory
> > is a thing in Linux, just account for it in mlocked and that is
> > enough.
It's not mlocked memory, it's mlocked memory and I can exfiltrate it.
Mlock is fine, exfiltration not so much. It's mlock, but a global pool and
if you didn't munlock then the next mlock from a completely different user
will alias with your stuff.
Or is there something that prevents that? Oded at least explain that gaudi
works like a gpu from 20 years ago, single user, no security at all within
the device.
-Daniel
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch