LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Daniel Vetter <daniel@ffwll.ch>
To: "Christian König" <ckoenig.leichtzumerken@gmail.com>
Cc: "Rob Clark" <robdclark@gmail.com>,
	dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org,
	"Michel Dänzer" <michel@daenzer.net>,
	"Pekka Paalanen" <ppaalanen@gmail.com>,
	"Rob Clark" <robdclark@chromium.org>,
	"David Airlie" <airlied@linux.ie>,
	"Sumit Semwal" <sumit.semwal@linaro.org>,
	"Christian König" <christian.koenig@amd.com>,
	"Tian Tao" <tiantao6@hisilicon.com>,
	"Steven Price" <steven.price@arm.com>,
	"Melissa Wen" <mwen@igalia.com>,
	"Luben Tuikov" <luben.tuikov@amd.com>,
	"Andrey Grodzovsky" <andrey.grodzovsky@amd.com>,
	"Boris Brezillon" <boris.brezillon@collabora.com>,
	"Jack Zhang" <Jack.Zhang1@amd.com>,
	"open list" <linux-kernel@vger.kernel.org>,
	"open list:DMA BUFFER SHARING FRAMEWORK"
	<linux-media@vger.kernel.org>
Subject: Re: [PATCH v3 4/9] drm/scheduler: Add fence deadline support
Date: Tue, 14 Sep 2021 15:38:53 +0200	[thread overview]
Message-ID: <YUCl7cHXpgGa6DhP@phenom.ffwll.local> (raw)
In-Reply-To: <0bb5f8e0-9c42-c97f-16bd-8dbe0f878157@gmail.com>

On Thu, Sep 09, 2021 at 08:22:59AM +0200, Christian König wrote:
> Am 08.09.21 um 19:45 schrieb Daniel Vetter:
> > On Fri, Sep 03, 2021 at 11:47:55AM -0700, Rob Clark wrote:
> > > From: Rob Clark <robdclark@chromium.org>
> > > 
> > > As the finished fence is the one that is exposed to userspace, and
> > > therefore the one that other operations, like atomic update, would
> > > block on, we need to propagate the deadline from from the finished
> > > fence to the actual hw fence.
> > > 
> > > v2: Split into drm_sched_fence_set_parent() (ckoenig)
> > > 
> > > Signed-off-by: Rob Clark <robdclark@chromium.org>
> > > ---
> > >   drivers/gpu/drm/scheduler/sched_fence.c | 34 +++++++++++++++++++++++++
> > >   drivers/gpu/drm/scheduler/sched_main.c  |  2 +-
> > >   include/drm/gpu_scheduler.h             |  8 ++++++
> > >   3 files changed, 43 insertions(+), 1 deletion(-)
> > > 
> > > diff --git a/drivers/gpu/drm/scheduler/sched_fence.c b/drivers/gpu/drm/scheduler/sched_fence.c
> > > index bcea035cf4c6..4fc41a71d1c7 100644
> > > --- a/drivers/gpu/drm/scheduler/sched_fence.c
> > > +++ b/drivers/gpu/drm/scheduler/sched_fence.c
> > > @@ -128,6 +128,30 @@ static void drm_sched_fence_release_finished(struct dma_fence *f)
> > >   	dma_fence_put(&fence->scheduled);
> > >   }
> > > +static void drm_sched_fence_set_deadline_finished(struct dma_fence *f,
> > > +						  ktime_t deadline)
> > > +{
> > > +	struct drm_sched_fence *fence = to_drm_sched_fence(f);
> > > +	unsigned long flags;
> > > +
> > > +	spin_lock_irqsave(&fence->lock, flags);
> > > +
> > > +	/* If we already have an earlier deadline, keep it: */
> > > +	if (test_bit(DMA_FENCE_FLAG_HAS_DEADLINE_BIT, &f->flags) &&
> > > +	    ktime_before(fence->deadline, deadline)) {
> > > +		spin_unlock_irqrestore(&fence->lock, flags);
> > > +		return;
> > > +	}
> > > +
> > > +	fence->deadline = deadline;
> > > +	set_bit(DMA_FENCE_FLAG_HAS_DEADLINE_BIT, &f->flags);
> > > +
> > > +	spin_unlock_irqrestore(&fence->lock, flags);
> > > +
> > > +	if (fence->parent)
> > > +		dma_fence_set_deadline(fence->parent, deadline);
> > > +}
> > > +
> > >   static const struct dma_fence_ops drm_sched_fence_ops_scheduled = {
> > >   	.get_driver_name = drm_sched_fence_get_driver_name,
> > >   	.get_timeline_name = drm_sched_fence_get_timeline_name,
> > > @@ -138,6 +162,7 @@ static const struct dma_fence_ops drm_sched_fence_ops_finished = {
> > >   	.get_driver_name = drm_sched_fence_get_driver_name,
> > >   	.get_timeline_name = drm_sched_fence_get_timeline_name,
> > >   	.release = drm_sched_fence_release_finished,
> > > +	.set_deadline = drm_sched_fence_set_deadline_finished,
> > >   };
> > >   struct drm_sched_fence *to_drm_sched_fence(struct dma_fence *f)
> > > @@ -152,6 +177,15 @@ struct drm_sched_fence *to_drm_sched_fence(struct dma_fence *f)
> > >   }
> > >   EXPORT_SYMBOL(to_drm_sched_fence);
> > > +void drm_sched_fence_set_parent(struct drm_sched_fence *s_fence,
> > > +				struct dma_fence *fence)
> > > +{
> > > +	s_fence->parent = dma_fence_get(fence);
> > > +	if (test_bit(DMA_FENCE_FLAG_HAS_DEADLINE_BIT,
> > > +		     &s_fence->finished.flags))
> > Don't you need the spinlock here too to avoid races? test_bit is very
> > unordered, so guarantees nothing. Spinlock would need to be both around
> > ->parent = and the test_bit.
> > 
> > Entirely aside, but there's discussions going on to preallocate the hw
> > fence somehow. If we do that we could make the deadline forwarding
> > lockless here. Having a spinlock just to set the parent is a bit annoying
> 
> Well previously we didn't needed the spinlock to set the parent because the
> parent wasn't used outside of the main thread.
> 
> This becomes an issue now because we can race with setting the deadline. So
> yeah we probably do need the spinlock here now.

Yeah tbh this is the part I don't like, because it means the scheduler
thread locking becomes more complex.

We might need to look at this again when we include stuff like boosting
priorities and things like that. Maybe it would be better if we have a
request queue which the scheduler thread could then pick up whenever it
gets around to the next scheduling decision?

I think for now just the spinlock in more places should be fine.
-Daniel


> Christian.
> 
> > ...
> > 
> > Alternative is that you do this locklessly with barriers and a _lot_ of
> > comments. Would be good to benchmark whether the overhead matters though
> > first.
> > -Daniel
> > 
> > > +		dma_fence_set_deadline(fence, s_fence->deadline);
> > > +}
> > > +
> > >   struct drm_sched_fence *drm_sched_fence_alloc(struct drm_sched_entity *entity,
> > >   					      void *owner)
> > >   {
> > > diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c
> > > index 595e47ff7d06..27bf0ac0625f 100644
> > > --- a/drivers/gpu/drm/scheduler/sched_main.c
> > > +++ b/drivers/gpu/drm/scheduler/sched_main.c
> > > @@ -978,7 +978,7 @@ static int drm_sched_main(void *param)
> > >   		drm_sched_fence_scheduled(s_fence);
> > >   		if (!IS_ERR_OR_NULL(fence)) {
> > > -			s_fence->parent = dma_fence_get(fence);
> > > +			drm_sched_fence_set_parent(s_fence, fence);
> > >   			r = dma_fence_add_callback(fence, &sched_job->cb,
> > >   						   drm_sched_job_done_cb);
> > >   			if (r == -ENOENT)
> > > diff --git a/include/drm/gpu_scheduler.h b/include/drm/gpu_scheduler.h
> > > index 7f77a455722c..158ddd662469 100644
> > > --- a/include/drm/gpu_scheduler.h
> > > +++ b/include/drm/gpu_scheduler.h
> > > @@ -238,6 +238,12 @@ struct drm_sched_fence {
> > >            */
> > >   	struct dma_fence		finished;
> > > +	/**
> > > +	 * @deadline: deadline set on &drm_sched_fence.finished which
> > > +	 * potentially needs to be propagated to &drm_sched_fence.parent
> > > +	 */
> > > +	ktime_t				deadline;
> > > +
> > >           /**
> > >            * @parent: the fence returned by &drm_sched_backend_ops.run_job
> > >            * when scheduling the job on hardware. We signal the
> > > @@ -505,6 +511,8 @@ void drm_sched_entity_set_priority(struct drm_sched_entity *entity,
> > >   				   enum drm_sched_priority priority);
> > >   bool drm_sched_entity_is_ready(struct drm_sched_entity *entity);
> > > +void drm_sched_fence_set_parent(struct drm_sched_fence *s_fence,
> > > +				struct dma_fence *fence);
> > >   struct drm_sched_fence *drm_sched_fence_alloc(
> > >   	struct drm_sched_entity *s_entity, void *owner);
> > >   void drm_sched_fence_init(struct drm_sched_fence *fence,
> > > -- 
> > > 2.31.1
> > > 
> 

-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

  reply	other threads:[~2021-09-14 13:39 UTC|newest]

Thread overview: 45+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-09-03 18:47 [PATCH v3 0/9] dma-fence: Deadline awareness Rob Clark
2021-09-03 18:47 ` [PATCH v3 1/9] dma-fence: Add deadline awareness Rob Clark
2021-09-08 17:55   ` Daniel Vetter
2021-09-03 18:47 ` [PATCH v3 2/9] drm/vblank: Add helper to get next vblank time Rob Clark
2021-09-03 18:47 ` [PATCH v3 3/9] drm/atomic-helper: Set fence deadline for vblank Rob Clark
2021-09-03 18:47 ` [PATCH v3 4/9] drm/scheduler: Add fence deadline support Rob Clark
2021-09-08 17:45   ` Daniel Vetter
2021-09-09  6:22     ` Christian König
2021-09-14 13:38       ` Daniel Vetter [this message]
2021-09-21 15:57     ` Rob Clark
2021-09-21 16:35       ` Rob Clark
2021-09-21 16:45         ` Christian König
2021-09-21 20:09   ` Andrey Grodzovsky
2021-09-21 20:47     ` Rob Clark
2021-09-22  2:18       ` Andrey Grodzovsky
2021-09-22  3:32         ` Rob Clark
2021-09-22 14:31           ` Andrey Grodzovsky
2021-09-22 15:01             ` Rob Clark
2021-09-03 18:47 ` [PATCH v3 5/9] drm/msm: Add deadline based boost support Rob Clark
2021-09-08 17:48   ` Daniel Vetter
2021-09-08 17:57     ` Rob Clark
2021-09-03 18:47 ` [PATCH v3 6/9] dma-buf/fence-array: Add fence deadline support Rob Clark
2021-09-08 18:00   ` Daniel Vetter
2021-09-09  6:55     ` Christian König
2021-09-03 18:47 ` [PATCH v3 7/9] dma-buf/fence-chain: " Rob Clark
2021-09-08 17:54   ` Daniel Vetter
2021-09-08 18:19     ` Rob Clark
2021-09-08 18:45       ` Daniel Vetter
2021-09-09  6:31         ` Christian König
2021-09-03 18:47 ` [PATCH v3 8/9] dma-buf/sync_file: Add SET_DEADLINE ioctl Rob Clark
2021-09-08 17:50   ` Daniel Vetter
2021-09-08 18:23     ` Rob Clark
2021-09-08 18:49       ` Daniel Vetter
2021-09-08 19:40         ` Rob Clark
2021-09-08 21:10           ` Daniel Vetter
2021-09-21 18:08             ` Rob Clark
2021-09-27  8:42   ` Pekka Paalanen
2021-09-27  8:53     ` Christian König
2021-09-27 14:36     ` Rob Clark
2021-09-28  7:57       ` Pekka Paalanen
2021-09-03 18:48 ` [PATCH v3 9/9] dma-buf/sw_sync: Add fence deadline support Rob Clark
2021-09-09 16:16 ` [PATCH v3 0/9] dma-fence: Deadline awareness Simon Ser
2021-09-09 16:35   ` Rob Clark
2021-09-09 16:42     ` Simon Ser
2021-09-09 17:08       ` Rob Clark

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=YUCl7cHXpgGa6DhP@phenom.ffwll.local \
    --to=daniel@ffwll.ch \
    --cc=Jack.Zhang1@amd.com \
    --cc=airlied@linux.ie \
    --cc=andrey.grodzovsky@amd.com \
    --cc=boris.brezillon@collabora.com \
    --cc=christian.koenig@amd.com \
    --cc=ckoenig.leichtzumerken@gmail.com \
    --cc=dri-devel@lists.freedesktop.org \
    --cc=linaro-mm-sig@lists.linaro.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-media@vger.kernel.org \
    --cc=luben.tuikov@amd.com \
    --cc=michel@daenzer.net \
    --cc=mwen@igalia.com \
    --cc=ppaalanen@gmail.com \
    --cc=robdclark@chromium.org \
    --cc=robdclark@gmail.com \
    --cc=steven.price@arm.com \
    --cc=sumit.semwal@linaro.org \
    --cc=tiantao6@hisilicon.com \
    --subject='Re: [PATCH v3 4/9] drm/scheduler: Add fence deadline support' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).