LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Gregory Haskins <ghaskins@novell.com>
To: mingo@elte.hu, a.p.zijlstra@chello.nl, tglx@linutronix.de,
	rostedt@goodmis.org, linux-rt-users@vger.kernel.org
Cc: linux-kernel@vger.kernel.org, bill.huey@gmail.com,
	kevin@hilman.org, cminyard@mvista.com, dsingleton@mvista.com,
	dwalker@mvista.com, npiggin@suse.de, dsaxena@plexity.net,
	ak@suse.de, pavel@ucw.cz, acme@redhat.com, gregkh@suse.de,
	sdietrich@novell.com, pmorreale@novell.com, mkohari@novell.com,
	ghaskins@novell.com
Subject: [(RT RFC) PATCH v2 4/9] optimize rt lock wakeup
Date: Mon, 25 Feb 2008 11:00:58 -0500	[thread overview]
Message-ID: <20080225160058.11268.96036.stgit@novell1.haskins.net> (raw)
In-Reply-To: <20080225155959.11268.35541.stgit@novell1.haskins.net>

It is redundant to wake the grantee task if it is already running

Credit goes to Peter for the general idea.

Signed-off-by: Gregory Haskins <ghaskins@novell.com>
Signed-off-by: Peter Morreale <pmorreale@novell.com>
---

 kernel/rtmutex.c |   45 ++++++++++++++++++++++++++++++++++++++++-----
 1 files changed, 40 insertions(+), 5 deletions(-)

diff --git a/kernel/rtmutex.c b/kernel/rtmutex.c
index ef52db6..bf9e230 100644
--- a/kernel/rtmutex.c
+++ b/kernel/rtmutex.c
@@ -531,6 +531,41 @@ static void wakeup_next_waiter(struct rt_mutex *lock, int savestate)
 	pendowner = waiter->task;
 	waiter->task = NULL;
 
+	/*
+	 * Do the wakeup before the ownership change to give any spinning
+	 * waiter grantees a headstart over the other threads that will
+	 * trigger once owner changes.
+	 */
+	if (!savestate)
+		wake_up_process(pendowner);
+	else {
+		/*
+		 * We can skip the actual (expensive) wakeup if the
+		 * waiter is already running, but we have to be careful
+		 * of race conditions because they may be about to sleep.
+		 *
+		 * The waiter-side protocol has the following pattern:
+		 * 1: Set state != RUNNING
+		 * 2: Conditionally sleep if waiter->task != NULL;
+		 *
+		 * And the owner-side has the following:
+		 * A: Set waiter->task = NULL
+		 * B: Conditionally wake if the state != RUNNING
+		 *
+		 * As long as we ensure 1->2 order, and A->B order, we
+		 * will never miss a wakeup.
+		 *
+		 * Therefore, this barrier ensures that waiter->task = NULL
+		 * is visible before we test the pendowner->state.  The
+		 * corresponding barrier is in the sleep logic.
+		 */
+		smp_mb();
+
+		if ((pendowner->state != TASK_RUNNING)
+		    && (pendowner->state != TASK_RUNNING_MUTEX))
+			wake_up_process_mutex(pendowner);
+	}
+
 	rt_mutex_set_owner(lock, pendowner, RT_MUTEX_OWNER_PENDING);
 
 	spin_unlock(&current->pi_lock);
@@ -557,11 +592,6 @@ static void wakeup_next_waiter(struct rt_mutex *lock, int savestate)
 		plist_add(&next->pi_list_entry, &pendowner->pi_waiters);
 	}
 	spin_unlock(&pendowner->pi_lock);
-
-	if (savestate)
-		wake_up_process_mutex(pendowner);
-	else
-		wake_up_process(pendowner);
 }
 
 /*
@@ -762,6 +792,11 @@ rt_spin_lock_slowlock(struct rt_mutex *lock)
 		debug_rt_mutex_print_deadlock(&waiter);
 
 		update_current(TASK_UNINTERRUPTIBLE, &saved_state);
+		/*
+		 * The xchg() in update_current() is an implicit barrier
+		 * which we rely upon to ensure current->state is visible
+		 * before we test waiter.task.
+		 */
 		if (waiter.task)
 			schedule_rt_mutex(lock);
 		else


  parent reply	other threads:[~2008-02-25 16:30 UTC|newest]

Thread overview: 36+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2008-02-25 16:00 [(RT RFC) PATCH v2 0/9] adaptive real-time locks Gregory Haskins
2008-02-25 16:00 ` [(RT RFC) PATCH v2 1/9] allow rt-mutex lock-stealing to include lateral priority Gregory Haskins
2008-03-03 15:13   ` Steven Rostedt
2008-03-03 15:41     ` [(RT RFC) PATCH v2 1/9] allow rt-mutex lock-stealing to includelateral priority Gregory Haskins
2008-03-03 15:55       ` Steven Rostedt
2008-03-03 15:55         ` [(RT RFC) PATCH v2 1/9] allow rt-mutex lock-stealing toincludelateral priority Gregory Haskins
2008-02-25 16:00 ` [(RT RFC) PATCH v2 2/9] sysctl for runtime-control of lateral mutex stealing Gregory Haskins
2008-02-25 21:53   ` Pavel Machek
2008-02-25 22:57     ` Sven-Thorsten Dietrich
2008-02-25 23:00       ` Pavel Machek
2008-02-25 23:40         ` Sven-Thorsten Dietrich
2008-02-26  1:15       ` Gregory Haskins
2008-02-25 16:00 ` [(RT RFC) PATCH v2 3/9] rearrange rt_spin_lock sleep Gregory Haskins
2008-02-25 21:54   ` Pavel Machek
2008-02-26  0:45     ` Gregory Haskins
2008-02-25 16:00 ` Gregory Haskins [this message]
2008-03-03 15:37   ` [(RT RFC) PATCH v2 4/9] optimize rt lock wakeup Steven Rostedt
2008-03-03 15:41     ` Gregory Haskins
2008-02-25 16:01 ` [(RT RFC) PATCH v2 5/9] adaptive real-time lock support Gregory Haskins
2008-02-25 22:03   ` Pavel Machek
2008-02-26  0:48     ` Gregory Haskins
2008-02-26 15:03     ` Gregory Haskins
2008-02-26 18:06       ` Pavel Machek
2008-02-26 18:01         ` Gregory Haskins
2008-02-25 16:01 ` [(RT RFC) PATCH v2 6/9] add a loop counter based timeout mechanism Gregory Haskins
2008-02-25 22:06   ` Pavel Machek
2008-02-25 22:19     ` Greg KH
2008-02-25 22:21       ` Pavel Machek
2008-02-25 22:39     ` Sven-Thorsten Dietrich
2008-02-26 15:09     ` Gregory Haskins
2008-02-25 16:01 ` [(RT RFC) PATCH v2 7/9] adaptive mutexes Gregory Haskins
2008-02-25 22:09   ` Pavel Machek
2008-02-26  0:52     ` Gregory Haskins
2008-02-25 16:01 ` [(RT RFC) PATCH v2 8/9] adjust pi_lock usage in wakeup Gregory Haskins
2008-02-25 22:10   ` Pavel Machek
2008-02-25 16:01 ` [(RT RFC) PATCH v2 9/9] remove the extra call to try_to_take_lock Gregory Haskins

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20080225160058.11268.96036.stgit@novell1.haskins.net \
    --to=ghaskins@novell.com \
    --cc=a.p.zijlstra@chello.nl \
    --cc=acme@redhat.com \
    --cc=ak@suse.de \
    --cc=bill.huey@gmail.com \
    --cc=cminyard@mvista.com \
    --cc=dsaxena@plexity.net \
    --cc=dsingleton@mvista.com \
    --cc=dwalker@mvista.com \
    --cc=gregkh@suse.de \
    --cc=kevin@hilman.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-rt-users@vger.kernel.org \
    --cc=mingo@elte.hu \
    --cc=mkohari@novell.com \
    --cc=npiggin@suse.de \
    --cc=pavel@ucw.cz \
    --cc=pmorreale@novell.com \
    --cc=rostedt@goodmis.org \
    --cc=sdietrich@novell.com \
    --cc=tglx@linutronix.de \
    --subject='Re: [(RT RFC) PATCH v2 4/9] optimize rt lock wakeup' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).