LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Davidlohr Bueso <dave@stgolabs.net>
To: Peter Zijlstra <peterz@infradead.org>, Ingo Molnar <mingo@kernel.org>
Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>,
	Jason Low <jason.low2@hp.com>,
	Michel Lespinasse <walken@google.com>,
	Tim Chen <tim.c.chen@linux.intel.com>,
	linux-kernel@vger.kernel.org, Davidlohr Bueso <dave@stgolabs.net>,
	Davidlohr Bueso <dbueso@suse.de>
Subject: [PATCH 4/6] locking/rwsem: Avoid deceiving lock spinners
Date: Sun, 25 Jan 2015 23:36:07 -0800	[thread overview]
Message-ID: <1422257769-14083-5-git-send-email-dave@stgolabs.net> (raw)
In-Reply-To: <1422257769-14083-1-git-send-email-dave@stgolabs.net>

When readers hold the semaphore, the ->owner is nil. As such,
and unlike mutexes, '!owner' does not necessarily imply that
the lock is free. This will cause writer spinners to potentially
spin excessively as they've been mislead to thinking they have
a chance of acquiring the lock, instead of blocking.

This patch therefore replaces this bogus check to solely rely on
the counter to know if the lock is available. Because we don't
hold the wait lock, we can obviously do this in an unqueued
manner.

Signed-off-by: Davidlohr Bueso <dbueso@suse.de>
---
 kernel/locking/rwsem-xadd.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/kernel/locking/rwsem-xadd.c b/kernel/locking/rwsem-xadd.c
index 5e425d8..18a50da 100644
--- a/kernel/locking/rwsem-xadd.c
+++ b/kernel/locking/rwsem-xadd.c
@@ -335,6 +335,8 @@ static inline bool owner_running(struct rw_semaphore *sem,
 static noinline
 bool rwsem_spin_on_owner(struct rw_semaphore *sem, struct task_struct *owner)
 {
+	long count;
+
 	rcu_read_lock();
 	while (owner_running(sem, owner)) {
 		if (need_resched())
@@ -347,9 +349,11 @@ bool rwsem_spin_on_owner(struct rw_semaphore *sem, struct task_struct *owner)
 	/*
 	 * We break out the loop above on need_resched() or when the
 	 * owner changed, which is a sign for heavy contention. Return
-	 * success only when sem->owner is NULL.
+	 * success only when the lock is available in order to attempt
+	 * another trylock.
 	 */
-	return sem->owner == NULL;
+	count = READ_ONCE(sem->count);
+	return count == 0 || count == RWSEM_WAITING_BIAS;
 }
 
 static bool rwsem_optimistic_spin(struct rw_semaphore *sem)
-- 
2.1.2


  parent reply	other threads:[~2015-01-26  7:37 UTC|newest]

Thread overview: 26+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-01-26  7:36 [PATCH -tip 0/6] rwsem: Fine tuning Davidlohr Bueso
2015-01-26  7:36 ` [PATCH 1/6] locking/rwsem: Use task->state helpers Davidlohr Bueso
2015-02-04 14:38   ` [tip:locking/core] " tip-bot for Davidlohr Bueso
2015-01-26  7:36 ` [PATCH 2/6] locking/rwsem: Document barrier need when waking tasks Davidlohr Bueso
2015-01-27 17:07   ` Peter Zijlstra
2015-01-27 20:30     ` Davidlohr Bueso
2015-01-26  7:36 ` [PATCH 3/6] locking/rwsem: Set lock ownership ASAP Davidlohr Bueso
2015-01-27 17:10   ` Peter Zijlstra
2015-01-27 19:18     ` Davidlohr Bueso
2015-01-26  7:36 ` Davidlohr Bueso [this message]
2015-01-27 17:23   ` [PATCH 4/6] locking/rwsem: Avoid deceiving lock spinners Jason Low
2015-01-28  3:54     ` Davidlohr Bueso
2015-01-28 17:01       ` Tim Chen
2015-01-28 21:03       ` Jason Low
2015-01-29  1:10         ` Davidlohr Bueso
2015-01-29 20:13           ` Jason Low
2015-01-29 20:18             ` Jason Low
2015-01-29 23:15               ` Davidlohr Bueso
2015-01-30  1:52                 ` Refactoring mutex spin on owner code Jason Low
2015-01-30  7:14                   ` Davidlohr Bueso
2015-01-30  7:51                     ` Peter Zijlstra
2015-01-26  7:36 ` [PATCH 5/6] locking/rwsem: Optimize slowpath/sleeping Davidlohr Bueso
2015-01-27 17:34   ` Peter Zijlstra
2015-01-27 21:57     ` Davidlohr Bueso
2015-01-26  7:36 ` [PATCH 6/6] locking/rwsem: Check for active lock before bailing on spinning Davidlohr Bueso
2015-01-27 18:11   ` Jason Low

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1422257769-14083-5-git-send-email-dave@stgolabs.net \
    --to=dave@stgolabs.net \
    --cc=dbueso@suse.de \
    --cc=jason.low2@hp.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mingo@kernel.org \
    --cc=paulmck@linux.vnet.ibm.com \
    --cc=peterz@infradead.org \
    --cc=tim.c.chen@linux.intel.com \
    --cc=walken@google.com \
    --subject='Re: [PATCH 4/6] locking/rwsem: Avoid deceiving lock spinners' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).