LKML Archive on
help / color / mirror / Atom feed
From: Gregory Haskins <>
Subject: [(RT RFC) PATCH v2 0/9] adaptive real-time locks
Date: Mon, 25 Feb 2008 11:00:38 -0500	[thread overview]
Message-ID: <> (raw)

You can download this series here:

Changes since v1:

*) Rebased from 24-rt1 to 24.2-rt2
*) Dropped controversial (and likely unecessary) printk patch
*) Dropped (internally) controversial PREEMPT_SPINLOCK_WAITERS config options
*) Incorporated review feedback for comment/config cleanup from Pavel/PeterZ
*) Moved lateral-steal to front of queue
*) Fixed compilation issue with !defined(LATERAL_STEAL)
*) Moved spinlock rework into a separate series:

*) Convert loop based timeouts to use nanoseconds
*) Tie into lockstat infrastructure.
*) Long-term: research adaptive-timeout algorithms so a fixed/one-size-
   -fits-all value is not necessary.


Adaptive real-time locks

The Real Time patches to the Linux kernel converts the architecture
specific SMP-synchronization primitives commonly referred to as
"spinlocks" to an "RT mutex" implementation that support a priority
inheritance protocol, and priority-ordered wait queues.  The RT mutex
implementation allows tasks that would otherwise busy-wait for a
contended lock to be preempted by higher priority tasks without
compromising the integrity of critical sections protected by the lock.
The unintended side-effect is that the -rt kernel suffers from
significant degradation of IO throughput (disk and net) due to the
extra overhead associated with managing pi-lists and context switching.
This has been generally accepted as a price to pay for low-latency

Our research indicates that it doesn't necessarily have to be this
way.  This patch set introduces an adaptive technology that retains both
the priority inheritance protocol as well as the preemptive nature of
spinlocks and mutexes and adds a 300+% throughput increase to the Linux
Real time kernel.  It applies to 2.6.24-rt1.  

These performance increases apply to disk IO as well as netperf UDP
benchmarks, without compromising RT preemption latency.  For more
complex applications, overall the I/O throughput seems to approach the
throughput on a PREEMPT_VOLUNTARY or PREEMPT_DESKTOP Kernel, as is
shipped by most distros.

Essentially, the RT Mutex has been modified to busy-wait under
contention for a limited (and configurable) time.  This works because
most locks are typically held for very short time spans.  Too often,
by the time a task goes to sleep on a mutex, the mutex is already being
released on another CPU.

The effect (on SMP) is that by polling a mutex for a limited time we
reduce context switch overhead by up to 90%, and therefore eliminate CPU
cycles as well as massive hot-spots in the scheduler / other bottlenecks
in the Kernel - even though we busy-wait (using CPU cycles) to poll the

We have put together some data from different types of benchmarks for
this patch series, which you can find here:

It compares a stock 2.6.24 (PREEMPT_DESKTOP), a stock
2.6.24-rt1 (PREEMPT_RT), and a 2.6.24-rt1 + adaptive-lock
(2.6.24-rt1-al) (PREEMPT_RT) kernel.  The machine is a 4-way (dual-core,
dual-socket) 2Ghz 5130 Xeon (core2duo-woodcrest) Dell Precision 490. 

Some tests show a marked improvement (for instance, ~450% more throughput
for dbench, and ~500% faster for hackbench), whereas some others
(make -j 128) the results were not as profound but they were still
net-positive. In all cases we have also verified that deterministic
latency is not impacted by using cyclic-test. 

This patch series depends on some re-work on the raw_spinlock
infrastructure, including Nick Piggin's x86-ticket-locks.  We found that
the increased pressure on the lock->wait_locks could cause rare but
serious latency spikes that are fixed by a fifo raw_spinlock_t
implementation.  Nick was gracious enough to allow us to re-use his
work (which is already accepted in 2.6.25).  Note that we also have a
C version of his protocol available if other architectures need
fifo-lock support as well, which we will gladly post upon request.

You can find this re-work as a separate series here:

Special thanks go to many people who were instrumental to this project,
  *) the -rt team here at Novell for research, development, and testing.
  *) Nick Piggin for his invaluable consultation/feedback and use of his
  *) The reviewers/testers at Suse, Montavista, and Bill Huey for their
     time and feedback on the early versions of these patches.

As always, comments/feedback/bug-fixes are welcome.


             reply	other threads:[~2008-02-25 16:29 UTC|newest]

Thread overview: 36+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2008-02-25 16:00 Gregory Haskins [this message]
2008-02-25 16:00 ` [(RT RFC) PATCH v2 1/9] allow rt-mutex lock-stealing to include lateral priority Gregory Haskins
2008-03-03 15:13   ` Steven Rostedt
2008-03-03 15:41     ` [(RT RFC) PATCH v2 1/9] allow rt-mutex lock-stealing to includelateral priority Gregory Haskins
2008-03-03 15:55       ` Steven Rostedt
2008-03-03 15:55         ` [(RT RFC) PATCH v2 1/9] allow rt-mutex lock-stealing toincludelateral priority Gregory Haskins
2008-02-25 16:00 ` [(RT RFC) PATCH v2 2/9] sysctl for runtime-control of lateral mutex stealing Gregory Haskins
2008-02-25 21:53   ` Pavel Machek
2008-02-25 22:57     ` Sven-Thorsten Dietrich
2008-02-25 23:00       ` Pavel Machek
2008-02-25 23:40         ` Sven-Thorsten Dietrich
2008-02-26  1:15       ` Gregory Haskins
2008-02-25 16:00 ` [(RT RFC) PATCH v2 3/9] rearrange rt_spin_lock sleep Gregory Haskins
2008-02-25 21:54   ` Pavel Machek
2008-02-26  0:45     ` Gregory Haskins
2008-02-25 16:00 ` [(RT RFC) PATCH v2 4/9] optimize rt lock wakeup Gregory Haskins
2008-03-03 15:37   ` Steven Rostedt
2008-03-03 15:41     ` Gregory Haskins
2008-02-25 16:01 ` [(RT RFC) PATCH v2 5/9] adaptive real-time lock support Gregory Haskins
2008-02-25 22:03   ` Pavel Machek
2008-02-26  0:48     ` Gregory Haskins
2008-02-26 15:03     ` Gregory Haskins
2008-02-26 18:06       ` Pavel Machek
2008-02-26 18:01         ` Gregory Haskins
2008-02-25 16:01 ` [(RT RFC) PATCH v2 6/9] add a loop counter based timeout mechanism Gregory Haskins
2008-02-25 22:06   ` Pavel Machek
2008-02-25 22:19     ` Greg KH
2008-02-25 22:21       ` Pavel Machek
2008-02-25 22:39     ` Sven-Thorsten Dietrich
2008-02-26 15:09     ` Gregory Haskins
2008-02-25 16:01 ` [(RT RFC) PATCH v2 7/9] adaptive mutexes Gregory Haskins
2008-02-25 22:09   ` Pavel Machek
2008-02-26  0:52     ` Gregory Haskins
2008-02-25 16:01 ` [(RT RFC) PATCH v2 8/9] adjust pi_lock usage in wakeup Gregory Haskins
2008-02-25 22:10   ` Pavel Machek
2008-02-25 16:01 ` [(RT RFC) PATCH v2 9/9] remove the extra call to try_to_take_lock Gregory Haskins

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \
    --subject='Re: [(RT RFC) PATCH v2 0/9] adaptive real-time locks' \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).