LKML Archive on
help / color / mirror / Atom feed
From: Joonsoo Kim <>
To: Mark Rutland <>
	Andrew Morton <>,
	Catalin Marinas <>,
	Christoph Lameter <>,
	David Rientjes <>,
	Jesper Dangaard Brouer <>,
	Linus Torvalds <>,
	Pekka Enberg <>,
	Steve Capper <>
Subject: Re: [PATCH] mm/slub: fix lockups on PREEMPT && !SMP kernels
Date: Tue, 17 Mar 2015 10:09:12 +0900	[thread overview]
Message-ID: <20150317010912.GA19483@js1304-P5Q-DELUXE> (raw)
In-Reply-To: <>


On Fri, Mar 13, 2015 at 03:47:12PM +0000, Mark Rutland wrote:
> Commit 9aabf810a67cd97e ("mm/slub: optimize alloc/free fastpath by
> removing preemption on/off") introduced an occasional hang for kernels
> The problem is the following loop the patch introduced to
> slab_alloc_node and slab_free:
> do {
>         tid = this_cpu_read(s->cpu_slab->tid);
>         c = raw_cpu_ptr(s->cpu_slab);
> } while (IS_ENABLED(CONFIG_PREEMPT) && unlikely(tid != c->tid));
> GCC 4.9 has been observed to hoist the load of c and c->tid above the
> loop for !SMP kernels (as in this case raw_cpu_ptr(x) is compile-time
> constant and does not force a reload). On arm64 the generated assembly
> looks like:
> ffffffc00016d3c4:       f9400404        ldr     x4, [x0,#8]
> ffffffc00016d3c8:       f9400401        ldr     x1, [x0,#8]
> ffffffc00016d3cc:       eb04003f        cmp     x1, x4
> ffffffc00016d3d0:       54ffffc1    ffffffc00016d3c8 <slab_alloc_node.constprop.82+0x30>
> If the thread is preempted between the load of c->tid (into x1) and tid
> (into x4), and and allocation or free occurs in another thread (bumping
> the cpu_slab's tid), the thread will be stuck in the loop until
> s->cpu_slab->tid wraps, which may be forever in the absence of
> allocations on the same CPU.

Is there any method to guarantee refetching these in each loop?

> The loop itself is somewhat pointless as the thread can be preempted at
> any point after the loop before the this_cpu_cmpxchg_double, and the
> window for preemption within the loop is far smaller. Given that we
> assume accessing c->tid is safe for the loop condition, and we retry
> when the cmpxchg fails, we can get rid of the loop entirely and just
> access c->tid via the raw_cpu_ptr for s->cpu_slab.

Hmm... IIUC, loop itself is not pointless. It guarantees that tid and
c (s->cpu_slab) are fetched on right and same processor and this is
for algorithm correctness.

Think about your code.

c = raw_cpu_ptr(s->cpu_slab);
tid = READ_ONCE(c->tid);

This doesn't guarantee that tid is fetched on the cpu where c is
fetched if preemption/migration happens between these operations.

If c->tid, c->freelist, c->page are fetched on the other cpu,
there is no ordering guarantee and c->freelist, c->page could be stale
value even if c->tid is recent one.

Think about following free case with your patch.

Assume initial cpu 0's state as following.
c->tid: 1, c->freelist: NULL, c->page: A

User X: try to free object X for page A
User X: fetch c (s->cpu_slab)

Preemtion and migration happens...
The other allocation/free happens... so cpu 0's state is as following.
c->tid: 3, c->freelist: NULL, c->page: B

User X: read c->tid: 3, c->freelist: NULL, c->page A (stale value)

Because tid and freelist are matched with current ones, free would
succeed, but, current c->page is B and object is for A so this success
is wrong.

Loop prevents this possibility.


  parent reply	other threads:[~2015-03-17  1:09 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-03-13 15:47 Mark Rutland
2015-03-13 16:29 ` Christoph Lameter
2015-03-13 18:16   ` Mark Rutland
2015-03-13 18:27     ` Christoph Lameter
2015-03-16 12:45   ` [PATCHv2] " Mark Rutland
2015-03-17  1:09 ` Joonsoo Kim [this message]
2015-03-17 12:00   ` [PATCH] " Mark Rutland
2015-03-17 12:15     ` [PATCHv2] " Mark Rutland
2015-03-17 18:57       ` Christoph Lameter
2015-03-18  5:59       ` Joonsoo Kim
2015-03-18 15:21         ` Mark Rutland
2015-03-19 12:13           ` Joonsoo Kim
2015-03-19 16:16             ` Christoph Lameter
2015-03-24 14:17       ` Ville Syrjälä

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20150317010912.GA19483@js1304-P5Q-DELUXE \ \ \ \ \ \ \ \ \ \ \ \
    --subject='Re: [PATCH] mm/slub: fix lockups on PREEMPT && '\!'SMP kernels' \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).