LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Mark Rutland <mark.rutland@arm.com>
To: Joonsoo Kim <iamjoonsoo.kim@lge.com>
Cc: "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	Andrew Morton <akpm@linux-foundation.org>,
	Catalin Marinas <Catalin.Marinas@arm.com>,
	Christoph Lameter <cl@linux.com>,
	David Rientjes <rientjes@google.com>,
	Jesper Dangaard Brouer <brouer@redhat.com>,
	Linus Torvalds <torvalds@linux-foundation.org>,
	Pekka Enberg <penberg@kernel.org>,
	Steve Capper <steve.capper@linaro.org>
Subject: Re: [PATCH] mm/slub: fix lockups on PREEMPT && !SMP kernels
Date: Tue, 17 Mar 2015 12:00:58 +0000	[thread overview]
Message-ID: <20150317120058.GC23340@leverpostej> (raw)
In-Reply-To: <20150317010912.GA19483@js1304-P5Q-DELUXE>

Hi,

> On Fri, Mar 13, 2015 at 03:47:12PM +0000, Mark Rutland wrote:
> > Commit 9aabf810a67cd97e ("mm/slub: optimize alloc/free fastpath by
> > removing preemption on/off") introduced an occasional hang for kernels
> > built with CONFIG_PREEMPT && !CONFIG_SMP.
> > 
> > The problem is the following loop the patch introduced to
> > slab_alloc_node and slab_free:
> > 
> > do {
> >         tid = this_cpu_read(s->cpu_slab->tid);
> >         c = raw_cpu_ptr(s->cpu_slab);
> > } while (IS_ENABLED(CONFIG_PREEMPT) && unlikely(tid != c->tid));
> > 
> > GCC 4.9 has been observed to hoist the load of c and c->tid above the
> > loop for !SMP kernels (as in this case raw_cpu_ptr(x) is compile-time
> > constant and does not force a reload). On arm64 the generated assembly
> > looks like:
> > 
> > ffffffc00016d3c4:       f9400404        ldr     x4, [x0,#8]
> > ffffffc00016d3c8:       f9400401        ldr     x1, [x0,#8]
> > ffffffc00016d3cc:       eb04003f        cmp     x1, x4
> > ffffffc00016d3d0:       54ffffc1        b.ne    ffffffc00016d3c8 <slab_alloc_node.constprop.82+0x30>
> > 
> > If the thread is preempted between the load of c->tid (into x1) and tid
> > (into x4), and and allocation or free occurs in another thread (bumping
> > the cpu_slab's tid), the thread will be stuck in the loop until
> > s->cpu_slab->tid wraps, which may be forever in the absence of
> > allocations on the same CPU.
> 
> Is there any method to guarantee refetching these in each loop?

We can use READ_ONCE(c->tid), e.g.

	while (IS_ENABLED(CONFIG_PREEMPT) &&
	       unlikely(tid != READ_ONCE(c->tid));

I will send a patch to that effect.

I previously thought that READ_ONCE wasn't guaranteed to be atomic, and
thought it could return torn values (even for a single load
instruction). I now understand that this is not the case, and a
READ_ONCE will be sufficient.

[...]

> If c->tid, c->freelist, c->page are fetched on the other cpu,
> there is no ordering guarantee and c->freelist, c->page could be stale
> value even if c->tid is recent one.

Ah. Good point.

> Think about following free case with your patch.
> 
> Assume initial cpu 0's state as following.
> c->tid: 1, c->freelist: NULL, c->page: A
> 
> User X: try to free object X for page A
> User X: fetch c (s->cpu_slab)
> 
> Preemtion and migration happens...
> The other allocation/free happens... so cpu 0's state is as following.
> c->tid: 3, c->freelist: NULL, c->page: B
> 
> User X: read c->tid: 3, c->freelist: NULL, c->page A (stale value)
> 
> Because tid and freelist are matched with current ones, free would
> succeed, but, current c->page is B and object is for A so this success
> is wrong.

Thanks for the example; it's extremely helpful!

Mark.

  reply	other threads:[~2015-03-17 12:01 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-03-13 15:47 Mark Rutland
2015-03-13 16:29 ` Christoph Lameter
2015-03-13 18:16   ` Mark Rutland
2015-03-13 18:27     ` Christoph Lameter
2015-03-16 12:45   ` [PATCHv2] " Mark Rutland
2015-03-17  1:09 ` [PATCH] " Joonsoo Kim
2015-03-17 12:00   ` Mark Rutland [this message]
2015-03-17 12:15     ` [PATCHv2] " Mark Rutland
2015-03-17 18:57       ` Christoph Lameter
2015-03-18  5:59       ` Joonsoo Kim
2015-03-18 15:21         ` Mark Rutland
2015-03-19 12:13           ` Joonsoo Kim
2015-03-19 16:16             ` Christoph Lameter
2015-03-24 14:17       ` Ville Syrjälä

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20150317120058.GC23340@leverpostej \
    --to=mark.rutland@arm.com \
    --cc=Catalin.Marinas@arm.com \
    --cc=akpm@linux-foundation.org \
    --cc=brouer@redhat.com \
    --cc=cl@linux.com \
    --cc=iamjoonsoo.kim@lge.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=penberg@kernel.org \
    --cc=rientjes@google.com \
    --cc=steve.capper@linaro.org \
    --cc=torvalds@linux-foundation.org \
    --subject='Re: [PATCH] mm/slub: fix lockups on PREEMPT && '\!'SMP kernels' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).