LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Mel Gorman <mgorman@suse.de>
To: Andrew Morton <akpm@linux-foundation.org>
Cc: linux-mm@kvack.org, Minchan Kim <minchan@kernel.org>,
	Vlastimil Babka <vbabka@suse.cz>,
	linux-kernel@vger.kernel.org
Subject: Re: [RFC PATCH] mm: madvise: Ignore repeated MADV_DONTNEED hints
Date: Mon, 2 Feb 2015 22:18:24 +0000	[thread overview]
Message-ID: <20150202221824.GN2395@suse.de> (raw)
In-Reply-To: <20150202140506.392ff6920743f19ea44cff59@linux-foundation.org>

On Mon, Feb 02, 2015 at 02:05:06PM -0800, Andrew Morton wrote:
> On Mon, 2 Feb 2015 16:55:25 +0000 Mel Gorman <mgorman@suse.de> wrote:
> 
> > glibc malloc changed behaviour in glibc 2.10 to have per-thread arenas
> > instead of creating new areans if the existing ones were contended.
> > The decision appears to have been made so the allocator scales better but the
> > downside is that madvise(MADV_DONTNEED) is now called for these per-thread
> > areans during free. This tears down pages that would have previously
> > remained. There is nothing wrong with this decision from a functional point
> > of view but any threaded application that frequently allocates/frees the
> > same-sized region is going to incur the full teardown and refault costs.
> 
> MADV_DONTNEED has been there for many years.  How could this problem
> not have been noticed during glibc 2.10 development/testing? 

I do not know. I only spotted it due to switching distributions. Looping
allocations and frees of the same sizes is considered inefficient and it
might have been dismissed on those grounds. It's probably less noticeable
when it only affects threaded applications.

> Is there
> some more recent kernel change which is triggering this?
> 

Not that I'm aware of.

> > This patch identifies when a thread is frequently calling MADV_DONTNEED
> > on the same region of memory and starts ignoring the hint.
> 
> That's pretty nasty-looking :(
> 

Yep, it is but we're very limited in terms of what we can do within the
kernel here.

> And presumably there are all sorts of behaviours which will still
> trigger the problem but which will avoid the start/end equality test in
> ignore_madvise_hint()?
> 

Yes. I would expect that a simple pattern of multiple allocs followed by
multiple frees in a loop would also trigger it.

> Really, this is a glibc problem and only a glibc problem. 
> MADV_DONTNEED is unavoidably expensive and glibc is calling
> MADV_DONTNEED for a region which it *does* need. 

To be fair to glibc, it calls it on a region it *thinks* it doesn't need only
to reuse it immediately afterwards because of how the benchmark is
implemented.

> Is there something
> preventing this from being addressed within glibc?
 
I doubt it other than I expect they'll punt it back and blame either the
application for being stupid or the kernel for being slow.

-- 
Mel Gorman
SUSE Labs

  reply	other threads:[~2015-02-02 22:18 UTC|newest]

Thread overview: 31+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-02-02 16:55 Mel Gorman
2015-02-02 22:05 ` Andrew Morton
2015-02-02 22:18   ` Mel Gorman [this message]
2015-02-02 22:35     ` Andrew Morton
2015-02-03  0:26       ` Davidlohr Bueso
2015-02-03 10:50       ` Mel Gorman
2015-02-05 21:44     ` Rik van Riel
2015-02-02 22:22 ` Dave Hansen
2015-02-03  8:19   ` MADV_DONTNEED semantics? Was: " Vlastimil Babka
2015-02-03 10:53     ` Kirill A. Shutemov
2015-02-03 11:42       ` Vlastimil Babka
2015-02-03 16:20         ` Michael Kerrisk (man-pages)
2015-02-04 13:46           ` Vlastimil Babka
2015-02-04 14:00             ` Michael Kerrisk (man-pages)
2015-02-04 17:02               ` Vlastimil Babka
2015-02-04 19:24                 ` Michael Kerrisk (man-pages)
2015-02-05  1:07                   ` Minchan Kim
2015-02-06 15:41                     ` Michael Kerrisk (man-pages)
2015-02-09  6:46                       ` Minchan Kim
2015-02-09  9:13                         ` Michael Kerrisk (man-pages)
2015-02-05 15:41                   ` Michal Hocko
2015-02-06 15:57                     ` Michael Kerrisk (man-pages)
2015-02-06 20:45                       ` Michal Hocko
2015-02-09  6:50                       ` Minchan Kim
2015-02-04  0:09         ` Minchan Kim
2015-02-03 11:16     ` Mel Gorman
2015-02-03 15:21       ` Michal Hocko
2015-02-03 16:25         ` Michael Kerrisk (man-pages)
2015-02-03  9:47   ` Mel Gorman
2015-02-03 10:47     ` Kirill A. Shutemov
2015-02-03 11:21       ` Mel Gorman

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20150202221824.GN2395@suse.de \
    --to=mgorman@suse.de \
    --cc=akpm@linux-foundation.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=minchan@kernel.org \
    --cc=vbabka@suse.cz \
    --subject='Re: [RFC PATCH] mm: madvise: Ignore repeated MADV_DONTNEED hints' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).