LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Rik van Riel <riel@redhat.com>
To: Mel Gorman <mgorman@suse.de>, Andrew Morton <akpm@linux-foundation.org>
Cc: linux-mm@kvack.org, Minchan Kim <minchan@kernel.org>,
	Vlastimil Babka <vbabka@suse.cz>,
	linux-kernel@vger.kernel.org
Subject: Re: [RFC PATCH] mm: madvise: Ignore repeated MADV_DONTNEED hints
Date: Thu, 05 Feb 2015 16:44:43 -0500	[thread overview]
Message-ID: <54D3E44B.7060501@redhat.com> (raw)
In-Reply-To: <20150202221824.GN2395@suse.de>

On 02/02/2015 05:18 PM, Mel Gorman wrote:
> On Mon, Feb 02, 2015 at 02:05:06PM -0800, Andrew Morton wrote:
>> On Mon, 2 Feb 2015 16:55:25 +0000 Mel Gorman <mgorman@suse.de> wrote:
>>
>>> glibc malloc changed behaviour in glibc 2.10 to have per-thread arenas
>>> instead of creating new areans if the existing ones were contended.
>>> The decision appears to have been made so the allocator scales better but the
>>> downside is that madvise(MADV_DONTNEED) is now called for these per-thread
>>> areans during free. This tears down pages that would have previously
>>> remained. There is nothing wrong with this decision from a functional point
>>> of view but any threaded application that frequently allocates/frees the
>>> same-sized region is going to incur the full teardown and refault costs.
>>
>> MADV_DONTNEED has been there for many years.  How could this problem
>> not have been noticed during glibc 2.10 development/testing? 
> 
> I do not know. I only spotted it due to switching distributions. Looping
> allocations and frees of the same sizes is considered inefficient and it
> might have been dismissed on those grounds. It's probably less noticeable
> when it only affects threaded applications.
> 
>> Is there
>> some more recent kernel change which is triggering this?
>>
> 
> Not that I'm aware of.
> 
>>> This patch identifies when a thread is frequently calling MADV_DONTNEED
>>> on the same region of memory and starts ignoring the hint.
>>
>> That's pretty nasty-looking :(
>>
> 
> Yep, it is but we're very limited in terms of what we can do within the
> kernel here.
> 
>> And presumably there are all sorts of behaviours which will still
>> trigger the problem but which will avoid the start/end equality test in
>> ignore_madvise_hint()?
>>
> 
> Yes. I would expect that a simple pattern of multiple allocs followed by
> multiple frees in a loop would also trigger it.
> 
>> Really, this is a glibc problem and only a glibc problem. 
>> MADV_DONTNEED is unavoidably expensive and glibc is calling
>> MADV_DONTNEED for a region which it *does* need. 
> 
> To be fair to glibc, it calls it on a region it *thinks* it doesn't need only
> to reuse it immediately afterwards because of how the benchmark is
> implemented.
> 
>> Is there something
>> preventing this from being addressed within glibc?
>  
> I doubt it other than I expect they'll punt it back and blame either the
> application for being stupid or the kernel for being slow.

This sounds like something that could benefit from Minchan's
MADV_FREE, instead of MADV_DONTNEED.

If non page aligned malloc/free does not depend on pages
being zeroed, I suspect an MADV_DONTNEED resulting from
a malloc/free loop also does not depend on it.


  parent reply	other threads:[~2015-02-05 21:44 UTC|newest]

Thread overview: 31+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-02-02 16:55 Mel Gorman
2015-02-02 22:05 ` Andrew Morton
2015-02-02 22:18   ` Mel Gorman
2015-02-02 22:35     ` Andrew Morton
2015-02-03  0:26       ` Davidlohr Bueso
2015-02-03 10:50       ` Mel Gorman
2015-02-05 21:44     ` Rik van Riel [this message]
2015-02-02 22:22 ` Dave Hansen
2015-02-03  8:19   ` MADV_DONTNEED semantics? Was: " Vlastimil Babka
2015-02-03 10:53     ` Kirill A. Shutemov
2015-02-03 11:42       ` Vlastimil Babka
2015-02-03 16:20         ` Michael Kerrisk (man-pages)
2015-02-04 13:46           ` Vlastimil Babka
2015-02-04 14:00             ` Michael Kerrisk (man-pages)
2015-02-04 17:02               ` Vlastimil Babka
2015-02-04 19:24                 ` Michael Kerrisk (man-pages)
2015-02-05  1:07                   ` Minchan Kim
2015-02-06 15:41                     ` Michael Kerrisk (man-pages)
2015-02-09  6:46                       ` Minchan Kim
2015-02-09  9:13                         ` Michael Kerrisk (man-pages)
2015-02-05 15:41                   ` Michal Hocko
2015-02-06 15:57                     ` Michael Kerrisk (man-pages)
2015-02-06 20:45                       ` Michal Hocko
2015-02-09  6:50                       ` Minchan Kim
2015-02-04  0:09         ` Minchan Kim
2015-02-03 11:16     ` Mel Gorman
2015-02-03 15:21       ` Michal Hocko
2015-02-03 16:25         ` Michael Kerrisk (man-pages)
2015-02-03  9:47   ` Mel Gorman
2015-02-03 10:47     ` Kirill A. Shutemov
2015-02-03 11:21       ` Mel Gorman

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=54D3E44B.7060501@redhat.com \
    --to=riel@redhat.com \
    --cc=akpm@linux-foundation.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mgorman@suse.de \
    --cc=minchan@kernel.org \
    --cc=vbabka@suse.cz \
    --subject='Re: [RFC PATCH] mm: madvise: Ignore repeated MADV_DONTNEED hints' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).