LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Andy Whitcroft <apw@shadowen.org>
To: Andrew Morton <akpm@osdl.org>, linux-mm@kvack.org
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>,
	Mel Gorman <mel@csn.ul.ie>, Andy Whitcroft <apw@shadowen.org>,
	linux-kernel@vger.kernel.org
Subject: [PATCH 0/4] Lumpy Reclaim V3
Date: Wed, 6 Dec 2006 16:59:04 +0000	[thread overview]
Message-ID: <exportbomb.1165424343@pinky> (raw)

This is a repost of the lumpy reclaim patch set.  This is
basically unchanged from the last post, other than being rebased
to 2.6.19-rc2-mm2.  This has passed basic stress testing on a range
of machines here.

[Sorry for the delay reposting, I had a test failure and needed
to confirm it was not due to lumpy before posting.]

As before, I have left the changes broken out for the time being
so as to make it clear what is the original and what is my fault.
I would expect to roll this up before acceptance.

-apw

=== 8< ===
Lumpy Reclaim (V3)

When we are out of memory of a suitable size we enter reclaim.
The current reclaim algorithm targets pages in LRU order, which
is great for fairness but highly unsuitable if you desire pages at
higher orders.  To get pages of higher order we must shoot down a
very high proportion of memory; >95% in a lot of cases.

This patch set adds a lumpy reclaim algorithm to the allocator.
It targets groups of pages at the specified order anchored at the
end of the active and inactive lists.  This encourages groups of
pages at the requested orders to move from active to inactive,
and active to free lists.  This behaviour is only triggered out of
direct reclaim when higher order pages have been requested.

This patch set is particularly effective when utilised with
an anti-fragmentation scheme which groups pages of similar
reclaimability together.

This patch set (against 2.6.19-rc6-mm2) is based on Peter Zijlstra's
lumpy reclaim V2 patch which forms the foundation.  It comprises
the following patches:

lumpy-reclaim-v2 -- Peter Zijlstra's lumpy reclaim prototype,

lumpy-cleanup-a-missplaced-comment-and-simplify-some-code --
  cleanups to move a comment back to where it came from, to make
  the area edge selection more comprehensible and also cleans up
  the switch coding style to match the concensus in mm/*.c,

lumpy-ensure-we-respect-zone-boundaries -- bug fix to ensure we do
  not attempt to take pages from adjacent zones, and

lumpy-take-the-other-active-inactive-pages-in-the-area -- patch to
  increase aggression over the targetted order.

Testing of this patch set under high fragmentation high allocation
load conditions shows significantly improved high order reclaim
rates than a standard kernel.  The stack here it is now within 5%
of the best case linear-reclaim figures.

It would be interesting to see if this setup is also successful in
reducing order-2 allocation failures that you have been seeing with
jumbo frames.

Please consider for -mm.

-apw
(lumpy-v3r7)

             reply	other threads:[~2006-12-06 16:59 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2006-12-06 16:59 Andy Whitcroft [this message]
2006-12-06 16:59 ` [PATCH 1/4] lumpy reclaim v2 Andy Whitcroft
2006-12-15  4:57   ` Andrew Morton
2007-01-26 11:00     ` Andrew Morton
2006-12-06 17:00 ` [PATCH 2/4] lumpy cleanup a missplaced comment and simplify some code Andy Whitcroft
2006-12-06 17:00 ` [PATCH 3/4] lumpy ensure we respect zone boundaries Andy Whitcroft
2006-12-06 17:01 ` [PATCH 4/4] lumpy take the other active inactive pages in the area Andy Whitcroft
2006-12-11 23:29 ` [PATCH 0/4] Lumpy Reclaim V3 Andrew Morton
2007-01-29 12:24   ` Andy Whitcroft
2006-12-12 11:13 ` Andrew Morton
2006-12-12 13:14   ` Andy Whitcroft
2007-01-29 12:25   ` Andy Whitcroft
  -- strict thread matches above, loose matches on Subject: below --
2006-11-23 16:48 Andy Whitcroft
2006-11-23 19:02 ` Peter Zijlstra

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=exportbomb.1165424343@pinky \
    --to=apw@shadowen.org \
    --cc=a.p.zijlstra@chello.nl \
    --cc=akpm@osdl.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mel@csn.ul.ie \
    --subject='Re: [PATCH 0/4] Lumpy Reclaim V3' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).