LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
* [PATCH] Allow compaction of unevictable pages
@ 2015-03-06 18:41 Eric B Munson
  2015-03-06 21:07 ` David Rientjes
  0 siblings, 1 reply; 3+ messages in thread
From: Eric B Munson @ 2015-03-06 18:41 UTC (permalink / raw)
  To: Andrew Morton
  Cc: Eric B Munson, Vlastimil Babka, Thomas Gleixner,
	Christoph Lameter, Peter Zijlstra, Mel Gorman, linux-mm,
	linux-kernel

Currently, pages which are marked as unevictable are protected from
compaction, but not from other types of migration.  The mlock
desctription does not promise that all page faults will be avoided, only
major ones so this protection is not necessary.  This extra protection
can cause problems for applications that are using mlock to avoid
swapping pages out, but require order > 0 allocations to continue to
succeed in a fragmented environment.  This patch changes the
isolate_mode used by the compaction code to allow compaction of
unevictable pages.

To illustrate this problem I wrote a quick test program that mmaps a
large number of 1MB files filled with random data.  These maps are
created locked and read only.  Then, every other mmap is unmapped and I
attempt to allocate huge pages to the static huge page pool.  Without
this patch I am unable to allocate any huge pages after fragmenting
memory.  With it, I can allocate almost all the space freed by unmapping
as huge pages.

Signed-off-by: Eric B Munson <emunson@akamai.com>
To: Andrew Morton <akpm@linux-foundation.org>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Christoph Lameter <cl@linux.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Mel Gorman <mgorman@suse.de>
Cc: linux-mm@kvack.org
Cc: linux-kernel@vger.kernel.org
---
 mm/compaction.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mm/compaction.c b/mm/compaction.c
index 8c0d945..33c81e1 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -1056,7 +1056,7 @@ static isolate_migrate_t isolate_migratepages(struct zone *zone,
 {
 	unsigned long low_pfn, end_pfn;
 	struct page *page;
-	const isolate_mode_t isolate_mode =
+	const isolate_mode_t isolate_mode = ISOLATE_UNEVICTABLE |
 		(cc->mode == MIGRATE_ASYNC ? ISOLATE_ASYNC_MIGRATE : 0);
 
 	/*
-- 
1.7.9.5


^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: [PATCH] Allow compaction of unevictable pages
  2015-03-06 18:41 [PATCH] Allow compaction of unevictable pages Eric B Munson
@ 2015-03-06 21:07 ` David Rientjes
  2015-03-09 17:05   ` Eric B Munson
  0 siblings, 1 reply; 3+ messages in thread
From: David Rientjes @ 2015-03-06 21:07 UTC (permalink / raw)
  To: Eric B Munson
  Cc: Andrew Morton, Vlastimil Babka, Thomas Gleixner,
	Christoph Lameter, Peter Zijlstra, Mel Gorman, linux-mm,
	linux-kernel

On Fri, 6 Mar 2015, Eric B Munson wrote:

> diff --git a/mm/compaction.c b/mm/compaction.c
> index 8c0d945..33c81e1 100644
> --- a/mm/compaction.c
> +++ b/mm/compaction.c
> @@ -1056,7 +1056,7 @@ static isolate_migrate_t isolate_migratepages(struct zone *zone,
>  {
>  	unsigned long low_pfn, end_pfn;
>  	struct page *page;
> -	const isolate_mode_t isolate_mode =
> +	const isolate_mode_t isolate_mode = ISOLATE_UNEVICTABLE |
>  		(cc->mode == MIGRATE_ASYNC ? ISOLATE_ASYNC_MIGRATE : 0);
>  
>  	/*

I agree that memory compaction should be isolating and migrating 
unevictable memory for better results, and we have been running with a 
similar patch internally for about a year for the same purpose as you, 
higher probability of allocating hugepages.

This would be better off removing the notion of ISOLATE_UNEVICTABLE 
entirely, however, since CMA and now memory compaction would be using it, 
so the check in __isolate_lru_page() is no longer necessary.  Has the 
added bonus of removing about 10 lines of soure code.

^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: [PATCH] Allow compaction of unevictable pages
  2015-03-06 21:07 ` David Rientjes
@ 2015-03-09 17:05   ` Eric B Munson
  0 siblings, 0 replies; 3+ messages in thread
From: Eric B Munson @ 2015-03-09 17:05 UTC (permalink / raw)
  To: David Rientjes
  Cc: Andrew Morton, Vlastimil Babka, Thomas Gleixner,
	Christoph Lameter, Peter Zijlstra, Mel Gorman, linux-mm,
	linux-kernel

[-- Attachment #1: Type: text/plain, Size: 1276 bytes --]

On Fri, 06 Mar 2015, David Rientjes wrote:

> On Fri, 6 Mar 2015, Eric B Munson wrote:
> 
> > diff --git a/mm/compaction.c b/mm/compaction.c
> > index 8c0d945..33c81e1 100644
> > --- a/mm/compaction.c
> > +++ b/mm/compaction.c
> > @@ -1056,7 +1056,7 @@ static isolate_migrate_t isolate_migratepages(struct zone *zone,
> >  {
> >  	unsigned long low_pfn, end_pfn;
> >  	struct page *page;
> > -	const isolate_mode_t isolate_mode =
> > +	const isolate_mode_t isolate_mode = ISOLATE_UNEVICTABLE |
> >  		(cc->mode == MIGRATE_ASYNC ? ISOLATE_ASYNC_MIGRATE : 0);
> >  
> >  	/*
> 
> I agree that memory compaction should be isolating and migrating 
> unevictable memory for better results, and we have been running with a 
> similar patch internally for about a year for the same purpose as you, 
> higher probability of allocating hugepages.
> 
> This would be better off removing the notion of ISOLATE_UNEVICTABLE 
> entirely, however, since CMA and now memory compaction would be using it, 
> so the check in __isolate_lru_page() is no longer necessary.  Has the 
> added bonus of removing about 10 lines of soure code.

Thanks for having a look, I will send out a V2 that removes
ISOLATE_UNEVICTABLE and the check in __isolate_lru_page().

Eric

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, other threads:[~2015-03-09 17:05 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-03-06 18:41 [PATCH] Allow compaction of unevictable pages Eric B Munson
2015-03-06 21:07 ` David Rientjes
2015-03-09 17:05   ` Eric B Munson

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).