LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Andrew Gallatin <gallatin@myri.com>
To: AndrewL733 <AndrewL733@aol.com>
Cc: linux-kernel@vger.kernel.org, Myricom Technical Support <help@myri.com>
Subject: Re: [Myricom help #56546] Re: 2.6.24 Page Allocation Failure
Date: Fri, 01 Feb 2008 10:07:54 -0500	[thread overview]
Message-ID: <47A335CA.1020301@myri.com> (raw)
In-Reply-To: <2517DBDB-3596-4815-89AB-E08B55F11E8E@aol.com>

[-- Attachment #1: Type: text/plain, Size: 1756 bytes --]

AndrewL733 wrote:
 > The cause of this problem seems to be compiling the Myricom driver with
 > the ALLOC_ORDER=2 option. When I use the in-kernel driver, (1.3.2) or
 > recompile the Myricom 1.4.0 driver WITHOUT the option, the problem seems
 > to go away even after heavy hammering of the system.
 >
 > The ALLOC_ORDER=2 compiling option doesn't seem to cause any problem for
 > the Myricom 1.4.0 driver in the 2.6.22 kernel but it does cause the
 > problem when I run it in 2.6.24.

High order allocations are much harder to satisfy than single page
allocations, which is why we default MYRI10GE_ALLOC_ORDER to zero.
Part of the reason why we receive into pages rather than plain skbs is
to be able to use jumbo frames without high order allocations.  In
your case, I suspect that "something" changed enough between 2.6.22
and 2.6.24 that caused the high order allocations to start failing
under the same workload.  I don't know enough about the VM system
to know what changed.

In some quick experiments here, it seems that most of the cost of
these failures is the console messages that they generate.  The driver
is written such that failures like this can be tolerated.  I think the
warning should just be disabled for our rx buffer allocations.

I've attached a patch (against 1.4.0) which does this.  When running a
memory hog which repeatedly reads one byte per page of an array larger
than physical memory, I still see line rate with this patch applied.
Without the patch, I see a few hundred Mb/s and constant page
allocation warnings.  Please note that this disables the warnings just
for our rx buffer allocations.  Other allocations done elsewhere in
the system may still generate warnings if the system is under heavy
memory pressure.

Drew

[-- Attachment #2: nowarn.diff --]
[-- Type: text/plain, Size: 652 bytes --]

Index: myri10ge.c
===================================================================
RCS file: /repository/myrige/linux/myri10ge.c,v
retrieving revision 1.313
diff -u -r1.313 myri10ge.c
--- myri10ge.c	17 Jan 2008 15:04:30 -0000	1.313
+++ myri10ge.c	1 Feb 2008 15:00:39 -0000
@@ -1655,7 +1655,8 @@
 			get_page(rx->page);
 		} else {
 			/* we need a new page */
-			page = myri10ge_alloc_pages(GFP_ATOMIC | __GFP_COMP, MYRI10GE_ALLOC_ORDER);
+			page = myri10ge_alloc_pages(GFP_ATOMIC | __GFP_COMP | __GFP_NOWARN,
+						    MYRI10GE_ALLOC_ORDER);
 			if (unlikely(page == NULL)) {
 				if (rx->fill_cnt - rx->cnt < 16)
 					rx->watchdog_needed = 1;

           reply	other threads:[~2008-02-01 15:08 UTC|newest]

Thread overview: expand[flat|nested]  mbox.gz  Atom feed
 [parent not found: <2517DBDB-3596-4815-89AB-E08B55F11E8E@aol.com>]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=47A335CA.1020301@myri.com \
    --to=gallatin@myri.com \
    --cc=AndrewL733@aol.com \
    --cc=help@myri.com \
    --cc=linux-kernel@vger.kernel.org \
    --subject='Re: [Myricom help #56546] Re: 2.6.24 Page Allocation Failure' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).