LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: "David Schwartz" <davids@webmaster.com>
To: <brettspamacct@fastclick.com>, "Martin J. Bligh" <mbligh@aracnet.com>
Cc: "linux-kernel mailing list" <linux-kernel@vger.kernel.org>,
	<jbarnes@engr.sgi.com>
Subject: RE: How can I optimize a process on a NUMA architecture(x86-64 specifically)?
Date: Sat, 22 May 2004 19:49:37 -0700	[thread overview]
Message-ID: <MDEHLPKNGKAHNMBLJOLKAENNMCAA.davids@webmaster.com> (raw)
In-Reply-To: <40AE3BF5.5080804@fastclick.com>


> Let's say I have a 2 way opteron and want to run 4 long-lived processes.
>   I fork and exec to create 1 of the processes, it chooses to run on
> processor 0 since processor 1 is overloaded at that time, so its
> homenode is processor 0.  I fork and exec another, it chooses processor
> 0 since processors 1 is overloaded at that time. .. Let's say an uneven
> distribution is chosen for all 4 processes, with all processes mapped to
> processor 0. So they allocate on node 0 yet the scheduler will map these
> to both processors since CPU should be balanced. In this case, you will
> have a situation where the second processor will have to fetch memory
> from the other processor's memory.
>
> So a better solution would be to use numactl to set the homenodes
> explicitly, choosing processor 0 for 2 processes, processor 1 for the 2
> other processes.
>
> Is this incorrect?

	Generally, yes, it is. Surprisingly so. If you assume everything is
perfect, then this seems true. But in the real world, it almost never works
that way.

	Consider, for example, if process 1 is responsible for most of the memory
load at any particular time. If it's on CPU #1 and all its memory is on CPU
#1, then the memory controller on CPU #2 is underused and memory bandwidth
suffers. This is why most BIOSes interlave the memory pages across the CPUs.
This gives you the best chance of being able to use both memory controllers
under load.

	I don't think we've reached the point yet where treating x86-64 systems as
NUMA machines makes very much sense.

	DS



  parent reply	other threads:[~2004-05-23  2:50 UTC|newest]

Thread overview: 22+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2004-05-21  0:51 Brett E.
2004-05-21  1:29 ` Jesse Barnes
2004-05-21  6:37 ` Martin J. Bligh
2004-05-21 17:27   ` Brett E.
2004-05-21 17:46     ` Martin J. Bligh
2004-05-21 18:14       ` Brett E.
2004-05-21 18:30         ` Martin J. Bligh
2004-05-21 18:58         ` Jesse Barnes
2004-05-21 19:08           ` Martin J. Bligh
2004-05-23  2:49     ` David Schwartz [this message]
     [not found] <1Y6yr-eM-11@gated-at.bofh.it>
     [not found] ` <1YbRm-4iF-11@gated-at.bofh.it>
     [not found]   ` <1Yma3-4cF-3@gated-at.bofh.it>
     [not found]     ` <1YmjP-4jX-37@gated-at.bofh.it>
2004-05-21 19:17       ` Andi Kleen
     [not found]       ` <1YmMN-4Kh-17@gated-at.bofh.it>
     [not found]         ` <1Yn67-50q-7@gated-at.bofh.it>
2004-05-21 19:19           ` Andi Kleen
2004-05-21 20:32             ` Martin J. Bligh
2004-05-21 23:42               ` Brett E.
2004-05-22  6:13                 ` Martin J. Bligh
2004-05-22  7:41                   ` Andi Kleen
2004-05-23  0:28                 ` Bryan O'Sullivan
2004-05-23 14:28                   ` Andi Kleen
2004-05-24 22:00                     ` Andrew Theurer
2004-05-25  0:27                       ` Scott Robert Ladd
2004-05-25  1:09                       ` Brett E.
     [not found]     ` <1YRnC-3vk-5@gated-at.bofh.it>
2004-05-23 11:57       ` Andi Kleen

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=MDEHLPKNGKAHNMBLJOLKAENNMCAA.davids@webmaster.com \
    --to=davids@webmaster.com \
    --cc=brettspamacct@fastclick.com \
    --cc=jbarnes@engr.sgi.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mbligh@aracnet.com \
    --subject='RE: How can I optimize a process on a NUMA architecture(x86-64 specifically)?' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).