Linux-Fsdevel Archive on lore.kernel.org
help / color / mirror / Atom feed
From: peterz@infradead.org
To: Xi Wang <xii@google.com>
Cc: Ingo Molnar <mingo@redhat.com>,
	Juri Lelli <juri.lelli@redhat.com>,
	Vincent Guittot <vincent.guittot@linaro.org>,
	Dietmar Eggemann <dietmar.eggemann@arm.com>,
	Steven Rostedt <rostedt@goodmis.org>,
	Mel Gorman <mgorman@suse.de>,
	linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org,
	suravee.suthikulpanit@amd.com, thomas.lendacky@amd.com
Subject: Re: [PATCH] sched: Make select_idle_sibling search domain configurable
Date: Tue, 28 Jul 2020 12:39:07 +0200	[thread overview]
Message-ID: <20200728103907.GT119549@hirez.programming.kicks-ass.net> (raw)
In-Reply-To: <20200728070131.1629670-1-xii@google.com>

On Tue, Jul 28, 2020 at 12:01:31AM -0700, Xi Wang wrote:
> The scope of select_idle_sibling idle cpu search is LLC. This
> becomes a problem for the AMD CCX architecture, as the sd_llc is only
> 4 cores. On a many core machine, the range of search is too small to
> reach a satisfactory level of statistical multiplexing / efficient
> utilization of short idle time slices.
> 
> With this patch idle sibling search is detached from LLC and it
> becomes run time configurable. To reduce search and migration
> overheads, a presearch domain is added. The presearch domain will be
> searched first before the "main search" domain, e.g.:
> 
> sysctl_sched_wake_idle_domain == 2 ("MC" domain)
> sysctl_sched_wake_idle_presearch_domain == 1 ("DIE" domain)
> 
> Presearch will go through 4 cores of a CCX. If no idle cpu is found
> during presearch, full search will go through the remaining cores of
> a cpu socket.

*groan*, this is horrific :-(

It is also in direct conflict with people wanting to make it smaller.

On top of that, a domain number is a terrible terrible interface. They
aren't even available without SCHED_DEBUG on.

What is the inter-L3 latency? Going by this that had better be awesome.
And if this Infinity Fabric stuff if highly effective in effectively
merging L3s -- analogous to what Intel does with it's cache slices, then
should we not change the AMD topology setup instead of this 'thing'?

Also, this commit:

  051f3ca02e46 ("sched/topology: Introduce NUMA identity node sched domain")

seems to suggest L3 is actually bigger. Suravee, can you please comment?

  parent reply	other threads:[~2020-07-28 10:39 UTC|newest]

Thread overview: 8+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-07-28  7:01 Xi Wang
2020-07-28 10:00 ` kernel test robot
2020-07-28 10:02 ` kernel test robot
2020-07-28 10:39 ` peterz [this message]
2020-07-28 17:54   ` Xi Wang
2020-08-03 22:38     ` Xi Wang
2020-08-04  1:19 ` kernel test robot
2020-08-04  1:19 ` [RFC PATCH] sched: __rebuild_sched_domains() can be static kernel test robot

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200728103907.GT119549@hirez.programming.kicks-ass.net \
    --to=peterz@infradead.org \
    --cc=dietmar.eggemann@arm.com \
    --cc=juri.lelli@redhat.com \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mgorman@suse.de \
    --cc=mingo@redhat.com \
    --cc=rostedt@goodmis.org \
    --cc=suravee.suthikulpanit@amd.com \
    --cc=thomas.lendacky@amd.com \
    --cc=vincent.guittot@linaro.org \
    --cc=xii@google.com \
    --subject='Re: [PATCH] sched: Make select_idle_sibling search domain configurable' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).