Re: [PATCH 5/5] sched/rt: Optimize find_lowest_rq() to select a cache hot cpu

[Peter Zijlstra <peterz@infradead.org>]

On Fri, Jan 30, 2015 at 12:42:47AM +0800, Xunlei Pang wrote:
> On 27 January 2015 at 22:56, Steven Rostedt <rostedt@goodmis.org> wrote:
> > On Tue, 27 Jan 2015 15:21:36 +0100
> > Peter Zijlstra <peterz@infradead.org> wrote:
> >
> >> On Mon, Jan 19, 2015 at 04:49:40AM +0000, Xunlei Pang wrote:
> >> > In find_lowest_rq(), if we can't find a wake_affine cpu from
> >> > sched_domain, then we can actually determine a cache hot cpu
> >> > instead of simply calling "cpumask_any(lowest_mask)" which
> >> > always returns the first cpu in the mask.
> >> >
> >> > So, we can determine the cache hot cpu during the interation of
> >> > sched_domain() in passing.
> >>
> >> Steve, I'm not getting this. Why are we using WAKE_AFFINE here?
> >>
> >
> > It originated from Gregory Haskins topology patches. See
> >  6e1254d2c41215da27025add8900ed187bca121d
> 
> Hi Peter, Steve,
> 
> I think the responsiveness is the most important feature for RT tasks,
> so I think:
> response latency > cache > SMT in significance.

No, deterministic execution time is the utmost important feature. And
for that SMT utterly blows. So much so in fact that rule #1 for -rt work
is to disable SMT on your hardware.

The same argument can be made for shared caches. If your !rt workload
blows away the cache of the rt workload, you loose.

> I was wondering if we can take the cpuidle state into account like
> current find_idlest_cpu() for CFS?
> cpupri_find() can be easily modified to indicate the CPUPRI_IDLE case,
> then we can select
> an optimal idle cpu to improve RT tasks' responsiveness. For other
> cases(mostly non-idle cpu),
> I think we can rely on the existent sched_domain iteraction to select
> a cache-hot cpu without
> caring too much about SMT.

your patch calls something 'cache-hot' when crossing large numa domains,
don't you think that's somewhat stretching the definition of hot?