Re: Integrating cpusets and cpu isolation [was Re: [CPUISOL] CPU isolation extensions]

[Paul Jackson <pj@sgi.com>]

Max wrote:
> Paul, I actually mentioned at the beginning of my email that I did read that thread
> started by Peter. I did learn quite a bit from it :)

Ah - sorry - I missed that part.  However, I'm still getting the feeling
that there were some key points in that thread that we have not managed
to communicate successfully.

> Sounds like at this point we're in agreement that sched_load_balance is not suitable
> for what I'd like to achieve.

I don't think we're in agreement; I think we're in confusion ;)

Yes, sched_load_balance does not *directly* have anything to do with
this.

But indirectly it is a critical element in what I think you'd like to
achieve.  It affects how the cpuset code sets up sched_domains, and
if I understand correctly, you require either (1) some sched_domains to
only contain RT tasks, or (2) some CPUs to be in no sched_domain at all.

Proper configuration of the cpuset hierarchy, including the setting of
the per-cpuset sched_load_balance flag, can provide either of these
sched_domain partitions, as desired.

> But how about making cpusets aware of the cpu_isolated_map ?

No.  That's confusing cpusets and the scheduler again.

The cpu_isolated_map is a file static variable known only within
the kernel/sched.c file; this should not change.

Presently, the boot parameter isolcpus= is just used to initialize
what CPUs are isolated at boot, and then the sched_domain partitioning,
as done in kernel/sched.c:partition_sched_domains() (the hook into
the sched code that cpusets uses) determines which CPUs are isolated
from that point forward.  I doubt that this should change either.

In that thread referenced above, did you see the part where RT is
achieved not by isolating CPUs from any scheduler, but rather by
polymorphically having several schedulers available to operate on each
sched_domain, and having RT threads self-select the RT scheduler?

-- 
                  I won't rest till it's the best ...
                  Programmer, Linux Scalability
                  Paul Jackson <pj@sgi.com> 1.940.382.4214