LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
* [ANNOUNCE] Linsched for 2.6.35 released
@ 2010-10-12 17:29 Ranjit Manomohan
  2010-10-19  4:52 ` Vaidyanathan Srinivasan
  0 siblings, 1 reply; 5+ messages in thread
From: Ranjit Manomohan @ 2010-10-12 17:29 UTC (permalink / raw)
  To: linux-kernel
  Cc: Mike Galbraith, Nikhil Rao, Salman Qazi, Dhaval Giani,
	Peter Zijlstra, Ingo Molnar, Thomas Gleixner,
	Venkatesh Pallipadi, Paul Turner

Hi,
  I would like to announce the availability of the Linux Scheduler Simulator
(Linsched) for 2.6.35.

Originally developed at the University of North Carolina, LinSched is a
user-space program that hosts the Linux scheduling subsystem.
Its purpose is to provide a tool for observing and modifying the behavior
of the Linux scheduler. This makes it a valuable tool in prototyping new
Linux scheduling policies in a way that may be easier (or otherwise
less painful or time-consuming) to many developers when compared
to working with real hardware.

Since Linsched allows arbitrary hardware topologies to be modeled,
it enables testing of scheduler changes on hardware that may not be
easily accessible to the developer. For example, most developers don't
have access to a quad-core quad-socket box, but they can use LinSched
to see how their changes affect the scheduler on such boxes.

When compared to alternatives such as using UML which may offer
similar benefits to developers, Linsched has fewer dependencies and
relies on a very small subset of files of the kernel. It also offers stable
and repeatable results that are not affected by the environment or
hardware that is used to run the simulation.

LinSched may be especially useful to those who are new to Linux
scheduler development since it brings all the benefits of user space
analysis and debugging tools (e.g. gdb) to understanding the
kernel scheduler code.

The code is available at:

http://google3-2.osuosl.org/?p=linsched/2.6.35.git;a=summary

You can clone the repository using:

git clone git://google3-2.osuosl.org/linsched/2.6.35.git

New features/functionality in this release are:
-- Based on the 2.6.35 kernel
-- support for group scheduling
-- Ability to specify arbitrary sleep/wakeup patterns for tasks
-- High resolution timers
-- Tickless scheduler (no hz)
-- sched domain support for all levels

At Google we have used Linsched as a testing tool to validate the
behavior of the kernel scheduler. Please see some of the basic tests
we have added at linsched/basic_tests.c. We could  cut down our
validation time from several days to a couple of hours due to the ability
to run multiple tests on several different types of hardware models.

Recently we have proposed a set of enhancements to the CFS scheduler
including:
 -- CFS bandwidth control
   (http://thread.gmane.org/gmane.linux.kernel/979066)
 -- Improved load balancing for low weight tasks
   (http://thread.gmane.org/gmane.linux.kernel/1041721)
These features have been extensively validated using this new infrastructure
which has helped weed out some bugs that would have been difficult to catch
otherwise. See http://thread.gmane.org/gmane.linux.kernel/1037521 for an
example of a load balancing bug uncovered during testing.

We are working on a port of Linsched to 2.6.36 and I would encourage folks
experimenting with the kernel scheduler to give it a try. Please see the
instructions specified in the linsched/README file to build and run tests.

-Please contact me if you run into any difficulties or have any questions

-Thanks,
-Ranjit Manomohan

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [ANNOUNCE] Linsched for 2.6.35 released
  2010-10-12 17:29 [ANNOUNCE] Linsched for 2.6.35 released Ranjit Manomohan
@ 2010-10-19  4:52 ` Vaidyanathan Srinivasan
  2010-11-16  1:52   ` Ranjit Manomohan
  0 siblings, 1 reply; 5+ messages in thread
From: Vaidyanathan Srinivasan @ 2010-10-19  4:52 UTC (permalink / raw)
  To: Ranjit Manomohan
  Cc: linux-kernel, Mike Galbraith, Nikhil Rao, Salman Qazi,
	Dhaval Giani, Peter Zijlstra, Ingo Molnar, Thomas Gleixner,
	Venkatesh Pallipadi, Paul Turner

* Ranjit Manomohan <ranjitm@google.com> [2010-10-12 10:29:54]:

> Hi,
>   I would like to announce the availability of the Linux Scheduler Simulator
> (Linsched) for 2.6.35.
> 
> Originally developed at the University of North Carolina, LinSched is a
> user-space program that hosts the Linux scheduling subsystem.
> Its purpose is to provide a tool for observing and modifying the behavior
> of the Linux scheduler. This makes it a valuable tool in prototyping new
> Linux scheduling policies in a way that may be easier (or otherwise
> less painful or time-consuming) to many developers when compared
> to working with real hardware.

The idea and framework looks very interesting.  I tried it out in
order to understand the workload model and verification model and it
worked fine for the test cases that you have provided.

> Since Linsched allows arbitrary hardware topologies to be modeled,
> it enables testing of scheduler changes on hardware that may not be
> easily accessible to the developer. For example, most developers don't
> have access to a quad-core quad-socket box, but they can use LinSched
> to see how their changes affect the scheduler on such boxes.

I am interested in trying this simulator in order to
design/study/verify task placement logic within the SMP loadbalancer.
Basically the effects of SD_POWERSAVINGS_BALANCE, SD_PREFER_SIBLING,
etc in various topologies.

The current interface and verification mechanism is to create tasks
and observe the runtime received by each task.  In an ideal
loadbalancer situation, all tasks should have received runtime
proportional to their priority.

Can you help me figure out how to get to kstat_cpu() or per-cpu
kernel_stat accounting/utilisation metrics within the simulation?

Thanks for sharing the framework.

--Vaidy


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [ANNOUNCE] Linsched for 2.6.35 released
  2010-10-19  4:52 ` Vaidyanathan Srinivasan
@ 2010-11-16  1:52   ` Ranjit Manomohan
  2011-02-08 18:10     ` Vaidyanathan Srinivasan
  0 siblings, 1 reply; 5+ messages in thread
From: Ranjit Manomohan @ 2010-11-16  1:52 UTC (permalink / raw)
  To: svaidy
  Cc: linux-kernel, Mike Galbraith, Nikhil Rao, Salman Qazi,
	Dhaval Giani, Peter Zijlstra, Ingo Molnar, Thomas Gleixner,
	Venkatesh Pallipadi, Paul Turner

On Mon, Oct 18, 2010 at 9:52 PM, Vaidyanathan Srinivasan
<svaidy@linux.vnet.ibm.com> wrote:
> * Ranjit Manomohan <ranjitm@google.com> [2010-10-12 10:29:54]:
>
>> Hi,
>>   I would like to announce the availability of the Linux Scheduler Simulator
>> (Linsched) for 2.6.35.
>>
>> Originally developed at the University of North Carolina, LinSched is a
>> user-space program that hosts the Linux scheduling subsystem.
>> Its purpose is to provide a tool for observing and modifying the behavior
>> of the Linux scheduler. This makes it a valuable tool in prototyping new
>> Linux scheduling policies in a way that may be easier (or otherwise
>> less painful or time-consuming) to many developers when compared
>> to working with real hardware.
>
> The idea and framework looks very interesting.  I tried it out in
> order to understand the workload model and verification model and it
> worked fine for the test cases that you have provided.

Thanks for evaluating it.

>
>> Since Linsched allows arbitrary hardware topologies to be modeled,
>> it enables testing of scheduler changes on hardware that may not be
>> easily accessible to the developer. For example, most developers don't
>> have access to a quad-core quad-socket box, but they can use LinSched
>> to see how their changes affect the scheduler on such boxes.
>
> I am interested in trying this simulator in order to
> design/study/verify task placement logic within the SMP loadbalancer.
> Basically the effects of SD_POWERSAVINGS_BALANCE, SD_PREFER_SIBLING,
> etc in various topologies.
>
> The current interface and verification mechanism is to create tasks
> and observe the runtime received by each task.  In an ideal
> loadbalancer situation, all tasks should have received runtime
> proportional to their priority.
>
> Can you help me figure out how to get to kstat_cpu() or per-cpu
> kernel_stat accounting/utilisation metrics within the simulation?

we don't use the kstat_cpu accounting in the simulation since it does
not really make sense in this environment.

We have a timer driven loop that advances time globally and kicks of
events scheduled to run at specified times on each CPU. The periodic
timer tick is one among these events. Since there is really no notion
of system vs user time in this scenario, the current code disables the
update_process_times routine. I am not sure how these times relate to
the task placement logic you are trying to verify. If you could let me
know how you plan to use these then I can try to accommodate that in
the simulation.

Sorry for the delay in response. My mail filters messed this up.

-Thanks,
Ranjit

>
> Thanks for sharing the framework.
>
> --Vaidy
>
>

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [ANNOUNCE] Linsched for 2.6.35 released
  2010-11-16  1:52   ` Ranjit Manomohan
@ 2011-02-08 18:10     ` Vaidyanathan Srinivasan
  2011-02-15 18:25       ` Ranjit Manomohan
  0 siblings, 1 reply; 5+ messages in thread
From: Vaidyanathan Srinivasan @ 2011-02-08 18:10 UTC (permalink / raw)
  To: Ranjit Manomohan
  Cc: linux-kernel, Mike Galbraith, Nikhil Rao, Salman Qazi,
	Dhaval Giani, Peter Zijlstra, Ingo Molnar, Thomas Gleixner,
	Venkatesh Pallipadi, Paul Turner

* Ranjit Manomohan <ranjitm@google.com> [2010-11-15 17:52:05]:

> On Mon, Oct 18, 2010 at 9:52 PM, Vaidyanathan Srinivasan
> <svaidy@linux.vnet.ibm.com> wrote:
> > * Ranjit Manomohan <ranjitm@google.com> [2010-10-12 10:29:54]:
> >

[snip]

> > Can you help me figure out how to get to kstat_cpu() or per-cpu
> > kernel_stat accounting/utilisation metrics within the simulation?
> 
> we don't use the kstat_cpu accounting in the simulation since it does
> not really make sense in this environment.
> 
> We have a timer driven loop that advances time globally and kicks of
> events scheduled to run at specified times on each CPU. The periodic
> timer tick is one among these events. Since there is really no notion
> of system vs user time in this scenario, the current code disables the
> update_process_times routine. I am not sure how these times relate to
> the task placement logic you are trying to verify. If you could let me
> know how you plan to use these then I can try to accommodate that in
> the simulation.

The current setup lets us find how much time each task was run.
I would like to use the kernel_stat information to understand 'which
cpu' ran the task.  Basically we could place nr_tasks < nr_cpus and
see them settle to the right CPUs within the sched domain topology.
This can be verified by checking the CPU's utilisation or run time at
the end of the simulation.  Like two tasks on the same socket of
a dual-socket dual-core system should settle to one task per socket.
The load balancer should be able to spread the tasks around slowly.

The ability to create diverse topology within linsched is very
useful to test these load balancer functions and corner cases.

> Sorry for the delay in response. My mail filters messed this up.

I got your reply earlier. No problem with the delay.
Do you have a new version to share?  Any new feature that you are planning?

--Vaidy


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [ANNOUNCE] Linsched for 2.6.35 released
  2011-02-08 18:10     ` Vaidyanathan Srinivasan
@ 2011-02-15 18:25       ` Ranjit Manomohan
  0 siblings, 0 replies; 5+ messages in thread
From: Ranjit Manomohan @ 2011-02-15 18:25 UTC (permalink / raw)
  To: svaidy
  Cc: linux-kernel, Mike Galbraith, Nikhil Rao, Salman Qazi,
	Dhaval Giani, Peter Zijlstra, Ingo Molnar, Thomas Gleixner,
	Venkatesh Pallipadi, Paul Turner

On Tue, Feb 8, 2011 at 10:10 AM, Vaidyanathan Srinivasan
<svaidy@linux.vnet.ibm.com> wrote:
> * Ranjit Manomohan <ranjitm@google.com> [2010-11-15 17:52:05]:
>
>> On Mon, Oct 18, 2010 at 9:52 PM, Vaidyanathan Srinivasan
>> <svaidy@linux.vnet.ibm.com> wrote:
>> > * Ranjit Manomohan <ranjitm@google.com> [2010-10-12 10:29:54]:
>> >
>
> [snip]
>
>> > Can you help me figure out how to get to kstat_cpu() or per-cpu
>> > kernel_stat accounting/utilisation metrics within the simulation?
>>
>> we don't use the kstat_cpu accounting in the simulation since it does
>> not really make sense in this environment.
>>
>> We have a timer driven loop that advances time globally and kicks of
>> events scheduled to run at specified times on each CPU. The periodic
>> timer tick is one among these events. Since there is really no notion
>> of system vs user time in this scenario, the current code disables the
>> update_process_times routine. I am not sure how these times relate to
>> the task placement logic you are trying to verify. If you could let me
>> know how you plan to use these then I can try to accommodate that in
>> the simulation.
>
> The current setup lets us find how much time each task was run.
> I would like to use the kernel_stat information to understand 'which
> cpu' ran the task.  Basically we could place nr_tasks < nr_cpus and
> see them settle to the right CPUs within the sched domain topology.
> This can be verified by checking the CPU's utilisation or run time at
> the end of the simulation.  Like two tasks on the same socket of
> a dual-socket dual-core system should settle to one task per socket.
> The load balancer should be able to spread the tasks around slowly.
>
> The ability to create diverse topology within linsched is very
> useful to test these load balancer functions and corner cases.

Ok, I understand what you are looking for. There does not seem to be
anything in the kernel that I can reuse. I can add a Linsched specific
per cpu task counter to get the stats. I will try to send out an
update in a couple of days.

>
>> Sorry for the delay in response. My mail filters messed this up.
>
> I got your reply earlier. No problem with the delay.
> Do you have a new version to share?  Any new feature that you are planning?

Unfortunately we have fallen a little behind in terms of keeping
linsched up to date with mainline kernel releases. Hopefully we can
get an updated version out this summer. Our current plans are to
include a record/replay type of option to the simulation that will
allow us to optimize a particular workload. Please let me know if
there is anything else that you would like to see added.

-Thanks,
Ranjit

>
> --Vaidy
>
>

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2011-02-15 18:25 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2010-10-12 17:29 [ANNOUNCE] Linsched for 2.6.35 released Ranjit Manomohan
2010-10-19  4:52 ` Vaidyanathan Srinivasan
2010-11-16  1:52   ` Ranjit Manomohan
2011-02-08 18:10     ` Vaidyanathan Srinivasan
2011-02-15 18:25       ` Ranjit Manomohan

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).