LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Michael Gerdau <mgd@technosis.de>
To: Ingo Molnar <mingo@elte.hu>
Cc: linux-kernel@vger.kernel.org,
	Linus Torvalds <torvalds@linux-foundation.org>,
	Nick Piggin <npiggin@suse.de>,
	Gene Heskett <gene.heskett@gmail.com>,
	Juliusz Chroboczek <jch@pps.jussieu.fr>,
	Mike Galbraith <efault@gmx.de>,
	Peter Williams <pwil3058@bigpond.net.au>,
	ck list <ck@vds.kolivas.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	William Lee Irwin III <wli@holomorphy.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	Bill Davidsen <davidsen@tmr.com>, Willy Tarreau <w@1wt.eu>,
	Arjan van de Ven <arjan@infradead.org>
Subject: Re: [REPORT] cfs-v6-rc2 vs sd-0.46 vs 2.6.21-rc7
Date: Thu, 26 Apr 2007 15:22:28 +0200	[thread overview]
Message-ID: <200704261522.43347.mgd@technosis.de> (raw)
In-Reply-To: <20070426120723.GA4092@elte.hu>

[-- Attachment #1: Type: text/plain, Size: 5860 bytes --]

> as a summary: i think your numbers demonstrate it nicely that the 
> shorter 'timeslice length' that both CFS and SD utilizes does not have a 
> measurable negative impact on your workload.

That's my impression as well. In particular I think that for this
type of workload it is pretty much irrelevant which scheduler I'm
using :-)

> To measure the total impact  
> of 'timeslicing' you might want to try the exact same workload with a 
> much higher 'timeslice length' of say 400 msecs, via:
> 
>     echo 400000000 > /proc/sys/kernel/sched_granularity_ns  # on CFS
>     echo 400 > /proc/sys/kernel/rr_interval                 # on SD

I just finished building cfs and sd against 2.6.21 and will try with
these over the next days.

> your existing numbers are a bit hard to analyze because the 3 workloads 
> were started at the same time and they overlapped differently and 
> utilized the system differently.

Right. However the differences w/r to which job finished when, in
particular the change in who "came in" 2. and 3. between sd and cfs
had been consistent with other similar such jobs I ran over the last
couple of days.

In general sd tends to finish all three such jobs at roughly the
same time while cfs clearly "favors" the LTMM-type jobs (which
admittedly involve the least computations). I don't really know
why that is so. However the jobs does some minimal I/O after each
successfully finish run and that might be different w/r to sd and
cfs. Not really knowing what both scheduler do internally I'm in
no position to discuss that with either you or Con :)

> i think the primary number that makes sense to look at (which is perhaps 
> the least sensitive to the 'overlap effect') is the 'combined user times 
> of all 3 workloads' (in order of performance):
> 
> > 2.6.21-rc7:                              20589.423    100.00%
> > 2.6.21-rc7-cfs-v6-rc2 (X @ nice -10):    20613.845     99.88%
> > 2.6.21-rc7-sd046:                        20617.945     99.86%
> > 2.6.21-rc7-cfs-v6-rc2 (X @ nice 0):      20743.564     99.25%
> 
> to me this gives the impression that it's all "within noise".

I think so too. But then I didn't seriously expect any different.

> In  
> particular the two CFS results suggest that there's at least a ~100 
> seconds noise in these results, because the renicing of X should have no 
> impact on the result (the workloads are pure number-crunchers, and all 
> use up the CPUs 100%, correct?),

Correct.

The differences could as well result from small fluctuations in my
work on the machine during the test (like reading mail, editing sources
and similar stuff).

> another (perhaps less reliable) number is the total wall-clock runtime 
> of all 3 jobs. Provided i did not make any mistakes in my calculations, 
> here are the results:
> 
> > 2.6.21-rc7-sd046:                        10512.611 seconds
> > 2.6.21-rc7-cfs-v6-rc2 (X @ nice -10):    10605.946 seconds
> > 2.6.21-rc7:                              10650.535 seconds
> > 2.6.21-rc7-cfs-v6-rc2 (X @ nice 0):      10788.125 seconds
> 
> (the numbers are lower than the first numbers because this is a 2 CPU 
> system)
> 
> both SD and CFS-nice-10 was faster than mainline, but i'd say this too 
> is noise - especially because this result highly depends on the way the 
> workloads overlap in general, which seems to be different for SD.

I don't think that's noise. As I wrote above IMO this is the one
difference I personally consider significant. While running I could
watch intermediate output of all 3 jobs intermixed on a console.

With sd the jobs were -within reason- head to head while with cfs
and mainline LTMM quickly got ahead and remained there.

As I speculated above this (IMO real) difference in the way sd and
cfs (and mainline) schedule the jobs might be related to the way
the schedulers react on I/O but I don't know.

> system time is interesting too:
> 
> > 2.6.21-rc7:                              35.379
> > 2.6.21-rc7-cfs-v6-rc2 (X @ nice -10):    40.399
> > 2.6.21-rc7-cfs-v6-rc2 (X @ nice 0):      44.239
> > 2.6.21-rc7-sd046:                        45.515

Over the weekend I might try to repeat these tests while doing
nothing else on the machine (i.e. no editing or reading mail).

> combined system+user time:
> 
> > 2.6.21-rc7:                              20624.802
> > 2.6.21-rc7-cfs-v6-rc2 (X @ nice 0):      20658.084
> > 2.6.21-rc7-sd046:                        20663.460
> > 2.6.21-rc7-cfs-v6-rc2 (X @ nice -10):    20783.963
> 
> perhaps it might make more sense to run the workloads serialized, to 
> have better comparabality of the individual workloads. (on a real system 
> you'd naturally want to overlap these workloads to utilize the CPUs, so 
> the numbers you did are very relevant too.)

Yes. In fact the workloads used to be run serialized and only
recently I tried to easily have them run on multicore systems.

The whole set of such jobs consists of 312 tasks and thus the
idea is to get a couple of PS3 and have them run there...
(once I found the time to port the software).

> The vmstat suggested there is occasional idle time in the system - is 
> the workload IO-bound (or memory bound) in those cases?

There is writing out stats after each run (i.e. every 5-8 sec) both
to the PostgreSQL DB as well as to the console.

I could get rid of the console I/O (i.e. make it conditional) and
for testing purpose could switch off the DB if that would make be
helpful.

Best,
Michael
-- 
 Technosis GmbH, Geschäftsführer: Michael Gerdau, Tobias Dittmar
 Sitz Hamburg; HRB 89145 Amtsgericht Hamburg
 Vote against SPAM - see http://www.politik-digital.de/spam/
 Michael Gerdau       email: mgd@technosis.de
 GPG-keys available on request or at public keyserver

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

  reply	other threads:[~2007-04-26 13:22 UTC|newest]

Thread overview: 7+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2007-04-26 11:12 Michael Gerdau
2007-04-26 12:07 ` Ingo Molnar
2007-04-26 13:22   ` Michael Gerdau [this message]
2007-04-26 13:55     ` Ingo Molnar
2007-04-26 22:59   ` [ck] " Con Kolivas
2007-04-27  5:59     ` Michael Gerdau
2007-04-27  6:52     ` Ingo Molnar

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=200704261522.43347.mgd@technosis.de \
    --to=mgd@technosis.de \
    --cc=akpm@linux-foundation.org \
    --cc=arjan@infradead.org \
    --cc=ck@vds.kolivas.org \
    --cc=davidsen@tmr.com \
    --cc=efault@gmx.de \
    --cc=gene.heskett@gmail.com \
    --cc=jch@pps.jussieu.fr \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mingo@elte.hu \
    --cc=npiggin@suse.de \
    --cc=pwil3058@bigpond.net.au \
    --cc=tglx@linutronix.de \
    --cc=torvalds@linux-foundation.org \
    --cc=w@1wt.eu \
    --cc=wli@holomorphy.com \
    --subject='Re: [REPORT] cfs-v6-rc2 vs sd-0.46 vs 2.6.21-rc7' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).