LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
* [REPORT] cfs-v6-rc2 vs sd-0.46 vs 2.6.21-rc7
@ 2007-04-26 11:12 Michael Gerdau
2007-04-26 12:07 ` Ingo Molnar
0 siblings, 1 reply; 7+ messages in thread
From: Michael Gerdau @ 2007-04-26 11:12 UTC (permalink / raw)
To: linux-kernel
Cc: Ingo Molnar, Linus Torvalds, Nick Piggin, Gene Heskett,
Juliusz Chroboczek, Mike Galbraith, Peter Williams, ck list,
Thomas Gleixner, William Lee Irwin III, Andrew Morton,
Bill Davidsen, Willy Tarreau, Arjan van de Ven
[-- Attachment #1: Type: text/plain, Size: 18956 bytes --]
Hi list,
find below a test comparing
2.6.21-rc7 (mainline)
2.6.21-rc7-sd046
2.6.21-rc7-cfs-v6-rc2(*) (X @ nice 0)
2.6.21-rc7-cfs-v6-rc2(*) (X @ nice -10)
running on a dualcore x86_64.
(*) At the time I started the tests, 2.6.21-rc7-cfs-v6 was not yet
available. I don't know whether the changes from -v6-rc2 would
have had an impact on the results.
The 3 tasks perform extensive computations on some data that is
initially loaded from disk. Each completed run (1711 per task in
total) in saved in a PostgreSQL DB running on the same machine
I do provide actual start and end timestamp of each task.
Each task is run via 'time'; those figures are provided as well.
Last not least the timevalues given from "within" the perl script
are provided by calling the perl builtin function "times" which
(hopefully) gives user time (among other things).
Note that both mainline and cfs did finish the jobs in the order
LTMM, LTMB, LTBM while sd had them finished in the order LTMM,
LTBM, LTMB. Also sd had the shortest time to have all three
finished while cfs was the first to finish LTMM (the fastest of
the three jobs).
I don't really know the reason for these differences:
LTMM, LTMB and LTBM all do similar things though there are differences.
LTMM is indeed the job that involves the least computations but I have
not yet tried to find out in what order of magnitude. I could if that
would be of interest though it would involve extensive analysis and
partly depends on the data processed and I'm not keen to do that.
I have not extensively checked how stable these figures are w/r to
repeated reruns as a single run takes roughly 3h, but I have repeated
one run and that produced basically the same timing.
One remark regarding cfs with X niced to -10 and 0:
I have not really seen much difference in terms of usage experience
between the two. If at all I have the impression that at nice -10
under KDE some apps (namely kontact and the helper it is using) have
a higher probability to actually "take over" the desktop but so far
this is just a vague impression (read: have not been able to reproduce).
On the other hand I have not seen degradations with nice 0 either and
thus my gut feeling prefers cfs nice 0. Thanks to the bootparam
privileged_nice_level=0 this is easy to achieve.
When comparing cfs nice 0 and sd I have not been able to see any
real difference w/r to desktop responsiveness. But then I have not
tried to stresstest the desktop while running the above benchmarks
either.
Anyway, here are the numbers. Feel free to ask whenever something
is unclear. GC2 is the name of the dataset processed.
2.6.21-rc7-cfs-v6-rc2 (X @ nice -10)
====================================
/proc/sys/kernel/sched_granularity_ns=6000000
Start of all 3 jobs: 10:03:01.933
Job LTMM End: 12:25:57.214
GC2 20070101 LTMM Thu Apr 26 12:25:20.844 2007: 142.42 Min., 4.9942 sec/run (1711 runs)
Ranking-3.pl fertig nach 5484.580s(CPU) und 8574 Sek.
times: bewerteUmgeb # 1 Su 0.79 ~ 0.7900
times: evalMarkt # 1711 Su 5469.91 ~ 3.1969
times: writeDB # 1711 Su 0.26 ~ 0.0002
real 142m56.806s
user 91m24.871s
sys 0m13.257s
Job LTMB End: 12:57:14.624
GC2 20070101 LTMB Thu Apr 26 12:56:52.295 2007: 173.90 Min., 6.0982 sec/run (1711 runs)
Ranking-3.pl fertig nach 7544.170s(CPU) und 10449 Sek.
times: bewerteUmgeb # 1 Su 1.37 ~ 1.3700
times: evalMarkt # 1711 Su 7529.12 ~ 4.4004
times: writeDB # 1711 Su 0.27 ~ 0.0002
real 174m10.404s
user 125m44.484s
sys 0m15.293s
Job LTBM End: 12:59:47.879
GC2 20070101 LTBM Thu Apr 26 12:59:30.257 2007: 176.55 Min., 6.1911 sec/run (1711 runs)
Ranking-3.pl fertig nach 7584.040s(CPU) und 10607 Sek.
times: bewerteUmgeb # 1 Su 0.82 ~ 0.8200
times: evalMarkt # 1711 Su 7569.56 ~ 4.4241
times: writeDB # 1711 Su 0.25 ~ 0.0001
real 176m50.446s
user 126m24.490s
sys 0m15.689s
From fairly early during the job:
top - 10:07:02 up 1:07, 9 users, load average: 3.18, 1.83, 0.84
Tasks: 3 total, 3 running, 0 sleeping, 0 stopped, 0 zombie
Cpu(s): 99.3%us, 0.2%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.2%hi, 0.3%si, 0.0%st
Mem: 3348632k total, 1377624k used, 1971008k free, 51748k buffers
Swap: 2097144k total, 0k used, 2097144k free, 674288k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
13938 mgd 20 0 92780 17m 3668 R 100 0.5 2:31.04 perl
13937 mgd 20 0 93884 18m 3672 R 50 0.6 2:51.31 perl
13940 mgd 20 0 93864 18m 3668 R 50 0.6 2:28.69 perl
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
3 0 0 1954320 52516 688484 0 0 0 71 430 927 100 0 0 0
4 0 0 1954412 52532 688496 0 0 0 49 433 942 100 0 0 0
3 0 0 1954012 52556 688492 0 0 0 159 452 1203 100 0 0 0
3 0 0 1953924 52584 688476 0 0 0 141 444 961 100 0 0 0
3 0 0 1953964 52584 688512 0 0 0 0 470 1170 100 0 0 0
3 0 0 1953980 52600 688516 0 0 0 83 490 1144 100 0 0 0
3 0 0 1953948 52600 688524 0 0 0 0 503 1303 100 0 0 0
3 0 0 1953852 52624 688524 0 0 0 97 503 1394 100 0 0 0
4 0 0 1953852 52640 688512 0 0 0 96 496 1474 100 0 0 0
3 0 0 1953332 52656 688524 0 0 3 93 487 1556 100 0 0 0
5 0 0 1953272 52672 688536 0 0 0 65 494 1483 100 0 0 0
3 0 0 1953264 52684 688548 0 0 0 39 486 1452 100 0 0 0
3 0 0 1953216 52692 688576 0 0 3 44 436 983 100 0 0 0
5 0 0 1953264 52708 688564 0 0 0 61 437 911 100 0 0 0
3 0 0 1953264 52708 688580 0 0 0 0 420 896 100 0 0 0
3 0 0 1953172 52732 688584 0 0 0 67 437 913 100 0 0 0
5 0 0 1952948 52740 688584 0 0 0 73 466 1720 99 1 0 0
3 0 0 1952996 52772 688572 0 0 1 109 470 1173 99 0 1 0
3 0 0 1952996 52788 688580 0 0 0 47 479 1491 89 10 1 0
3 0 0 1952996 52796 688588 0 0 0 19 464 1101 95 5 0 0
3 0 0 1952964 52836 688584 0 0 0 179 461 1072 100 0 0 0
2.6.21-rc7-cfs-v6-rc2 (X @ nice 0)
==================================
/proc/sys/kernel/sched_granularity_ns=6000000
Start of all 3 jobs: 15:13:57.802
Job LTMM End: 17:19:53.494
GC2 20070101 LTMM Wed Apr 25 17:19:18.802 2007: 125.45 Min., 4.3992 sec/run (1711 runs)
Ranking-3.pl finished after 5472.800s(CPU) and 7555 Sec.
times: bewerteUmgeb # 1 Su 0.79 ~ 0.7900
times: evalMarkt # 1711 Su 5458.31 ~ 3.1901
times: writeDB # 1711 Su 0.25 ~ 0.0001
real 125m58.235s
user 91m13.034s
sys 0m15.925s
Job LTMB End: 18:04:45.556
GC2 20070101 LTMB Wed Apr 25 18:04:22.411 2007: 170.47 Min., 5.9778 sec/run (1711 runs)
Ranking-3.pl finished after 7646.700s(CPU) and 10243 Sec.
times: bewerteUmgeb # 1 Su 0.98 ~ 0.9800
times: evalMarkt # 1711 Su 7632.08 ~ 4.4606
times: writeDB # 1711 Su 0.19 ~ 0.0001
real 170m48.445s
user 127m26.950s
sys 0m13.705s
Job LTBM End: 18:13:45.927
GC2 20070101 LTBM Wed Apr 25 18:13:27.382 2007: 179.57 Min., 6.2969 sec/run (1711 runs)
Ranking-3.pl finished after 7623.300s(CPU) and 10789 Sec.
times: bewerteUmgeb # 1 Su 0.85 ~ 0.8500
times: evalMarkt # 1711 Su 7607.95 ~ 4.4465
times: writeDB # 1711 Su 0.26 ~ 0.0002
real 179m51.230s
user 127m3.580s
sys 0m10.769s
From fairly early during the job:
top - 15:17:43 up 21 min, 9 users, load average: 3.35, 2.58, 2.10
Tasks: 3 total, 3 running, 0 sleeping, 0 stopped, 0 zombie
Cpu(s): 99.8%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.2%si, 0.0%st
Mem: 3348632k total, 1624044k used, 1724588k free, 64272k buffers
Swap: 2097144k total, 0k used, 2097144k free, 940176k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
30042 mgd 20 0 92752 17m 3668 R 100 0.5 2:51.25 perl
30038 mgd 20 0 93884 18m 3668 R 50 0.6 1:59.39 perl
30040 mgd 20 0 93624 18m 3668 R 50 0.6 2:32.13 perl
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
4 0 0 1720424 65132 940632 0 0 0 1 424 963 100 0 0 0
6 0 0 1720448 65152 940612 0 0 0 79 441 1030 100 0 0 0
7 0 0 1720448 65152 940636 0 0 0 45 442 1003 100 0 0 0
3 0 0 1720424 65168 940620 0 0 0 76 451 1130 100 0 0 0
5 0 0 1720384 65168 940640 0 0 0 0 431 913 100 0 0 0
3 0 0 1720880 65200 940644 0 0 0 105 455 1080 100 0 0 0
3 0 0 1719936 65208 940656 0 0 0 87 442 1302 100 0 0 0
3 0 0 1720060 65216 940660 0 0 0 81 438 973 100 0 0 0
3 0 0 1720560 65244 940644 0 0 0 88 437 998 100 0 0 0
3 0 0 1720548 65252 940664 0 0 0 59 435 955 100 0 0 0
3 0 0 1720548 65260 940664 0 0 0 39 429 918 100 0 0 0
3 0 0 1720572 65268 940688 0 0 3 33 434 935 100 0 0 0
3 0 0 1720572 65276 940696 0 0 0 31 433 969 100 0 0 0
3 0 0 1719928 65296 940700 0 0 0 155 453 1362 99 1 0 0
3 0 0 1719944 65316 940680 0 0 0 69 436 1025 100 0 0 0
4 0 0 1719772 65372 940712 0 0 1 277 460 1207 100 0 0 0
3 0 0 1719756 65380 940704 0 0 0 44 441 1142 100 0 0 0
3 0 0 1719756 65388 940720 0 0 0 47 436 1094 100 0 0 0
3 0 0 1719780 65396 940736 0 0 3 35 439 1043 100 0 0 0
3 0 0 1719400 65412 940736 0 0 0 87 616 1306 100 1 0 0
3 0 0 1719400 65420 940736 0 0 0 51 440 1033 100 0 0 0
2.6.21-rc7-sd046
================
/proc/sys/kernel/rr_interval=16
Start of all 3 jobs: 11:39:54.650
Job LTMM End: 14:00:07.868
GC2 20070101 LTMM Wed Apr 25 13:59:32.209 2007: 139.73 Min., 4.9001 sec/run (1711 runs)
Ranking-3.pl finished after 5448.340s(CPU) and 8413 Sec.
times: bewerteUmgeb # 1 Su 0.38 ~ 0.3800
times: evalMarkt # 1711 Su 5434.63 ~ 3.1763
times: writeDB # 1711 Su 0.21 ~ 0.0001
real 140m16.127s
user 90m48.597s
sys 0m12.389s
Job LTBM End: 14:32:40.408
GC2 20070101 LTBM Wed Apr 25 14:32:19.259 2007: 172.48 Min., 6.0485 sec/run (1711 runs)
Ranking-3.pl finished after 7555.780s(CPU) and 10365 Sec.
times: bewerteUmgeb # 1 Su 0.39 ~ 0.3900
times: evalMarkt # 1711 Su 7541.47 ~ 4.4076
times: writeDB # 1711 Su 0.18 ~ 0.0001
real 172m48.009s
user 125m56.188s
sys 0m16.689s
Job LTMB End: 14:35:07.261
GC2 20070101 LTMB Wed Apr 25 14:34:50.896 2007: 175.02 Min., 6.1373 sec/run (1711 runs)
Ranking-3.pl finished after 7612.750s(CPU) and 10514 Sec.
times: bewerteUmgeb # 1 Su 0.33 ~ 0.3300
times: evalMarkt # 1711 Su 7599.43 ~ 4.4415
times: writeDB # 1711 Su 0.29 ~ 0.0002
real 175m17.481s
user 126m53.160s
sys 0m16.437s
From fairly early during the job:
top - 11:43:24 up 1:43, 9 users, load average: 3.86, 2.35, 1.21
Tasks: 3 total, 3 running, 0 sleeping, 0 stopped, 0 zombie
Cpu(s): 99.5%us, 0.2%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.3%si, 0.0%st
Mem: 3348628k total, 3212640k used, 135988k free, 435744k buffers
Swap: 2097144k total, 0k used, 2097144k free, 1089812k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
17563 mgd 11 0 92872 17m 3668 R 100 0.5 2:43.97 perl
17568 mgd 33 0 93932 18m 3668 R 50 0.6 1:48.49 perl
17570 mgd 31 0 93652 18m 3668 R 50 0.6 2:14.16 perl
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
4 0 0 125052 436832 1093420 0 0 0 87 932 1464 99 1 0 0
7 0 0 125060 436832 1093428 0 0 0 0 933 1306 99 1 0 0
3 0 0 125256 436852 1093408 0 0 0 120 831 1336 99 1 0 0
3 0 0 124804 436864 1093440 0 0 0 165 443 1374 99 1 0 0
3 0 0 124804 436864 1093440 0 0 0 0 421 955 100 0 0 0
4 0 0 124812 436884 1093440 0 0 0 113 440 989 100 0 0 0
4 0 0 124812 436884 1093440 0 0 0 63 436 936 100 0 0 0
3 0 0 124804 436900 1093448 0 0 0 64 485 1142 100 0 0 0
3 0 0 124804 436912 1093436 0 0 0 47 434 992 100 0 0 0
3 0 0 124804 436920 1093448 0 0 0 24 448 978 100 0 0 0
3 0 0 124656 436932 1093448 0 0 0 120 443 1850 99 1 0 0
3 0 0 124648 436932 1093464 0 0 0 0 441 1081 99 1 0 0
8 0 0 124688 436952 1093464 0 0 0 156 432 924 100 0 0 0
4 0 0 124976 436960 1093472 0 0 0 43 439 1028 100 0 0 0
3 0 0 125100 436968 1093472 0 0 0 27 426 961 100 0 0 0
10 0 0 125132 436976 1093480 0 0 0 43 427 876 100 0 0 0
6 0 0 125132 436984 1093480 0 0 0 28 440 1087 100 0 0 0
3 0 0 124868 437000 1093484 0 0 0 133 445 1629 100 0 0 0
3 0 0 124392 437056 1093456 0 0 0 356 483 1072 99 1 0 0
3 0 0 124400 437056 1093496 0 0 0 0 429 1069 100 0 0 0
3 0 0 124400 437072 1093488 0 0 0 80 441 1000 100 0 0 0
2.6.21-rc7
==========
Start of all 3 jobs: 20:11:45.971
Job LTMM End: 22:25:33.108
GC2 20070101 LTMM Wed Apr 25 22:25:07.427 2007: 133.45 Min., 4.6797 sec/run (1711 runs)
Ranking-3.pl finished after 5462.240s(CPU) and 8026 Sek.
times: bewerteUmgeb # 1 Su 0.77 ~ 0.7700
times: evalMarkt # 1711 Su 5447.89 ~ 3.1840
times: writeDB # 1711 Su 0.29 ~ 0.0002
real 133m50.274s
user 91m2.485s
sys 0m10.801s
Job LTMB End: 23:01:54.763
GC2 20070101 LTMB Wed Apr 25 23:01:35.274 2007: 169.88 Min., 5.9573 sec/run (1711 runs)
Ranking-3.pl finished after 7452.300s(CPU) and 10208 Sek.
times: bewerteUmgeb # 1 Su 0.76 ~ 0.7600
times: evalMarkt # 1711 Su 7438.33 ~ 4.3474
times: writeDB # 1711 Su 0.25 ~ 0.0001
real 170m12.885s
user 124m12.562s
sys 0m14.705s
Job LTBM End: 23:09:16.506
GC2 20070101 LTBM Wed Apr 25 23:08:59.376 2007: 177.28 Min., 6.2168 sec/run (1711 runs)
Ranking-3.pl finished after 7673.940s(CPU) and 10652 Sek.
times: bewerteUmgeb # 1 Su 0.78 ~ 0.7800
times: evalMarkt # 1711 Su 7659.10 ~ 4.4764
times: writeDB # 1711 Su 0.30 ~ 0.0002
real 177m35.890s
user 127m54.376s
sys 0m9.873s
From fairly early during the job:
top - 20:15:13 up 12 min, 9 users, load average: 3.89, 2.61, 1.52
Tasks: 3 total, 3 running, 0 sleeping, 0 stopped, 0 zombie
Cpu(s): 99.8%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.2%si, 0.0%st
Mem: 3348628k total, 1109020k used, 2239608k free, 28820k buffers
Swap: 2097144k total, 0k used, 2097144k free, 511352k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
6389 mgd 25 0 92776 17m 3668 R 100 0.5 2:50.23 perl
6390 mgd 25 0 93932 18m 3668 R 53 0.6 2:08.47 perl
6393 mgd 25 0 93880 18m 3668 R 47 0.6 1:42.53 perl
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
6 0 0 2211984 30780 525612 0 0 0 0 490 1318 100 0 0 0
4 0 0 2211460 30872 525576 0 0 8 795 489 1052 100 0 0 0
3 0 0 2211736 30896 525620 0 0 0 129 467 1003 99 1 0 0
7 0 0 2211024 30896 525624 0 0 0 97 469 1234 100 0 0 0
4 0 0 2211080 30924 525624 0 0 0 147 472 1117 100 0 0 0
4 0 0 2211008 30936 525616 0 0 0 35 467 1086 100 0 0 0
5 0 0 2210792 30976 525632 0 0 0 144 480 1120 100 0 0 0
5 0 0 2210932 30992 525620 0 0 0 47 462 1099 99 1 0 0
3 0 0 2210948 31000 525644 0 0 0 40 460 1087 100 0 0 0
4 0 0 2210980 31016 525648 0 0 0 41 453 898 100 0 0 0
4 0 0 2210716 31036 525648 0 0 1 99 471 1322 100 0 0 0
4 0 0 2210748 31052 525648 0 0 0 60 454 1085 100 0 0 0
5 0 0 2210764 31080 525652 0 0 0 68 463 1007 100 0 0 0
5 0 0 2210796 31080 525656 0 0 0 68 462 962 100 0 0 0
5 0 0 2210812 31104 525664 0 0 0 97 470 1193 100 0 0 0
5 0 0 2210812 31104 525664 0 0 0 0 451 911 100 0 0 0
4 0 0 2210812 31128 525672 0 0 0 69 465 1100 100 0 0 0
3 0 0 2210300 31152 525660 0 0 0 143 466 1350 99 1 0 0
4 0 0 2210292 31152 525676 0 0 3 9 513 2025 100 0 0 0
6 0 0 2210624 31196 525700 0 0 3 133 492 1207 100 0 0 0
4 0 0 2210460 31196 525720 0 0 0 0 458 1086 100 0 0 0
Best,
Michael
--
Technosis GmbH, Geschäftsführer: Michael Gerdau, Tobias Dittmar
Sitz Hamburg; HRB 89145 Amtsgericht Hamburg
Vote against SPAM - see http://www.politik-digital.de/spam/
Michael Gerdau email: mgd@technosis.de
GPG-keys available on request or at public keyserver
[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: [REPORT] cfs-v6-rc2 vs sd-0.46 vs 2.6.21-rc7
2007-04-26 11:12 [REPORT] cfs-v6-rc2 vs sd-0.46 vs 2.6.21-rc7 Michael Gerdau
@ 2007-04-26 12:07 ` Ingo Molnar
2007-04-26 13:22 ` Michael Gerdau
2007-04-26 22:59 ` [ck] " Con Kolivas
0 siblings, 2 replies; 7+ messages in thread
From: Ingo Molnar @ 2007-04-26 12:07 UTC (permalink / raw)
To: Michael Gerdau
Cc: linux-kernel, Linus Torvalds, Nick Piggin, Gene Heskett,
Juliusz Chroboczek, Mike Galbraith, Peter Williams, ck list,
Thomas Gleixner, William Lee Irwin III, Andrew Morton,
Bill Davidsen, Willy Tarreau, Arjan van de Ven
* Michael Gerdau <mgd@technosis.de> wrote:
> Hi list,
>
> find below a test comparing
> 2.6.21-rc7 (mainline)
> 2.6.21-rc7-sd046
> 2.6.21-rc7-cfs-v6-rc2(*) (X @ nice 0)
> 2.6.21-rc7-cfs-v6-rc2(*) (X @ nice -10)
> running on a dualcore x86_64.
thanks for the testing!
as a summary: i think your numbers demonstrate it nicely that the
shorter 'timeslice length' that both CFS and SD utilizes does not have a
measurable negative impact on your workload. To measure the total impact
of 'timeslicing' you might want to try the exact same workload with a
much higher 'timeslice length' of say 400 msecs, via:
echo 400000000 > /proc/sys/kernel/sched_granularity_ns # on CFS
echo 400 > /proc/sys/kernel/rr_interval # on SD
your existing numbers are a bit hard to analyze because the 3 workloads
were started at the same time and they overlapped differently and
utilized the system differently.
i think the primary number that makes sense to look at (which is perhaps
the least sensitive to the 'overlap effect') is the 'combined user times
of all 3 workloads' (in order of performance):
> 2.6.21-rc7: 20589.423 100.00%
> 2.6.21-rc7-cfs-v6-rc2 (X @ nice -10): 20613.845 99.88%
> 2.6.21-rc7-sd046: 20617.945 99.86%
> 2.6.21-rc7-cfs-v6-rc2 (X @ nice 0): 20743.564 99.25%
to me this gives the impression that it's all "within noise". In
particular the two CFS results suggest that there's at least a ~100
seconds noise in these results, because the renicing of X should have no
impact on the result (the workloads are pure number-crunchers, and all
use up the CPUs 100%, correct?), and even if it has an impact, renicing
X to nice 0 should _speed up_ the result - not slow it down a bit like
the numbers suggest.
another (perhaps less reliable) number is the total wall-clock runtime
of all 3 jobs. Provided i did not make any mistakes in my calculations,
here are the results:
> 2.6.21-rc7-sd046: 10512.611 seconds
> 2.6.21-rc7-cfs-v6-rc2 (X @ nice -10): 10605.946 seconds
> 2.6.21-rc7: 10650.535 seconds
> 2.6.21-rc7-cfs-v6-rc2 (X @ nice 0): 10788.125 seconds
(the numbers are lower than the first numbers because this is a 2 CPU
system)
both SD and CFS-nice-10 was faster than mainline, but i'd say this too
is noise - especially because this result highly depends on the way the
workloads overlap in general, which seems to be different for SD.
system time is interesting too:
> 2.6.21-rc7: 35.379
> 2.6.21-rc7-cfs-v6-rc2 (X @ nice -10): 40.399
> 2.6.21-rc7-cfs-v6-rc2 (X @ nice 0): 44.239
> 2.6.21-rc7-sd046: 45.515
here too the two CFS results seem to suggest that there's at least
around 5 seconds of noise. So i'd not necessarily call it systematic
that vanilla had the lowest system time and SD had the highest.
combined system+user time:
> 2.6.21-rc7: 20624.802
> 2.6.21-rc7-cfs-v6-rc2 (X @ nice 0): 20658.084
> 2.6.21-rc7-sd046: 20663.460
> 2.6.21-rc7-cfs-v6-rc2 (X @ nice -10): 20783.963
perhaps it might make more sense to run the workloads serialized, to
have better comparabality of the individual workloads. (on a real system
you'd naturally want to overlap these workloads to utilize the CPUs, so
the numbers you did are very relevant too.)
The vmstat suggested there is occasional idle time in the system - is
the workload IO-bound (or memory bound) in those cases?
Ingo
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: [REPORT] cfs-v6-rc2 vs sd-0.46 vs 2.6.21-rc7
2007-04-26 12:07 ` Ingo Molnar
@ 2007-04-26 13:22 ` Michael Gerdau
2007-04-26 13:55 ` Ingo Molnar
2007-04-26 22:59 ` [ck] " Con Kolivas
1 sibling, 1 reply; 7+ messages in thread
From: Michael Gerdau @ 2007-04-26 13:22 UTC (permalink / raw)
To: Ingo Molnar
Cc: linux-kernel, Linus Torvalds, Nick Piggin, Gene Heskett,
Juliusz Chroboczek, Mike Galbraith, Peter Williams, ck list,
Thomas Gleixner, William Lee Irwin III, Andrew Morton,
Bill Davidsen, Willy Tarreau, Arjan van de Ven
[-- Attachment #1: Type: text/plain, Size: 5860 bytes --]
> as a summary: i think your numbers demonstrate it nicely that the
> shorter 'timeslice length' that both CFS and SD utilizes does not have a
> measurable negative impact on your workload.
That's my impression as well. In particular I think that for this
type of workload it is pretty much irrelevant which scheduler I'm
using :-)
> To measure the total impact
> of 'timeslicing' you might want to try the exact same workload with a
> much higher 'timeslice length' of say 400 msecs, via:
>
> echo 400000000 > /proc/sys/kernel/sched_granularity_ns # on CFS
> echo 400 > /proc/sys/kernel/rr_interval # on SD
I just finished building cfs and sd against 2.6.21 and will try with
these over the next days.
> your existing numbers are a bit hard to analyze because the 3 workloads
> were started at the same time and they overlapped differently and
> utilized the system differently.
Right. However the differences w/r to which job finished when, in
particular the change in who "came in" 2. and 3. between sd and cfs
had been consistent with other similar such jobs I ran over the last
couple of days.
In general sd tends to finish all three such jobs at roughly the
same time while cfs clearly "favors" the LTMM-type jobs (which
admittedly involve the least computations). I don't really know
why that is so. However the jobs does some minimal I/O after each
successfully finish run and that might be different w/r to sd and
cfs. Not really knowing what both scheduler do internally I'm in
no position to discuss that with either you or Con :)
> i think the primary number that makes sense to look at (which is perhaps
> the least sensitive to the 'overlap effect') is the 'combined user times
> of all 3 workloads' (in order of performance):
>
> > 2.6.21-rc7: 20589.423 100.00%
> > 2.6.21-rc7-cfs-v6-rc2 (X @ nice -10): 20613.845 99.88%
> > 2.6.21-rc7-sd046: 20617.945 99.86%
> > 2.6.21-rc7-cfs-v6-rc2 (X @ nice 0): 20743.564 99.25%
>
> to me this gives the impression that it's all "within noise".
I think so too. But then I didn't seriously expect any different.
> In
> particular the two CFS results suggest that there's at least a ~100
> seconds noise in these results, because the renicing of X should have no
> impact on the result (the workloads are pure number-crunchers, and all
> use up the CPUs 100%, correct?),
Correct.
The differences could as well result from small fluctuations in my
work on the machine during the test (like reading mail, editing sources
and similar stuff).
> another (perhaps less reliable) number is the total wall-clock runtime
> of all 3 jobs. Provided i did not make any mistakes in my calculations,
> here are the results:
>
> > 2.6.21-rc7-sd046: 10512.611 seconds
> > 2.6.21-rc7-cfs-v6-rc2 (X @ nice -10): 10605.946 seconds
> > 2.6.21-rc7: 10650.535 seconds
> > 2.6.21-rc7-cfs-v6-rc2 (X @ nice 0): 10788.125 seconds
>
> (the numbers are lower than the first numbers because this is a 2 CPU
> system)
>
> both SD and CFS-nice-10 was faster than mainline, but i'd say this too
> is noise - especially because this result highly depends on the way the
> workloads overlap in general, which seems to be different for SD.
I don't think that's noise. As I wrote above IMO this is the one
difference I personally consider significant. While running I could
watch intermediate output of all 3 jobs intermixed on a console.
With sd the jobs were -within reason- head to head while with cfs
and mainline LTMM quickly got ahead and remained there.
As I speculated above this (IMO real) difference in the way sd and
cfs (and mainline) schedule the jobs might be related to the way
the schedulers react on I/O but I don't know.
> system time is interesting too:
>
> > 2.6.21-rc7: 35.379
> > 2.6.21-rc7-cfs-v6-rc2 (X @ nice -10): 40.399
> > 2.6.21-rc7-cfs-v6-rc2 (X @ nice 0): 44.239
> > 2.6.21-rc7-sd046: 45.515
Over the weekend I might try to repeat these tests while doing
nothing else on the machine (i.e. no editing or reading mail).
> combined system+user time:
>
> > 2.6.21-rc7: 20624.802
> > 2.6.21-rc7-cfs-v6-rc2 (X @ nice 0): 20658.084
> > 2.6.21-rc7-sd046: 20663.460
> > 2.6.21-rc7-cfs-v6-rc2 (X @ nice -10): 20783.963
>
> perhaps it might make more sense to run the workloads serialized, to
> have better comparabality of the individual workloads. (on a real system
> you'd naturally want to overlap these workloads to utilize the CPUs, so
> the numbers you did are very relevant too.)
Yes. In fact the workloads used to be run serialized and only
recently I tried to easily have them run on multicore systems.
The whole set of such jobs consists of 312 tasks and thus the
idea is to get a couple of PS3 and have them run there...
(once I found the time to port the software).
> The vmstat suggested there is occasional idle time in the system - is
> the workload IO-bound (or memory bound) in those cases?
There is writing out stats after each run (i.e. every 5-8 sec) both
to the PostgreSQL DB as well as to the console.
I could get rid of the console I/O (i.e. make it conditional) and
for testing purpose could switch off the DB if that would make be
helpful.
Best,
Michael
--
Technosis GmbH, Geschäftsführer: Michael Gerdau, Tobias Dittmar
Sitz Hamburg; HRB 89145 Amtsgericht Hamburg
Vote against SPAM - see http://www.politik-digital.de/spam/
Michael Gerdau email: mgd@technosis.de
GPG-keys available on request or at public keyserver
[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [REPORT] cfs-v6-rc2 vs sd-0.46 vs 2.6.21-rc7
2007-04-26 13:22 ` Michael Gerdau
@ 2007-04-26 13:55 ` Ingo Molnar
0 siblings, 0 replies; 7+ messages in thread
From: Ingo Molnar @ 2007-04-26 13:55 UTC (permalink / raw)
To: Michael Gerdau
Cc: linux-kernel, Linus Torvalds, Nick Piggin, Gene Heskett,
Juliusz Chroboczek, Mike Galbraith, Peter Williams, ck list,
Thomas Gleixner, William Lee Irwin III, Andrew Morton,
Bill Davidsen, Willy Tarreau, Arjan van de Ven
* Michael Gerdau <mgd@technosis.de> wrote:
> In general sd tends to finish all three such jobs at roughly the same
> time while cfs clearly "favors" the LTMM-type jobs (which admittedly
> involve the least computations). I don't really know why that is so.
for the reason of this, look at the raw user runtimes the 3 jobs have:
5498.128 secs # LTMM
7559.777 secs
7600.179 secs
the "perfect scheduler" would run each of the jobs at 33.33% of capacity
for ~5500 CPU-seconds, and would then run the remaining two jobs at
50.0% capacity for about ~2075 CPU-seconds.
Why? Because the scheduler how no idea how much each job takes! So a
fair scheduler would run: 3 jobs at 33.33% capacity for as long as the
shortest job ends, then the remaining 2 jobs at 50% capacity for as the
shorter one of the remaining 2 finishes, and the remaining one at 100%.
in your case that means that the best scheduling would be roughly the
following ideal timeline:
5500*3 / 2 == 8250 seconds for the LTMM to finish
2075*2 / 2 == +2075 more seconds for the other two jobs to finish.
the various runs showed the following wallclock-time timeline for the
LTMM phase:
CFS #1: real 142m56.806s
CFS #2: real 125m58.235s
SD: real 140m16.127s
vanilla: real 133m50.274s
the "ideal" is ~137 minutes (8250 seconds) for the LTMM phase. The
closest was indeed SD, but vanilla and cfs #1 wasnt too far away from it
either. [ and the variance between CFS #1 and #2 seems to suggest that
the noise of this particular metric is significant. The average does
come in at 134.5, which is quite close to the ideal number. ]
Ingo
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [ck] Re: [REPORT] cfs-v6-rc2 vs sd-0.46 vs 2.6.21-rc7
2007-04-26 12:07 ` Ingo Molnar
2007-04-26 13:22 ` Michael Gerdau
@ 2007-04-26 22:59 ` Con Kolivas
2007-04-27 5:59 ` Michael Gerdau
2007-04-27 6:52 ` Ingo Molnar
1 sibling, 2 replies; 7+ messages in thread
From: Con Kolivas @ 2007-04-26 22:59 UTC (permalink / raw)
To: ck
Cc: Ingo Molnar, Michael Gerdau, Nick Piggin, Bill Davidsen,
Juliusz Chroboczek, Mike Galbraith, linux-kernel,
William Lee Irwin III, Peter Williams, Gene Heskett,
Willy Tarreau, Thomas Gleixner, Linus Torvalds, Andrew Morton,
Arjan van de Ven
On Thursday 26 April 2007 22:07, Ingo Molnar wrote:
> * Michael Gerdau <mgd@technosis.de> wrote:
> > Hi list,
> >
> > find below a test comparing
> > 2.6.21-rc7 (mainline)
> > 2.6.21-rc7-sd046
> > 2.6.21-rc7-cfs-v6-rc2(*) (X @ nice 0)
> > 2.6.21-rc7-cfs-v6-rc2(*) (X @ nice -10)
> > running on a dualcore x86_64.
>
> thanks for the testing!
Very interesting indeed but fairly complicated as well.
> as a summary: i think your numbers demonstrate it nicely that the
> shorter 'timeslice length' that both CFS and SD utilizes does not have a
> measurable negative impact on your workload. To measure the total impact
> of 'timeslicing' you might want to try the exact same workload with a
> much higher 'timeslice length' of say 400 msecs, via:
>
> echo 400000000 > /proc/sys/kernel/sched_granularity_ns # on CFS
> echo 400 > /proc/sys/kernel/rr_interval # on SD
I thought that the effective "timeslice" on CFS was double the
sched_granularity_ns so wouldn't this make the effective timeslice double
that of what you're setting SD to? Anyway the difference between 400 and
800ms timeslices is unlikely to be significant so I don't mind.
--
-ck
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [ck] Re: [REPORT] cfs-v6-rc2 vs sd-0.46 vs 2.6.21-rc7
2007-04-26 22:59 ` [ck] " Con Kolivas
@ 2007-04-27 5:59 ` Michael Gerdau
2007-04-27 6:52 ` Ingo Molnar
1 sibling, 0 replies; 7+ messages in thread
From: Michael Gerdau @ 2007-04-27 5:59 UTC (permalink / raw)
To: Con Kolivas
Cc: ck, Ingo Molnar, Nick Piggin, Bill Davidsen, Juliusz Chroboczek,
Mike Galbraith, linux-kernel, William Lee Irwin III,
Peter Williams, Gene Heskett, Willy Tarreau, Thomas Gleixner,
Linus Torvalds, Andrew Morton, Arjan van de Ven
[-- Attachment #1: Type: text/plain, Size: 1650 bytes --]
> Very interesting indeed but fairly complicated as well.
Sorry for that -- I've taken these figures from the 3MB logfile
that each job creates and "reading" them on a regular basis tend
to forget that probably everyody else does not find them as
obvious as I do. Also I'm don't really have lots of experience
with how a scheduler is properly tested.
For any upcoming tests I will restrict the numbers to wallclock
and what time provides which probably is better suited anyway.
> > as a summary: i think your numbers demonstrate it nicely that the
> > shorter 'timeslice length' that both CFS and SD utilizes does not have a
> > measurable negative impact on your workload. To measure the total impact
> > of 'timeslicing' you might want to try the exact same workload with a
> > much higher 'timeslice length' of say 400 msecs, via:
> >
> > echo 400000000 > /proc/sys/kernel/sched_granularity_ns # on CFS
> > echo 400 > /proc/sys/kernel/rr_interval # on SD
>
> I thought that the effective "timeslice" on CFS was double the
> sched_granularity_ns so wouldn't this make the effective timeslice double
> that of what you're setting SD to? Anyway the difference between 400 and
> 800ms timeslices is unlikely to be significant so I don't mind.
I'm happy to do that, hopefully over the weekend.
Best,
Michael
--
Technosis GmbH, Geschäftsführer: Michael Gerdau, Tobias Dittmar
Sitz Hamburg; HRB 89145 Amtsgericht Hamburg
Vote against SPAM - see http://www.politik-digital.de/spam/
Michael Gerdau email: mgd@technosis.de
GPG-keys available on request or at public keyserver
[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [ck] Re: [REPORT] cfs-v6-rc2 vs sd-0.46 vs 2.6.21-rc7
2007-04-26 22:59 ` [ck] " Con Kolivas
2007-04-27 5:59 ` Michael Gerdau
@ 2007-04-27 6:52 ` Ingo Molnar
1 sibling, 0 replies; 7+ messages in thread
From: Ingo Molnar @ 2007-04-27 6:52 UTC (permalink / raw)
To: Con Kolivas
Cc: ck, Michael Gerdau, Nick Piggin, Bill Davidsen,
Juliusz Chroboczek, Mike Galbraith, linux-kernel,
William Lee Irwin III, Peter Williams, Gene Heskett,
Willy Tarreau, Thomas Gleixner, Linus Torvalds, Andrew Morton,
Arjan van de Ven
* Con Kolivas <kernel@kolivas.org> wrote:
> > as a summary: i think your numbers demonstrate it nicely that the
> > shorter 'timeslice length' that both CFS and SD utilizes does not have a
> > measurable negative impact on your workload. To measure the total impact
> > of 'timeslicing' you might want to try the exact same workload with a
> > much higher 'timeslice length' of say 400 msecs, via:
> >
> > echo 400000000 > /proc/sys/kernel/sched_granularity_ns # on CFS
> > echo 400 > /proc/sys/kernel/rr_interval # on SD
>
> I thought that the effective "timeslice" on CFS was double the
> sched_granularity_ns so wouldn't this make the effective timeslice
> double that of what you're setting SD to? [...]
The two settings are not really comparable. The "effective timeslice is
the double of the granularity" thing i mentioned before is really a
special-case: only true for a really undisturbed 100% CPU-using
_two-task_ workload, if and only if the workload would not reschedule
otherwise, but that is clearly not the case here: and if you look at the
vmstat output provided by Michael you'll see that all 3 schedulers
rescheduled this workload at around 1000/sec or 1 msec per scheduling
atom. (But i'd agree that to be on the safe side the context-switch rate
has to be monitored and if it seems too high on SD, the rr_interval
should be increased.)
> [...] Anyway the difference between 400 and 800ms timeslices is
> unlikely to be significant so I don't mind.
even on a totally idle system there's at least a 10 Hz 'background
sound' of various activities, so any setting above 100 msecs rarely has
any effect.
Ingo
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2007-04-27 6:53 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2007-04-26 11:12 [REPORT] cfs-v6-rc2 vs sd-0.46 vs 2.6.21-rc7 Michael Gerdau
2007-04-26 12:07 ` Ingo Molnar
2007-04-26 13:22 ` Michael Gerdau
2007-04-26 13:55 ` Ingo Molnar
2007-04-26 22:59 ` [ck] " Con Kolivas
2007-04-27 5:59 ` Michael Gerdau
2007-04-27 6:52 ` Ingo Molnar
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).