LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
* [REPORT] cfs-v6-rc2 vs sd-0.46 vs 2.6.21-rc7
@ 2007-04-26 11:12 Michael Gerdau
  2007-04-26 12:07 ` Ingo Molnar
  0 siblings, 1 reply; 7+ messages in thread
From: Michael Gerdau @ 2007-04-26 11:12 UTC (permalink / raw)
  To: linux-kernel
  Cc: Ingo Molnar, Linus Torvalds, Nick Piggin, Gene Heskett,
	Juliusz Chroboczek, Mike Galbraith, Peter Williams, ck list,
	Thomas Gleixner, William Lee Irwin III, Andrew Morton,
	Bill Davidsen, Willy Tarreau, Arjan van de Ven

[-- Attachment #1: Type: text/plain, Size: 18956 bytes --]

Hi list,

find below a test comparing
    2.6.21-rc7 (mainline)
    2.6.21-rc7-sd046
    2.6.21-rc7-cfs-v6-rc2(*) (X @ nice 0)
    2.6.21-rc7-cfs-v6-rc2(*) (X @ nice -10)
running on a dualcore x86_64.

(*) At the time I started the tests, 2.6.21-rc7-cfs-v6 was not yet
available. I don't know whether the changes from -v6-rc2 would
have had an impact on the results.


The 3 tasks perform extensive computations on some data that is
initially loaded from disk. Each completed run (1711 per task in
total) in saved in a PostgreSQL DB running on the same machine

I do provide actual start and end timestamp of each task.
Each task is run via 'time'; those figures are provided as well.
Last not least the timevalues given from "within" the perl script
are provided by calling the perl builtin function "times" which
(hopefully) gives user time (among other things).

Note that both mainline and cfs did finish the jobs in the order
LTMM, LTMB, LTBM while sd had them finished in the order LTMM,
LTBM, LTMB. Also sd had the shortest time to have all three
finished while cfs was the first to finish LTMM (the fastest of
the three jobs).

I don't really know the reason for these differences:
LTMM, LTMB and LTBM all do similar things though there are differences.
LTMM is indeed the job that involves the least computations but I have
not yet tried to find out in what order of magnitude. I could if that
would be of interest though it would involve extensive analysis and
partly depends on the data processed and I'm not keen to do that.

I have not extensively checked how stable these figures are w/r to
repeated reruns as a single run takes roughly 3h, but I have repeated
one run and that produced basically the same timing.

One remark regarding cfs with X niced to -10 and 0:
I have not really seen much difference in terms of usage experience
between the two. If at all I have the impression that at nice -10
under KDE some apps (namely kontact and the helper it is using) have
a higher probability to actually "take over" the desktop but so far
this is just a vague impression (read: have not been able to reproduce).
On the other hand I have not seen degradations with nice 0 either and
thus my gut feeling prefers cfs nice 0. Thanks to the bootparam
privileged_nice_level=0 this is easy to achieve.

When comparing cfs nice 0 and sd I have not been able to see any
real difference w/r to desktop responsiveness. But then I have not
tried to stresstest the desktop while running the above benchmarks
either.

Anyway, here are the numbers. Feel free to ask whenever something
is unclear. GC2 is the name of the dataset processed.


2.6.21-rc7-cfs-v6-rc2 (X @ nice -10)
====================================
/proc/sys/kernel/sched_granularity_ns=6000000

Start of all 3 jobs: 10:03:01.933

Job LTMM End: 12:25:57.214
GC2 20070101 LTMM Thu Apr 26 12:25:20.844 2007: 142.42 Min.,  4.9942 sec/run (1711 runs)
Ranking-3.pl fertig nach 5484.580s(CPU) und 8574 Sek.
times:                   bewerteUmgeb #        1 Su    0.79 ~  0.7900
times:                      evalMarkt #     1711 Su 5469.91 ~  3.1969
times:                        writeDB #     1711 Su    0.26 ~  0.0002
real	142m56.806s
user	91m24.871s
sys	0m13.257s


Job LTMB End: 12:57:14.624
GC2 20070101 LTMB Thu Apr 26 12:56:52.295 2007: 173.90 Min.,  6.0982 sec/run (1711 runs)
Ranking-3.pl fertig nach 7544.170s(CPU) und 10449 Sek.
times:                   bewerteUmgeb #        1 Su    1.37 ~  1.3700
times:                      evalMarkt #     1711 Su 7529.12 ~  4.4004
times:                        writeDB #     1711 Su    0.27 ~  0.0002
real	174m10.404s
user	125m44.484s
sys	0m15.293s


Job LTBM End: 12:59:47.879
GC2 20070101 LTBM Thu Apr 26 12:59:30.257 2007: 176.55 Min.,  6.1911 sec/run (1711 runs)
Ranking-3.pl fertig nach 7584.040s(CPU) und 10607 Sek.
times:                   bewerteUmgeb #        1 Su    0.82 ~  0.8200
times:                      evalMarkt #     1711 Su 7569.56 ~  4.4241
times:                        writeDB #     1711 Su    0.25 ~  0.0001
real	176m50.446s
user	126m24.490s
sys	0m15.689s


From fairly early during the job:
top - 10:07:02 up  1:07,  9 users,  load average: 3.18, 1.83, 0.84
Tasks:   3 total,   3 running,   0 sleeping,   0 stopped,   0 zombie
Cpu(s): 99.3%us,  0.2%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.2%hi,  0.3%si,  0.0%st
Mem:   3348632k total,  1377624k used,  1971008k free,    51748k buffers
Swap:  2097144k total,        0k used,  2097144k free,   674288k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
13938 mgd       20   0 92780  17m 3668 R  100  0.5   2:31.04 perl
13937 mgd       20   0 93884  18m 3672 R   50  0.6   2:51.31 perl
13940 mgd       20   0 93864  18m 3668 R   50  0.6   2:28.69 perl


procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
 3  0      0 1954320  52516 688484    0    0     0    71  430  927 100  0  0  0
 4  0      0 1954412  52532 688496    0    0     0    49  433  942 100  0  0  0
 3  0      0 1954012  52556 688492    0    0     0   159  452 1203 100  0  0  0
 3  0      0 1953924  52584 688476    0    0     0   141  444  961 100  0  0  0
 3  0      0 1953964  52584 688512    0    0     0     0  470 1170 100  0  0  0
 3  0      0 1953980  52600 688516    0    0     0    83  490 1144 100  0  0  0
 3  0      0 1953948  52600 688524    0    0     0     0  503 1303 100  0  0  0
 3  0      0 1953852  52624 688524    0    0     0    97  503 1394 100  0  0  0
 4  0      0 1953852  52640 688512    0    0     0    96  496 1474 100  0  0  0
 3  0      0 1953332  52656 688524    0    0     3    93  487 1556 100  0  0  0
 5  0      0 1953272  52672 688536    0    0     0    65  494 1483 100  0  0  0
 3  0      0 1953264  52684 688548    0    0     0    39  486 1452 100  0  0  0
 3  0      0 1953216  52692 688576    0    0     3    44  436  983 100  0  0  0
 5  0      0 1953264  52708 688564    0    0     0    61  437  911 100  0  0  0
 3  0      0 1953264  52708 688580    0    0     0     0  420  896 100  0  0  0
 3  0      0 1953172  52732 688584    0    0     0    67  437  913 100  0  0  0
 5  0      0 1952948  52740 688584    0    0     0    73  466 1720 99  1  0  0
 3  0      0 1952996  52772 688572    0    0     1   109  470 1173 99  0  1  0
 3  0      0 1952996  52788 688580    0    0     0    47  479 1491 89 10  1  0
 3  0      0 1952996  52796 688588    0    0     0    19  464 1101 95  5  0  0
 3  0      0 1952964  52836 688584    0    0     0   179  461 1072 100  0  0  0



2.6.21-rc7-cfs-v6-rc2 (X @ nice 0)
==================================
/proc/sys/kernel/sched_granularity_ns=6000000

Start of all 3 jobs: 15:13:57.802

Job LTMM End: 17:19:53.494
GC2 20070101 LTMM Wed Apr 25 17:19:18.802 2007: 125.45 Min.,  4.3992 sec/run (1711 runs)
Ranking-3.pl finished after 5472.800s(CPU) and 7555 Sec.
times:                   bewerteUmgeb #        1 Su    0.79 ~  0.7900
times:                      evalMarkt #     1711 Su 5458.31 ~  3.1901
times:                        writeDB #     1711 Su    0.25 ~  0.0001
real	125m58.235s
user	91m13.034s
sys	0m15.925s


Job LTMB End: 18:04:45.556
GC2 20070101 LTMB Wed Apr 25 18:04:22.411 2007: 170.47 Min.,  5.9778 sec/run (1711 runs)
Ranking-3.pl finished after 7646.700s(CPU) and 10243 Sec.
times:                   bewerteUmgeb #        1 Su    0.98 ~  0.9800
times:                      evalMarkt #     1711 Su 7632.08 ~  4.4606
times:                        writeDB #     1711 Su    0.19 ~  0.0001
real	170m48.445s
user	127m26.950s
sys	0m13.705s


Job LTBM End: 18:13:45.927
GC2 20070101 LTBM Wed Apr 25 18:13:27.382 2007: 179.57 Min.,  6.2969 sec/run (1711 runs)
Ranking-3.pl finished after 7623.300s(CPU) and 10789 Sec.
times:                   bewerteUmgeb #        1 Su    0.85 ~  0.8500
times:                      evalMarkt #     1711 Su 7607.95 ~  4.4465
times:                        writeDB #     1711 Su    0.26 ~  0.0002
real	179m51.230s
user	127m3.580s
sys	0m10.769s


From fairly early during the job:
top - 15:17:43 up 21 min,  9 users,  load average: 3.35, 2.58, 2.10
Tasks:   3 total,   3 running,   0 sleeping,   0 stopped,   0 zombie
Cpu(s): 99.8%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.2%si,  0.0%st
Mem:   3348632k total,  1624044k used,  1724588k free,    64272k buffers
Swap:  2097144k total,        0k used,  2097144k free,   940176k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
30042 mgd       20   0 92752  17m 3668 R  100  0.5   2:51.25 perl
30038 mgd       20   0 93884  18m 3668 R   50  0.6   1:59.39 perl
30040 mgd       20   0 93624  18m 3668 R   50  0.6   2:32.13 perl


procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
 4  0      0 1720424  65132 940632    0    0     0     1  424  963 100  0  0  0
 6  0      0 1720448  65152 940612    0    0     0    79  441 1030 100  0  0  0
 7  0      0 1720448  65152 940636    0    0     0    45  442 1003 100  0  0  0
 3  0      0 1720424  65168 940620    0    0     0    76  451 1130 100  0  0  0
 5  0      0 1720384  65168 940640    0    0     0     0  431  913 100  0  0  0
 3  0      0 1720880  65200 940644    0    0     0   105  455 1080 100  0  0  0
 3  0      0 1719936  65208 940656    0    0     0    87  442 1302 100  0  0  0
 3  0      0 1720060  65216 940660    0    0     0    81  438  973 100  0  0  0
 3  0      0 1720560  65244 940644    0    0     0    88  437  998 100  0  0  0
 3  0      0 1720548  65252 940664    0    0     0    59  435  955 100  0  0  0
 3  0      0 1720548  65260 940664    0    0     0    39  429  918 100  0  0  0
 3  0      0 1720572  65268 940688    0    0     3    33  434  935 100  0  0  0
 3  0      0 1720572  65276 940696    0    0     0    31  433  969 100  0  0  0
 3  0      0 1719928  65296 940700    0    0     0   155  453 1362 99  1  0  0
 3  0      0 1719944  65316 940680    0    0     0    69  436 1025 100  0  0  0
 4  0      0 1719772  65372 940712    0    0     1   277  460 1207 100  0  0  0
 3  0      0 1719756  65380 940704    0    0     0    44  441 1142 100  0  0  0
 3  0      0 1719756  65388 940720    0    0     0    47  436 1094 100  0  0  0
 3  0      0 1719780  65396 940736    0    0     3    35  439 1043 100  0  0  0
 3  0      0 1719400  65412 940736    0    0     0    87  616 1306 100  1  0  0
 3  0      0 1719400  65420 940736    0    0     0    51  440 1033 100  0  0  0



2.6.21-rc7-sd046
================
/proc/sys/kernel/rr_interval=16

Start of all 3 jobs: 11:39:54.650

Job LTMM End: 14:00:07.868
GC2 20070101 LTMM Wed Apr 25 13:59:32.209 2007: 139.73 Min.,  4.9001 sec/run (1711 runs)
Ranking-3.pl finished after 5448.340s(CPU) and 8413 Sec.
times:                   bewerteUmgeb #        1 Su    0.38 ~  0.3800
times:                      evalMarkt #     1711 Su 5434.63 ~  3.1763
times:                        writeDB #     1711 Su    0.21 ~  0.0001
real	140m16.127s
user	90m48.597s
sys	0m12.389s


Job LTBM End: 14:32:40.408
GC2 20070101 LTBM Wed Apr 25 14:32:19.259 2007: 172.48 Min.,  6.0485 sec/run (1711 runs)
Ranking-3.pl finished after 7555.780s(CPU) and 10365 Sec.
times:                   bewerteUmgeb #        1 Su    0.39 ~  0.3900
times:                      evalMarkt #     1711 Su 7541.47 ~  4.4076
times:                        writeDB #     1711 Su    0.18 ~  0.0001
real	172m48.009s
user	125m56.188s
sys	0m16.689s


Job LTMB End: 14:35:07.261
GC2 20070101 LTMB Wed Apr 25 14:34:50.896 2007: 175.02 Min.,  6.1373 sec/run (1711 runs)
Ranking-3.pl finished after 7612.750s(CPU) and 10514 Sec.
times:                   bewerteUmgeb #        1 Su    0.33 ~  0.3300
times:                      evalMarkt #     1711 Su 7599.43 ~  4.4415
times:                        writeDB #     1711 Su    0.29 ~  0.0002
real	175m17.481s
user	126m53.160s
sys	0m16.437s


From fairly early during the job:
top - 11:43:24 up  1:43,  9 users,  load average: 3.86, 2.35, 1.21
Tasks:   3 total,   3 running,   0 sleeping,   0 stopped,   0 zombie
Cpu(s): 99.5%us,  0.2%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.3%si,  0.0%st
Mem:   3348628k total,  3212640k used,   135988k free,   435744k buffers
Swap:  2097144k total,        0k used,  2097144k free,  1089812k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
17563 mgd       11   0 92872  17m 3668 R  100  0.5   2:43.97 perl
17568 mgd       33   0 93932  18m 3668 R   50  0.6   1:48.49 perl
17570 mgd       31   0 93652  18m 3668 R   50  0.6   2:14.16 perl


procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
 4  0      0 125052 436832 1093420    0    0     0    87  932 1464 99  1  0  0
 7  0      0 125060 436832 1093428    0    0     0     0  933 1306 99  1  0  0
 3  0      0 125256 436852 1093408    0    0     0   120  831 1336 99  1  0  0
 3  0      0 124804 436864 1093440    0    0     0   165  443 1374 99  1  0  0
 3  0      0 124804 436864 1093440    0    0     0     0  421  955 100  0  0  0
 4  0      0 124812 436884 1093440    0    0     0   113  440  989 100  0  0  0
 4  0      0 124812 436884 1093440    0    0     0    63  436  936 100  0  0  0
 3  0      0 124804 436900 1093448    0    0     0    64  485 1142 100  0  0  0
 3  0      0 124804 436912 1093436    0    0     0    47  434  992 100  0  0  0
 3  0      0 124804 436920 1093448    0    0     0    24  448  978 100  0  0  0
 3  0      0 124656 436932 1093448    0    0     0   120  443 1850 99  1  0  0
 3  0      0 124648 436932 1093464    0    0     0     0  441 1081 99  1  0  0
 8  0      0 124688 436952 1093464    0    0     0   156  432  924 100  0  0  0
 4  0      0 124976 436960 1093472    0    0     0    43  439 1028 100  0  0  0
 3  0      0 125100 436968 1093472    0    0     0    27  426  961 100  0  0  0
10  0      0 125132 436976 1093480    0    0     0    43  427  876 100  0  0  0
 6  0      0 125132 436984 1093480    0    0     0    28  440 1087 100  0  0  0
 3  0      0 124868 437000 1093484    0    0     0   133  445 1629 100  0  0  0
 3  0      0 124392 437056 1093456    0    0     0   356  483 1072 99  1  0  0
 3  0      0 124400 437056 1093496    0    0     0     0  429 1069 100  0  0  0
 3  0      0 124400 437072 1093488    0    0     0    80  441 1000 100  0  0  0



2.6.21-rc7
==========
Start of all 3 jobs: 20:11:45.971

Job LTMM End: 22:25:33.108
GC2 20070101 LTMM Wed Apr 25 22:25:07.427 2007: 133.45 Min.,  4.6797 sec/run (1711 runs)
Ranking-3.pl finished after 5462.240s(CPU) and 8026 Sek.
times:                   bewerteUmgeb #        1 Su    0.77 ~  0.7700
times:                      evalMarkt #     1711 Su 5447.89 ~  3.1840
times:                        writeDB #     1711 Su    0.29 ~  0.0002
real	133m50.274s
user	91m2.485s
sys	0m10.801s


Job LTMB End: 23:01:54.763
GC2 20070101 LTMB Wed Apr 25 23:01:35.274 2007: 169.88 Min.,  5.9573 sec/run (1711 runs)
Ranking-3.pl finished after 7452.300s(CPU) and 10208 Sek.
times:                   bewerteUmgeb #        1 Su    0.76 ~  0.7600
times:                      evalMarkt #     1711 Su 7438.33 ~  4.3474
times:                        writeDB #     1711 Su    0.25 ~  0.0001
real	170m12.885s
user	124m12.562s
sys	0m14.705s


Job LTBM End: 23:09:16.506
GC2 20070101 LTBM Wed Apr 25 23:08:59.376 2007: 177.28 Min.,  6.2168 sec/run (1711 runs)
Ranking-3.pl finished after 7673.940s(CPU) and 10652 Sek.
times:                   bewerteUmgeb #        1 Su    0.78 ~  0.7800
times:                      evalMarkt #     1711 Su 7659.10 ~  4.4764
times:                        writeDB #     1711 Su    0.30 ~  0.0002
real	177m35.890s
user	127m54.376s
sys	0m9.873s


From fairly early during the job:
top - 20:15:13 up 12 min,  9 users,  load average: 3.89, 2.61, 1.52
Tasks:   3 total,   3 running,   0 sleeping,   0 stopped,   0 zombie
Cpu(s): 99.8%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.2%si,  0.0%st
Mem:   3348628k total,  1109020k used,  2239608k free,    28820k buffers
Swap:  2097144k total,        0k used,  2097144k free,   511352k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 6389 mgd       25   0 92776  17m 3668 R  100  0.5   2:50.23 perl
 6390 mgd       25   0 93932  18m 3668 R   53  0.6   2:08.47 perl
 6393 mgd       25   0 93880  18m 3668 R   47  0.6   1:42.53 perl


procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
 6  0      0 2211984  30780 525612    0    0     0     0  490 1318 100  0  0  0
 4  0      0 2211460  30872 525576    0    0     8   795  489 1052 100  0  0  0
 3  0      0 2211736  30896 525620    0    0     0   129  467 1003 99  1  0  0
 7  0      0 2211024  30896 525624    0    0     0    97  469 1234 100  0  0  0
 4  0      0 2211080  30924 525624    0    0     0   147  472 1117 100  0  0  0
 4  0      0 2211008  30936 525616    0    0     0    35  467 1086 100  0  0  0
 5  0      0 2210792  30976 525632    0    0     0   144  480 1120 100  0  0  0
 5  0      0 2210932  30992 525620    0    0     0    47  462 1099 99  1  0  0
 3  0      0 2210948  31000 525644    0    0     0    40  460 1087 100  0  0  0
 4  0      0 2210980  31016 525648    0    0     0    41  453  898 100  0  0  0
 4  0      0 2210716  31036 525648    0    0     1    99  471 1322 100  0  0  0
 4  0      0 2210748  31052 525648    0    0     0    60  454 1085 100  0  0  0
 5  0      0 2210764  31080 525652    0    0     0    68  463 1007 100  0  0  0
 5  0      0 2210796  31080 525656    0    0     0    68  462  962 100  0  0  0
 5  0      0 2210812  31104 525664    0    0     0    97  470 1193 100  0  0  0
 5  0      0 2210812  31104 525664    0    0     0     0  451  911 100  0  0  0
 4  0      0 2210812  31128 525672    0    0     0    69  465 1100 100  0  0  0
 3  0      0 2210300  31152 525660    0    0     0   143  466 1350 99  1  0  0
 4  0      0 2210292  31152 525676    0    0     3     9  513 2025 100  0  0  0
 6  0      0 2210624  31196 525700    0    0     3   133  492 1207 100  0  0  0
 4  0      0 2210460  31196 525720    0    0     0     0  458 1086 100  0  0  0

Best,
Michael
-- 
 Technosis GmbH, Geschäftsführer: Michael Gerdau, Tobias Dittmar
 Sitz Hamburg; HRB 89145 Amtsgericht Hamburg
 Vote against SPAM - see http://www.politik-digital.de/spam/
 Michael Gerdau       email: mgd@technosis.de
 GPG-keys available on request or at public keyserver

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [REPORT] cfs-v6-rc2 vs sd-0.46 vs 2.6.21-rc7
  2007-04-26 11:12 [REPORT] cfs-v6-rc2 vs sd-0.46 vs 2.6.21-rc7 Michael Gerdau
@ 2007-04-26 12:07 ` Ingo Molnar
  2007-04-26 13:22   ` Michael Gerdau
  2007-04-26 22:59   ` [ck] " Con Kolivas
  0 siblings, 2 replies; 7+ messages in thread
From: Ingo Molnar @ 2007-04-26 12:07 UTC (permalink / raw)
  To: Michael Gerdau
  Cc: linux-kernel, Linus Torvalds, Nick Piggin, Gene Heskett,
	Juliusz Chroboczek, Mike Galbraith, Peter Williams, ck list,
	Thomas Gleixner, William Lee Irwin III, Andrew Morton,
	Bill Davidsen, Willy Tarreau, Arjan van de Ven


* Michael Gerdau <mgd@technosis.de> wrote:

> Hi list,
> 
> find below a test comparing
>     2.6.21-rc7 (mainline)
>     2.6.21-rc7-sd046
>     2.6.21-rc7-cfs-v6-rc2(*) (X @ nice 0)
>     2.6.21-rc7-cfs-v6-rc2(*) (X @ nice -10)
> running on a dualcore x86_64.

thanks for the testing!

as a summary: i think your numbers demonstrate it nicely that the 
shorter 'timeslice length' that both CFS and SD utilizes does not have a 
measurable negative impact on your workload. To measure the total impact 
of 'timeslicing' you might want to try the exact same workload with a 
much higher 'timeslice length' of say 400 msecs, via:

    echo 400000000 > /proc/sys/kernel/sched_granularity_ns  # on CFS
    echo 400 > /proc/sys/kernel/rr_interval                 # on SD

your existing numbers are a bit hard to analyze because the 3 workloads 
were started at the same time and they overlapped differently and 
utilized the system differently.

i think the primary number that makes sense to look at (which is perhaps 
the least sensitive to the 'overlap effect') is the 'combined user times 
of all 3 workloads' (in order of performance):

> 2.6.21-rc7:                              20589.423    100.00%
> 2.6.21-rc7-cfs-v6-rc2 (X @ nice -10):    20613.845     99.88%
> 2.6.21-rc7-sd046:                        20617.945     99.86%
> 2.6.21-rc7-cfs-v6-rc2 (X @ nice 0):      20743.564     99.25%

to me this gives the impression that it's all "within noise". In 
particular the two CFS results suggest that there's at least a ~100 
seconds noise in these results, because the renicing of X should have no 
impact on the result (the workloads are pure number-crunchers, and all 
use up the CPUs 100%, correct?), and even if it has an impact, renicing 
X to nice 0 should _speed up_ the result - not slow it down a bit like 
the numbers suggest.

another (perhaps less reliable) number is the total wall-clock runtime 
of all 3 jobs. Provided i did not make any mistakes in my calculations, 
here are the results:

> 2.6.21-rc7-sd046:                        10512.611 seconds
> 2.6.21-rc7-cfs-v6-rc2 (X @ nice -10):    10605.946 seconds
> 2.6.21-rc7:                              10650.535 seconds
> 2.6.21-rc7-cfs-v6-rc2 (X @ nice 0):      10788.125 seconds

(the numbers are lower than the first numbers because this is a 2 CPU 
system)

both SD and CFS-nice-10 was faster than mainline, but i'd say this too 
is noise - especially because this result highly depends on the way the 
workloads overlap in general, which seems to be different for SD.

system time is interesting too:

> 2.6.21-rc7:                              35.379
> 2.6.21-rc7-cfs-v6-rc2 (X @ nice -10):    40.399
> 2.6.21-rc7-cfs-v6-rc2 (X @ nice 0):      44.239
> 2.6.21-rc7-sd046:                        45.515

here too the two CFS results seem to suggest that there's at least 
around 5 seconds of noise. So i'd not necessarily call it systematic 
that vanilla had the lowest system time and SD had the highest.

combined system+user time:

> 2.6.21-rc7:                              20624.802
> 2.6.21-rc7-cfs-v6-rc2 (X @ nice 0):      20658.084
> 2.6.21-rc7-sd046:                        20663.460
> 2.6.21-rc7-cfs-v6-rc2 (X @ nice -10):    20783.963

perhaps it might make more sense to run the workloads serialized, to 
have better comparabality of the individual workloads. (on a real system 
you'd naturally want to overlap these workloads to utilize the CPUs, so 
the numbers you did are very relevant too.)
 
The vmstat suggested there is occasional idle time in the system - is 
the workload IO-bound (or memory bound) in those cases?

	Ingo

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [REPORT] cfs-v6-rc2 vs sd-0.46 vs 2.6.21-rc7
  2007-04-26 12:07 ` Ingo Molnar
@ 2007-04-26 13:22   ` Michael Gerdau
  2007-04-26 13:55     ` Ingo Molnar
  2007-04-26 22:59   ` [ck] " Con Kolivas
  1 sibling, 1 reply; 7+ messages in thread
From: Michael Gerdau @ 2007-04-26 13:22 UTC (permalink / raw)
  To: Ingo Molnar
  Cc: linux-kernel, Linus Torvalds, Nick Piggin, Gene Heskett,
	Juliusz Chroboczek, Mike Galbraith, Peter Williams, ck list,
	Thomas Gleixner, William Lee Irwin III, Andrew Morton,
	Bill Davidsen, Willy Tarreau, Arjan van de Ven

[-- Attachment #1: Type: text/plain, Size: 5860 bytes --]

> as a summary: i think your numbers demonstrate it nicely that the 
> shorter 'timeslice length' that both CFS and SD utilizes does not have a 
> measurable negative impact on your workload.

That's my impression as well. In particular I think that for this
type of workload it is pretty much irrelevant which scheduler I'm
using :-)

> To measure the total impact  
> of 'timeslicing' you might want to try the exact same workload with a 
> much higher 'timeslice length' of say 400 msecs, via:
> 
>     echo 400000000 > /proc/sys/kernel/sched_granularity_ns  # on CFS
>     echo 400 > /proc/sys/kernel/rr_interval                 # on SD

I just finished building cfs and sd against 2.6.21 and will try with
these over the next days.

> your existing numbers are a bit hard to analyze because the 3 workloads 
> were started at the same time and they overlapped differently and 
> utilized the system differently.

Right. However the differences w/r to which job finished when, in
particular the change in who "came in" 2. and 3. between sd and cfs
had been consistent with other similar such jobs I ran over the last
couple of days.

In general sd tends to finish all three such jobs at roughly the
same time while cfs clearly "favors" the LTMM-type jobs (which
admittedly involve the least computations). I don't really know
why that is so. However the jobs does some minimal I/O after each
successfully finish run and that might be different w/r to sd and
cfs. Not really knowing what both scheduler do internally I'm in
no position to discuss that with either you or Con :)

> i think the primary number that makes sense to look at (which is perhaps 
> the least sensitive to the 'overlap effect') is the 'combined user times 
> of all 3 workloads' (in order of performance):
> 
> > 2.6.21-rc7:                              20589.423    100.00%
> > 2.6.21-rc7-cfs-v6-rc2 (X @ nice -10):    20613.845     99.88%
> > 2.6.21-rc7-sd046:                        20617.945     99.86%
> > 2.6.21-rc7-cfs-v6-rc2 (X @ nice 0):      20743.564     99.25%
> 
> to me this gives the impression that it's all "within noise".

I think so too. But then I didn't seriously expect any different.

> In  
> particular the two CFS results suggest that there's at least a ~100 
> seconds noise in these results, because the renicing of X should have no 
> impact on the result (the workloads are pure number-crunchers, and all 
> use up the CPUs 100%, correct?),

Correct.

The differences could as well result from small fluctuations in my
work on the machine during the test (like reading mail, editing sources
and similar stuff).

> another (perhaps less reliable) number is the total wall-clock runtime 
> of all 3 jobs. Provided i did not make any mistakes in my calculations, 
> here are the results:
> 
> > 2.6.21-rc7-sd046:                        10512.611 seconds
> > 2.6.21-rc7-cfs-v6-rc2 (X @ nice -10):    10605.946 seconds
> > 2.6.21-rc7:                              10650.535 seconds
> > 2.6.21-rc7-cfs-v6-rc2 (X @ nice 0):      10788.125 seconds
> 
> (the numbers are lower than the first numbers because this is a 2 CPU 
> system)
> 
> both SD and CFS-nice-10 was faster than mainline, but i'd say this too 
> is noise - especially because this result highly depends on the way the 
> workloads overlap in general, which seems to be different for SD.

I don't think that's noise. As I wrote above IMO this is the one
difference I personally consider significant. While running I could
watch intermediate output of all 3 jobs intermixed on a console.

With sd the jobs were -within reason- head to head while with cfs
and mainline LTMM quickly got ahead and remained there.

As I speculated above this (IMO real) difference in the way sd and
cfs (and mainline) schedule the jobs might be related to the way
the schedulers react on I/O but I don't know.

> system time is interesting too:
> 
> > 2.6.21-rc7:                              35.379
> > 2.6.21-rc7-cfs-v6-rc2 (X @ nice -10):    40.399
> > 2.6.21-rc7-cfs-v6-rc2 (X @ nice 0):      44.239
> > 2.6.21-rc7-sd046:                        45.515

Over the weekend I might try to repeat these tests while doing
nothing else on the machine (i.e. no editing or reading mail).

> combined system+user time:
> 
> > 2.6.21-rc7:                              20624.802
> > 2.6.21-rc7-cfs-v6-rc2 (X @ nice 0):      20658.084
> > 2.6.21-rc7-sd046:                        20663.460
> > 2.6.21-rc7-cfs-v6-rc2 (X @ nice -10):    20783.963
> 
> perhaps it might make more sense to run the workloads serialized, to 
> have better comparabality of the individual workloads. (on a real system 
> you'd naturally want to overlap these workloads to utilize the CPUs, so 
> the numbers you did are very relevant too.)

Yes. In fact the workloads used to be run serialized and only
recently I tried to easily have them run on multicore systems.

The whole set of such jobs consists of 312 tasks and thus the
idea is to get a couple of PS3 and have them run there...
(once I found the time to port the software).

> The vmstat suggested there is occasional idle time in the system - is 
> the workload IO-bound (or memory bound) in those cases?

There is writing out stats after each run (i.e. every 5-8 sec) both
to the PostgreSQL DB as well as to the console.

I could get rid of the console I/O (i.e. make it conditional) and
for testing purpose could switch off the DB if that would make be
helpful.

Best,
Michael
-- 
 Technosis GmbH, Geschäftsführer: Michael Gerdau, Tobias Dittmar
 Sitz Hamburg; HRB 89145 Amtsgericht Hamburg
 Vote against SPAM - see http://www.politik-digital.de/spam/
 Michael Gerdau       email: mgd@technosis.de
 GPG-keys available on request or at public keyserver

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [REPORT] cfs-v6-rc2 vs sd-0.46 vs 2.6.21-rc7
  2007-04-26 13:22   ` Michael Gerdau
@ 2007-04-26 13:55     ` Ingo Molnar
  0 siblings, 0 replies; 7+ messages in thread
From: Ingo Molnar @ 2007-04-26 13:55 UTC (permalink / raw)
  To: Michael Gerdau
  Cc: linux-kernel, Linus Torvalds, Nick Piggin, Gene Heskett,
	Juliusz Chroboczek, Mike Galbraith, Peter Williams, ck list,
	Thomas Gleixner, William Lee Irwin III, Andrew Morton,
	Bill Davidsen, Willy Tarreau, Arjan van de Ven


* Michael Gerdau <mgd@technosis.de> wrote:

> In general sd tends to finish all three such jobs at roughly the same 
> time while cfs clearly "favors" the LTMM-type jobs (which admittedly 
> involve the least computations). I don't really know why that is so.

for the reason of this, look at the raw user runtimes the 3 jobs have:

  5498.128  secs            # LTMM
  7559.777  secs
  7600.179  secs

the "perfect scheduler" would run each of the jobs at 33.33% of capacity 
for ~5500 CPU-seconds, and would then run the remaining two jobs at 
50.0% capacity for about ~2075 CPU-seconds.

Why? Because the scheduler how no idea how much each job takes! So a 
fair scheduler would run: 3 jobs at 33.33% capacity for as long as the 
shortest job ends, then the remaining 2 jobs at 50% capacity for as the 
shorter one of the remaining 2 finishes, and the remaining one at 100%.

in your case that means that the best scheduling would be roughly the 
following ideal timeline:

   5500*3 / 2 ==  8250 seconds for the LTMM to finish
   2075*2 / 2 == +2075 more seconds for the other two jobs to finish.

the various runs showed the following wallclock-time timeline for the 
LTMM phase:

   CFS #1:  real    142m56.806s
   CFS #2:  real    125m58.235s
   SD:      real    140m16.127s
   vanilla: real    133m50.274s

the "ideal" is ~137 minutes (8250 seconds) for the LTMM phase. The 
closest was indeed SD, but vanilla and cfs #1 wasnt too far away from it 
either. [ and the variance between CFS #1 and #2 seems to suggest that 
the noise of this particular metric is significant. The average does 
come in at 134.5, which is quite close to the ideal number. ]

	Ingo

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [ck] Re: [REPORT] cfs-v6-rc2 vs sd-0.46 vs 2.6.21-rc7
  2007-04-26 12:07 ` Ingo Molnar
  2007-04-26 13:22   ` Michael Gerdau
@ 2007-04-26 22:59   ` Con Kolivas
  2007-04-27  5:59     ` Michael Gerdau
  2007-04-27  6:52     ` Ingo Molnar
  1 sibling, 2 replies; 7+ messages in thread
From: Con Kolivas @ 2007-04-26 22:59 UTC (permalink / raw)
  To: ck
  Cc: Ingo Molnar, Michael Gerdau, Nick Piggin, Bill Davidsen,
	Juliusz Chroboczek, Mike Galbraith, linux-kernel,
	William Lee Irwin III, Peter Williams, Gene Heskett,
	Willy Tarreau, Thomas Gleixner, Linus Torvalds, Andrew Morton,
	Arjan van de Ven

On Thursday 26 April 2007 22:07, Ingo Molnar wrote:
> * Michael Gerdau <mgd@technosis.de> wrote:
> > Hi list,
> >
> > find below a test comparing
> >     2.6.21-rc7 (mainline)
> >     2.6.21-rc7-sd046
> >     2.6.21-rc7-cfs-v6-rc2(*) (X @ nice 0)
> >     2.6.21-rc7-cfs-v6-rc2(*) (X @ nice -10)
> > running on a dualcore x86_64.
>
> thanks for the testing!

Very interesting indeed but fairly complicated as well.

> as a summary: i think your numbers demonstrate it nicely that the
> shorter 'timeslice length' that both CFS and SD utilizes does not have a
> measurable negative impact on your workload. To measure the total impact
> of 'timeslicing' you might want to try the exact same workload with a
> much higher 'timeslice length' of say 400 msecs, via:
>
>     echo 400000000 > /proc/sys/kernel/sched_granularity_ns  # on CFS
>     echo 400 > /proc/sys/kernel/rr_interval                 # on SD

I thought that the effective "timeslice" on CFS was double the 
sched_granularity_ns so wouldn't this make the effective timeslice double 
that of what you're setting SD to? Anyway the difference between 400 and 
800ms timeslices is unlikely to be significant so I don't mind.

-- 
-ck

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [ck] Re: [REPORT] cfs-v6-rc2 vs sd-0.46 vs 2.6.21-rc7
  2007-04-26 22:59   ` [ck] " Con Kolivas
@ 2007-04-27  5:59     ` Michael Gerdau
  2007-04-27  6:52     ` Ingo Molnar
  1 sibling, 0 replies; 7+ messages in thread
From: Michael Gerdau @ 2007-04-27  5:59 UTC (permalink / raw)
  To: Con Kolivas
  Cc: ck, Ingo Molnar, Nick Piggin, Bill Davidsen, Juliusz Chroboczek,
	Mike Galbraith, linux-kernel, William Lee Irwin III,
	Peter Williams, Gene Heskett, Willy Tarreau, Thomas Gleixner,
	Linus Torvalds, Andrew Morton, Arjan van de Ven

[-- Attachment #1: Type: text/plain, Size: 1650 bytes --]

> Very interesting indeed but fairly complicated as well.

Sorry for that -- I've taken these figures from the 3MB logfile
that each job creates and "reading" them on a regular basis tend
to forget that probably everyody else does not find them as
obvious as I do. Also I'm don't really have lots of experience
with how a scheduler is properly tested.

For any upcoming tests I will restrict the numbers to wallclock
and what time provides which probably is better suited anyway.

> > as a summary: i think your numbers demonstrate it nicely that the
> > shorter 'timeslice length' that both CFS and SD utilizes does not have a
> > measurable negative impact on your workload. To measure the total impact
> > of 'timeslicing' you might want to try the exact same workload with a
> > much higher 'timeslice length' of say 400 msecs, via:
> >
> >     echo 400000000 > /proc/sys/kernel/sched_granularity_ns  # on CFS
> >     echo 400 > /proc/sys/kernel/rr_interval                 # on SD
> 
> I thought that the effective "timeslice" on CFS was double the 
> sched_granularity_ns so wouldn't this make the effective timeslice double 
> that of what you're setting SD to? Anyway the difference between 400 and 
> 800ms timeslices is unlikely to be significant so I don't mind.

I'm happy to do that, hopefully over the weekend.

Best,
Michael
-- 
 Technosis GmbH, Geschäftsführer: Michael Gerdau, Tobias Dittmar
 Sitz Hamburg; HRB 89145 Amtsgericht Hamburg
 Vote against SPAM - see http://www.politik-digital.de/spam/
 Michael Gerdau       email: mgd@technosis.de
 GPG-keys available on request or at public keyserver

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [ck] Re: [REPORT] cfs-v6-rc2 vs sd-0.46 vs 2.6.21-rc7
  2007-04-26 22:59   ` [ck] " Con Kolivas
  2007-04-27  5:59     ` Michael Gerdau
@ 2007-04-27  6:52     ` Ingo Molnar
  1 sibling, 0 replies; 7+ messages in thread
From: Ingo Molnar @ 2007-04-27  6:52 UTC (permalink / raw)
  To: Con Kolivas
  Cc: ck, Michael Gerdau, Nick Piggin, Bill Davidsen,
	Juliusz Chroboczek, Mike Galbraith, linux-kernel,
	William Lee Irwin III, Peter Williams, Gene Heskett,
	Willy Tarreau, Thomas Gleixner, Linus Torvalds, Andrew Morton,
	Arjan van de Ven


* Con Kolivas <kernel@kolivas.org> wrote:

> > as a summary: i think your numbers demonstrate it nicely that the
> > shorter 'timeslice length' that both CFS and SD utilizes does not have a
> > measurable negative impact on your workload. To measure the total impact
> > of 'timeslicing' you might want to try the exact same workload with a
> > much higher 'timeslice length' of say 400 msecs, via:
> >
> >     echo 400000000 > /proc/sys/kernel/sched_granularity_ns  # on CFS
> >     echo 400 > /proc/sys/kernel/rr_interval                 # on SD
> 
> I thought that the effective "timeslice" on CFS was double the 
> sched_granularity_ns so wouldn't this make the effective timeslice 
> double that of what you're setting SD to? [...]

The two settings are not really comparable. The "effective timeslice is 
the double of the granularity" thing i mentioned before is really a 
special-case: only true for a really undisturbed 100% CPU-using 
_two-task_ workload, if and only if the workload would not reschedule 
otherwise, but that is clearly not the case here: and if you look at the 
vmstat output provided by Michael you'll see that all 3 schedulers 
rescheduled this workload at around 1000/sec or 1 msec per scheduling 
atom. (But i'd agree that to be on the safe side the context-switch rate 
has to be monitored and if it seems too high on SD, the rr_interval 
should be increased.)

> [...] Anyway the difference between 400 and 800ms timeslices is 
> unlikely to be significant so I don't mind.

even on a totally idle system there's at least a 10 Hz 'background 
sound' of various activities, so any setting above 100 msecs rarely has 
any effect.

	Ingo

^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2007-04-27  6:53 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2007-04-26 11:12 [REPORT] cfs-v6-rc2 vs sd-0.46 vs 2.6.21-rc7 Michael Gerdau
2007-04-26 12:07 ` Ingo Molnar
2007-04-26 13:22   ` Michael Gerdau
2007-04-26 13:55     ` Ingo Molnar
2007-04-26 22:59   ` [ck] " Con Kolivas
2007-04-27  5:59     ` Michael Gerdau
2007-04-27  6:52     ` Ingo Molnar

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).