VM: 2.4.10ac4 vs. 2.4.11pre2

VM: 2.4.10ac4 vs. 2.4.11pre2

[chris]

Hi,

Here are some test results. Results are averaged over multiple runs.
Comments and conclusions below.

                     2.4.11pre2        2.4.10ac4
dbench 8             34Mbyte/sec       40Mbyte/sec
dbench 32            7.7Mbyte/sec      14Mbyte/sec
bonnie++ write       17.5Mbyte/sec     18Mbyte/sec
bonnie++ rewrite     5.6Mbyte/sec      5.8Mbyte/sec
bonnie++ read        24Mbyte/sec       24.5Mbyte/sec
kernel stress build  212min24s         229m54s
linear swap test     1m30s             2m15s
bonnie++ creat()     7200              9600  [*]
bonnie++ stat()      2100              9000  [*]
bonnie++ unlink()    5300              30000 [*]

[*] either the ext2 directory optimization in 2.4.10ac is influencing the
test, or 2.4.11pre2 VM has a problem caching inodes.

Comments + conclusions
----------------------

- The 2.4.11pre2 VM is considerably more stable, where "stable" is defined
as repeatable test scores and consistent performance. The 2.4.10ac4 VM is
all over the place.

- Both kernels exhibit similar interactive response under load.

- The 2.4.11pre2 VM performs substantially better in tests which invoke
swapping.

- Surprisingly, the 2.4.10ac4 kernel does much much better at dbench. The
2.4.11pre2 performance is alleged to have regressed since 2.4.10pre10?

- I have not tried 2.4.11pre4, but the report of streaming i/o causing
swapping is concerning.


Note that the above results were generated using a very simple (and
extensible) script. VM developers would do well to spend the 30 seconds
writing a similar script, and post results along with proposed VM patches.

Cheers
Chris

Re: VM: 2.4.10ac4 vs. 2.4.11pre2

[Mike Fedyk]

On Sat, Oct 06, 2001 at 01:20:23PM +0100, chris@scary.beasts.org wrote:
> 
> Note that the above results were generated using a very simple (and
> extensible) script. VM developers would do well to spend the 30 seconds
> writing a similar script, and post results along with proposed VM patches.
> 

Note to all Benchmark posters:

If you have a script, you should either post it, or provide a link to where
the script is available.

Even if it's simple, it will help give reproducable results, or see if there
may be a better way to test...

Mike

Re: VM: 2.4.10ac4 vs. 2.4.11pre2

[Rik van Riel]

On Sat, 6 Oct 2001, Mike Fedyk wrote:
> On Sat, Oct 06, 2001 at 01:20:23PM +0100, chris@scary.beasts.org wrote:
> >
> > Note that the above results were generated using a very simple (and
> > extensible) script. VM developers would do well to spend the 30 seconds
> > writing a similar script, and post results along with proposed VM patches.
>
> Note to all Benchmark posters:
>
> If you have a script, you should either post it, or provide a link to
> where the script is available.

Absolutely.  The VM hackers could have a full-time job
rewriting all the benchmark scripts which weren't posted
here ... without leaving time to even run them once.

cheers,

Rik
-- 
DMCA, SSSCA, W3C?  Who cares?  http://thefreeworld.net/  (volunteers needed)

http://www.surriel.com/		http://distro.conectiva.com/

Re: VM: 2.4.10ac4 vs. 2.4.11pre2

[Andrea Arcangeli]

On Sat, Oct 06, 2001 at 01:20:23PM +0100, chris@scary.beasts.org wrote:
> 
> dbench 8             34Mbyte/sec       40Mbyte/sec

dbench wants a very unfair vm behaviour. We must penalize all tasks
except one.  I measured a x2 slowdown after Linus introduced
mark_page_accessed after 2.4.10pre11 (but still it was faster than
pre10).

If you want patches to make dbench much faster I can provide them, just
ask, at the moment I just think dbench is a very bad benchmark. I can
implement a /proc/sys/vm/dbench if people wants to post nice numbers
without having to apply patches :).

I'd also like to know what you get using -aa instead of mainline, there
are a few changes that can make a difference in the numbers.

Andrea

Re: VM: 2.4.10ac4 vs. 2.4.11pre2

[Alexei Podtelezhnikov]

Chris, 

Can you elaborate on "all over the place" by posting standard deviations 
of your measurements, or the entire sets? I guess you did about 6-10 of 
each. I think it's important since - who cares about good average 
performance if once in a while VM fails miserably. 

To emphasize the importance of this: I would expect that large deviations 
are more likely during short spikes of activity. It would be interesting 
to see how start-up times of mozilla vary having your linear swap 
test on the background. Too much to ask anyway.

Your conclusion about poor swapping with ac kernels is consistent with 
earlier posts ("VM: more numbers").

Alexei