LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Nick Piggin <nickpiggin@yahoo.com.au>
To: Dmitry Antipov <dmantipov@yandex.ru>
Cc: Eric Dumazet <dada1@cosmosbay.com>,
	David Miller <davem@davemloft.net>,
	linux-kernel@vger.kernel.org
Subject: Re: Are Linux pipes slower than the FreeBSD ones ?
Date: Mon, 17 Mar 2008 23:53:23 +1100	[thread overview]
Message-ID: <200803172353.23965.nickpiggin@yahoo.com.au> (raw)
In-Reply-To: <47CFDF55.6030103@yandex.ru>

On Thursday 06 March 2008 23:11, Dmitry Antipov wrote:
> Nick Piggin wrote:
> > One thing to try is pinning both processes on the same CPU. This
> > may be what the FreeBSD scheduler is preferring to do, and it ends
> > up being really a tradeoff that helps some workloads and hurts
> > others. With a very unscientific test with an old kernel, the
> > pipe.c test gets anywhere from about 1.5 to 3 times faster when
> > running it as taskset 1 ./pipe
>
> Sounds interesting. What kernel version did you tried? Can you
> send your .config to me?
>
> I've tried this trick on 2.6.25-rc4, and got ~20% more throughput for
> large (> 8K) buffers at the cost of going ~30% down for the small ones.

Seems some people are still concerned about this benchmark. OK I
tried with Linux 2.6.25-rc6 (just because it's what I've got on
this system). Versus FreeBSD 7.0.

Unfortunately, I don't think FreeBSD supports binding a process to a
CPU, and on either system when the scheduler is allowed to choose
what happens, results are more variable than you would like.

That being said, I found that Linux often outscored FreeBSD in all 3
tests of pipe_v3. FreeBSD does appear like it can get a slightly higher
throughput at 64K in test #1, so maybe it's data copy routines are
slightly better. OTOH, I found Linux is better at 64K in test #2. For
the low sizes, I found Linux was usually faster than FreeBSD in tests
1 and 2, and around the same on test 3.

The other thing is that this test is pretty well a random context
switch benchmark that really depends on slight variations in how the
scheduler scheduler runs things. If you happen to be able to keep the
pipe from filling or emptying completely, you can run both processes
at the same time on different CPUs. If you run both processes on the
same CPU, then you want to avoid preempting the producer until it
fills the pipe, then you want to avoid preempting the consumer until
it empties the pipe in order to minimise context switches.

For example, if I call a nice(20) in the start of the reader processes
in tests #1 and #2, I get around a 5x speedup in Linux when running
reader and writer on the same CPU.

I won't bother posting actual numbers... if anybody is interested I
can mail raw results offline.

But again, it isn't such a great test because a higher number doesn't
really mean you'll do better with any real program, and optimising for
a higher number here could actually harm real programs.

pipe test v3 also is doing funny things with "simulating" real
accesses. It should generally write into the buffer before
write(2)ing it, and read from the buffer after read(2)ing it. Instead
it writes to the buffer after write(2) and after read(2). Also, it
should probably touch a significant number of the pages and
cachelines transferred in each case, rather than the 1/2 stores it
does right now. There are a lot of ways you can copy data around, so
even if you defeat page flipping (for small transfers), then you
still don't know if one method of copying data around is better than
another.

Basically I will just reiterate what I said before that it is really
difficult to draw any conclusions from a test like this, and from the
numbers I see, you certainly can't say FreeBSD is faster than Linux.

If you want to run this kind of microbenchmark, something like lmbench
at least has been around for a long time and been reviewed (whether or
not it is any more meaningful, I don't know). Or do you have a real
workload that this pipe test simulates?

Thanks,
Nick


      reply	other threads:[~2008-03-17 12:53 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2008-03-05  7:46 Antipov Dmitry
2008-03-05  8:00 ` David Miller
2008-03-05  9:47   ` Eric Dumazet
2008-03-05 10:19     ` Andi Kleen
2008-03-05 12:12     ` David Newall
2008-03-05 14:20     ` Nick Piggin
2008-03-05 14:55       ` Eric Dumazet
2008-03-05 15:38         ` Nick Piggin
2008-03-05 15:55           ` Ray Lee
2008-03-05 16:02             ` Nick Piggin
2008-03-06 12:11       ` Dmitry Antipov
2008-03-17 12:53         ` Nick Piggin [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=200803172353.23965.nickpiggin@yahoo.com.au \
    --to=nickpiggin@yahoo.com.au \
    --cc=dada1@cosmosbay.com \
    --cc=davem@davemloft.net \
    --cc=dmantipov@yandex.ru \
    --cc=linux-kernel@vger.kernel.org \
    --subject='Re: Are Linux pipes slower than the FreeBSD ones ?' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).