Re: AIO, FIO and Threads ...

[Davide Libenzi <davidel@xmailserver.org>]

On Wed, 21 Mar 2007, Jens Axboe wrote:

> On Tue, Mar 20 2007, Davide Libenzi wrote:
> > 
> > I was looking at Jens FIO stuff, and I decided to cook a quick patch for 
> > FIO to support GUASI (Generic Userspace Asyncronous Syscall Interface):
> > 
> > http://www.xmailserver.org/guasi-lib.html
> > 
> > I then ran a few tests on my Dual Opteron 252 with SATA drives (sata_nv) 
> > and 8GB of RAM.
> > Mind that I'm not FIO expert, like at all, but I got some interesting 
> > results when comparing GUASI with libaio at 8/1000/10000 depths.
> > If I read those result correctly (Jens may help), GUASI output is more 
> > then double the libaio one.
> > Lots of context switches, yes. But the throughput looks like 2+ times.
> > Can someone try to repeat the measures and/or spot the error?
> > Or tell me which other tests to run?
> > This is kinda a suprise for me ...
> 
> I don't know guasi at all, but libaio requires O_DIRECT to be async. I'm
> sure you know this, but you may not know that fio default to buffered IO
> so you have to tell it to use O_DIRECT :-)
> 
> So try adding a --direct=1 (or --buffered=0, same thing) as an extra
> option when comparing depths > 1.

I knew about AIO and O_DIRECT, but I thought FIO was using it by default :)
I used it for the first time yesterday night, and there are a pretty wide 
set of options. Will re-run today with --direct.
I was pretty surprised though. Since libaio was matching with syslets, I 
was thinking that a userspace version using a queue-always design (hard to 
do the cachehit optimization if you're not inside the scheduler ;), was 
going to be considerably slower.


> I'll add your guasi engine, but disable it. Unfortunately fio still
> doesn't have a nifty configure setup, so these things are still
> manual...

Well, you do have your own HAVE_*, you just need to make autoconf/automake 
do the checks for you. Of course, then you'll be pissed every time an 
autoconf/automake update break your setup, but lately it seems to be going 
better. Really :)


- Davide