Re: Strange NFS write performance Linux->Solaris-10/VXFS, maybe VW related

[Martin Knoblauch <spamtrap@knobisoft.de>]

----- Original Message ----
> From: Martin Knoblauch <spamtrap@knobisoft.de>
> To: Chris Snook <csnook@redhat.com>
> Cc: linux-kernel@vger.kernel.org; linux-nfs@vger.kernel.org; spam trap <spamtrap@knobisoft.de>
> Sent: Saturday, December 29, 2007 12:11:08 PM
> Subject: Re: Strange NFS write performance Linux->Solaris-10/VXFS, maybe VW related
> 
> ----- Original Message ----
> > From: Chris Snook 
> > To: Martin Knoblauch 
> > Cc: linux-kernel@vger.kernel.org; linux-nfs@vger.kernel.org
> > Sent: Friday, December 28, 2007 7:45:13 PM
> > Subject: Re: Strange NFS write performance
> Linux->Solaris-10/VXFS,
> 
 maybe VW related
> > 
> > Martin Knoblauch wrote:
> > > Hi,
> > > 
> > > currently I am tracking down an "interesting" effect when writing
> 
> > 3) It sounds like the bottleneck is the vxfs filesystem.  It
> > only *appears* on  the client side because writes up
> until
> 
 dirty_ratio
> > get buffered on the client. 
> >   If you can confirm that the server is actually writing stuff to
> > disk slower  when the client is in writeback mode, then it's possible
> > the Linux NFSclient is  doing something inefficient in
> writeback
> 
 mode.
> > 
> 
> so, is the output of "iostat -d -l1 d111" during two runs. The
> first
> 
 run is with 750 MB, the second with 850MB.
> 
> // 750MB
> $ iostat -d -l 1 md111 2
>    md111
> kps tps serv
>  22   0   14
>   0   0    0
>   0   0   13
> 29347 468   12
> 37040 593   17
> 30938 492   25
> 30421 491   25
> 41626 676   16
> 42913 703   14
> 39890 647   15
> 9009 141    7
> 8963 141    7
> 5143  81    7
> 34814 547   10
> 49323 775   12
> 28624 451    6
>  22   1    6
> #### finish
>   0   0    0
>   0   0    0
> 
>  Here it seems that the disk is writing for 26-28 seconds with avg.
> 29
> 
 MB/sec. Fine.
> 
> // 850MB
> $ iostat -d -l 1 md111 2
>    md111
> kps tps serv
>   0   0    0
> 11275 180   10
> 39874 635   14
> 37403 587   17
> 24341 392   30
> 25989 423   26
> 22464 375   30
> 21922 361   32
> 27924 450   26
> 21507 342   21
> 9217 153   15
> 9260 150   15
> 9544 155   15
> 9298 150   14
> 10118 162   11
> 15505 250   12
> 27513 448   14
> 26698 436   15
> 26144 431   15
> 25201 412   14
> #### 38 seconds in run
>   0   0    0
>   0   0    0
> 579  17   12
>   0   0    0
>   0   0    0
>   0   0    0
>   0   0    0
> 518   9   16
> 485   8    6
>   9   1    7
> 514   9    7
>   0   0    0
>   0   0    0
> 541   9    8
> 532  10    6
>   0   0    0
>   0   0    0
> 650  12    7
>   0   0    0
> 242   8    9
> 1023  18    5
> 304   5    6
> 418   8    7
> 283   5    5
> 303   5    8
> 527  10    6
>   0   0    0
>   0   0    0
>   0   0    0
>   5   1   13
>   0   0    0
>   0   0    0
>   0   0    0
>   0   0    0
>   0   0    0
>   0   0   11
>   0   0    0
>   0   0    0
>   0   0    0
>   1   0   15
>   0   0    0
>  96   2   15
> 138   3   10
> 11057 175    6
> 17549 280    6
> 351   8    5
>   0   0    0
> ##### 218 seconds in run, finish.
> 
>  So, for the first 38 seconds everything looks similar to the 750
> MB case. For the next about 180 seconds most time nothing happens.
> Averaging 4.1 MB/sec.
> 
> Maybe it is time to capture the traffic. What are the best
> tcpdump parameters for NFS? I always forget :-(
> 
> Cheers
> Martin
> 
> 
Hi,

 now that the seasonal festivities are over - Happy New Year btw. - any comments/suggestions on my problem?

Cheers
Martin