LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
* 2.6.20: stripe_cache_size goes boom with 32mb
@ 2007-02-23 11:41 Justin Piszcz
  2007-02-23 12:41 ` Jason Rainforest
  2007-02-23 13:11 ` Jan Engelhardt
  0 siblings, 2 replies; 5+ messages in thread
From: Justin Piszcz @ 2007-02-23 11:41 UTC (permalink / raw)
  To: linux-kernel; +Cc: linux-raid

Each of these are averaged over three runs with 6 SATA disks in a SW RAID 
5 configuration:

(dd if=/dev/zero of=file_1 bs=1M count=2000)

128k_stripe: 69.2MB/s
256k_stripe: 105.3MB/s
512k_stripe: 142.0MB/s
1024k_stripe: 144.6MB/s
2048k_stripe: 208.3MB/s
4096k_stripe: 223.6MB/s
8192k_stripe: 226.0MB/s
16384k_stripe: 215.0MB/s

When I tried a 32768k stripe, this happened:
p34:~# echo 32768 > /sys/block/md4/md/stripe_cache_size
Connection to p34 closed

I was able to Alt-SysRQ+b but I could not access the console/X/etc, it 
appeared to be frozen.

FYI.

Justin.


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: 2.6.20: stripe_cache_size goes boom with 32mb
  2007-02-23 11:41 2.6.20: stripe_cache_size goes boom with 32mb Justin Piszcz
@ 2007-02-23 12:41 ` Jason Rainforest
  2007-02-23 12:42   ` Justin Piszcz
  2007-02-23 13:11 ` Jan Engelhardt
  1 sibling, 1 reply; 5+ messages in thread
From: Jason Rainforest @ 2007-02-23 12:41 UTC (permalink / raw)
  To: Justin Piszcz; +Cc: linux-kernel, linux-raid

Hi Justin,

I'm not a RAID or kernel developer, but .. do you have enough RAM to
support a 32mb stripe_cache_size?! Here on my 7*250Gb SW RAID5 array,
decreasing a stripe_cache_size of 8192 to 4096 frees up no less than
120mb of RAM. Using that as a calculation tool, a 32mb stripe_cache_size
would require approximately 960mb of RAM! My RAID box only has 1Gb of
RAM, so I'm not game to test such a thing. Others on these lists would
definitely have a good idea on what's happening :-)

Cheers,
Jason


On Fri, 2007-02-23 at 06:41 -0500, Justin Piszcz wrote:
> Each of these are averaged over three runs with 6 SATA disks in a SW RAID 
> 5 configuration:
> 
> (dd if=/dev/zero of=file_1 bs=1M count=2000)
> 
> 128k_stripe: 69.2MB/s
> 256k_stripe: 105.3MB/s
> 512k_stripe: 142.0MB/s
> 1024k_stripe: 144.6MB/s
> 2048k_stripe: 208.3MB/s
> 4096k_stripe: 223.6MB/s
> 8192k_stripe: 226.0MB/s
> 16384k_stripe: 215.0MB/s
> 
> When I tried a 32768k stripe, this happened:
> p34:~# echo 32768 > /sys/block/md4/md/stripe_cache_size
> Connection to p34 closed
> 
> I was able to Alt-SysRQ+b but I could not access the console/X/etc, it 
> appeared to be frozen.
> 
> FYI.
> 
> Justin.
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: 2.6.20: stripe_cache_size goes boom with 32mb
  2007-02-23 12:41 ` Jason Rainforest
@ 2007-02-23 12:42   ` Justin Piszcz
  2007-02-23 19:11     ` Dan Williams
  0 siblings, 1 reply; 5+ messages in thread
From: Justin Piszcz @ 2007-02-23 12:42 UTC (permalink / raw)
  To: Jason Rainforest; +Cc: linux-kernel, linux-raid

I have 2GB On this machine.  For me, 8192 seems to be the sweet spot, I 
will probably keep it at 8mb.

On Fri, 23 Feb 2007, Jason Rainforest wrote:

> Hi Justin,
>
> I'm not a RAID or kernel developer, but .. do you have enough RAM to
> support a 32mb stripe_cache_size?! Here on my 7*250Gb SW RAID5 array,
> decreasing a stripe_cache_size of 8192 to 4096 frees up no less than
> 120mb of RAM. Using that as a calculation tool, a 32mb stripe_cache_size
> would require approximately 960mb of RAM! My RAID box only has 1Gb of
> RAM, so I'm not game to test such a thing. Others on these lists would
> definitely have a good idea on what's happening :-)
>
> Cheers,
> Jason
>
>
> On Fri, 2007-02-23 at 06:41 -0500, Justin Piszcz wrote:
>> Each of these are averaged over three runs with 6 SATA disks in a SW RAID
>> 5 configuration:
>>
>> (dd if=/dev/zero of=file_1 bs=1M count=2000)
>>
>> 128k_stripe: 69.2MB/s
>> 256k_stripe: 105.3MB/s
>> 512k_stripe: 142.0MB/s
>> 1024k_stripe: 144.6MB/s
>> 2048k_stripe: 208.3MB/s
>> 4096k_stripe: 223.6MB/s
>> 8192k_stripe: 226.0MB/s
>> 16384k_stripe: 215.0MB/s
>>
>> When I tried a 32768k stripe, this happened:
>> p34:~# echo 32768 > /sys/block/md4/md/stripe_cache_size
>> Connection to p34 closed
>>
>> I was able to Alt-SysRQ+b but I could not access the console/X/etc, it
>> appeared to be frozen.
>>
>> FYI.
>>
>> Justin.
>>
>> -
>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: 2.6.20: stripe_cache_size goes boom with 32mb
  2007-02-23 11:41 2.6.20: stripe_cache_size goes boom with 32mb Justin Piszcz
  2007-02-23 12:41 ` Jason Rainforest
@ 2007-02-23 13:11 ` Jan Engelhardt
  1 sibling, 0 replies; 5+ messages in thread
From: Jan Engelhardt @ 2007-02-23 13:11 UTC (permalink / raw)
  To: Justin Piszcz; +Cc: linux-kernel, linux-raid


On Feb 23 2007 06:41, Justin Piszcz wrote:
>
> I was able to Alt-SysRQ+b but I could not access the console/X/etc, it appeared
> to be frozen.

No sysrq+t? (Ah, unblanking might hang.) Well, netconsole/serial to the rescue,
then ;-)


Jan
-- 

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: 2.6.20: stripe_cache_size goes boom with 32mb
  2007-02-23 12:42   ` Justin Piszcz
@ 2007-02-23 19:11     ` Dan Williams
  0 siblings, 0 replies; 5+ messages in thread
From: Dan Williams @ 2007-02-23 19:11 UTC (permalink / raw)
  To: Justin Piszcz; +Cc: Jason Rainforest, linux-kernel, linux-raid

On 2/23/07, Justin Piszcz <jpiszcz@lucidpixels.com> wrote:
> I have 2GB On this machine.  For me, 8192 seems to be the sweet spot, I
> will probably keep it at 8mb.
>
Just a note stripe_cache_size = 8192 = 192MB with six disks.

The calculation is:
stripe_cache_size * num_disks * PAGE_SIZE = stripe_cache_size_bytes

--
Dan

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2007-02-23 19:11 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2007-02-23 11:41 2.6.20: stripe_cache_size goes boom with 32mb Justin Piszcz
2007-02-23 12:41 ` Jason Rainforest
2007-02-23 12:42   ` Justin Piszcz
2007-02-23 19:11     ` Dan Williams
2007-02-23 13:11 ` Jan Engelhardt

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).