EXT4-fs: not enough memory for 522250 flex groups

EXT4-fs: not enough memory for 522250 flex groups

[Christian Borntraeger]

Hello,

I tried to use ext4 on my s390 system. Converting the old ext3 file systems 
(tune2fs -O extents) worked fine. When I tried to create a new ext4 file 
system on a logical volume I got the "EXT4-fs: not enough memory for 522250 
flex groups" error messageduring mount:


# mke2fs -t ext4 /dev/mapper/space-test
mke2fs 1.41.3 (12-Oct-2008)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
262144 inodes, 1048576 blocks
52428 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=1073741824
32 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376, 294912, 819200, 884736
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done
This filesystem will be automatically checked every 22 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.


# mount -t ext4 /dev/mapper/space-test /mnt/
mount: wrong fs type, bad option, bad superblock on /dev/mapper/space-test,
       missing codepage or helper program, or other error
       In some cases useful info is found in syslog - try
       dmesg | tail  or so



# dmesg
[...]
[6923911.715968] EXT4-fs: not enough memory for 522250 flex groups
[6923911.715973] EXT4-fs: unable to initialize flex_bg meta info!



# lvdisplay
  --- Logical volume ---
  LV Name                /dev/space/test
  VG Name                space
  LV UUID                FPhFLU-1xTK-XtG4-STmI-u3D5-KhCR-FmlcCO
  LV Write Access        read/write
  LV Status              available
  # open                 0
  LV Size                4.00 GB
  Current LE             1024
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           253:54



# free
             total       used       free     shared    buffers     cached
Mem:       3081168     822972    2258196          0     154784     445420
-/+ buffers/cache:     222768    2858400
Swap:      9615904          0    9615904


# uname -a
Linux 2.6.28-rc2 #64 SMP Thu Nov 6 15:22:04 CET 2008 s390x s390x s390x 
GNU/Linux



# git describe
kvm-78-2-g6e6c459


Any ideas?

[PATCH]: ext4: fix big endian (was: not enough memory for 522250 flex groups)

[Christian Borntraeger]

Ok, I think I found it:

On big endianess plattforms newly created ext4 file systems cannot be mounted
and show messages like:
[6923911.715968] EXT4-fs: not enough memory for 522250 flex groups
[6923911.715973] EXT4-fs: unable to initialize flex_bg meta info!

We have to access s_reserved_gdb_blocks with le16_to_cpu.

Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
---
 fs/ext4/super.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Index: kvm/fs/ext4/super.c
===================================================================
--- kvm.orig/fs/ext4/super.c
+++ kvm/fs/ext4/super.c
@@ -1455,7 +1455,7 @@ static int ext4_fill_flex_info(struct su
 
 	/* We allocate both existing and potentially added groups */
 	flex_group_count = ((sbi->s_groups_count + groups_per_flex - 1) +
-			    ((sbi->s_es->s_reserved_gdt_blocks +1 ) <<
+			    (le16_to_cpu(sbi->s_es->s_reserved_gdt_blocks + 1) <<
 			      EXT4_DESC_PER_BLOCK_BITS(sb))) /
 			   groups_per_flex;
 	sbi->s_flex_groups = kzalloc(flex_group_count *

Re: [PATCH]: ext4: fix big endian

[Peter Staubach]

Christian Borntraeger wrote:
> Ok, I think I found it:
>
> On big endianess plattforms newly created ext4 file systems cannot be mounted
> and show messages like:
> [6923911.715968] EXT4-fs: not enough memory for 522250 flex groups
> [6923911.715973] EXT4-fs: unable to initialize flex_bg meta info!
>
> We have to access s_reserved_gdb_blocks with le16_to_cpu.
>
> Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
> ---
>  fs/ext4/super.c |    2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> Index: kvm/fs/ext4/super.c
> ===================================================================
> --- kvm.orig/fs/ext4/super.c
> +++ kvm/fs/ext4/super.c
> @@ -1455,7 +1455,7 @@ static int ext4_fill_flex_info(struct su
>  
>  	/* We allocate both existing and potentially added groups */
>  	flex_group_count = ((sbi->s_groups_count + groups_per_flex - 1) +
> -			    ((sbi->s_es->s_reserved_gdt_blocks +1 ) <<
> +			    (le16_to_cpu(sbi->s_es->s_reserved_gdt_blocks + 1) <<
>   

I suspect that you want to do the le16_to_cpu() and _then_
add the 1.  Otherwise, adding 1 to a different byte order
value won't do quite what is expected or hoped for...

       ps

>  			      EXT4_DESC_PER_BLOCK_BITS(sb))) /
>  			   groups_per_flex;
>  	sbi->s_flex_groups = kzalloc(flex_group_count *
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>   

Re: [PATCH]: ext4: fix big endian (was: not enough memory for 522250 flex groups)

[Andreas Schwab]

Christian Borntraeger <borntraeger@de.ibm.com> writes:

> Index: kvm/fs/ext4/super.c
> ===================================================================
> --- kvm.orig/fs/ext4/super.c
> +++ kvm/fs/ext4/super.c
> @@ -1455,7 +1455,7 @@ static int ext4_fill_flex_info(struct su
>  
>  	/* We allocate both existing and potentially added groups */
>  	flex_group_count = ((sbi->s_groups_count + groups_per_flex - 1) +
> -			    ((sbi->s_es->s_reserved_gdt_blocks +1 ) <<
> +			    (le16_to_cpu(sbi->s_es->s_reserved_gdt_blocks + 1) <<

You need to put the "+ 1" after the swap.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: [PATCH]: ext4: fix big endian

[Christian Borntraeger]

Am Donnerstag, 6. November 2008 schrieb Peter Staubach:
> >  	/* We allocate both existing and potentially added groups */
> >  	flex_group_count = ((sbi->s_groups_count + groups_per_flex - 1) +
> > -			    ((sbi->s_es->s_reserved_gdt_blocks +1 ) <<
> > +			    (le16_to_cpu(sbi->s_es->s_reserved_gdt_blocks + 1) <<
> >   
> 
> I suspect that you want to do the le16_to_cpu() and _then_
> add the 1.  Otherwise, adding 1 to a different byte order
> value won't do quite what is expected or hoped for...

errm. Right.
Something like this maybe?

[PATCH]: ext4: fix big endian access for flex groups

On big endianess plattforms newly created ext4 file systems cannot be mounted
and show messages like:
[6923911.715968] EXT4-fs: not enough memory for 522250 flex groups
[6923911.715973] EXT4-fs: unable to initialize flex_bg meta info!

We have to access s_reserved_gdb_blocks with le16_to_cpu.

Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
---
 fs/ext4/super.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

Index: kvm/fs/ext4/super.c
===================================================================
--- kvm.orig/fs/ext4/super.c
+++ kvm/fs/ext4/super.c
@@ -1455,8 +1455,8 @@ static int ext4_fill_flex_info(struct su
 
 	/* We allocate both existing and potentially added groups */
 	flex_group_count = ((sbi->s_groups_count + groups_per_flex - 1) +
-			    ((sbi->s_es->s_reserved_gdt_blocks +1 ) <<
-			      EXT4_DESC_PER_BLOCK_BITS(sb))) /
+			    ((le16_to_cpu(sbi->s_es->s_reserved_gdt_blocks) +
+			      1) << EXT4_DESC_PER_BLOCK_BITS(sb))) /
 			   groups_per_flex;
 	sbi->s_flex_groups = kzalloc(flex_group_count *
 				     sizeof(struct flex_groups), GFP_KERNEL);

Re: [PATCH]: ext4: fix big endian

[Aneesh Kumar K.V]

On Thu, Nov 06, 2008 at 05:15:53PM +0100, Christian Borntraeger wrote:
> Am Donnerstag, 6. November 2008 schrieb Peter Staubach:
> > >  	/* We allocate both existing and potentially added groups */
> > >  	flex_group_count = ((sbi->s_groups_count + groups_per_flex - 1) +
> > > -			    ((sbi->s_es->s_reserved_gdt_blocks +1 ) <<
> > > +			    (le16_to_cpu(sbi->s_es->s_reserved_gdt_blocks + 1) <<
> > >   
> > 
> > I suspect that you want to do the le16_to_cpu() and _then_
> > add the 1.  Otherwise, adding 1 to a different byte order
> > value won't do quite what is expected or hoped for...
> 
> errm. Right.
> Something like this maybe?
> 
> [PATCH]: ext4: fix big endian access for flex groups
> 
> On big endianess plattforms newly created ext4 file systems cannot be mounted
> and show messages like:
> [6923911.715968] EXT4-fs: not enough memory for 522250 flex groups
> [6923911.715973] EXT4-fs: unable to initialize flex_bg meta info!
> 
> We have to access s_reserved_gdb_blocks with le16_to_cpu.
> 
> Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>

There is already a patch in the patchqueue that fix this
http://repo.or.cz/w/ext4-patch-queue.git?a=blob;f=convert-to-host-order-in-ext4_fill_flex_info;h=ebe7efb3ee78b6cab4db49391f4a005d78a26f03;hb=HEAD


-aneesh

Re: [PATCH]: ext4: fix big endian

[Christoph Hellwig]

On Thu, Nov 06, 2008 at 11:06:25AM -0500, Peter Staubach wrote:
>> Index: kvm/fs/ext4/super.c
>> ===================================================================
>> --- kvm.orig/fs/ext4/super.c
>> +++ kvm/fs/ext4/super.c
>> @@ -1455,7 +1455,7 @@ static int ext4_fill_flex_info(struct su
>>   	/* We allocate both existing and potentially added groups */
>>  	flex_group_count = ((sbi->s_groups_count + groups_per_flex - 1) +
>> -			    ((sbi->s_es->s_reserved_gdt_blocks +1 ) <<
>> +			    (le16_to_cpu(sbi->s_es->s_reserved_gdt_blocks + 1) <<
>>   
>
> I suspect that you want to do the le16_to_cpu() and _then_
> add the 1.  Otherwise, adding 1 to a different byte order
> value won't do quite what is expected or hoped for...

Yes.  And if someone ran sparse over the code both the initial error
and this varaint would be trivial to spot..

Re: [PATCH]: ext4: fix big endian (was: not enough memory for 522250 flex groups)

[Theodore Tso]

On Thu, Nov 06, 2008 at 04:51:49PM +0100, Christian Borntraeger wrote:
> Ok, I think I found it:
> 
> On big endianess plattforms newly created ext4 file systems cannot be mounted
> and show messages like:
> [6923911.715968] EXT4-fs: not enough memory for 522250 flex groups
> [6923911.715973] EXT4-fs: unable to initialize flex_bg meta info!
> 
> We have to access s_reserved_gdb_blocks with le16_to_cpu.

Aneesh has already submitted a patch to fix this; I'll be pushing it
to Linus shortly.

						- Ted

Re: [PATCH]: ext4: fix big endian

[Christian Borntraeger]

Am Donnerstag, 6. November 2008 schrieb Christoph Hellwig:
> > I suspect that you want to do the le16_to_cpu() and _then_
> > add the 1.  Otherwise, adding 1 to a different byte order
> > value won't do quite what is expected or hoped for...
> 
> Yes.  And if someone ran sparse over the code both the initial error
> and this varaint would be trivial to spot..

I think the problem is, that sparse now requires
__CHECK_ENDIAN__
to check for endianess problems. Seems that lots of people are not aware of 
this.

Christian

Re: [PATCH]: ext4: fix big endian

[Theodore Tso]

On Thu, Nov 06, 2008 at 11:38:46AM -0500, Christoph Hellwig wrote:
> 
> Yes.  And if someone ran sparse over the code both the initial error
> and this varaint would be trivial to spot..
> 

That's how Aneesh found it.  :-)

					- Ted

Re: [PATCH]: ext4: fix big endian

[Christoph Hellwig]

On Thu, Nov 06, 2008 at 12:33:27PM -0500, Theodore Tso wrote:
> On Thu, Nov 06, 2008 at 11:38:46AM -0500, Christoph Hellwig wrote:
> > 
> > Yes.  And if someone ran sparse over the code both the initial error
> > and this varaint would be trivial to spot..
> > 
> 
> That's how Aneesh found it.  :-)

Heh, okay.  We really should try to find some annotations that run
sparse with endian checking by default for those parts of the tree
where it makes sense..

Re: [PATCH]: ext4: fix big endian

[Alexey Dobriyan]

On Thu, Nov 06, 2008 at 12:39:44PM -0500, Christoph Hellwig wrote:
> On Thu, Nov 06, 2008 at 12:33:27PM -0500, Theodore Tso wrote:
> > On Thu, Nov 06, 2008 at 11:38:46AM -0500, Christoph Hellwig wrote:
> > > 
> > > Yes.  And if someone ran sparse over the code both the initial error
> > > and this varaint would be trivial to spot..
> > > 
> > 
> > That's how Aneesh found it.  :-)
> 
> Heh, okay.  We really should try to find some annotations that run
> sparse with endian checking by default for those parts of the tree
> where it makes sense..

In fact endian warnings were off by default, because there were too much of
them in networking stack and elsewhere, but situations definitely improved.
Out of head, only drivers/ieee1394/ is not done.

Re: [PATCH]: ext4: fix big endian

[Jörn Engel]

On Thu, 6 November 2008 12:39:44 -0500, Christoph Hellwig wrote:
> 
> Heh, okay.  We really should try to find some annotations that run
> sparse with endian checking by default for those parts of the tree
> where it makes sense..

#define __CHECK_ENDIAN__ in some ext4 header.  Made life much easier for
me.

Jörn

-- 
A surrounded army must be given a way out.
-- Sun Tzu

Re: [PATCH]: ext4: fix big endian

[Harvey Harrison]

On Sat, 2008-11-08 at 23:35 +0100, Jörn Engel wrote:
> On Thu, 6 November 2008 12:39:44 -0500, Christoph Hellwig wrote:
> > 
> > Heh, okay.  We really should try to find some annotations that run
> > sparse with endian checking by default for those parts of the tree
> > where it makes sense..
> 
> #define __CHECK_ENDIAN__ in some ext4 header.  Made life much easier for
> me.
> 

I'd say that it's getting close to just being able to turn it on by
default.  A lot of the really verbose offenders have been annotated
now, drivers/ieee1394, drivers/scsi, drivers/message and some of the
older areas of drivers/net will get noisy, I was going to send an RFC
for 2.6.29 and continue to chip away at the output.

Harvey

Re: [PATCH]: ext4: fix big endian

[Jörn Engel]

On Mon, 10 November 2008 10:09:38 -0800, Harvey Harrison wrote:
> 
> I'd say that it's getting close to just being able to turn it on by
> default.  A lot of the really verbose offenders have been annotated
> now, drivers/ieee1394, drivers/scsi, drivers/message and some of the
> older areas of drivers/net will get noisy, I was going to send an RFC
> for 2.6.29 and continue to chip away at the output.

My argument for the define is that it doesn't hurt much if endian checks
are enabled globally, but helps a lot if they are not.  So until it is
clear that endian checks will be enabled, it may still be a good idea.

Jörn

-- 
It's not whether you win or lose, it's how you place the blame.
-- unknown