v2.6.28-rc1: Regression in ext3/jbd

v2.6.28-rc1: Regression in ext3/jbd

[Vegard Nossum]

[Sorry in advance for the huge Cc, most of it is from the commit.]

Hi,

This commit:

commit 2d7c820e56ce83b23daee9eb5343730fb309418e
Author: Hidehiro Kawai <hidehiro.kawai.ez@hitachi.com>
Date:   Wed Oct 22 14:15:01 2008 -0700

    ext3: add checks for errors from jbd

introduces a regression which was discovered by kmemcheck:

WARNING: kmemcheck: Caught 32-bit read from freed memory (f4f1b804)
00b0f1f4fbffffff404439ef008830f20200000097970000ad4eaddeffffffff
 i i i i f f f f f f f f f f f f f f f f f f f f f f f f f f f f
         ^

Pid: 9550, comm: umount Not tainted (2.6.28-rc1 #58) 945P-A
EIP: 0060:[<c05bdf38>] EFLAGS: 00010246 CPU: 0
EIP is at __journal_abort_soft+0x18/0xa0
EAX: f4f1b800 EBX: f4f1b800 ECX: c0462799 EDX: fffffffb
ESI: fffffffb EDI: f4f1a800 EBP: f145dea8 ESP: c25699c8
 DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
CR0: 8005003b CR2: f6c1d704 CR3: 31448000 CR4: 00000650
DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
DR6: ffff4ff0 DR7: 00000400
 [<c05bdfc8>] journal_abort+0x8/0x10
 [<c0589eb5>] ext3_abort+0xb5/0xc0
 [<c058a300>] ext3_put_super+0x160/0x230
 [<c04ec02a>] generic_shutdown_super+0x5a/0xe0

In particular, this hunk is guilty:

-       journal_destroy(sbi->s_journal);
+       if (journal_destroy(sbi->s_journal) < 0)
+               ext3_abort(sb, __func__, "Couldn't clean up the journal");

because journal_destroy() will free the journal regardless of whether
it returned < 0 or not. And then ext3_abort() makes some calls that
dereference the (freed) journal. These are the line numbers for the
backtrace:

addr2line -e vmlinux -i c05bdf38 c05bdfc8 c0589eb5 c058a300 c04ec02a
fs/jbd/journal.c:1502
fs/jbd/journal.c:1560
fs/ext3/super.c:284
fs/ext3/super.c:397
fs/super.c:307

(as of e013e13bf605b9e6b702adffbe2853cfc60e7806 in Linus's tree).

I hope this helps.


Vegard

[PATCH 1/2] ext3: fix a bug accessing freed memory in ext3_abort

[Hidehiro Kawai]

Hi Vegard,

Vegard Nossum wrote:

> commit 2d7c820e56ce83b23daee9eb5343730fb309418e
> Author: Hidehiro Kawai <hidehiro.kawai.ez@hitachi.com>
> Date:   Wed Oct 22 14:15:01 2008 -0700
> 
>     ext3: add checks for errors from jbd
> 
> introduces a regression which was discovered by kmemcheck:

Thank you for reporting this regression.  It's my fault.
I send patches to fix.

Regards,


Subject: [PATCH 1/2] ext3: fix a bug accessing freed memory in ext3_abort

Vegard Nossum reported a bug which accesses freed memory.  When
journal has been aborted, ext3_put_super() calls ext3_abort() after
freeing the journal_t object, and then ext3_abort() accesses it.
This patch fix it.

Signed-off-by: Hidehiro Kawai <hidehiro.kawai.ez@hitachi.com>
---
 fs/ext3/super.c |   10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

Index: linux-2.6.28-rc2/fs/ext3/super.c
===================================================================
--- linux-2.6.28-rc2.orig/fs/ext3/super.c
+++ linux-2.6.28-rc2/fs/ext3/super.c
@@ -281,7 +281,8 @@ void ext3_abort (struct super_block * sb
 	EXT3_SB(sb)->s_mount_state |= EXT3_ERROR_FS;
 	sb->s_flags |= MS_RDONLY;
 	EXT3_SB(sb)->s_mount_opt |= EXT3_MOUNT_ABORT;
-	journal_abort(EXT3_SB(sb)->s_journal, -EIO);
+	if (EXT3_SB(sb)->s_journal)
+		journal_abort(EXT3_SB(sb)->s_journal, -EIO);
 }
 
 void ext3_warning (struct super_block * sb, const char * function,
@@ -390,11 +391,14 @@ static void ext3_put_super (struct super
 {
 	struct ext3_sb_info *sbi = EXT3_SB(sb);
 	struct ext3_super_block *es = sbi->s_es;
-	int i;
+	int i, err;
 
 	ext3_xattr_put_super(sb);
-	if (journal_destroy(sbi->s_journal) < 0)
+	err = journal_destroy(sbi->s_journal);
+	sbi->s_journal = NULL;
+	if (err < 0)
 		ext3_abort(sb, __func__, "Couldn't clean up the journal");
+
 	if (!(sb->s_flags & MS_RDONLY)) {
 		EXT3_CLEAR_INCOMPAT_FEATURE(sb, EXT3_FEATURE_INCOMPAT_RECOVER);
 		es->s_state = cpu_to_le16(sbi->s_mount_state);

[PATCH 1/2] ext3: fix a bug accessing freed memory in ext3_abort

[Hidehiro Kawai]

Hi Vegard,

Vegard Nossum wrote:

> commit 2d7c820e56ce83b23daee9eb5343730fb309418e
> Author: Hidehiro Kawai <hidehiro.kawai.ez@hitachi.com>
> Date:   Wed Oct 22 14:15:01 2008 -0700
> 
>     ext3: add checks for errors from jbd
> 
> introduces a regression which was discovered by kmemcheck:

Thank you for reporting this regression.  It's my fault.
I send patches to fix.

Regards,


Subject: [PATCH 1/2] ext3: fix a bug accessing freed memory in ext3_abort

Vegard Nossum reported a bug which accesses freed memory.  When
journal has been aborted, ext3_put_super() calls ext3_abort() after
freeing the journal_t object, and then ext3_abort() accesses it.
This patch fix it.

Signed-off-by: Hidehiro Kawai <hidehiro.kawai.ez@hitachi.com>
---
 fs/ext3/super.c |   10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

Index: linux-2.6.28-rc2/fs/ext3/super.c
===================================================================
--- linux-2.6.28-rc2.orig/fs/ext3/super.c
+++ linux-2.6.28-rc2/fs/ext3/super.c
@@ -281,7 +281,8 @@ void ext3_abort (struct super_block * sb
 	EXT3_SB(sb)->s_mount_state |= EXT3_ERROR_FS;
 	sb->s_flags |= MS_RDONLY;
 	EXT3_SB(sb)->s_mount_opt |= EXT3_MOUNT_ABORT;
-	journal_abort(EXT3_SB(sb)->s_journal, -EIO);
+	if (EXT3_SB(sb)->s_journal)
+		journal_abort(EXT3_SB(sb)->s_journal, -EIO);
 }
 
 void ext3_warning (struct super_block * sb, const char * function,
@@ -390,11 +391,14 @@ static void ext3_put_super (struct super
 {
 	struct ext3_sb_info *sbi = EXT3_SB(sb);
 	struct ext3_super_block *es = sbi->s_es;
-	int i;
+	int i, err;
 
 	ext3_xattr_put_super(sb);
-	if (journal_destroy(sbi->s_journal) < 0)
+	err = journal_destroy(sbi->s_journal);
+	sbi->s_journal = NULL;
+	if (err < 0)
 		ext3_abort(sb, __func__, "Couldn't clean up the journal");
+
 	if (!(sb->s_flags & MS_RDONLY)) {
 		EXT3_CLEAR_INCOMPAT_FEATURE(sb, EXT3_FEATURE_INCOMPAT_RECOVER);
 		es->s_state = cpu_to_le16(sbi->s_mount_state);

[PATCH 2/2] ext4: fix a bug accessing freed memory in ext4_abort

[Hidehiro Kawai]

Vegard Nossum reported a bug which accesses freed memory.  When
journal has been aborted, ext4_put_super() calls ext4_abort() after
freeing the journal_t object, and then ext4_abort() accesses it.
This patch fix it.

Signed-off-by: Hidehiro Kawai <hidehiro.kawai.ez@hitachi.com>
---
 fs/ext4/super.c |   11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

Index: linux-2.6.28-rc2/fs/ext4/super.c
===================================================================
--- linux-2.6.28-rc2.orig/fs/ext4/super.c
+++ linux-2.6.28-rc2/fs/ext4/super.c
@@ -333,7 +333,8 @@ void ext4_abort(struct super_block *sb, 
 	EXT4_SB(sb)->s_mount_state |= EXT4_ERROR_FS;
 	sb->s_flags |= MS_RDONLY;
 	EXT4_SB(sb)->s_mount_opt |= EXT4_MOUNT_ABORT;
-	jbd2_journal_abort(EXT4_SB(sb)->s_journal, -EIO);
+	if (EXT4_SB(sb)->s_journal)
+		jbd2_journal_abort(EXT4_SB(sb)->s_journal, -EIO);
 }
 
 void ext4_warning(struct super_block *sb, const char *function,
@@ -442,14 +443,16 @@ static void ext4_put_super(struct super_
 {
 	struct ext4_sb_info *sbi = EXT4_SB(sb);
 	struct ext4_super_block *es = sbi->s_es;
-	int i;
+	int i, err;
 
 	ext4_mb_release(sb);
 	ext4_ext_release(sb);
 	ext4_xattr_put_super(sb);
-	if (jbd2_journal_destroy(sbi->s_journal) < 0)
-		ext4_abort(sb, __func__, "Couldn't clean up the journal");
+	err = jbd2_journal_destroy(sbi->s_journal);
 	sbi->s_journal = NULL;
+	if (err < 0)
+		ext4_abort(sb, __func__, "Couldn't clean up the journal");
+
 	if (!(sb->s_flags & MS_RDONLY)) {
 		EXT4_CLEAR_INCOMPAT_FEATURE(sb, EXT4_FEATURE_INCOMPAT_RECOVER);
 		es->s_state = cpu_to_le16(sbi->s_mount_state);

Re: [PATCH 1/2] ext3: fix a bug accessing freed memory in ext3_abort

[Jan Kara]

> Hi Vegard,
> 
> Vegard Nossum wrote:
> 
> > commit 2d7c820e56ce83b23daee9eb5343730fb309418e
> > Author: Hidehiro Kawai <hidehiro.kawai.ez@hitachi.com>
> > Date:   Wed Oct 22 14:15:01 2008 -0700
> > 
> >     ext3: add checks for errors from jbd
> > 
> > introduces a regression which was discovered by kmemcheck:
> 
> Thank you for reporting this regression.  It's my fault.
> I send patches to fix.
> 
> Regards,
> 
> 
> Subject: [PATCH 1/2] ext3: fix a bug accessing freed memory in ext3_abort
> 
> Vegard Nossum reported a bug which accesses freed memory.  When
> journal has been aborted, ext3_put_super() calls ext3_abort() after
> freeing the journal_t object, and then ext3_abort() accesses it.
> This patch fix it.
> 
> Signed-off-by: Hidehiro Kawai <hidehiro.kawai.ez@hitachi.com>
  The patch looks fine. Acked-by: Jan Kara <jack@suse.cz>

								Honza
> ---
>  fs/ext3/super.c |   10 +++++++---
>  1 file changed, 7 insertions(+), 3 deletions(-)
> 
> Index: linux-2.6.28-rc2/fs/ext3/super.c
> ===================================================================
> --- linux-2.6.28-rc2.orig/fs/ext3/super.c
> +++ linux-2.6.28-rc2/fs/ext3/super.c
> @@ -281,7 +281,8 @@ void ext3_abort (struct super_block * sb
>  	EXT3_SB(sb)->s_mount_state |= EXT3_ERROR_FS;
>  	sb->s_flags |= MS_RDONLY;
>  	EXT3_SB(sb)->s_mount_opt |= EXT3_MOUNT_ABORT;
> -	journal_abort(EXT3_SB(sb)->s_journal, -EIO);
> +	if (EXT3_SB(sb)->s_journal)
> +		journal_abort(EXT3_SB(sb)->s_journal, -EIO);
>  }
>  
>  void ext3_warning (struct super_block * sb, const char * function,
> @@ -390,11 +391,14 @@ static void ext3_put_super (struct super
>  {
>  	struct ext3_sb_info *sbi = EXT3_SB(sb);
>  	struct ext3_super_block *es = sbi->s_es;
> -	int i;
> +	int i, err;
>  
>  	ext3_xattr_put_super(sb);
> -	if (journal_destroy(sbi->s_journal) < 0)
> +	err = journal_destroy(sbi->s_journal);
> +	sbi->s_journal = NULL;
> +	if (err < 0)
>  		ext3_abort(sb, __func__, "Couldn't clean up the journal");
> +
>  	if (!(sb->s_flags & MS_RDONLY)) {
>  		EXT3_CLEAR_INCOMPAT_FEATURE(sb, EXT3_FEATURE_INCOMPAT_RECOVER);
>  		es->s_state = cpu_to_le16(sbi->s_mount_state);
> 
-- 
Jan Kara <jack@suse.cz>
SuSE CR Labs

Re: [PATCH 2/2] ext4: fix a bug accessing freed memory in ext4_abort

[Jan Kara]

> Vegard Nossum reported a bug which accesses freed memory.  When
> journal has been aborted, ext4_put_super() calls ext4_abort() after
> freeing the journal_t object, and then ext4_abort() accesses it.
> This patch fix it.
> 
> Signed-off-by: Hidehiro Kawai <hidehiro.kawai.ez@hitachi.com>
  And this one as well. Acked-by: Jan Kara <jack@suse.cz>
> ---
>  fs/ext4/super.c |   11 +++++++----
>  1 file changed, 7 insertions(+), 4 deletions(-)
> 
> Index: linux-2.6.28-rc2/fs/ext4/super.c
> ===================================================================
> --- linux-2.6.28-rc2.orig/fs/ext4/super.c
> +++ linux-2.6.28-rc2/fs/ext4/super.c
> @@ -333,7 +333,8 @@ void ext4_abort(struct super_block *sb, 
>  	EXT4_SB(sb)->s_mount_state |= EXT4_ERROR_FS;
>  	sb->s_flags |= MS_RDONLY;
>  	EXT4_SB(sb)->s_mount_opt |= EXT4_MOUNT_ABORT;
> -	jbd2_journal_abort(EXT4_SB(sb)->s_journal, -EIO);
> +	if (EXT4_SB(sb)->s_journal)
> +		jbd2_journal_abort(EXT4_SB(sb)->s_journal, -EIO);
>  }
>  
>  void ext4_warning(struct super_block *sb, const char *function,
> @@ -442,14 +443,16 @@ static void ext4_put_super(struct super_
>  {
>  	struct ext4_sb_info *sbi = EXT4_SB(sb);
>  	struct ext4_super_block *es = sbi->s_es;
> -	int i;
> +	int i, err;
>  
>  	ext4_mb_release(sb);
>  	ext4_ext_release(sb);
>  	ext4_xattr_put_super(sb);
> -	if (jbd2_journal_destroy(sbi->s_journal) < 0)
> -		ext4_abort(sb, __func__, "Couldn't clean up the journal");
> +	err = jbd2_journal_destroy(sbi->s_journal);
>  	sbi->s_journal = NULL;
> +	if (err < 0)
> +		ext4_abort(sb, __func__, "Couldn't clean up the journal");
> +
>  	if (!(sb->s_flags & MS_RDONLY)) {
>  		EXT4_CLEAR_INCOMPAT_FEATURE(sb, EXT4_FEATURE_INCOMPAT_RECOVER);
>  		es->s_state = cpu_to_le16(sbi->s_mount_state);
> 
-- 
Jan Kara <jack@suse.cz>
SuSE CR Labs

Re: [PATCH 2/2] ext4: fix a bug accessing freed memory in ext4_abort

[Ingo Molnar]

* Hidehiro Kawai <hidehiro.kawai.ez@hitachi.com> wrote:

> Vegard Nossum reported a bug which accesses freed memory.  When
> journal has been aborted, ext4_put_super() calls ext4_abort() after
> freeing the journal_t object, and then ext4_abort() accesses it.
> This patch fix it.
> 
> Signed-off-by: Hidehiro Kawai <hidehiro.kawai.ez@hitachi.com>

could you please put the word "kmemcheck" into the commit log? ("found 
via kmemcheck" or so) We want to have an easily git-greppable track 
record of upstream kernel bugs that were found via kmemcheck. (and 
there's already a long list)

	Ingo

[PATCH 1/2 take 2] ext3: fix a bug accessing freed memory in ext3_abort

[Hidehiro Kawai]

Ingo Molnar wrote:

> * Hidehiro Kawai <hidehiro.kawai.ez@hitachi.com> wrote:
> 
>>Vegard Nossum reported a bug which accesses freed memory.  When
>>journal has been aborted, ext4_put_super() calls ext4_abort() after
>>freeing the journal_t object, and then ext4_abort() accesses it.
>>This patch fix it.
>>
>>Signed-off-by: Hidehiro Kawai <hidehiro.kawai.ez@hitachi.com>
> 
> could you please put the word "kmemcheck" into the commit log? ("found 
> via kmemcheck" or so) We want to have an easily git-greppable track 
> record of upstream kernel bugs that were found via kmemcheck. (and 
> there's already a long list)

Ah, OK.  I fixed the commit log.

Thanks,


Subject: [PATCH 1/2] ext3: fix a bug accessing freed memory in ext3_abort

Vegard Nossum reported a bug which accesses freed memory (found via
kmemcheck).  When journal has been aborted, ext3_put_super() calls
ext3_abort() after freeing the journal_t object, and then ext3_abort()
accesses it.  This patch fix it.

Signed-off-by: Hidehiro Kawai <hidehiro.kawai.ez@hitachi.com>
Acked-by: Jan Kara <jack@suse.cz>
---
 fs/ext3/super.c |   10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

Index: linux-2.6.28-rc2/fs/ext3/super.c
===================================================================
--- linux-2.6.28-rc2.orig/fs/ext3/super.c
+++ linux-2.6.28-rc2/fs/ext3/super.c
@@ -281,7 +281,8 @@ void ext3_abort (struct super_block * sb
 	EXT3_SB(sb)->s_mount_state |= EXT3_ERROR_FS;
 	sb->s_flags |= MS_RDONLY;
 	EXT3_SB(sb)->s_mount_opt |= EXT3_MOUNT_ABORT;
-	journal_abort(EXT3_SB(sb)->s_journal, -EIO);
+	if (EXT3_SB(sb)->s_journal)
+		journal_abort(EXT3_SB(sb)->s_journal, -EIO);
 }
 
 void ext3_warning (struct super_block * sb, const char * function,
@@ -390,11 +391,14 @@ static void ext3_put_super (struct super
 {
 	struct ext3_sb_info *sbi = EXT3_SB(sb);
 	struct ext3_super_block *es = sbi->s_es;
-	int i;
+	int i, err;
 
 	ext3_xattr_put_super(sb);
-	if (journal_destroy(sbi->s_journal) < 0)
+	err = journal_destroy(sbi->s_journal);
+	sbi->s_journal = NULL;
+	if (err < 0)
 		ext3_abort(sb, __func__, "Couldn't clean up the journal");
+
 	if (!(sb->s_flags & MS_RDONLY)) {
 		EXT3_CLEAR_INCOMPAT_FEATURE(sb, EXT3_FEATURE_INCOMPAT_RECOVER);
 		es->s_state = cpu_to_le16(sbi->s_mount_state);

[PATCH 2/2 take 2] ext4: fix a bug accessing freed memory in ext4_abort

[Hidehiro Kawai]

Vegard Nossum reported a bug which accesses freed memory (found via
kmemcheck).  When journal has been aborted, ext4_put_super() calls
ext4_abort() after freeing the journal_t object, and then ext4_abort()
accesses it.  This patch fix it.

Signed-off-by: Hidehiro Kawai <hidehiro.kawai.ez@hitachi.com>
Acked-by: Jan Kara <jack@suse.cz>
---
 fs/ext4/super.c |   11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

Index: linux-2.6.28-rc2/fs/ext4/super.c
===================================================================
--- linux-2.6.28-rc2.orig/fs/ext4/super.c
+++ linux-2.6.28-rc2/fs/ext4/super.c
@@ -333,7 +333,8 @@ void ext4_abort(struct super_block *sb, 
 	EXT4_SB(sb)->s_mount_state |= EXT4_ERROR_FS;
 	sb->s_flags |= MS_RDONLY;
 	EXT4_SB(sb)->s_mount_opt |= EXT4_MOUNT_ABORT;
-	jbd2_journal_abort(EXT4_SB(sb)->s_journal, -EIO);
+	if (EXT4_SB(sb)->s_journal)
+		jbd2_journal_abort(EXT4_SB(sb)->s_journal, -EIO);
 }
 
 void ext4_warning(struct super_block *sb, const char *function,
@@ -442,14 +443,16 @@ static void ext4_put_super(struct super_
 {
 	struct ext4_sb_info *sbi = EXT4_SB(sb);
 	struct ext4_super_block *es = sbi->s_es;
-	int i;
+	int i, err;
 
 	ext4_mb_release(sb);
 	ext4_ext_release(sb);
 	ext4_xattr_put_super(sb);
-	if (jbd2_journal_destroy(sbi->s_journal) < 0)
-		ext4_abort(sb, __func__, "Couldn't clean up the journal");
+	err = jbd2_journal_destroy(sbi->s_journal);
 	sbi->s_journal = NULL;
+	if (err < 0)
+		ext4_abort(sb, __func__, "Couldn't clean up the journal");
+
 	if (!(sb->s_flags & MS_RDONLY)) {
 		EXT4_CLEAR_INCOMPAT_FEATURE(sb, EXT4_FEATURE_INCOMPAT_RECOVER);
 		es->s_state = cpu_to_le16(sbi->s_mount_state);

Re: [PATCH 2/2 take 2] ext4: fix a bug accessing freed memory in ext4_abort

[Theodore Tso]

Many thanks, Kawai-san, it looks good.  I'll get this queued for
mainline.

		   	 	     	 - Ted