LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: David Chinner <dgc@sgi.com>
To: "Ramy M. Hassan " <ramy@gawab.com>
Cc: linux-kernel@vger.kernel.org, Ahmed El Zein <a.elzein@gawab.net>,
	xfs@oss.sgi.com
Subject: Re: xfs internal error on a new filesystem
Date: Thu, 15 Feb 2007 20:16:10 +1100	[thread overview]
Message-ID: <20070215091610.GB5743@melbourne.sgi.com> (raw)
In-Reply-To: <20070214102432.6346.qmail@info6.gawab.com>

On Wed, Feb 14, 2007 at 10:24:27AM +0000, Ramy M. Hassan  wrote:
> Hello,
> We got the following xfs internal error on one of our production servers:
> 
> Feb 14 08:28:52 info6 kernel: [238186.676483] Filesystem "sdd8": XFS
> internal error xfs_trans_cancel at line 1138 of file fs/xfs/xfs_trans.c. 
> Caller 0xf8b906e7

Real stack looks to be:

 xfs_trans_cancel
 xfs_mkdir
 xfs_vn_mknod
 xfs_vn_mkdir
 vfs_mkdir
 sys_mkdirat
 sys_mkdir

We aborted a transaction for some reason. We got an error somewhere in
a mkdir while we had a dirty transaction.  Unfortunately, this tells us very
little about the error that actually caused the shutdown.

What is your filessytem layout? (xfs_info <mntpt>) How much memory
do you have and were you near enomem conditions?

> We were able to unmount/remount the volume (didn't do xfs_repair because we
> thought it might take long time, and the server was already in production
> at the moement)

Risky to run a production system on a filesystem that might be corrupted.
You risk further problems if you don't run repair....

> The file system was created less than 48hours ago, and 370G of sensitve
> production data was moved to the server before it xfs crash.

So that's not a "new" filesystem at all...

FWIW, did you do any offline testing before you put it into production?

> System details :
> Kernel: 2.6.18
> Controller: 3ware 9550SX-8LP (RAID 10)

Can you describe your dm/md volume layout?

> We are wondering here if this problem is an indicator to data corruption on
> disk ?

It might be. You didn't run xfs_check or xfs_repair, so we don't know if
there is any on disk corruption here.

> is it really necessary to run xfs_repair ?

If you want to know if you haven't left any landmines around for the
filesystem to trip over again. i.e. You should run repair after any
sort of XFS shutdown to make sure nothing is corrupted on disk.
If nothing is corrupted on disk, then we are looking at an in-memory
problem....

> Do u recommend that we switch back to reiserfs ?

Not yet.

> Could it be a hardware related problems  ?

Yes. Do you have ECC memory on your server? Have you run memtest86?
Were there any I/O errors in the log prior to the shutdown message?

Cheers,

Dave.
-- 
Dave Chinner
Principal Engineer
SGI Australian Software Group

  parent reply	other threads:[~2007-02-15  9:16 UTC|newest]

Thread overview: 7+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2007-02-14 10:24 Ramy M. Hassan 
2007-02-14 10:40 ` Jan-Benedict Glaw
2007-02-14 10:48 ` Patrick Ale
2007-02-15  9:16 ` David Chinner [this message]
2007-02-15 16:19 Ahmed El Zein
2007-02-16 17:58 ` David Chinner
2007-02-18 13:56 ` Leon Kolchinsky

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20070215091610.GB5743@melbourne.sgi.com \
    --to=dgc@sgi.com \
    --cc=a.elzein@gawab.net \
    --cc=linux-kernel@vger.kernel.org \
    --cc=ramy@gawab.com \
    --cc=xfs@oss.sgi.com \
    --subject='Re: xfs internal error on a new filesystem' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).