LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Miklos Szeredi <miklos@szeredi.hu>
To: akpm@linux-foundation.org
Cc: miklos@szeredi.hu, linux-kernel@vger.kernel.org, linux-mm@kvack.org
Subject: Re: dirty balancing deadlock
Date: Thu, 22 Feb 2007 09:02:24 +0100	[thread overview]
Message-ID: <E1HK8uG-0005OT-00@dorka.pomaz.szeredi.hu> (raw)
In-Reply-To: <20070221235532.2361f827.akpm@linux-foundation.org> (message from Andrew Morton on Wed, 21 Feb 2007 23:55:32 -0800)

> > On Thu, 22 Feb 2007 08:42:26 +0100 Miklos Szeredi <miklos@szeredi.hu> wrote:
> > > > 
> > > > Index: linux/mm/page-writeback.c
> > > > ===================================================================
> > > > --- linux.orig/mm/page-writeback.c	2007-02-19 17:32:41.000000000 +0100
> > > > +++ linux/mm/page-writeback.c	2007-02-19 18:05:28.000000000 +0100
> > > > @@ -198,6 +198,25 @@ static void balance_dirty_pages(struct a
> > > >  			dirty_thresh)
> > > >  				break;
> > > >  
> > > > +		/*
> > > > +		 * Acquit this producer if there's little or nothing
> > > > +		 * to write back to this particular queue
> > > > +		 *
> > > > +		 * Without this check a deadlock is possible in the
> > > > +		 * following case:
> > > > +		 *
> > > > +		 * - filesystem A writes data through filesystem B
> > > > +		 * - filesystem A has dirty pages over dirty_thresh
> > > > +		 * - writeback is started, this triggers a write in B
> > > > +		 * - balance_dirty_pages() is called synchronously
> > > > +		 * - the write to B blocks
> > > > +		 * - the writeback completes, but dirty is still over threshold
> > > > +		 * - the blocking write prevents futher writes from happening
> > > > +		 */
> > > > +		if (atomic_long_read(&bdi->nr_dirty) +
> > > > +		    atomic_long_read(&bdi->nr_writeback) < 16)
> > > > +			break;
> > > > +
> > > 
> > > The problem seems to that little "- the write to B blocks".
> > > 
> > > How come it blocks?  I mean, if we cannot retire writes to that filesystem
> > > then we're screwed anyway.
> > 
> > Sorry about the sloppy description.  I mean, it's not the lowlevel
> > write that will block, but rather the VFS one
> > (generic_file_aio_write).  It will block (or rather loop forever with
> > 0.1 second sleeps) in balance_dirty_pages().  That means, that for
> > this inode, i_mutex is held and no other writer can continue the work.
> 
> "this inode" I assume is the inode against filesystem A?

No, the one in B.

> Why does holding that inode's i_mutex prevent further writeback of
> pages in A?

It is generic_file_aio_write() that is holding the mutex.

Here's the stack for the filesystem daemon trying to write back a page:

08dcfb40:  [<08182fe6>] schedule+0x246/0x547
08dcfb98:  [<08183a03>] schedule_timeout+0x4e/0xb6
08dcfbcc:  [<08183991>] io_schedule_timeout+0x11/0x20
08dcfbd4:  [<080a0cf2>] congestion_wait+0x72/0x87
08dcfc04:  [<0809c693>] balance_dirty_pages+0xa8/0x153
08dcfc5c:  [<0809c7bf>] balance_dirty_pages_ratelimited_nr+0x43/0x45
08dcfc68:  [<080992b5>] generic_file_buffered_write+0x3e3/0x6f5
08dcfd20:  [<0809988e>] __generic_file_aio_write_nolock+0x2c7/0x5dd
08dcfda8:  [<08099cb6>] generic_file_aio_write+0x55/0xc7
08dcfddc:  [<080ea1e6>] ext3_file_write+0x39/0xaf
08dcfe04:  [<080b060b>] do_sync_write+0xd8/0x10e
08dcfebc:  [<080b06e3>] vfs_write+0xa2/0x1cb
08dcfeec:  [<080b09b8>] sys_pwrite64+0x65/0x69

Miklos

      reply	other threads:[~2007-02-22  8:02 UTC|newest]

Thread overview: 26+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2007-02-18 18:28 Miklos Szeredi
2007-02-18 20:53 ` Andrew Morton
2007-02-18 21:25   ` Rik van Riel
2007-02-18 22:54     ` Miklos Szeredi
2007-02-18 22:50   ` Miklos Szeredi
2007-02-18 22:59     ` Andrew Morton
2007-02-18 23:22       ` Miklos Szeredi
2007-02-18 23:59         ` Andrew Morton
2007-02-19  0:25           ` Miklos Szeredi
2007-02-19  0:30             ` Miklos Szeredi
2007-02-19  0:45             ` Miklos Szeredi
2007-02-19  0:45             ` Chris Mason
2007-02-19  0:54               ` Miklos Szeredi
2007-02-19  1:01                 ` Chris Mason
2007-02-19  1:14                   ` Miklos Szeredi
2007-02-20  0:16                     ` Chris Mason
2007-02-20  8:53                       ` Miklos Szeredi
2007-02-19 17:11           ` Miklos Szeredi
2007-02-19 23:12             ` Miklos Szeredi
2007-02-20  0:13             ` Chris Mason
2007-02-20  8:47               ` Miklos Szeredi
2007-02-20 11:30                 ` Chris Mason
2007-02-21 21:36             ` Andrew Morton
2007-02-22  7:42               ` Miklos Szeredi
2007-02-22  7:55                 ` Andrew Morton
2007-02-22  8:02                   ` Miklos Szeredi [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=E1HK8uG-0005OT-00@dorka.pomaz.szeredi.hu \
    --to=miklos@szeredi.hu \
    --cc=akpm@linux-foundation.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --subject='Re: dirty balancing deadlock' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).