LKML Archive on
help / color / mirror / Atom feed
From: Andrew Morton <>
To: Miklos Szeredi <>
Cc: "Russ Cox" <>,,
Subject: Re: [PATCH] fix quadratic behavior of shrink_dcache_parent()
Date: Fri, 9 Feb 2007 16:00:55 -0800	[thread overview]
Message-ID: <> (raw)
In-Reply-To: <>

On Fri, 09 Feb 2007 23:01:06 +0100
Miklos Szeredi <> wrote:

> From: Miklos Szeredi <>
> The time shrink_dcache_parent() takes, grows quadratically with the
> depth of the tree under 'parent'.  This starts to get noticable at
> about 10,000.
> These kinds of depths don't occur normally, and filesystems which
> invoke shrink_dcache_parent() via d_invalidate() seem to have other
> depth dependent timings, so it's not even easy to expose this problem.
> However with FUSE it's easy to create a deep tree and d_invalidate()
> will also get called.  This can make a syscall hang for a very long
> time.
> This is the original discovery of the problem by Russ Cox:

"The file system mounted on /tmp/z in the example contains 2^50
directories".   heh.

I do wonder how realistic this problem is in real life.

> The following patch fixes the quadratic behavior, by optionally
> allowing prune_dcache() to prune ancestors of a dentry in one go,
> instead of doing it one at a time.
> Common code in dput() and prune_one_dentry() is extracted into a new
> helper function d_kill().
> shrink_dcache_parent() as well as shrink_dcache_sb() are converted to
> use the ancestry-pruner option.  Only for shrink_dcache_memory() is
> this behavior not desirable, so it keeps using the old algorithm.

I wonder if we should be setting shrink_parents=1 in
shrink_dcache_memory()?  Because we have this problem where the dentry
slabs suffer lots of internal fragmentation and we end up with whole slab
pages pinned by a single directory dentry.  I expect that if
shrink_dcache_memory() were aggressive about reaping newly-childless
directory dentries, some improvements might be realised there.

If so, we should change prune_dcache() to return the number pruned, so that
shrink_dcache_memory() can keep its arithmetic correct.  Would require some
careful testing and is out of scope for your work.

  reply	other threads:[~2007-02-10  0:01 UTC|newest]

Thread overview: 8+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2007-02-09 22:01 [PATCH] fix quadratic behavior of shrink_dcache_parent() Miklos Szeredi
2007-02-10  0:00 ` Andrew Morton [this message]
2007-02-10  0:23   ` Russ Cox
2007-02-10  0:40     ` Andrew Morton
2007-02-10  8:46     ` Miklos Szeredi
2007-02-10 10:51       ` Andi Kleen
2007-02-10 12:57       ` Russ Cox
2007-02-11 12:13         ` Miklos Szeredi

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \ \ \ \ \ \ \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).