Linux-Fsdevel Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Junxiao Bi <junxiao.bi@oracle.com>
To: Konstantin Khlebnikov <koct9i@gmail.com>
Cc: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	linux-fsdevel <linux-fsdevel@vger.kernel.org>,
	linux-mm@kvack.org, Alexander Viro <viro@zeniv.linux.org.uk>,
	Waiman Long <longman@redhat.com>,
	Gautham Ananthakrishna <gautham.ananthakrishna@oracle.com>,
	matthew.wilcox@oracle.com
Subject: Re: [PATCH RFC 0/8] dcache: increase poison resistance
Date: Mon, 14 Dec 2020 15:10:25 -0800	[thread overview]
Message-ID: <04b4d5cf-780d-83a9-2b2b-80ae6029ae2c@oracle.com> (raw)
In-Reply-To: <CALYGNiM8Fp=ZV8S6c2L50ne1cGhE30PrT-C=4nfershvfAgP+Q@mail.gmail.com>

On 12/13/20 11:43 PM, Konstantin Khlebnikov wrote:

>
>
> On Sun, Dec 13, 2020 at 9:52 PM Junxiao Bi <junxiao.bi@oracle.com 
> <mailto:junxiao.bi@oracle.com>> wrote:
>
>     On 12/11/20 11:32 PM, Konstantin Khlebnikov wrote:
>
>     > On Thu, Dec 10, 2020 at 2:01 AM Junxiao Bi
>     <junxiao.bi@oracle.com <mailto:junxiao.bi@oracle.com>
>     > <mailto:junxiao.bi@oracle.com <mailto:junxiao.bi@oracle.com>>>
>     wrote:
>     >
>     >     Hi Konstantin,
>     >
>     >     We tested this patch set recently and found it limiting negative
>     >     dentry
>     >     to a small part of total memory. And also we don't see any
>     >     performance
>     >     regression on it. Do you have any plan to integrate it into
>     >     mainline? It
>     >     will help a lot on memory fragmentation issue causing by
>     dentry slab,
>     >     there were a lot of customer cases where sys% was very high
>     since
>     >     most
>     >     cpu were doing memory compaction, dentry slab was taking too
>     much
>     >     memory
>     >     and nearly all dentry there were negative.
>     >
>     >
>     > Right now I don't have any plans for this. I suspect such
>     problems will
>     > appear much more often since machines are getting bigger.
>     > So, somebody will take care of it.
>     We already had a lot of customer cases. It made no sense to leave so
>     many negative dentry in the system, it caused memory fragmentation
>     and
>     not much benefit.
>
>
> Dcache could grow so big only if the system lacks of memory pressure.
>
> Simplest solution is a cronjob which provinces such pressure by
> creating sparse file on disk-based fs and then reading it.
> This should wash away all inactive caches with no IO and zero chance 
> of oom.
Sound good, will try.
>
>     >
>     > First part which collects negative dentries at the end list of
>     > siblings could be
>     > done in a more obvious way by splitting the list in two.
>     > But this touches much more code.
>     That would add new field to dentry?
>
>
> Yep. Decision is up to maintainers.
>
>     >
>     > Last patch isn't very rigid but does non-trivial changes.
>     > Probably it's better to call some garbage collector thingy
>     periodically.
>     > Lru list needs pressure to age and reorder entries properly.
>
>     Swap the negative dentry to the head of hash list when it get
>     accessed?
>     Extra ones can be easily trimmed when swapping, using GC is to reduce
>     perf impact?
>
>
> Reclaimer/shrinker scans denties in LRU lists, it's an another list.

Ah, you mean GC to reclaim from LRU list. I am not sure it could catch 
up the speed of negative dentry generating.

Thanks,

Junxiao.

> My patch used order in hash lists is a very unusual way. Don't be 
> confused.
>
> There are four lists
> parent - siblings
> hashtable - hashchain
> LRU
> inode - alias
>
>
>     Thanks,
>
>     Junxioao.
>
>     >
>     > Gc could be off by default or thresholds set very high (50% of
>     ram for
>     > example).
>     > Final setup could be left up to owners of large systems, which
>     needs
>     > fine tuning.
>

  parent reply	other threads:[~2020-12-14 23:12 UTC|newest]

Thread overview: 24+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-05-08 12:23 Konstantin Khlebnikov
2020-05-08 12:23 ` [PATCH RFC 1/8] dcache: show count of hash buckets in sysctl fs.dentry-state Konstantin Khlebnikov
2020-05-08 14:49   ` Waiman Long
2020-05-08 16:16     ` Konstantin Khlebnikov
2020-05-08 19:05       ` Waiman Long
2020-05-08 19:38         ` Konstantin Khlebnikov
2020-05-08 20:00           ` Waiman Long
2020-05-08 20:03             ` Matthew Wilcox
2020-05-08 12:23 ` [PATCH RFC 2/8] selftests: add stress testing tool for dcache Konstantin Khlebnikov
2020-05-13  1:52   ` Dave Chinner
2020-05-08 12:23 ` [PATCH RFC 3/8] dcache: sweep cached negative dentries to the end of list of siblings Konstantin Khlebnikov
2020-05-08 19:38   ` Waiman Long
2020-05-08 12:23 ` [PATCH RFC 4/8] fsnotify: stop walking child dentries if remaining tail is negative Konstantin Khlebnikov
2020-05-08 12:23 ` [PATCH RFC 5/8] dcache: add action D_WALK_SKIP_SIBLINGS to d_walk() Konstantin Khlebnikov
2020-05-08 12:23 ` [PATCH RFC 6/8] dcache: stop walking siblings if remaining dentries all negative Konstantin Khlebnikov
2020-05-08 12:23 ` [PATCH RFC 7/8] dcache: push releasing dentry lock into sweep_negative Konstantin Khlebnikov
2020-05-08 12:23 ` [PATCH RFC 8/8] dcache: prevent flooding with negative dentries Konstantin Khlebnikov
2020-05-08 14:56   ` Matthew Wilcox
2020-05-08 16:29     ` Konstantin Khlebnikov
2020-05-08 21:07   ` Waiman Long
2020-12-09 23:01 ` [PATCH RFC 0/8] dcache: increase poison resistance Junxiao Bi
     [not found]   ` <CALYGNiN2F8gcKX+2nKOi1tapquJWfyzUkajWxTqgd9xvd7u1AA@mail.gmail.com>
2020-12-13 18:49     ` Junxiao Bi
     [not found]       ` <CALYGNiM8Fp=ZV8S6c2L50ne1cGhE30PrT-C=4nfershvfAgP+Q@mail.gmail.com>
2020-12-14 23:10         ` Junxiao Bi [this message]
2020-12-16 18:46           ` Junxiao Bi

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=04b4d5cf-780d-83a9-2b2b-80ae6029ae2c@oracle.com \
    --to=junxiao.bi@oracle.com \
    --cc=gautham.ananthakrishna@oracle.com \
    --cc=khlebnikov@yandex-team.ru \
    --cc=koct9i@gmail.com \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=longman@redhat.com \
    --cc=matthew.wilcox@oracle.com \
    --cc=viro@zeniv.linux.org.uk \
    --subject='Re: [PATCH RFC 0/8] dcache: increase poison resistance' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).