Linux-Fsdevel Archive on lore.kernel.org
help / color / mirror / Atom feed
* [PATCH RFC 0/8] dcache: increase poison resistance
@ 2020-05-08 12:23 Konstantin Khlebnikov
  2020-05-08 12:23 ` [PATCH RFC 1/8] dcache: show count of hash buckets in sysctl fs.dentry-state Konstantin Khlebnikov
                   ` (8 more replies)
  0 siblings, 9 replies; 24+ messages in thread
From: Konstantin Khlebnikov @ 2020-05-08 12:23 UTC (permalink / raw)
  To: linux-kernel, linux-fsdevel, linux-mm, Alexander Viro; +Cc: Waiman Long

For most filesystems result of every negative lookup is cached, content of
directories is usually cached too. Production of negative dentries isn't
limited with disk speed. It's really easy to generate millions of them if
system has enough memory.

Getting this memory back ins't that easy because slab frees pages only when
all related objects are gone. While dcache shrinker works in LRU order.

Typical scenario is an idle system where some process periodically creates
temporary files and removes them. After some time, memory will be filled
with negative dentries for these random file names.

Simple lookup of random names also generates negative dentries very fast.
Constant flow of such negative denries drains all other inactive caches.

Negative dentries are linked into siblings list along with normal positive
dentries. Some operations walks dcache tree but looks only for positive
dentries: most important is fsnotify/inotify. Hordes of negative dentries
slow down these operations significantly.

Time of dentry lookup is usually unaffected because hash table grows along
with size of memory. Unless somebody especially crafts hash collisions.

This patch set solves all of these problems:

Move negative denries to the end of sliblings list, thus walkers could
skip them at first sight (patches 3-6).

Keep in dcache at most three unreferenced negative denties in row in each
hash bucket (patches 7-8).

---

Konstantin Khlebnikov (8):
      dcache: show count of hash buckets in sysctl fs.dentry-state
      selftests: add stress testing tool for dcache
      dcache: sweep cached negative dentries to the end of list of siblings
      fsnotify: stop walking child dentries if remaining tail is negative
      dcache: add action D_WALK_SKIP_SIBLINGS to d_walk()
      dcache: stop walking siblings if remaining dentries all negative
      dcache: push releasing dentry lock into sweep_negative
      dcache: prevent flooding with negative dentries


 fs/dcache.c                                   | 144 +++++++++++-
 fs/libfs.c                                    |  10 +-
 fs/notify/fsnotify.c                          |   6 +-
 include/linux/dcache.h                        |   6 +
 tools/testing/selftests/filesystems/Makefile  |   1 +
 .../selftests/filesystems/dcache_stress.c     | 210 ++++++++++++++++++
 6 files changed, 370 insertions(+), 7 deletions(-)
 create mode 100644 tools/testing/selftests/filesystems/dcache_stress.c

--
Signature

^ permalink raw reply	[flat|nested] 24+ messages in thread

end of thread, other threads:[~2020-12-16 18:47 UTC | newest]

Thread overview: 24+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-05-08 12:23 [PATCH RFC 0/8] dcache: increase poison resistance Konstantin Khlebnikov
2020-05-08 12:23 ` [PATCH RFC 1/8] dcache: show count of hash buckets in sysctl fs.dentry-state Konstantin Khlebnikov
2020-05-08 14:49   ` Waiman Long
2020-05-08 16:16     ` Konstantin Khlebnikov
2020-05-08 19:05       ` Waiman Long
2020-05-08 19:38         ` Konstantin Khlebnikov
2020-05-08 20:00           ` Waiman Long
2020-05-08 20:03             ` Matthew Wilcox
2020-05-08 12:23 ` [PATCH RFC 2/8] selftests: add stress testing tool for dcache Konstantin Khlebnikov
2020-05-13  1:52   ` Dave Chinner
2020-05-08 12:23 ` [PATCH RFC 3/8] dcache: sweep cached negative dentries to the end of list of siblings Konstantin Khlebnikov
2020-05-08 19:38   ` Waiman Long
2020-05-08 12:23 ` [PATCH RFC 4/8] fsnotify: stop walking child dentries if remaining tail is negative Konstantin Khlebnikov
2020-05-08 12:23 ` [PATCH RFC 5/8] dcache: add action D_WALK_SKIP_SIBLINGS to d_walk() Konstantin Khlebnikov
2020-05-08 12:23 ` [PATCH RFC 6/8] dcache: stop walking siblings if remaining dentries all negative Konstantin Khlebnikov
2020-05-08 12:23 ` [PATCH RFC 7/8] dcache: push releasing dentry lock into sweep_negative Konstantin Khlebnikov
2020-05-08 12:23 ` [PATCH RFC 8/8] dcache: prevent flooding with negative dentries Konstantin Khlebnikov
2020-05-08 14:56   ` Matthew Wilcox
2020-05-08 16:29     ` Konstantin Khlebnikov
2020-05-08 21:07   ` Waiman Long
2020-12-09 23:01 ` [PATCH RFC 0/8] dcache: increase poison resistance Junxiao Bi
     [not found]   ` <CALYGNiN2F8gcKX+2nKOi1tapquJWfyzUkajWxTqgd9xvd7u1AA@mail.gmail.com>
2020-12-13 18:49     ` Junxiao Bi
     [not found]       ` <CALYGNiM8Fp=ZV8S6c2L50ne1cGhE30PrT-C=4nfershvfAgP+Q@mail.gmail.com>
2020-12-14 23:10         ` Junxiao Bi
2020-12-16 18:46           ` Junxiao Bi

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).