LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
* [PATCH 00/10] Misc block layer patches for bcachefs
@ 2018-05-09  1:33 Kent Overstreet
  2018-05-09  1:33 ` [PATCH 01/10] mempool: Add mempool_init()/mempool_exit() Kent Overstreet
                   ` (12 more replies)
  0 siblings, 13 replies; 40+ messages in thread
From: Kent Overstreet @ 2018-05-09  1:33 UTC (permalink / raw)
  To: linux-kernel, linux-block, linux-mm, Jens Axboe, Ingo Molnar
  Cc: Kent Overstreet

 - Add separately allowed mempools, biosets: bcachefs uses both all over the
   place

 - Bit of utility code - bio_copy_data_iter(), zero_fill_bio_iter()

 - bio_list_copy_data(), the bi_next check - defensiveness because of a bug I
   had fun chasing down at one point

 - add some exports, because bcachefs does dio its own way
 - show whether fua is supported in sysfs, because I don't know of anything that
   exports whether the _block layer_ specifically thinks fua is supported.

Kent Overstreet (10):
  mempool: Add mempool_init()/mempool_exit()
  block: Convert bio_set to mempool_init()
  block: Add bioset_init()/bioset_exit()
  block: Use bioset_init() for fs_bio_set
  block: Add bio_copy_data_iter(), zero_fill_bio_iter()
  block: Split out bio_list_copy_data()
  block: Add missing flush_dcache_page() call
  block: Add warning for bi_next not NULL in bio_endio()
  block: Export bio check/set pages_dirty
  block: Add sysfs entry for fua support

 block/bio-integrity.c               |  29 ++--
 block/bio.c                         | 226 ++++++++++++++++++----------
 block/blk-core.c                    |  10 +-
 block/blk-sysfs.c                   |  11 ++
 drivers/block/pktcdvd.c             |   2 +-
 drivers/target/target_core_iblock.c |   2 +-
 include/linux/bio.h                 |  35 +++--
 include/linux/mempool.h             |  34 +++++
 mm/mempool.c                        | 108 +++++++++----
 9 files changed, 320 insertions(+), 137 deletions(-)

-- 
2.17.0

^ permalink raw reply	[flat|nested] 40+ messages in thread
* [PATCH 00/10] RFC: assorted bcachefs patches
@ 2018-05-18  7:48 Kent Overstreet
  2018-05-18  7:48 ` [PATCH 01/10] mempool: Add mempool_init()/mempool_exit() Kent Overstreet
  0 siblings, 1 reply; 40+ messages in thread
From: Kent Overstreet @ 2018-05-18  7:48 UTC (permalink / raw)
  To: linux-kernel, linux-fsdevel
  Cc: Kent Overstreet, Andrew Morton, Dave Chinner, darrick.wong,
	tytso, linux-btrfs, clm, jbacik, viro, willy, peterz

These are all the remaining patches in my bcachefs tree that touch stuff outside
fs/bcachefs. Not all of them are suitable for inclusion as is, I wanted to get
some discussion first.

 * pagecache add lock

This is the only one that touches existing code in nontrivial ways.  The problem
it's solving is that there is no existing general mechanism for shooting down
pages in the page and keeping them removed, which is a real problem if you're
doing anything that modifies file data and isn't buffered writes.

Historically, the only problematic case has been direct IO, and people have been
willing to say "well, if you mix buffered and direct IO you get what you
deserve", and that's probably not unreasonable. But now we have fallocate insert
range and collapse range, and those are broken in ways I frankly don't want to
think about if they can't ensure consistency with the page cache.

Also, the mechanism truncate uses (i_size and sacrificing a goat) has
historically been rather fragile, IMO it might be a good think if we switched it
to a more general rigorous mechanism.

I need this solved for bcachefs because without this mechanism, the page cache
inconsistencies lead to various assertions popping (primarily when we didn't
think we need to get a disk reservation going by page cache state, but then do
the actual write and disk space accounting says oops, we did need one). And
having to reason about what can happen without a locking mechanism for this is
not something I care to spend brain cycles on.

That said, my patch is kind of ugly, and it requires filesystem changes for
other filesystems to take advantage of it. And unfortunately, since one of the
code paths that needs locking is readahead, I don't see any realistic way of
implementing the locking within just bcachefs code.

So I'm hoping someone has an idea for something cleaner (I think I recall
Matthew Wilcox saying he had an idea for how to use xarray to solve this), but
if not I'll polish up my pagecache add lock patch and see what I can do to make
it less ugly, and hopefully other people find it palatable or at least useful.

 * lglocks

They were removed by Peter Zijlstra when the last in kernel user was removed,
but I've found them useful. His commit message seems to imply he doesn't think
people should be using them, but I'm not sure why. They are a bit niche though,
I can move them to fs/bcachefs if people would prefer. 

 * Generic radix trees

This is a very simple radix tree implementation that can store types of
arbitrary size, not just pointers/unsigned long. It could probably replace
flex arrays.

 * Dynamic fault injection

I've actually had this code sitting in my tree since forever... I know we have
an existing fault injection framework, but I think this one is quite a bit nicer
to actually use.

It works very much like the dynamic debug infrastructure - for those who aren't
familiar, dynamic debug makes it so you can list and individually enable/disable
every pr_debug() callsite in debugfs.

So to add a fault injection site with this, you just stick a call to
dynamic_fault("foobar") somewhere in your code - dynamic_fault() returns true if
you should fail whatever it is you're testing. And then it'll show up in
debugfs, where you can enable/disable faults by file/linenumber, module, name,
etc.

The patch then also adds macros that wrap all the various memory allocation
functions and fail if dynamic_fault("memory") returns true - which means you can
see in debugfs every place you're allocating memory and fail all of them or just
individually (I have tests that iterate over all the faults and flip them on one
by one). I also use it in bcachefs to add fault injection points for uncommon
error paths in the filesystem startup/recovery path, and for various hard to
test slowpaths that only happen if we race in weird ways (race_fault()).

Kent Overstreet (10):
  mm: pagecache add lock
  mm: export find_get_pages()
  locking: bring back lglocks
  locking: export osq_lock()/osq_unlock()
  don't use spin_lock_irqsave() unnecessarily
  Generic radix trees
  bcache: optimize continue_at_nobarrier()
  bcache: move closures to lib/
  closures: closure_wait_event()
  Dynamic fault injection

 drivers/md/bcache/Kconfig                     |  10 +-
 drivers/md/bcache/Makefile                    |   6 +-
 drivers/md/bcache/bcache.h                    |   2 +-
 drivers/md/bcache/super.c                     |   1 -
 drivers/md/bcache/util.h                      |   3 +-
 fs/inode.c                                    |   1 +
 include/asm-generic/vmlinux.lds.h             |   4 +
 .../md/bcache => include/linux}/closure.h     |  50 +-
 include/linux/dynamic_fault.h                 | 117 +++
 include/linux/fs.h                            |  23 +
 include/linux/generic-radix-tree.h            | 131 +++
 include/linux/lglock.h                        |  97 +++
 include/linux/sched.h                         |   4 +
 init/init_task.c                              |   1 +
 kernel/locking/Makefile                       |   1 +
 kernel/locking/lglock.c                       | 105 +++
 kernel/locking/osq_lock.c                     |   2 +
 lib/Kconfig                                   |   3 +
 lib/Kconfig.debug                             |  14 +
 lib/Makefile                                  |   7 +-
 {drivers/md/bcache => lib}/closure.c          |  17 +-
 lib/dynamic_fault.c                           | 760 ++++++++++++++++++
 lib/generic-radix-tree.c                      | 167 ++++
 mm/filemap.c                                  |  92 ++-
 mm/page-writeback.c                           |   5 +-
 25 files changed, 1577 insertions(+), 46 deletions(-)
 rename {drivers/md/bcache => include/linux}/closure.h (92%)
 create mode 100644 include/linux/dynamic_fault.h
 create mode 100644 include/linux/generic-radix-tree.h
 create mode 100644 include/linux/lglock.h
 create mode 100644 kernel/locking/lglock.c
 rename {drivers/md/bcache => lib}/closure.c (95%)
 create mode 100644 lib/dynamic_fault.c
 create mode 100644 lib/generic-radix-tree.c

-- 
2.17.0

^ permalink raw reply	[flat|nested] 40+ messages in thread

end of thread, other threads:[~2018-05-22 22:02 UTC | newest]

Thread overview: 40+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-05-09  1:33 [PATCH 00/10] Misc block layer patches for bcachefs Kent Overstreet
2018-05-09  1:33 ` [PATCH 01/10] mempool: Add mempool_init()/mempool_exit() Kent Overstreet
2018-05-09  7:54   ` Johannes Thumshirn
2018-05-11 21:11   ` Jens Axboe
2018-05-14 19:11     ` Kent Overstreet
2018-05-09  1:33 ` [PATCH 02/10] block: Convert bio_set to mempool_init() Kent Overstreet
2018-05-18 16:20   ` Christoph Hellwig
2018-05-18 16:21     ` Christoph Hellwig
2018-05-18 17:36     ` Kent Overstreet
2018-05-09  1:33 ` [PATCH 03/10] block: Add bioset_init()/bioset_exit() Kent Overstreet
2018-05-09  1:33 ` [PATCH 04/10] block: Use bioset_init() for fs_bio_set Kent Overstreet
2018-05-09  1:33 ` [PATCH 05/10] block: Add bio_copy_data_iter(), zero_fill_bio_iter() Kent Overstreet
2018-05-09  1:33 ` [PATCH 06/10] block: Split out bio_list_copy_data() Kent Overstreet
2018-05-09  1:33 ` [PATCH 07/10] block: Add missing flush_dcache_page() call Kent Overstreet
2018-05-09  1:33 ` [PATCH 08/10] block: Add warning for bi_next not NULL in bio_endio() Kent Overstreet
2018-05-09  1:33 ` [PATCH 09/10] block: Export bio check/set pages_dirty Kent Overstreet
2018-05-09  1:33 ` [PATCH 10/10] block: Add sysfs entry for fua support Kent Overstreet
2018-05-11 21:13 ` [PATCH 00/10] Misc block layer patches for bcachefs Jens Axboe
2018-05-18 16:23   ` Christoph Hellwig
2018-05-18 16:33     ` Jens Axboe
2018-05-14 19:24 ` Jens Axboe
2018-05-14 19:24   ` Kent Overstreet
2018-05-17 20:54 ` Bart Van Assche
2018-05-18  9:06   ` Kent Overstreet
2018-05-18 15:12     ` Bart Van Assche
2018-05-20 22:17       ` Kent Overstreet
2018-05-20 22:19         ` Bart Van Assche
2018-05-20 22:31           ` Kent Overstreet
2018-05-20 22:35             ` Bart Van Assche
2018-05-20 23:00               ` Kent Overstreet
2018-05-20 23:10                 ` Bart Van Assche
2018-05-20 23:21               ` Kent Overstreet
2018-05-20 23:40                 ` Bart Van Assche
2018-05-20 23:58                   ` Kent Overstreet
2018-05-21 15:11                     ` Bart Van Assche
2018-05-21 18:37                       ` Omar Sandoval
2018-05-21 18:46                         ` Bart Van Assche
2018-05-22 22:01         ` Bart Van Assche
2018-05-18  7:48 [PATCH 00/10] RFC: assorted bcachefs patches Kent Overstreet
2018-05-18  7:48 ` [PATCH 01/10] mempool: Add mempool_init()/mempool_exit() Kent Overstreet
2018-05-18  8:21   ` Johannes Thumshirn

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).