Linux-Fsdevel Archive on lore.kernel.org
help / color / mirror / Atom feed
* [RFC v6 00/51] Large pages in the page cache
@ 2020-06-10 20:12 Matthew Wilcox
  2020-06-10 20:12 ` [PATCH v6 01/51] mm: Print head flags in dump_page Matthew Wilcox
                   ` (52 more replies)
  0 siblings, 53 replies; 60+ messages in thread
From: Matthew Wilcox @ 2020-06-10 20:12 UTC (permalink / raw)
  To: linux-fsdevel; +Cc: Matthew Wilcox (Oracle), linux-mm, linux-kernel

From: "Matthew Wilcox (Oracle)" <willy@infradead.org>

Another fortnight, another dump of my current large pages work.
I've squished a lot of bugs this time.  xfstests is much happier now,
running for 1631 seconds and getting as far as generic/086.  This patchset
is getting a little big, so I'm going to try to get some bits of it
upstream soon (the bits that make sense regardless of whether the rest
of this is merged).

It's now based on linus' master (6f630784cc0d), and you can get it from
http://git.infradead.org/users/willy/linux-dax.git/shortlog/refs/heads/xarray-pagecache
if you'd rather see it there (this branch is force-pushed frequently)

The primary idea here is that a large part of the overhead in dealing
with individual pages is that there's just so darned many of them.
We would be better off dealing with fewer, larger pages, even if they
don't get to be the size necessary for the CPU to use a larger TLB entry.

The approach taken is to make THPs support arbitrary power-of-two sizes
(instead of just PMDs).  There's probably some tuning to be done to decide
what sizes are worth using, but we're a fair way from doing performance
work with this patchset yet.

TODO:
 - Fix arc/arm/arm64/mips/powerpc/space flush_dcache_page() to
   support THPs natively
 - Actually create large pages for sufficiently large writes
 - Copy in larger chunks for write() in iomap
 - More bug fixing

v6:
 - Improved debug output for large pages (will send to Andrew soon)
 - Make compound_nr() more efficient (will send to Andrew soon)
 - Renamed hpage_nr_pages() to thp_nr_pages()
 - Added thp_head()
 - Set the THP_SUPPORT flag in shmfs
 - Change zero_user_segments() to call flush_dcache_page() once for the
   head page instead of once for each subpage.  The architectures listed
   above need to be fixed.
 - Fix shmem & truncate to call zero_user_segment() with the head page
 - Fix page_is_mergeable() for THPs
 - Fix a bug in iomap_iop_set_range_uptodate() where I was assuming that
   the offset was block-aligned
 - Fix a few more places that assume unsigned int is large enough to hold
   offset/length within a page
 - Fix doing writeback of a page after discarding its iop due to a partial
   truncate
 - Convert the iomap write paths more comprehensively.  That's now four
   separate patches
v5:
 - Add a mapping AS_LARGE_PAGES flag to reduce the levels of indirection
   (Dave Chinner)
 - Change iomap_invalidate_page() to handle subpages of a THP being punched
 - Ensure we don't call page_cache_async_readahead() with a tail page
 - Revert to Bill's original patch for thp_get_unmapped_area() to allow
   for hardware page sizes other than PMD to be supported more easily
 - Remove a few more HPAGE_PMD_NR
 - Move shmem_punch_compound() to truncate.c and rename it to punch_thp()
 - Add support for page_private to punch_thp()

v4:
 - Fix thp_size typo
 - Fix the iomap page_mkwrite() path to operate on the head page, even
   though the vm_fault has a pointer to the tail page
 - Fix iomap_finish_ioend() to use bio_for_each_thp_segment_all()
 - Rework PageDoubleMap (see first two patches for details)
 - Fix page_cache_delete() to handle shadow entries being stored to a THP
 - Fix the assertion in pagecache_get_page() to handle tail pages
 - Change PageReadahead from NO_COMPOUND to ONLY_HEAD
 - Handle PageReadahead being set on head pages
 - Handle total_mapcount correctly (Kirill)
 - Pull the FS_LARGE_PAGES check out into mapping_large_pages()
 - Fix page size assumption in truncate_cleanup_page()
 - Avoid splitting large pages unnecessarily on truncate
 - Disable the page cache truncation introduced as part of the read-only
   THP patch set
 - Call compound_head() in iomap buffered write paths -- we retrieve a
   (potentially) tail page from the page cache and need to use that for
   flush_dcache_page(), but we expect to operate on a head page in most
   of the iomap code



Kirill A. Shutemov (1):
  mm: Fix total_mapcount assumption of page size

Matthew Wilcox (Oracle) (49):
  mm: Print head flags in dump_page
  mm: Print the inode number in dump_page
  mm: Print hashed address of struct page
  mm: Move PageDoubleMap bit
  mm: Simplify PageDoubleMap with PF_SECOND policy
  mm: Store compound_nr as well as compound_order
  mm: Move page-flags include to top of file
  mm: Add thp_order
  mm: Add thp_size
  mm: Replace hpage_nr_pages with thp_nr_pages
  mm: Add thp_head
  mm: Introduce offset_in_thp
  mm: Support arbitrary THP sizes
  fs: Add a filesystem flag for THPs
  fs: Do not update nr_thps for mappings which support THPs
  fs: Introduce i_blocks_per_page
  fs: Make page_mkwrite_check_truncate thp-aware
  mm: Support THPs in zero_user_segments
  mm: Zero the head page, not the tail page
  block: Add bio_for_each_thp_segment_all
  block: Support THPs in page_is_mergeable
  iomap: Support arbitrarily many blocks per page
  iomap: Support THPs in iomap_adjust_read_range
  iomap: Support THPs in invalidatepage
  iomap: Support THPs in read paths
  iomap: Convert iomap_write_end types
  iomap: Change calling convention for zeroing
  iomap: Change iomap_write_begin calling convention
  iomap: Support THPs in write paths
  iomap: Inline data shouldn't see THPs
  iomap: Handle tail pages in iomap_page_mkwrite
  xfs: Support THPs
  mm: Make prep_transhuge_page return its argument
  mm: Add __page_cache_alloc_order
  mm: Allow THPs to be added to the page cache
  mm: Allow THPs to be removed from the page cache
  mm: Remove page fault assumption of compound page size
  mm: Remove assumptions of THP size
  mm: Avoid splitting THPs
  mm: Fix truncation for pages of arbitrary size
  mm: Handle truncates that split THPs
  mm: Support storing shadow entries for THPs
  mm: Support retrieving tail pages from the page cache
  mm: Support tail pages in wait_for_stable_page
  mm: Add DEFINE_READAHEAD
  mm: Make page_cache_readahead_unbounded take a readahead_control
  mm: Make __do_page_cache_readahead take a readahead_control
  mm: Allow PageReadahead to be set on head pages
  mm: Add THP readahead

William Kucharski (1):
  mm: Align THP mappings for non-DAX

 block/bio.c                |   2 +-
 drivers/nvdimm/btt.c       |   4 +-
 drivers/nvdimm/pmem.c      |   6 +-
 fs/dax.c                   |  13 +-
 fs/ext4/verity.c           |   4 +-
 fs/f2fs/verity.c           |   4 +-
 fs/inode.c                 |   2 +
 fs/iomap/buffered-io.c     | 250 +++++++++++++++++++------------------
 fs/jfs/jfs_metapage.c      |   2 +-
 fs/xfs/xfs_aops.c          |   4 +-
 fs/xfs/xfs_super.c         |   2 +-
 include/linux/bio.h        |  13 ++
 include/linux/bvec.h       |  23 ++++
 include/linux/dax.h        |   3 +-
 include/linux/fs.h         |  28 +----
 include/linux/highmem.h    |  11 +-
 include/linux/huge_mm.h    |  65 ++++++++--
 include/linux/mm.h         |  46 +++----
 include/linux/mm_inline.h  |   6 +-
 include/linux/mm_types.h   |   1 +
 include/linux/page-flags.h |  46 ++-----
 include/linux/pagemap.h    | 102 ++++++++++++---
 mm/compaction.c            |   2 +-
 mm/debug.c                 |  23 ++--
 mm/filemap.c               | 101 +++++++++------
 mm/gup.c                   |   2 +-
 mm/highmem.c               |  62 ++++++++-
 mm/huge_memory.c           |  38 +++---
 mm/hugetlb.c               |   2 +-
 mm/internal.h              |  17 +--
 mm/memcontrol.c            |  10 +-
 mm/memory.c                |   7 +-
 mm/memory_hotplug.c        |   7 +-
 mm/mempolicy.c             |   2 +-
 mm/migrate.c               |  16 +--
 mm/mlock.c                 |   9 +-
 mm/page-writeback.c        |   1 +
 mm/page_alloc.c            |   5 +-
 mm/page_io.c               |   4 +-
 mm/page_vma_mapped.c       |   6 +-
 mm/readahead.c             | 145 ++++++++++++++++-----
 mm/rmap.c                  |  18 +--
 mm/shmem.c                 |  39 ++----
 mm/swap.c                  |  16 +--
 mm/swap_state.c            |   6 +-
 mm/swapfile.c              |   2 +-
 mm/truncate.c              |  70 ++++++++++-
 mm/vmscan.c                |  12 +-
 48 files changed, 795 insertions(+), 464 deletions(-)

-- 
2.26.2

^ permalink raw reply	[flat|nested] 60+ messages in thread

end of thread, other threads:[~2020-06-15 13:32 UTC | newest]

Thread overview: 60+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-06-10 20:12 [RFC v6 00/51] Large pages in the page cache Matthew Wilcox
2020-06-10 20:12 ` [PATCH v6 01/51] mm: Print head flags in dump_page Matthew Wilcox
2020-06-10 20:12 ` [PATCH v6 02/51] mm: Print the inode number " Matthew Wilcox
2020-06-10 20:12 ` [PATCH v6 03/51] mm: Print hashed address of struct page Matthew Wilcox
2020-06-10 20:12 ` [PATCH v6 04/51] mm: Move PageDoubleMap bit Matthew Wilcox
2020-06-11 15:03   ` Zi Yan
2020-06-10 20:12 ` [PATCH v6 05/51] mm: Simplify PageDoubleMap with PF_SECOND policy Matthew Wilcox
2020-06-11 15:14   ` Zi Yan
2020-06-10 20:13 ` [PATCH v6 06/51] mm: Store compound_nr as well as compound_order Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 07/51] mm: Move page-flags include to top of file Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 08/51] mm: Add thp_order Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 09/51] mm: Add thp_size Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 10/51] mm: Replace hpage_nr_pages with thp_nr_pages Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 11/51] mm: Add thp_head Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 12/51] mm: Introduce offset_in_thp Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 13/51] mm: Support arbitrary THP sizes Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 14/51] fs: Add a filesystem flag for THPs Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 15/51] fs: Do not update nr_thps for mappings which support THPs Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 16/51] fs: Introduce i_blocks_per_page Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 17/51] fs: Make page_mkwrite_check_truncate thp-aware Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 18/51] mm: Support THPs in zero_user_segments Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 19/51] mm: Zero the head page, not the tail page Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 20/51] block: Add bio_for_each_thp_segment_all Matthew Wilcox
2020-06-11 18:20   ` Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 21/51] block: Support THPs in page_is_mergeable Matthew Wilcox
2020-06-12 16:17   ` Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 22/51] iomap: Support arbitrarily many blocks per page Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 23/51] iomap: Support THPs in iomap_adjust_read_range Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 24/51] iomap: Support THPs in invalidatepage Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 25/51] iomap: Support THPs in read paths Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 26/51] iomap: Convert iomap_write_end types Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 27/51] iomap: Change calling convention for zeroing Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 28/51] iomap: Change iomap_write_begin calling convention Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 29/51] iomap: Support THPs in write paths Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 30/51] iomap: Inline data shouldn't see THPs Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 31/51] iomap: Handle tail pages in iomap_page_mkwrite Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 32/51] xfs: Support THPs Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 33/51] mm: Make prep_transhuge_page return its argument Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 34/51] mm: Add __page_cache_alloc_order Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 35/51] mm: Allow THPs to be added to the page cache Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 36/51] mm: Allow THPs to be removed from " Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 37/51] mm: Remove page fault assumption of compound page size Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 38/51] mm: Fix total_mapcount assumption of " Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 39/51] mm: Remove assumptions of THP size Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 40/51] mm: Avoid splitting THPs Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 41/51] mm: Fix truncation for pages of arbitrary size Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 42/51] mm: Handle truncates that split THPs Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 43/51] mm: Support storing shadow entries for THPs Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 44/51] mm: Support retrieving tail pages from the page cache Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 45/51] mm: Support tail pages in wait_for_stable_page Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 46/51] mm: Add DEFINE_READAHEAD Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 47/51] mm: Make page_cache_readahead_unbounded take a readahead_control Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 48/51] mm: Make __do_page_cache_readahead " Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 49/51] mm: Allow PageReadahead to be set on head pages Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 50/51] mm: Add THP readahead Matthew Wilcox
2020-06-10 20:13 ` [PATCH v6 51/51] mm: Align THP mappings for non-DAX Matthew Wilcox
2020-06-11  6:59 ` [RFC v6 00/51] Large pages in the page cache Christoph Hellwig
2020-06-11 11:24   ` Matthew Wilcox
2020-06-15 13:32     ` Christoph Hellwig
2020-06-14 16:26 ` Matthew Wilcox

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).