LKML Archive on lore.kernel.org help / color / mirror / Atom feed
From: "Kirill A. Shutemov" <email@example.com> To: Matthew Wilcox <firstname.lastname@example.org> Cc: Kent Overstreet <email@example.com>, firstname.lastname@example.org, email@example.com, firstname.lastname@example.org, Johannes Weiner <email@example.com>, Linus Torvalds <firstname.lastname@example.org>, Andrew Morton <email@example.com>, "Darrick J. Wong" <firstname.lastname@example.org>, Christoph Hellwig <email@example.com>, David Howells <firstname.lastname@example.org>, Mike Kravetz <email@example.com> Subject: Re: Mapcount of subpages Date: Thu, 23 Sep 2021 15:45:02 +0300 [thread overview] Message-ID: <firstname.lastname@example.org> (raw) In-Reply-To: <YUxnnq7uFBAtJ3rT@casper.infradead.org> On Thu, Sep 23, 2021 at 12:40:14PM +0100, Matthew Wilcox wrote: > On Thu, Sep 23, 2021 at 01:15:16AM -0400, Kent Overstreet wrote: > > On Thu, Sep 23, 2021 at 04:23:12AM +0100, Matthew Wilcox wrote: > > > (compiling that list reminds me that we'll need to sort out mapcount > > > on subpages when it comes time to do this. ask me if you don't know > > > what i'm talking about here.) > > > > I am curious why we would ever need a mapcount for just part of a page, tell me > > more. > > I would say Kirill is the expert here. My understanding: > > We have three different approaches to allocating 2MB pages today; > anon THP, shmem THP and hugetlbfs. Hugetlbfs can only be mapped on a > 2MB boundary, so it has no special handling of mapcount . Anon THP > always starts out as being mapped exclusively on a 2MB boundary, but > then it can be split by, eg, munmap(). If it is, then the mapcount in > the head page is distributed to the subpages. One more complication for anon THP is that it can be shared across fork() and one process may split it while other have it mapped with PMD. > Shmem THP is the tricky one. You might have a 2MB page in the page cache, > but then have processes which only ever map part of it. Or you might > have some processes mapping it with a 2MB entry and others mapping part > or all of it with 4kB entries. And then someone truncates the file to > midway through this page; we split it, and now we need to figure out what > the mapcount should be on each of the subpages. We handle this by using > ->mapcount on each subpage to record how many non-2MB mappings there are > of that specific page and using ->compound_mapcount to record how many 2MB > mappings there are of the entire 2MB page. Then, when we split, we just > need to distribute the compound_mapcount to each page to make it correct. > We also have the PageDoubleMap flag to tell us whether anybody has this > 2MB page mapped with 4kB entries, so we can skip all the summing of 4kB > mapcounts if nobody has done that. Possible future complication comes from 1G THP effort. With 1G THP we would have whole hierarchy of mapcounts: 1 PUD mapcount, 512 PMD mapcounts and 262144 PTE mapcounts. (That's one of the reasons I don't think 1G THP is viable.) Note that there are places where exact mapcount accounting is critical: try_to_unmap() may finish prematurely if we underestimate mapcount and overestimating mapcount may lead to superfluous CoW that breaks GUP. > >  Mike is looking to change this, but I'm not sure where he is with it. > -- Kirill A. Shutemov
next prev parent reply other threads:[~2021-09-23 12:46 UTC|newest] Thread overview: 35+ messages / expand[flat|nested] mbox.gz Atom feed top 2021-09-23 1:21 Struct page proposal Kent Overstreet 2021-09-23 3:23 ` Matthew Wilcox 2021-09-23 5:15 ` Kent Overstreet 2021-09-23 11:40 ` Mapcount of subpages Matthew Wilcox 2021-09-23 12:45 ` Kirill A. Shutemov [this message] 2021-09-23 21:10 ` Hugh Dickins 2021-09-23 21:54 ` Yang Shi 2021-09-23 22:23 ` Zi Yan 2021-09-23 23:48 ` Hugh Dickins 2021-09-24 0:25 ` Zi Yan 2021-09-24 0:57 ` Hugh Dickins 2021-09-24 1:11 ` Yang Shi 2021-09-24 1:31 ` Matthew Wilcox 2021-09-24 3:26 ` Yang Shi 2021-09-24 23:05 ` Kirill A. Shutemov 2021-09-23 18:56 ` Mike Kravetz 2021-09-23 9:03 ` Struct page proposal David Hildenbrand 2021-09-23 15:22 ` Kent Overstreet 2021-09-23 15:34 ` David Hildenbrand 2021-09-27 17:48 ` Vlastimil Babka 2021-09-27 17:53 ` Kent Overstreet 2021-09-27 18:34 ` Linus Torvalds 2021-09-27 20:45 ` David Hildenbrand 2021-09-27 18:05 ` Matthew Wilcox 2021-09-27 18:09 ` Kent Overstreet 2021-09-27 18:12 ` Matthew Wilcox 2021-09-27 18:16 ` David Hildenbrand 2021-09-27 18:53 ` Vlastimil Babka 2021-09-27 19:04 ` Linus Torvalds 2021-09-27 18:16 ` Kent Overstreet 2021-09-28 3:19 ` Matthew Wilcox 2021-09-27 19:07 ` Vlastimil Babka 2021-09-27 20:14 ` Kent Overstreet 2021-09-28 11:21 ` David Laight 2021-09-27 18:33 ` Kirill A. Shutemov
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).