LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: David Hildenbrand <david@redhat.com>
To: Matthew Wilcox <willy@infradead.org>, Vlastimil Babka <vbabka@suse.cz>
Cc: linux-mm@kvack.org, Steven Rostedt <rostedt@goodmis.org>,
	Ingo Molnar <mingo@redhat.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	Michal Hocko <mhocko@suse.com>, Huang Ying <ying.huang@intel.com>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	Pavel Tatashin <pasha.tatashin@oracle.com>,
	Miles Chen <miles.chen@mediatek.com>,
	Mel Gorman <mgorman@techsingularity.net>,
	Rik van Riel <riel@redhat.com>, James Hogan <jhogan@kernel.org>,
	"Levin, Alexander (Sasha Levin)" <alexander.levin@verizon.com>,
	open list <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH RFC 2/8] mm: introduce PG_offline
Date: Sun, 22 Apr 2018 10:17:31 +0200	[thread overview]
Message-ID: <7db70df4-c714-574c-5b14-898c1cf49af6@redhat.com> (raw)
In-Reply-To: <20180422030130.GG14610@bombadil.infradead.org>

On 22.04.2018 05:01, Matthew Wilcox wrote:
> On Sat, Apr 21, 2018 at 06:52:18PM +0200, Vlastimil Babka wrote:
>> On 04/13/2018 07:11 PM, Matthew Wilcox wrote:
>>> On Fri, Apr 13, 2018 at 03:16:26PM +0200, David Hildenbrand wrote:
>>>> online_pages()/offline_pages() theoretically allows us to work on
>>>> sub-section sizes. This is especially relevant in the context of
>>>> virtualization. It e.g. allows us to add/remove memory to Linux in a VM in
>>>> 4MB chunks.
>>>>
>>>> While the whole section is marked as online/offline, we have to know
>>>> the state of each page. E.g. to not read memory that is not online
>>>> during kexec() or to properly mark a section as offline as soon as all
>>>> contained pages are offline.
>>>
>>> Can you not use PG_reserved for this purpose?
>>
>> Sounds like your newly introduced "page types" could be useful here? I
>> don't suppose those offline pages would be using mapcount which is
>> aliased there?
> 
> Oh, that's a good point!  Yes, this is a perfect use for page_type.
> We have something like twenty bits available there.
> 
> Now you've got me thinking that we can move PG_hwpoison and PG_reserved
> to be page_type flags too.  That'll take us from 23 to 21 bits (on 32-bit,
> with PG_UNCACHED)
> 

Some things to clarify here. I modified the current RFC to also allow
PG_offline on allocated (ballooned) pages (e.g. virtio-balloon).

kdump based dump tools can then easily identify which pages are not to
be dumped (either because the content is invalid or not accessible).

I previously stated that ballooned pages would be marked as PG_reserved,
which is not true (at least not for virtio-balloon). However this allows
me to detect if all pages in a section are offline by looking at
(PG_reserved && PG_offline). So I can actually tell if a page is marked
as offline and allocated or really offline.


1. The location (not the number!) of PG_hwpoison is basically ABI and
cannot be changed. Moving it around will most probably break dump tools.
(see kernel/crash_core.c)

2. Exposing PG_offline via kdump will make it ABI as well. And we don't
want any complicated validity checks ("is the bit valid or not?"),
because that would imply having to make these bits ABI as well. So
having PG_offline just like PG_hwpoison part of page_flags is the right
thing to do. (see patch nr 4)

3. For determining if all pages of a section are offline (see patch nr
5), I will have to be able to check 1. PG_offline and 2. PG_reserved on
any page. Will this be possible by moving e.g. PG_reserved to page
types? (especially if some field is suddenly aliased?)


I was wondering if we could reuse PG_hwpoison to mark "this page is not
to be read by dump tools" and store the real reason (page offline/page
has an hw error/page is part of a balloon ...) somewhere else (page
types?). But I am not sure if changing the semantic of PG_hwpoison
(visible to dump tools) is okay, and if we can then always "read out"
the type (especially: when is the page type field valid and can be used?)

Thanks!

-- 

Thanks,

David / dhildenb

  reply	other threads:[~2018-04-22  8:17 UTC|newest]

Thread overview: 23+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <20180413131632.1413-1-david@redhat.com>
2018-04-13 13:16 ` [PATCH RFC 1/8] mm/memory_hotplug: Revert "mm/memory_hotplug: optimize memory hotplug" David Hildenbrand
2018-04-13 13:16 ` [PATCH RFC 2/8] mm: introduce PG_offline David Hildenbrand
2018-04-13 13:40   ` Michal Hocko
2018-04-13 13:46     ` David Hildenbrand
2018-04-17 11:50     ` David Hildenbrand
2018-04-13 17:11   ` Matthew Wilcox
2018-04-16  8:31     ` David Hildenbrand
2018-04-21 16:52     ` Vlastimil Babka
2018-04-22  3:01       ` Matthew Wilcox
2018-04-22  8:17         ` David Hildenbrand [this message]
2018-04-22 14:02           ` Matthew Wilcox
2018-04-22 15:13             ` David Hildenbrand
2018-04-29 21:08               ` Michal Hocko
2018-04-30  6:31                 ` David Hildenbrand
2018-04-20  7:30   ` David Hildenbrand
2018-04-13 13:16 ` [PATCH RFC 3/8] mm: use PG_offline in online/offlining code David Hildenbrand
2018-04-13 13:31 ` [PATCH RFC 4/8] kdump: expose PG_offline David Hildenbrand
2018-04-13 13:33 ` [PATCH RFC 5/8] mm: only mark section offline when all pages are offline David Hildenbrand
2018-04-13 13:33 ` [PATCH RFC 6/8] mm: offline_pages() is also limited by MAX_ORDER David Hildenbrand
2018-04-13 13:33 ` [PATCH RFC 7/8] mm: allow to control onlining/offlining of memory by a driver David Hildenbrand
2018-04-13 15:59   ` Michal Hocko
2018-04-13 16:32     ` David Hildenbrand
2018-04-13 13:33 ` [PATCH RFC 8/8] mm: export more functions used to online/offline memory David Hildenbrand

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=7db70df4-c714-574c-5b14-898c1cf49af6@redhat.com \
    --to=david@redhat.com \
    --cc=akpm@linux-foundation.org \
    --cc=alexander.levin@verizon.com \
    --cc=gregkh@linuxfoundation.org \
    --cc=jhogan@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mgorman@techsingularity.net \
    --cc=mhocko@suse.com \
    --cc=miles.chen@mediatek.com \
    --cc=mingo@redhat.com \
    --cc=pasha.tatashin@oracle.com \
    --cc=riel@redhat.com \
    --cc=rostedt@goodmis.org \
    --cc=vbabka@suse.cz \
    --cc=willy@infradead.org \
    --cc=ying.huang@intel.com \
    --subject='Re: [PATCH RFC 2/8] mm: introduce PG_offline' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).