LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Mike Kravetz <mike.kravetz@oracle.com>
To: Vlastimil Babka <vbabka@suse.cz>, Michal Hocko <mhocko@suse.com>
Cc: Hillf Danton <hdanton@sina.com>,
	linux-mm@kvack.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH RESEND 0/8] hugetlb: add demote/split page functionality
Date: Thu, 9 Sep 2021 14:31:43 -0700	[thread overview]
Message-ID: <4607345b-27c2-0c0b-4a04-8a3b054cb5a1@oracle.com> (raw)
In-Reply-To: <71f855ac-ff61-1eed-454f-909c0e4210b2@suse.cz>

On 9/9/21 6:45 AM, Vlastimil Babka wrote:
> On 9/9/21 13:54, Michal Hocko wrote:
>> On Wed 08-09-21 14:00:19, Mike Kravetz wrote:
>>> On 9/7/21 1:50 AM, Hillf Danton wrote:
>>>> On Mon, 6 Sep 2021 16:40:28 +0200 Vlastimil Babka wrote:
>>>>
>>>> And/or clamp reclaim retries for costly orders
>>>>
>>>> 	reclaim retries = MAX_RECLAIM_RETRIES - order;
>>>>
>>>> to pull down the chance for stall as low as possible.
>>>
>>> Thanks, and sorry for not replying quickly.  I only get back to this as
>>> time allows.
>>>
>>> We could clamp the number of compaction and reclaim retries in
>>> __alloc_pages_slowpath as suggested.  However, I noticed that a single
>>> reclaim call could take a bunch of time.  As a result, I instrumented
>>> shrink_node to see what might be happening.  Here is some information
>>> from a long stall.  Note that I only dump stats when jiffies > 100000.
>>>
>>> [ 8136.874706] shrink_node: 507654 total jiffies,  3557110 tries
>>> [ 8136.881130]              130596341 reclaimed, 32 nr_to_reclaim
>>> [ 8136.887643]              compaction_suitable results:
>>> [ 8136.893276]     idx COMPACT_SKIPPED, 3557109
>>
>> Can you get a more detailed break down of where the time is spent. Also
>> How come the number of reclaimed pages is so excessive comparing to the
>> reclaim target? There is something fishy going on here.
> 
> I would say it's simply should_continue_reclaim() behaving similarly to
> should_compact_retry(). We'll get compaction_suitable() returning
> COMPACT_SKIPPED because the reclaimed pages have been immediately stolen,
> and compaction indicates there's not enough base pages to begin with to form
> a high-order pages. Since the stolen pages will appear on inactive lru, it
> seems to be worth continuing reclaim to make enough free base pages for
> compaction to no longer be skipped, because "inactive_lru_pages >
> pages_for_compaction" is true.
> 
> So, both should_continue_reclaim() and should_compact_retry() are unable to
> recognize that reclaimed pages are being stolen and limit the retries in
> that case. The scenario seems to be uncommon, otherwise we'd be getting more
> reports of that.

Yes, I believe this is what is happening.

I honestly do not know if my test/recreation scenario is realistic.

I do know that our DB team has had issues with allocating a number of
hugetlb pages (after much uptime) taking forever or a REALLY long time.
These are all 2MB huge page allocations, so going through the normal
page allocation code path.  No idea what else is running on the system
at the time of the allocation stalls.  Unfortunately, this can not be
reproduced at will in their environment.  As a result, I have no data
and just this brief description of the issue.  When I stumbled on an
easy way to recreate, I thought it would be worth investigating/fixing.

It certainly does not seem to be a common scenario.
-- 
Mike Kravetz

  reply	other threads:[~2021-09-09 21:32 UTC|newest]

Thread overview: 38+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-08-16 22:49 Mike Kravetz
2021-08-16 22:49 ` [PATCH 1/8] hugetlb: add demote hugetlb page sysfs interfaces Mike Kravetz
2021-08-16 22:49 ` [PATCH 2/8] hugetlb: add HPageCma flag and code to free non-gigantic pages in CMA Mike Kravetz
2021-08-16 22:49 ` [PATCH 3/8] hugetlb: add demote bool to gigantic page routines Mike Kravetz
2021-08-16 22:49 ` [PATCH 4/8] hugetlb: add hugetlb demote page support Mike Kravetz
2021-08-16 22:49 ` [PATCH 5/8] hugetlb: document the demote sysfs interfaces Mike Kravetz
2021-08-16 23:28   ` Andrew Morton
2021-08-17  1:04     ` Mike Kravetz
2021-09-21 13:52   ` Aneesh Kumar K.V
2021-09-21 17:17     ` Mike Kravetz
2021-08-16 22:49 ` [PATCH 6/8] hugetlb: vmemmap optimizations when demoting hugetlb pages Mike Kravetz
2021-08-16 22:49 ` [PATCH 7/8] hugetlb: prepare destroy and prep routines for vmemmap optimized pages Mike Kravetz
2021-08-16 22:49 ` [PATCH 8/8] hugetlb: Optimized demote vmemmap optimizatized pages Mike Kravetz
2021-08-16 23:23 ` [PATCH RESEND 0/8] hugetlb: add demote/split page functionality Andrew Morton
2021-08-17  0:17   ` Mike Kravetz
2021-08-17  0:39     ` Andrew Morton
2021-08-17  0:58       ` Mike Kravetz
2021-08-16 23:27 ` Andrew Morton
2021-08-17  0:46   ` Mike Kravetz
2021-08-17  1:46     ` Andrew Morton
2021-08-17  7:30       ` David Hildenbrand
2021-08-17 16:19         ` Mike Kravetz
2021-08-17 18:49           ` David Hildenbrand
2021-08-24 22:08       ` Mike Kravetz
2021-08-27 17:22         ` Vlastimil Babka
2021-08-27 23:04           ` Mike Kravetz
2021-08-30 10:11             ` Vlastimil Babka
2021-09-02 18:17               ` Mike Kravetz
2021-09-06 14:40                 ` Vlastimil Babka
     [not found]                 ` <20210907085001.3773-1-hdanton@sina.com>
2021-09-08 21:00                   ` Mike Kravetz
2021-09-09 11:54                     ` Michal Hocko
2021-09-09 13:45                       ` Vlastimil Babka
2021-09-09 21:31                         ` Mike Kravetz [this message]
2021-09-10  8:20                         ` Michal Hocko
2021-09-11  0:11                           ` Mike Kravetz
2021-09-13 15:50                             ` Michal Hocko
2021-09-15 16:57                               ` Mike Kravetz
2021-09-17 20:44                                 ` Mike Kravetz

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=4607345b-27c2-0c0b-4a04-8a3b054cb5a1@oracle.com \
    --to=mike.kravetz@oracle.com \
    --cc=hdanton@sina.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mhocko@suse.com \
    --cc=vbabka@suse.cz \
    --subject='Re: [PATCH RESEND 0/8] hugetlb: add demote/split page functionality' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).