LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Michal Hocko <mhocko@kernel.org>
To: Robert Kolchmeyer <rkolchmeyer@google.com>,
	David Rientjes <rientjes@google.com>
Cc: Andrew Morton <akpm@linux-foundation.org>,
	Vlastimil Babka <vbabka@suse.cz>,
	linux-kernel@vger.kernel.org, linux-mm@kvack.org
Subject: Re: [patch] mm, oom: make a last minute check to prevent unnecessary memcg oom kills
Date: Tue, 17 Mar 2020 08:59:39 +0100	[thread overview]
Message-ID: <20200317075939.GD26018@dhcp22.suse.cz> (raw)
In-Reply-To: <20200311083900.GC23944@dhcp22.suse.cz>

On Wed 11-03-20 09:39:01, Michal Hocko wrote:
> On Tue 10-03-20 15:54:44, David Rientjes wrote:
> > On Tue, 10 Mar 2020, Michal Hocko wrote:
> > 
> > > On Tue 10-03-20 14:55:50, David Rientjes wrote:
> > > > Killing a user process as a result of hitting memcg limits is a serious
> > > > decision that is unfortunately needed only when no forward progress in
> > > > reclaiming memory can be made.
> > > > 
> > > > Deciding the appropriate oom victim can take a sufficient amount of time
> > > > that allows another process that is exiting to actually uncharge to the
> > > > same memcg hierarchy and prevent unnecessarily killing user processes.
> > > > 
> > > > An example is to prevent *multiple* unnecessary oom kills on a system
> > > > with two cores where the oom kill occurs when there is an abundance of
> > > > free memory available:
> > > > 
> > > > Memory cgroup out of memory: Killed process 628 (repro) total-vm:41944kB, anon-rss:40888kB, file-rss:496kB, shmem-rss:0kB, UID:0 pgtables:116kB oom_score_adj:0
> > > > <immediately after>
> > > > repro invoked oom-killer: gfp_mask=0xcc0(GFP_KERNEL), order=0, oom_score_adj=0
> > > > CPU: 1 PID: 629 Comm: repro Not tainted 5.6.0-rc5+ #130
> > > > Call Trace:
> > > >  dump_stack+0x78/0xb6
> > > >  dump_header+0x55/0x240
> > > >  oom_kill_process+0xc5/0x170
> > > >  out_of_memory+0x305/0x4a0
> > > >  try_charge+0x77b/0xac0
> > > >  mem_cgroup_try_charge+0x10a/0x220
> > > >  mem_cgroup_try_charge_delay+0x1e/0x40
> > > >  handle_mm_fault+0xdf2/0x15f0
> > > >  do_user_addr_fault+0x21f/0x420
> > > >  async_page_fault+0x2f/0x40
> > > > memory: usage 61336kB, limit 102400kB, failcnt 74
> > > > 
> > > > Notice the second memcg oom kill shows usage is >40MB below its limit of
> > > > 100MB but a process is still unnecessarily killed because the decision has
> > > > already been made to oom kill by calling out_of_memory() before the
> > > > initial victim had a chance to uncharge its memory.
> > > 
> > > Could you be more specific about the specific workload please?
> > > 
> > 
> > Robert, could you elaborate on the user-visible effects of this issue that 
> > caused it to initially get reported?
> 
> Yes please, real life usecases are important when adding hacks like this
> one and we should have a clear data to support the check actually helps
> (in how many instances etc...)

Friendly ping.
-- 
Michal Hocko
SUSE Labs

  reply	other threads:[~2020-03-17  7:59 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-03-10 21:55 David Rientjes
2020-03-10 22:19 ` Michal Hocko
2020-03-10 22:54   ` David Rientjes
2020-03-11  8:39     ` Michal Hocko
2020-03-17  7:59       ` Michal Hocko [this message]
2020-03-11 11:41     ` Tetsuo Handa
2020-03-11 19:51       ` David Rientjes
2020-03-17 18:25     ` Robert Kolchmeyer
2020-03-17 19:00       ` Ami Fischman
2020-03-18  9:57         ` Michal Hocko
2020-03-18 15:20           ` Ami Fischman
2020-03-18  9:55       ` Michal Hocko

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200317075939.GD26018@dhcp22.suse.cz \
    --to=mhocko@kernel.org \
    --cc=akpm@linux-foundation.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=rientjes@google.com \
    --cc=rkolchmeyer@google.com \
    --cc=vbabka@suse.cz \
    --subject='Re: [patch] mm, oom: make a last minute check to prevent unnecessary memcg oom kills' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).