LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Andrew Morton <akpm@linux-foundation.org>
To: Christoph Lameter <cl@linux-foundation.org>
Cc: peterz@infradead.org, rientjes@google.com, npiggin@suse.de,
	menage@google.com, dfults@sgi.com, linux-kernel@vger.kernel.org,
	containers@lists.osdl.org
Subject: Re: [patch 0/7] cpuset writeback throttling
Date: Wed, 5 Nov 2008 10:41:45 -0800	[thread overview]
Message-ID: <20081105104145.abd6fc91.akpm@linux-foundation.org> (raw)
In-Reply-To: <Pine.LNX.4.64.0811050747040.11867@quilx.com>

On Wed, 5 Nov 2008 07:52:44 -0600 (CST)
Christoph Lameter <cl@linux-foundation.org> wrote:

> On Tue, 4 Nov 2008, Andrew Morton wrote:
> 
> >> That is one aspect. When performing writeback then we need to figure out
> >> which inodes have dirty pages in the memcg and we need to start writeout
> >> on those inodes and not on others that have their dirty pages elsewhere.
> >> There are two components of this that are in this patch and that would
> >> also have to be implemented for a memcg.
> >
> > Doable.  lru->page->mapping->host is a good start.
> 
> The block layer has a list of inodes that are dirty. From that we need to 
> select ones that will improve the situation from the cpuset/memcg. How 
> does the LRU come into this?

In the simplest case, dirty-memory throttling can just walk the LRU
writing back pages in the same way that kswapd does.

There would probably be performance benefits in doing
address_space-ordered writeback, so the dirty-memory throttling could
pick a dirty page off the LRU, go find its inode and then feed that
into __sync_single_inode().

> >> This patch would solve the problem if the calculation of the dirty pages
> >> would consider the active memcg and be able to determine the amount of
> >> dirty pages (through some sort of additional memcg counters). That is just
> >> the first part though. The second part of finding the inodes that have
> >> dirty pages for writeback would require an association between memcgs and
> >> inodes.
> >
> > We presently have that via the LRU.  It has holes, but so does this per-cpuset
> > scheme.
> 
> How do I get to the LRU from the dirtied list of inodes?

Don't need it.

It'll be approximate and has obvious scenarios of great inaccuraracy
but it'll suffice for the workloads which this patchset addresses.



It sounds like any memcg-based approach just won't be suitable for the
people who are hitting this problem.

But _are_ people hitting this problem?  I haven't seen any real-looking
reports in ages.  Is there some workaround?  If so, what is it?  How
serious is this problem now?


  reply	other threads:[~2008-11-05 18:42 UTC|newest]

Thread overview: 45+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2008-10-30 19:23 David Rientjes
2008-10-30 19:23 ` [patch 1/7] cpusets: add dirty map to struct address_space David Rientjes
2008-11-04 21:09   ` Andrew Morton
2008-11-04 21:20     ` Christoph Lameter
2008-11-04 21:42       ` Andrew Morton
2008-10-30 19:23 ` [patch 2/7] pdflush: allow the passing of a nodemask parameter David Rientjes
2008-10-30 19:23 ` [patch 3/7] mm: make page writeback obey cpuset constraints David Rientjes
2008-10-30 19:23 ` [patch 4/7] mm: cpuset aware reclaim writeout David Rientjes
2008-10-30 19:23 ` [patch 5/7] mm: throttle writeout with cpuset awareness David Rientjes
2008-10-30 19:23 ` [patch 6/7] cpusets: per cpuset dirty ratios David Rientjes
2008-10-30 19:23 ` [patch 7/7] cpusets: update documentation for writeback throttling David Rientjes
2008-10-30 21:08 ` [patch 0/7] cpuset " Dave Chinner
2008-10-30 21:33   ` Christoph Lameter
2008-10-30 22:03     ` Dave Chinner
2008-10-31 13:47       ` Christoph Lameter
2008-10-31 16:36       ` David Rientjes
2008-11-04 20:47 ` Andrew Morton
2008-11-04 20:53   ` Peter Zijlstra
2008-11-04 20:58     ` Christoph Lameter
2008-11-04 21:10     ` David Rientjes
2008-11-04 21:16     ` Andrew Morton
2008-11-04 21:21       ` Peter Zijlstra
2008-11-04 21:50         ` Andrew Morton
2008-11-04 22:17           ` Christoph Lameter
2008-11-04 22:35             ` Andrew Morton
2008-11-04 22:52               ` Christoph Lameter
2008-11-04 23:36                 ` Andrew Morton
2008-11-05  1:31                   ` KAMEZAWA Hiroyuki
2008-11-05  3:09                     ` Andrew Morton
2008-11-05  2:45                   ` Christoph Lameter
2008-11-05  3:05                     ` Andrew Morton
2008-11-05  4:31                       ` KAMEZAWA Hiroyuki
2008-11-10  9:02                         ` Andrea Righi
2008-11-10 10:02                           ` David Rientjes
2008-11-05 13:52                       ` Christoph Lameter
2008-11-05 18:41                         ` Andrew Morton [this message]
2008-11-05 20:21                           ` Christoph Lameter
2008-11-05 20:31                             ` Andrew Morton
2008-11-05 20:40                               ` Christoph Lameter
2008-11-05 20:56                                 ` Andrew Morton
2008-11-05 21:28                                   ` Christoph Lameter
2008-11-05 21:55                                   ` Paul Menage
2008-11-05 22:04                                   ` David Rientjes
2008-11-06  1:34                                     ` KAMEZAWA Hiroyuki
2008-11-06 20:35                                       ` David Rientjes

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20081105104145.abd6fc91.akpm@linux-foundation.org \
    --to=akpm@linux-foundation.org \
    --cc=cl@linux-foundation.org \
    --cc=containers@lists.osdl.org \
    --cc=dfults@sgi.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=menage@google.com \
    --cc=npiggin@suse.de \
    --cc=peterz@infradead.org \
    --cc=rientjes@google.com \
    --subject='Re: [patch 0/7] cpuset writeback throttling' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).