LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Muchun Song <songmuchun@bytedance.com>
To: guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org,
	akpm@linux-foundation.org, shakeelb@google.com,
	vdavydov.dev@gmail.com
Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org,
	duanxiongchun@bytedance.com, fam.zheng@bytedance.com,
	bsingharora@gmail.com, shy828301@gmail.com, alexs@kernel.org,
	smuchun@gmail.com, zhengqi.arch@bytedance.com,
	Muchun Song <songmuchun@bytedance.com>
Subject: [PATCH v1 02/12] mm: memcontrol: introduce compact_folio_lruvec_lock_irqsave
Date: Sat, 14 Aug 2021 13:25:09 +0800	[thread overview]
Message-ID: <20210814052519.86679-3-songmuchun@bytedance.com> (raw)
In-Reply-To: <20210814052519.86679-1-songmuchun@bytedance.com>

If we reuse the objcg APIs to charge LRU pages, the folio_memcg()
can be changed when the LRU pages reparented. In this case, we need
to acquire the new lruvec lock.

    lruvec = folio_lruvec(folio);

    // The page is reparented.

    compact_lock_irqsave(&lruvec->lru_lock, &flags, cc);

    // Acquired the wrong lruvec lock and need to retry.

But compact_lock_irqsave() only take lruvec lock as the parameter,
we cannot aware this change. If it can take the page as parameter
to acquire the lruvec lock. When the page memcg is changed, we can
use the folio_memcg() detect whether we need to reacquire the new
lruvec lock. So compact_lock_irqsave() is not suitable for us.
Similar to folio_lruvec_lock_irqsave(), introduce
compact_folio_lruvec_lock_irqsave() to acquire the lruvec lock in
the compaction routine.

Signed-off-by: Muchun Song <songmuchun@bytedance.com>
Acked-by: Roman Gushchin <guro@fb.com>
---
 mm/compaction.c | 31 +++++++++++++++++++++++++++----
 1 file changed, 27 insertions(+), 4 deletions(-)

diff --git a/mm/compaction.c b/mm/compaction.c
index fbc60f964c38..3131558a7c31 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -509,6 +509,29 @@ static bool compact_lock_irqsave(spinlock_t *lock, unsigned long *flags,
 	return true;
 }
 
+static struct lruvec *
+compact_folio_lruvec_lock_irqsave(struct folio *folio, unsigned long *flags,
+				  struct compact_control *cc)
+{
+	struct lruvec *lruvec;
+
+	lruvec = folio_lruvec(folio);
+
+	/* Track if the lock is contended in async mode */
+	if (cc->mode == MIGRATE_ASYNC && !cc->contended) {
+		if (spin_trylock_irqsave(&lruvec->lru_lock, *flags))
+			goto out;
+
+		cc->contended = true;
+	}
+
+	spin_lock_irqsave(&lruvec->lru_lock, *flags);
+out:
+	lruvec_memcg_debug(lruvec, folio);
+
+	return lruvec;
+}
+
 /*
  * Compaction requires the taking of some coarse locks that are potentially
  * very heavily contended. The lock should be periodically unlocked to avoid
@@ -837,6 +860,7 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
 
 	/* Time to isolate some pages for migration */
 	for (; low_pfn < end_pfn; low_pfn++) {
+		struct folio *folio;
 
 		if (skip_on_failure && low_pfn >= next_skip_pfn) {
 			/*
@@ -1022,18 +1046,17 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
 		if (!TestClearPageLRU(page))
 			goto isolate_fail_put;
 
-		lruvec = folio_lruvec(page_folio(page));
+		folio = page_folio(page);
+		lruvec = folio_lruvec(folio);
 
 		/* If we already hold the lock, we can skip some rechecking */
 		if (lruvec != locked) {
 			if (locked)
 				unlock_page_lruvec_irqrestore(locked, flags);
 
-			compact_lock_irqsave(&lruvec->lru_lock, &flags, cc);
+			lruvec = compact_folio_lruvec_lock_irqsave(folio, &flags, cc);
 			locked = lruvec;
 
-			lruvec_memcg_debug(lruvec, page_folio(page));
-
 			/* Try get exclusive access under lock */
 			if (!skip_updated) {
 				skip_updated = true;
-- 
2.11.0


  parent reply	other threads:[~2021-08-14  5:25 UTC|newest]

Thread overview: 22+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-08-14  5:25 [PATCH v1 00/12] Use obj_cgroup APIs to charge the LRU pages Muchun Song
2021-08-14  5:25 ` [PATCH v1 01/12] mm: memcontrol: prepare objcg API for non-kmem usage Muchun Song
2021-08-14 22:23   ` kernel test robot
2021-08-18  3:01   ` Roman Gushchin
2021-08-20  6:44     ` Muchun Song
2021-08-14  5:25 ` Muchun Song [this message]
2021-08-14  5:25 ` [PATCH v1 03/12] mm: memcontrol: make lruvec lock safe when LRU pages are reparented Muchun Song
2021-08-18  3:18   ` Roman Gushchin
2021-08-18  4:28     ` Muchun Song
2021-08-18  4:47       ` Roman Gushchin
2021-08-14  5:25 ` [PATCH v1 04/12] mm: vmscan: rework move_pages_to_lru() Muchun Song
2021-08-14  5:25 ` [PATCH v1 05/12] mm: thp: introduce folio_split_queue_lock{_irqsave}() Muchun Song
2021-08-14  8:22   ` kernel test robot
2021-08-14 10:38   ` kernel test robot
2021-08-14  5:25 ` [PATCH v1 06/12] mm: thp: make split queue lock safe when LRU pages are reparented Muchun Song
2021-08-14  5:25 ` [PATCH v1 07/12] mm: memcontrol: make all the callers of {folio,page}_memcg() safe Muchun Song
2021-08-14  5:25 ` [PATCH v1 08/12] mm: memcontrol: introduce memcg_reparent_ops Muchun Song
2021-08-14  5:25 ` [PATCH v1 09/12] mm: memcontrol: use obj_cgroup APIs to charge the LRU pages Muchun Song
2021-08-14 14:08   ` kernel test robot
2021-08-14  5:25 ` [PATCH v1 10/12] mm: memcontrol: rename {un}lock_page_memcg() to {un}lock_page_objcg() Muchun Song
2021-08-14  5:25 ` [PATCH v1 11/12] mm: lru: add VM_BUG_ON_FOLIO to lru maintenance function Muchun Song
2021-08-14  5:25 ` [PATCH v1 12/12] mm: lru: use lruvec lock to serialize memcg changes Muchun Song

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210814052519.86679-3-songmuchun@bytedance.com \
    --to=songmuchun@bytedance.com \
    --cc=akpm@linux-foundation.org \
    --cc=alexs@kernel.org \
    --cc=bsingharora@gmail.com \
    --cc=duanxiongchun@bytedance.com \
    --cc=fam.zheng@bytedance.com \
    --cc=guro@fb.com \
    --cc=hannes@cmpxchg.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mhocko@kernel.org \
    --cc=shakeelb@google.com \
    --cc=shy828301@gmail.com \
    --cc=smuchun@gmail.com \
    --cc=vdavydov.dev@gmail.com \
    --cc=zhengqi.arch@bytedance.com \
    --subject='Re: [PATCH v1 02/12] mm: memcontrol: introduce compact_folio_lruvec_lock_irqsave' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).