LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Jeremy Linton <jeremy.linton@arm.com>
To: Geert Uytterhoeven <geert@linux-m68k.org>,
	Will Deacon <will.deacon@arm.com>
Cc: Sudeep Holla <sudeep.holla@arm.com>,
	Catalin Marinas <catalin.marinas@arm.com>,
	ACPI Devel Maling List <linux-acpi@vger.kernel.org>,
	Mark Rutland <Mark.Rutland@arm.com>,
	austinwc@codeaurora.org, tnowicki@caviumnetworks.com,
	Palmer Dabbelt <palmer@sifive.com>,
	linux-riscv@lists.infradead.org, Morten.Rasmussen@arm.com,
	vkilari@codeaurora.org,
	Lorenzo Pieralisi <Lorenzo.Pieralisi@arm.com>,
	jhugo@codeaurora.org, Al Stone <ahs3@redhat.com>,
	Len Brown <lenb@kernel.org>, John Garry <john.garry@huawei.com>,
	wangxiongfeng2@huawei.com,
	Dietmar Eggemann <Dietmar.Eggemann@arm.com>,
	Linux ARM <linux-arm-kernel@lists.infradead.org>,
	Ard Biesheuvel <ard.biesheuvel@linaro.org>,
	Greg KH <gregkh@linuxfoundation.org>,
	"Rafael J. Wysocki" <rjw@rjwysocki.net>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	Hanjun Guo <hanjun.guo@linaro.org>,
	Linux-Renesas <linux-renesas-soc@vger.kernel.org>
Subject: Re: [PATCH v9 00/12] Support PPTT for ARM64
Date: Tue, 29 May 2018 16:52:04 -0500	[thread overview]
Message-ID: <a79f5acc-51f2-cd7c-5abf-c7c1df8064a1@arm.com> (raw)
In-Reply-To: <CAMuHMdU0yET6+-FfS8e9HJdKB7h0gDn7kzWGpJZV=UiWn5fLkA@mail.gmail.com>

Hi,

On 05/29/2018 10:51 AM, Geert Uytterhoeven wrote:
> Hi Will,
> 
> On Tue, May 29, 2018 at 5:08 PM, Will Deacon <will.deacon@arm.com> wrote:
>> On Tue, May 29, 2018 at 02:18:40PM +0100, Sudeep Holla wrote:
>>> On 29/05/18 12:56, Geert Uytterhoeven wrote:
>>>> On Tue, May 29, 2018 at 1:14 PM, Sudeep Holla <sudeep.holla@arm.com> wrote:
>>>>> On 29/05/18 11:48, Geert Uytterhoeven wrote:
>>>>>> System supend still works fine on systems with big cores only:
>>>>>>
>>>>>>      R-Car H3 ES1.0 (4xCA57 (4xCA53 disabled in firmware))
>>>>>>      R-Car M3-N (2xCA57)
>>>>>>
>>>>>> Reverting this commit fixes the issue for me.
>>>>>
>>>>> I can't find anything that relates to system suspend in these patches
>>>>> unless they are messing with something during CPU hot plug-in back
>>>>> during resume.
>>>>
>>>> It's only the last patch that introduces the breakage.
>>>>
>>>
>>> As specified in the commit log, it won't change any behavior for DT
>>> systems if it's non-NUMA or single node system. So I am still wondering
>>> what could trigger this regression.
>>
>> I wonder if we're somehow giving an uninitialised/invalid NUMA configuration
>> to the scheduler, although I can't see how this would happen.
>>
>> Geert -- if you enable CONFIG_DEBUG_PER_CPU_MAPS=y and apply the diff below
>> do you see anything shouting in dmesg?
> 
> Thanks, but unfortunately it doesn't help.
> I added some debug code to print cpumask, but so far I don't see anything
> suspicious.

I suspect most of the problem is related to the node mask changing at 
unexpected times (particularly cores being removed from the mask). Once 
I understand that more, there may be a simpler patch.

OTOH, I've been testing with this, and with it, I can't seem to 
duplicate the problem with CONFIG_NUMA disabled I found.

diff --git a/arch/arm64/include/asm/topology.h 
b/arch/arm64/include/asm/topology.h
index df48212f767b..7450ef5ed733 100644
--- a/arch/arm64/include/asm/topology.h
+++ b/arch/arm64/include/asm/topology.h
@@ -12,6 +12,7 @@ struct cpu_topology {
         cpumask_t thread_sibling;
         cpumask_t core_sibling;
         cpumask_t llc_siblings;
+       cpumask_t node_siblings; /* maintain a stable node sibling list */
  };

  extern struct cpu_topology cpu_topology[NR_CPUS];
diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
index f3e2e3aec0b0..f4eb80852d78 100644
--- a/arch/arm64/kernel/smp.c
+++ b/arch/arm64/kernel/smp.c
@@ -677,8 +677,9 @@ void __init smp_prepare_cpus(unsigned int max_cpus)
         init_cpu_topology();

         this_cpu = smp_processor_id();
-       store_cpu_topology(this_cpu);
         numa_store_cpu_info(this_cpu);
+       store_cpu_topology(this_cpu);
+

         /*
          * If UP is mandated by "nosmp" (which implies "maxcpus=0"), 
don't set
diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c
index 7415c166281f..6819c764537d 100644
--- a/arch/arm64/kernel/topology.c
+++ b/arch/arm64/kernel/topology.c
@@ -215,7 +215,7 @@ EXPORT_SYMBOL_GPL(cpu_topology);

  const struct cpumask *cpu_coregroup_mask(int cpu)
  {
-       const cpumask_t *core_mask = cpumask_of_node(cpu_to_node(cpu));
+       const cpumask_t *core_mask = &cpu_topology[cpu].node_siblings;

         /* Find the smaller of NUMA, core or LLC siblings */
         if (cpumask_subset(&cpu_topology[cpu].core_sibling, core_mask)) {
@@ -233,12 +233,16 @@ const struct cpumask *cpu_coregroup_mask(int cpu)
  static void update_siblings_masks(unsigned int cpuid)
  {
         struct cpu_topology *cpu_topo, *cpuid_topo = &cpu_topology[cpuid];
+       int node = cpu_to_node(cpuid);
         int cpu;

         /* update core and thread sibling masks */
         for_each_possible_cpu(cpu) {
                 cpu_topo = &cpu_topology[cpu];

+               if (cpu_to_node(cpu) == node)
+                       cpumask_set_cpu(cpu, &cpu_topo->node_siblings);
+
                 if (cpuid_topo->llc_id == cpu_topo->llc_id)
                         cpumask_set_cpu(cpu, &cpuid_topo->llc_siblings);

@@ -311,6 +315,9 @@ static void __init reset_cpu_topology(void)
                 cpumask_clear(&cpu_topo->llc_siblings);
                 cpumask_set_cpu(cpu, &cpu_topo->llc_siblings);

+               cpumask_clear(&cpu_topo->node_siblings);
+               cpumask_set_cpu(cpu, &cpu_topo->node_siblings);
+
                 cpumask_clear(&cpu_topo->core_sibling);
                 cpumask_set_cpu(cpu, &cpu_topo->core_sibling);
                 cpumask_clear(&cpu_topo->thread_sibling);
-- 
2.14.3

  parent reply	other threads:[~2018-05-29 21:52 UTC|newest]

Thread overview: 56+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-05-11 23:57 Jeremy Linton
2018-05-11 23:57 ` [PATCH v9 01/12] drivers: base: cacheinfo: move cache_setup_of_node() Jeremy Linton
2018-05-11 23:57 ` [PATCH v9 02/12] drivers: base: cacheinfo: setup DT cache properties early Jeremy Linton
2018-05-15 17:15   ` Jeremy Linton
2018-05-15 19:32     ` Andy Shevchenko
2018-05-16 10:56       ` Sudeep Holla
2018-05-17 15:47         ` Sudeep Holla
2018-05-18 21:50           ` Andy Shevchenko
2018-05-21  9:27             ` Sudeep Holla
2018-05-21 10:15               ` Sudeep Holla
2018-05-21 10:32       ` [PATCH] drivers: base: cacheinfo: use OF property_read_u64 instead of get_property,read_number Sudeep Holla
2018-05-21 12:53         ` [PATCH v2] drivers: base: cacheinfo: use OF property_read_u32 " Sudeep Holla
2018-06-05 16:21           ` Andy Shevchenko
2018-06-05 16:26             ` Sudeep Holla
2018-06-05 16:34               ` Andy Shevchenko
2018-05-17  6:54     ` [PATCH v9 02/12] drivers: base: cacheinfo: setup DT cache properties early Greg KH
2018-05-17  9:08       ` Sudeep Holla
2018-05-17  9:35         ` Greg KH
2018-05-11 23:57 ` [PATCH v9 03/12] cacheinfo: rename of_node to fw_token Jeremy Linton
2018-05-11 23:57 ` [PATCH v9 04/12] arm64/acpi: Create arch specific cpu to acpi id helper Jeremy Linton
2018-05-14 14:41   ` Sudeep Holla
2018-05-11 23:58 ` [PATCH v9 05/12] ACPI/PPTT: Add Processor Properties Topology Table parsing Jeremy Linton
2018-05-12 10:09   ` Rafael J. Wysocki
2018-05-15 21:42     ` Jeremy Linton
2018-05-16  8:24       ` Rafael J. Wysocki
2018-05-11 23:58 ` [PATCH v9 06/12] ACPI: Enable PPTT support on ARM64 Jeremy Linton
2018-05-11 23:58 ` [PATCH v9 07/12] drivers: base cacheinfo: Add support for ACPI based firmware tables Jeremy Linton
2018-05-11 23:58 ` [PATCH v9 08/12] arm64: " Jeremy Linton
2018-05-11 23:58 ` [PATCH v9 09/12] arm64: topology: rename cluster_id Jeremy Linton
2018-05-11 23:58 ` [PATCH v9 10/12] arm64: topology: enable ACPI/PPTT based CPU topology Jeremy Linton
2018-05-11 23:58 ` [PATCH v9 11/12] ACPI: Add PPTT to injectable table list Jeremy Linton
2018-05-12 10:10   ` Rafael J. Wysocki
2018-05-11 23:58 ` [PATCH v9 12/12] arm64: topology: divorce MC scheduling domain from core_siblings Jeremy Linton
2018-05-17 17:05 ` [PATCH v9 00/12] Support PPTT for ARM64 Catalin Marinas
2018-05-29 10:48   ` Geert Uytterhoeven
2018-05-29 11:14     ` Sudeep Holla
2018-05-29 11:56       ` Geert Uytterhoeven
2018-05-29 13:18         ` Sudeep Holla
2018-05-29 15:08           ` Will Deacon
2018-05-29 15:51             ` Geert Uytterhoeven
2018-05-29 17:08               ` Robin Murphy
2018-05-29 17:18                 ` Geert Uytterhoeven
2018-05-29 17:31                 ` Sudeep Holla
2018-05-29 20:16               ` Will Deacon
2018-05-29 20:48                 ` Jeremy Linton
2018-05-29 21:52               ` Jeremy Linton [this message]
2018-05-30 13:24                 ` Sudeep Holla
2018-05-29 15:23           ` Jeremy Linton
2018-05-29 15:50           ` Geert Uytterhoeven
2018-05-30  8:52             ` Morten Rasmussen
2018-06-05 13:55     ` [PATCH 1/3] Revert "arm64: topology: divorce MC scheduling domain from core_siblings" Sudeep Holla
2018-06-05 13:55       ` [PATCH 2/3] ACPI / PPTT: fix build when CONFIG_ACPI_PPTT is not enabled Sudeep Holla
2018-06-05 13:55       ` [PATCH 3/3] arm64: disable ACPI PPTT support temporarily Sudeep Holla
2018-06-05 14:09       ` [PATCH 1/3] Revert "arm64: topology: divorce MC scheduling domain from core_siblings" Geert Uytterhoeven
2018-06-05 14:12         ` Sudeep Holla
2018-06-04 15:12   ` [PATCH v9 00/12] Support PPTT for ARM64 Catalin Marinas

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=a79f5acc-51f2-cd7c-5abf-c7c1df8064a1@arm.com \
    --to=jeremy.linton@arm.com \
    --cc=Dietmar.Eggemann@arm.com \
    --cc=Lorenzo.Pieralisi@arm.com \
    --cc=Mark.Rutland@arm.com \
    --cc=Morten.Rasmussen@arm.com \
    --cc=ahs3@redhat.com \
    --cc=ard.biesheuvel@linaro.org \
    --cc=austinwc@codeaurora.org \
    --cc=catalin.marinas@arm.com \
    --cc=geert@linux-m68k.org \
    --cc=gregkh@linuxfoundation.org \
    --cc=hanjun.guo@linaro.org \
    --cc=jhugo@codeaurora.org \
    --cc=john.garry@huawei.com \
    --cc=lenb@kernel.org \
    --cc=linux-acpi@vger.kernel.org \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-renesas-soc@vger.kernel.org \
    --cc=linux-riscv@lists.infradead.org \
    --cc=palmer@sifive.com \
    --cc=rjw@rjwysocki.net \
    --cc=sudeep.holla@arm.com \
    --cc=tnowicki@caviumnetworks.com \
    --cc=vkilari@codeaurora.org \
    --cc=wangxiongfeng2@huawei.com \
    --cc=will.deacon@arm.com \
    --subject='Re: [PATCH v9 00/12] Support PPTT for ARM64' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).