[PATCH] cpuidle/powernv : init all present cpus for deep states

[PATCH] cpuidle/powernv : init all present cpus for deep states

[Akshay Adiga]

Init all present cpus for deep states instead of "all possible" cpus.
Init fails if the possible cpu is gaurded. Resulting in making only
non-deep states available for cpuidle/hotplug.

Signed-off-by: Akshay Adiga <akshay.adiga@linux.vnet.ibm.com>
---
 arch/powerpc/platforms/powernv/idle.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/powerpc/platforms/powernv/idle.c b/arch/powerpc/platforms/powernv/idle.c
index 1f12ab1..1c5d067 100644
--- a/arch/powerpc/platforms/powernv/idle.c
+++ b/arch/powerpc/platforms/powernv/idle.c
@@ -79,7 +79,7 @@ static int pnv_save_sprs_for_deep_states(void)
 	uint64_t msr_val = MSR_IDLE;
 	uint64_t psscr_val = pnv_deepest_stop_psscr_val;
 
-	for_each_possible_cpu(cpu) {
+	for_each_present_cpu(cpu) {
 		uint64_t pir = get_hard_smp_processor_id(cpu);
 		uint64_t hsprg0_val = (uint64_t)paca_ptrs[cpu];
 
@@ -814,7 +814,7 @@ static int __init pnv_init_idle_states(void)
 		int cpu;
 
 		pr_info("powernv: idle: Saving PACA pointers of all CPUs in their thread sibling PACA\n");
-		for_each_possible_cpu(cpu) {
+		for_each_present_cpu(cpu) {
 			int base_cpu = cpu_first_thread_sibling(cpu);
 			int idx = cpu_thread_in_core(cpu);
 			int i;
-- 
2.5.5

Re: [PATCH] cpuidle/powernv : init all present cpus for deep states

[Stewart Smith]

Akshay Adiga <akshay.adiga@linux.vnet.ibm.com> writes:
> Init all present cpus for deep states instead of "all possible" cpus.
> Init fails if the possible cpu is gaurded. Resulting in making only
> non-deep states available for cpuidle/hotplug.

Should this also head to stable? It means that for single threaded
workloads, if you guard out a CPU core you'll not get WoF, which means
that performance goes down when you wouldn't expect it to. Right?

-- 
Stewart Smith
OPAL Architect, IBM.

Re: [PATCH] cpuidle/powernv : init all present cpus for deep states

[Akshay Adiga]

Yes this needs to be sent to  stable. 

Fixes: d405a98c ("powerpc/powernv: Move cpuidle related code from setup.c
to new file")

Re: [PATCH] cpuidle/powernv : init all present cpus for deep states

[Michael Ellerman]

Akshay Adiga <akshay.adiga@linux.vnet.ibm.com> writes:

> Yes this needs to be sent to  stable. 
>
> Fixes: d405a98c ("powerpc/powernv: Move cpuidle related code from setup.c
> to new file")

Is that really the commit that introduced the bug? :)

Seems like it's more likely this one:

  Fixes: 77b54e9f213f ("powernv/powerpc: Add winkle support for offline cpus")


It's true that because the code was moved in d405a98c we won't be able
to automatically backport the fix past that commit, but we should still
identify the right commit in the Fixes tag.

cheers

Re: [PATCH] cpuidle/powernv : init all present cpus for deep states

[Akshay Adiga]

On Wed, May 16, 2018 at 05:32:14PM +0530, Akshay Adiga wrote:
> Init all present cpus for deep states instead of "all possible" cpus.
> Init fails if the possible cpu is gaurded. Resulting in making only
> non-deep states available for cpuidle/hotplug.
> 
> Signed-off-by: Akshay Adiga <akshay.adiga@linux.vnet.ibm.com>
> ---
>  arch/powerpc/platforms/powernv/idle.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/arch/powerpc/platforms/powernv/idle.c b/arch/powerpc/platforms/powernv/idle.c
> index 1f12ab1..1c5d067 100644
> --- a/arch/powerpc/platforms/powernv/idle.c
> +++ b/arch/powerpc/platforms/powernv/idle.c
> @@ -79,7 +79,7 @@ static int pnv_save_sprs_for_deep_states(void)
>  	uint64_t msr_val = MSR_IDLE;
>  	uint64_t psscr_val = pnv_deepest_stop_psscr_val;
> 
> -	for_each_possible_cpu(cpu) {
> +	for_each_present_cpu(cpu) {
>  		uint64_t pir = get_hard_smp_processor_id(cpu);
>  		uint64_t hsprg0_val = (uint64_t)paca_ptrs[cpu];
> 
> @@ -814,7 +814,7 @@ static int __init pnv_init_idle_states(void)
>  		int cpu;
> 
>  		pr_info("powernv: idle: Saving PACA pointers of all CPUs in their thread sibling PACA\n");
> -		for_each_possible_cpu(cpu) {
> +		for_each_present_cpu(cpu) {
>  			int base_cpu = cpu_first_thread_sibling(cpu);
>  			int idx = cpu_thread_in_core(cpu);
>  			int i;
> -- 
> 2.5.5
>

Missed CC'ing to  mpe@ellerman.id.au 

Re: [PATCH] cpuidle/powernv : init all present cpus for deep states

[Michael Ellerman]

Akshay Adiga <akshay.adiga@linux.vnet.ibm.com> writes:

> Init all present cpus for deep states instead of "all possible" cpus.
> Init fails if the possible cpu is gaurded. Resulting in making only
> non-deep states available for cpuidle/hotplug.

This is basically the opposite of what we just did for IMC.

There we switched from present to possible, to make it work when some
CPUs are guarded.

Which makes me think we need a better way of dealing with guarded CPUs,
because working out which code should use present or possible seems to
be basically trial-and-error.

I'm not actually sure why Guarded CPUs are showing up as possible but
not present, did we do that on purpose or is it just happening by
accident?

I can merge this, but we need to make this less bug prone in future.

cheers

> diff --git a/arch/powerpc/platforms/powernv/idle.c b/arch/powerpc/platforms/powernv/idle.c
> index 1f12ab1..1c5d067 100644
> --- a/arch/powerpc/platforms/powernv/idle.c
> +++ b/arch/powerpc/platforms/powernv/idle.c
> @@ -79,7 +79,7 @@ static int pnv_save_sprs_for_deep_states(void)
>  	uint64_t msr_val = MSR_IDLE;
>  	uint64_t psscr_val = pnv_deepest_stop_psscr_val;
>  
> -	for_each_possible_cpu(cpu) {
> +	for_each_present_cpu(cpu) {
>  		uint64_t pir = get_hard_smp_processor_id(cpu);
>  		uint64_t hsprg0_val = (uint64_t)paca_ptrs[cpu];
>  
> @@ -814,7 +814,7 @@ static int __init pnv_init_idle_states(void)
>  		int cpu;
>  
>  		pr_info("powernv: idle: Saving PACA pointers of all CPUs in their thread sibling PACA\n");
> -		for_each_possible_cpu(cpu) {
> +		for_each_present_cpu(cpu) {
>  			int base_cpu = cpu_first_thread_sibling(cpu);
>  			int idx = cpu_thread_in_core(cpu);
>  			int i;
> -- 
> 2.5.5

Re: [PATCH] cpuidle/powernv : init all present cpus for deep states

[Stewart Smith]

Michael Ellerman <mpe@ellerman.id.au> writes:
> Akshay Adiga <akshay.adiga@linux.vnet.ibm.com> writes:
>
>> Init all present cpus for deep states instead of "all possible" cpus.
>> Init fails if the possible cpu is gaurded. Resulting in making only
>> non-deep states available for cpuidle/hotplug.
>
> This is basically the opposite of what we just did for IMC.
>
> There we switched from present to possible, to make it work when some
> CPUs are guarded.
>
> Which makes me think we need a better way of dealing with guarded CPUs,
> because working out which code should use present or possible seems to
> be basically trial-and-error.
>
> I'm not actually sure why Guarded CPUs are showing up as possible but
> not present, did we do that on purpose or is it just happening by
> accident?

My guess is that it flows through from firmware putting the guarded out
CPUs in the device tree with a not "okay" status (which, I just
realised, we're putting something in 'status' that isn't what the
current DeviceTree spec says we should... gah -
https://github.com/open-power/skiboot/issues/178 filed for that one).

The idea behind that is that you can answer "where did all my CPUs go?"
by looking at the device tree rather than having to know the platform
specific way of how guards are stored.

-- 
Stewart Smith
OPAL Architect, IBM.

Re: cpuidle/powernv : init all present cpus for deep states

[Michael Ellerman]

On Wed, 2018-05-16 at 12:02:14 UTC, Akshay Adiga wrote:
> Init all present cpus for deep states instead of "all possible" cpus.
> Init fails if the possible cpu is gaurded. Resulting in making only
> non-deep states available for cpuidle/hotplug.
> 
> Signed-off-by: Akshay Adiga <akshay.adiga@linux.vnet.ibm.com>

Applied to powerpc next, thanks.

https://git.kernel.org/powerpc/c/ac9816dcbab53c57bcf1d7b15370b0

cheers