LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
* [PATCH] genirq: Set initial default irq affinity to just CPU0
@ 2008-10-24 15:57 Kumar Gala
  2008-10-24 23:18 ` David Miller
  0 siblings, 1 reply; 8+ messages in thread
From: Kumar Gala @ 2008-10-24 15:57 UTC (permalink / raw)
  To: Linus Torvalds; +Cc: linux-kernel, linuxppc-dev, maxk, Andrew Morton, tglx

Commit 18404756765c713a0be4eb1082920c04822ce588 introduced a regression
on a subset of SMP based PPC systems whose interrupt controller only
allow setting an irq to a single processor.  The previous behavior
was only CPU0 was initially setup to get interrupts.  Revert back
to that behavior.

Signed-off-by: Kumar Gala <galak@kernel.crashing.org>
---
 kernel/irq/manage.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/kernel/irq/manage.c b/kernel/irq/manage.c
index c498a1b..728d36a 100644
--- a/kernel/irq/manage.c
+++ b/kernel/irq/manage.c
@@ -17,7 +17,7 @@
 
 #ifdef CONFIG_SMP
 
-cpumask_t irq_default_affinity = CPU_MASK_ALL;
+cpumask_t irq_default_affinity = CPU_MASK_CPU0;
 
 /**
  *	synchronize_irq - wait for pending IRQ handlers (on other CPUs)
-- 
1.5.5.1


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH] genirq: Set initial default irq affinity to just CPU0
  2008-10-24 15:57 [PATCH] genirq: Set initial default irq affinity to just CPU0 Kumar Gala
@ 2008-10-24 23:18 ` David Miller
  2008-10-25 21:33   ` Benjamin Herrenschmidt
  0 siblings, 1 reply; 8+ messages in thread
From: David Miller @ 2008-10-24 23:18 UTC (permalink / raw)
  To: galak; +Cc: torvalds, linux-kernel, linuxppc-dev, maxk, akpm, tglx

From: Kumar Gala <galak@kernel.crashing.org>
Date: Fri, 24 Oct 2008 10:57:38 -0500

> Commit 18404756765c713a0be4eb1082920c04822ce588 introduced a regression
> on a subset of SMP based PPC systems whose interrupt controller only
> allow setting an irq to a single processor.  The previous behavior
> was only CPU0 was initially setup to get interrupts.  Revert back
> to that behavior.
> 
> Signed-off-by: Kumar Gala <galak@kernel.crashing.org>

I really don't remember getting all of my interrupts only on cpu 0
on sparc64 before any of these changes.  I therefore find all of
this quite mysterious. :-)

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH] genirq: Set initial default irq affinity to just CPU0
  2008-10-24 23:18 ` David Miller
@ 2008-10-25 21:33   ` Benjamin Herrenschmidt
  2008-10-26  4:04     ` David Miller
  0 siblings, 1 reply; 8+ messages in thread
From: Benjamin Herrenschmidt @ 2008-10-25 21:33 UTC (permalink / raw)
  To: David Miller
  Cc: galak, akpm, linux-kernel, linuxppc-dev, torvalds, maxk, tglx

On Fri, 2008-10-24 at 16:18 -0700, David Miller wrote:
> From: Kumar Gala <galak@kernel.crashing.org>
> Date: Fri, 24 Oct 2008 10:57:38 -0500
> 
> > Commit 18404756765c713a0be4eb1082920c04822ce588 introduced a regression
> > on a subset of SMP based PPC systems whose interrupt controller only
> > allow setting an irq to a single processor.  The previous behavior
> > was only CPU0 was initially setup to get interrupts.  Revert back
> > to that behavior.
> > 
> > Signed-off-by: Kumar Gala <galak@kernel.crashing.org>
> 
> I really don't remember getting all of my interrupts only on cpu 0
> on sparc64 before any of these changes.  I therefore find all of
> this quite mysterious. :-)

Well, I don't know how you do it but on powerpc, we explicitely fill the
affinity masks at boot time when we can spread interrupts... Maybe we
should change it the other way around and limit the mask when we can't ?
It's hard to tell for sure at this stage.

Ben.



^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH] genirq: Set initial default irq affinity to just CPU0
  2008-10-25 21:33   ` Benjamin Herrenschmidt
@ 2008-10-26  4:04     ` David Miller
  2008-10-26  6:33       ` Benjamin Herrenschmidt
  0 siblings, 1 reply; 8+ messages in thread
From: David Miller @ 2008-10-26  4:04 UTC (permalink / raw)
  To: benh; +Cc: galak, akpm, linux-kernel, linuxppc-dev, torvalds, maxk, tglx

From: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Date: Sun, 26 Oct 2008 08:33:09 +1100

> Well, I don't know how you do it but on powerpc, we explicitely fill the
> affinity masks at boot time when we can spread interrupts... Maybe we
> should change it the other way around and limit the mask when we can't ?
> It's hard to tell for sure at this stage.

On sparc64 we look at the cpu mask configured for the interrupt and do
one of two things:

1) If all bits are set, we round robin assign a cpu at IRQ enable time.

2) Else we pick the first bit set in the mask.

One modification I want to make is to make case #1 NUMA aware.

But back to my original wonder, since I've always tipped off of this
generic IRQ layer cpu mask, when was it ever defaulting to zero
and causing the behvaior your powerpc guys actually want? :-)


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH] genirq: Set initial default irq affinity to just CPU0
  2008-10-26  4:04     ` David Miller
@ 2008-10-26  6:33       ` Benjamin Herrenschmidt
  2008-10-27 13:43         ` Kumar Gala
  0 siblings, 1 reply; 8+ messages in thread
From: Benjamin Herrenschmidt @ 2008-10-26  6:33 UTC (permalink / raw)
  To: David Miller
  Cc: galak, akpm, linux-kernel, linuxppc-dev, torvalds, maxk, tglx

On Sat, 2008-10-25 at 21:04 -0700, David Miller wrote:
> But back to my original wonder, since I've always tipped off of this
> generic IRQ layer cpu mask, when was it ever defaulting to zero
> and causing the behvaior your powerpc guys actually want? :-)

Well, I'm not sure what Kumar wants. Most powerpc SMP setups actually
want to spread interrupts to all CPUs, and those who can't tend to just
not implement set_affinity... So Kumar must have a special case of MPIC
usage here on FSL platforms.

In any case, the platform limitations should be dealt with there or the
user could break it by manipulating affinity via /proc anyway.

By yeah, I do expect default affinity to be all CPUs and in fact, I even
have an -OLD- comment in the code that says

 	/* let the mpic know we want intrs. default affinitya is 0xffffffff ...

Now, I've tried to track that down but it's hard because the generic code
seem to have changed in many ways around affinity handling...

So it looks like nowadays, the generic setup_irq() will call
irq_select_affinity() when an interrupt is first requested. Unless
you set CONFIG_AUTO_IRQ_AFFINITY and implement your own
irq_select_affinity(), thus, you will get the default one which copies
the content of this global irq_default_affinity to the interrupt.

However it does that _after_ your IRQ startup() has been called
(yes, this is very fishy), and so after you did your irq_choose_cpu()...

This is all very messy, along with hooks for balancing and other confusing
stuff that I suspect keeps changing. I'll have to spend more time next
week to sort out what exactly is happening on powerpc and whether we
get our interrupts spread or not...

That's the downside of having more generic irq code I suppose: now people
keep rewriting half of the generic code with x86 exclusively in mind and
we have to be extra careful :-)

Cheers,
Ben.



^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH] genirq: Set initial default irq affinity to just CPU0
  2008-10-26  6:33       ` Benjamin Herrenschmidt
@ 2008-10-27 13:43         ` Kumar Gala
  2008-10-27 20:27           ` Benjamin Herrenschmidt
  0 siblings, 1 reply; 8+ messages in thread
From: Kumar Gala @ 2008-10-27 13:43 UTC (permalink / raw)
  To: benh; +Cc: David Miller, akpm, linux-kernel, linuxppc-dev, torvalds, maxk, tglx


On Oct 26, 2008, at 1:33 AM, Benjamin Herrenschmidt wrote:

> On Sat, 2008-10-25 at 21:04 -0700, David Miller wrote:
>> But back to my original wonder, since I've always tipped off of this
>> generic IRQ layer cpu mask, when was it ever defaulting to zero
>> and causing the behvaior your powerpc guys actually want? :-)
>
> Well, I'm not sure what Kumar wants. Most powerpc SMP setups actually
> want to spread interrupts to all CPUs, and those who can't tend to  
> just
> not implement set_affinity... So Kumar must have a special case of  
> MPIC
> usage here on FSL platforms.
>
> In any case, the platform limitations should be dealt with there or  
> the
> user could break it by manipulating affinity via /proc anyway.
>
> By yeah, I do expect default affinity to be all CPUs and in fact, I  
> even
> have an -OLD- comment in the code that says
>
> 	/* let the mpic know we want intrs. default affinitya is  
> 0xffffffff ...

While we have the comment the code appears not to really follow it.   
We appear to write 1 << hard_smp_processor_id().

- k

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH] genirq: Set initial default irq affinity to just CPU0
  2008-10-27 13:43         ` Kumar Gala
@ 2008-10-27 20:27           ` Benjamin Herrenschmidt
  2008-10-27 20:45             ` Kumar Gala
  0 siblings, 1 reply; 8+ messages in thread
From: Benjamin Herrenschmidt @ 2008-10-27 20:27 UTC (permalink / raw)
  To: Kumar Gala
  Cc: David Miller, akpm, linux-kernel, linuxppc-dev, torvalds, maxk, tglx

On Mon, 2008-10-27 at 08:43 -0500, Kumar Gala wrote:
> 
> While we have the comment the code appears not to really follow it.   
> We appear to write 1 << hard_smp_processor_id().

That code is called by each CPU that gets onlined and OR's it's
bit in the mask.

Ben.



^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH] genirq: Set initial default irq affinity to just CPU0
  2008-10-27 20:27           ` Benjamin Herrenschmidt
@ 2008-10-27 20:45             ` Kumar Gala
  0 siblings, 0 replies; 8+ messages in thread
From: Kumar Gala @ 2008-10-27 20:45 UTC (permalink / raw)
  To: benh; +Cc: David Miller, akpm, linux-kernel, linuxppc-dev, torvalds, maxk, tglx


On Oct 27, 2008, at 3:27 PM, Benjamin Herrenschmidt wrote:

> On Mon, 2008-10-27 at 08:43 -0500, Kumar Gala wrote:
>>
>> While we have the comment the code appears not to really follow it.
>> We appear to write 1 << hard_smp_processor_id().
>
> That code is called by each CPU that gets onlined and OR's it's
> bit in the mask.

ahh, I see now.

- k

^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2008-10-27 20:47 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2008-10-24 15:57 [PATCH] genirq: Set initial default irq affinity to just CPU0 Kumar Gala
2008-10-24 23:18 ` David Miller
2008-10-25 21:33   ` Benjamin Herrenschmidt
2008-10-26  4:04     ` David Miller
2008-10-26  6:33       ` Benjamin Herrenschmidt
2008-10-27 13:43         ` Kumar Gala
2008-10-27 20:27           ` Benjamin Herrenschmidt
2008-10-27 20:45             ` Kumar Gala

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).