LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Max Krasnyansky <maxk@qualcomm.com>
To: Thomas Gleixner <tglx@linutronix.de>
Cc: LKML <linux-kernel@vger.kernel.org>,
	Peter Zijlstra <a.p.zijlstra@chello.nl>
Subject: [RFC] Genirq and CPU isolation
Date: Fri, 22 Feb 2008 22:19:18 -0800	[thread overview]
Message-ID: <47BFBAE6.3040606@qualcomm.com> (raw)

Hi Thomas,

While reviewing CPU isolation patches Peter pointed out that instead of
changing arch specific irq handling I should be extending genirq code.
Which makes perfect sense. Why didn't I think of that before :)
Basically the idea is that by default isolated CPUs must not get HW
irqs routed to them (besides IPIs and stuff of course). 
Does the patch included below look like the right approach ?

btw select_smp_affinity() which is currently used only by alpha seemed
out of place. It's called multiple times for shared irqs. ie Every time
new handler is registered irq is moved to a different CPU.
So I moved it under "if (!shared)" check inside setup_irq().

The patch introduces generic version of the select_smp_affinity() that
sets the affinity mask to "online_cpus - isolated_cpus", and updates 
x86_32 and alpha load balancers to ignore isolated cpus.
Booted on Core2 laptop and dual Opteron boxes with and w/o isolcpus=
options and everything seems to work as expected.

I wanted to run this by you before I include it in my patch series.
Thanx
Max


diff --git a/arch/alpha/kernel/irq.c b/arch/alpha/kernel/irq.c
index facf82a..6b01702 100644
--- a/arch/alpha/kernel/irq.c
+++ b/arch/alpha/kernel/irq.c
@@ -51,7 +51,7 @@ select_smp_affinity(unsigned int irq)
 	if (!irq_desc[irq].chip->set_affinity || irq_user_affinity[irq])
 		return 1;
 
-	while (!cpu_possible(cpu))
+	while (!cpu_possible(cpu) || cpu_isolated(cpu))
 		cpu = (cpu < (NR_CPUS-1) ? cpu + 1 : 0);
 	last_cpu = cpu;
 
diff --git a/arch/x86/kernel/genapic_flat_64.c b/arch/x86/kernel/genapic_flat_64.c
index e02e58c..07352b7 100644
--- a/arch/x86/kernel/genapic_flat_64.c
+++ b/arch/x86/kernel/genapic_flat_64.c
@@ -21,9 +21,7 @@
 
 static cpumask_t flat_target_cpus(void)
 {
-	cpumask_t target;
-	cpus_andnot(target, cpu_online_map, cpu_isolated_map);
-	return target;
+	return cpu_online_map;
 }
 
 static cpumask_t flat_vector_allocation_domain(int cpu)
diff --git a/arch/x86/kernel/io_apic_32.c b/arch/x86/kernel/io_apic_32.c
index 4ca5486..9c8816f 100644
--- a/arch/x86/kernel/io_apic_32.c
+++ b/arch/x86/kernel/io_apic_32.c
@@ -468,7 +468,7 @@ static void do_irq_balance(void)
 	for_each_possible_cpu(i) {
 		int package_index;
 		CPU_IRQ(i) = 0;
-		if (!cpu_online(i))
+		if (!cpu_online(i) || cpu_isolated(i))
 			continue;
 		package_index = CPU_TO_PACKAGEINDEX(i);
 		for (j = 0; j < NR_IRQS; j++) {
diff --git a/include/linux/irq.h b/include/linux/irq.h
index 176e5e7..287bc64 100644
--- a/include/linux/irq.h
+++ b/include/linux/irq.h
@@ -253,14 +253,7 @@ static inline void set_balance_irq_affinity(unsigned int irq, cpumask_t mask)
 }
 #endif
 
-#ifdef CONFIG_AUTO_IRQ_AFFINITY
 extern int select_smp_affinity(unsigned int irq);
-#else
-static inline int select_smp_affinity(unsigned int irq)
-{
-	return 1;
-}
-#endif
 
 extern int no_irq_affinity;
 
diff --git a/kernel/irq/manage.c b/kernel/irq/manage.c
index 438a014..e74db94 100644
--- a/kernel/irq/manage.c
+++ b/kernel/irq/manage.c
@@ -376,6 +376,9 @@ int setup_irq(unsigned int irq, struct irqaction *new)
 		} else
 			/* Undo nested disables: */
 			desc->depth = 1;
+
+		/* Set default affinity mask once everything is setup */
+		select_smp_affinity(irq);
 	}
 	/* Reset broken irq detection when installing new handler */
 	desc->irq_count = 0;
@@ -488,6 +491,26 @@ void free_irq(unsigned int irq, void *dev_id)
 }
 EXPORT_SYMBOL(free_irq);
 
+#ifndef CONFIG_AUTO_IRQ_AFFINITY
+/**
+ * Generic version of the affinity autoselector.
+ * Called under desc->lock from setup_irq().
+ * btw Should we rename this to select_irq_affinity() ?
+ */
+int select_smp_affinity(unsigned int irq)
+{
+	cpumask_t usable_cpus;
+
+	if (!irq_can_set_affinity(irq))
+		return 0;
+
+	cpus_andnot(usable_cpus, cpu_online_map, cpu_isolated_map);
+	irq_desc[irq].affinity = usable_cpus;
+	irq_desc[irq].chip->set_affinity(irq, usable_cpus);
+	return 0;
+}
+#endif
+
 /**
  *	request_irq - allocate an interrupt line
  *	@irq: Interrupt line to allocate
@@ -555,8 +578,6 @@ int request_irq(unsigned int irq, irq_handler_t handler,
 	action->next = NULL;
 	action->dev_id = dev_id;
 
-	select_smp_affinity(irq);
-
 #ifdef CONFIG_DEBUG_SHIRQ
 	if (irqflags & IRQF_SHARED) {
 		/*

             reply	other threads:[~2008-02-23  6:19 UTC|newest]

Thread overview: 4+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2008-02-23  6:19 Max Krasnyansky [this message]
2008-02-25  1:47 ` Randy Dunlap
2008-02-25 18:34   ` Max Krasnyanskiy
2008-02-27 20:41 ` Max Krasnyanskiy

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=47BFBAE6.3040606@qualcomm.com \
    --to=maxk@qualcomm.com \
    --cc=a.p.zijlstra@chello.nl \
    --cc=linux-kernel@vger.kernel.org \
    --cc=tglx@linutronix.de \
    --subject='Re: [RFC] Genirq and CPU isolation' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).