LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: David Rientjes <rientjes@google.com>
To: Paul Jackson <pj@sgi.com>
Cc: mingo@elte.hu, a.p.zijlstra@chello.nl, tglx@linutronix.de,
	oleg@tv-sign.ru, rostedt@goodmis.org, maxk@qualcomm.com,
	linux-kernel@vger.kernel.org
Subject: Re: [RFC/PATCH 0/4] CPUSET driven CPU isolation
Date: Thu, 28 Feb 2008 20:00:43 -0800 (PST)	[thread overview]
Message-ID: <alpine.DEB.1.00.0802281945060.31783@chino.kir.corp.google.com> (raw)
In-Reply-To: <20080228213402.7260906f.pj@sgi.com>

On Thu, 28 Feb 2008, Paul Jackson wrote:

> Apparently, you thought I was trying to reject the patch on the grounds
> that no such problem could ever occur, and you were showing how such a
> problem could occur.  I wasn't trying to reject the patch, and I agree
> that the check made sense, and I agree that such a problem could occur,
> as your example shows.
> 

Ok, good.  We're in agreement that my patch fixes a problem in allowing 
kthreads to move to cpusets that either currently or eventually deny them 
access to the cpu to which they are bound.

Now do you have any preference without extensive cpusets or cgroups 
hacking where we can gracefully handle -EINVAL being returned from 
set_cpus_allowed() so the task doesn't end up in a different cpuset?  The 
return value of set_cpus_allowed() is currently ignored by the cpuset 
implementation and, regardless, cpuset_attach() returns void.

cpuset_can_attach() does some sanity checking, but we need a call to 
set_cpus_allowed() to check the new logic and will actually change 
tsk->cpus_allowed if correctly invoked.

So the cpusets implementation needs a similar check in cpuset_can_attach() 
so that it is expressly denied from moving before we ever call 
cpuset_attach().  It doesn't make sense that a kthread is going to move 
itself through the cpusets interface, so this can simply be done by 
checking for tsk->flags & PF_THREAD_BOUND.



sched: prevent bound kthreads from changing cpus_allowed

Kthreads that have called kthread_bind() are bound to specific cpus, so 
other tasks should not be able to change their cpus_allowed from under 
them.  Otherwise, it is possible to move kthreads, such as the migration 
or watchdog threads, so they are not allowed access to the cpu they work 
on.

Signed-off-by: David Rientjes <rientjes@google.com>
---
 include/linux/sched.h |    1 +
 kernel/cpuset.c       |    2 ++
 kernel/kthread.c      |    1 +
 kernel/sched.c        |    6 ++++++
 4 files changed, 10 insertions(+), 0 deletions(-)

diff --git a/include/linux/sched.h b/include/linux/sched.h
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -1464,6 +1464,7 @@ static inline void put_task_struct(struct task_struct *t)
 #define PF_SWAPWRITE	0x00800000	/* Allowed to write to swap */
 #define PF_SPREAD_PAGE	0x01000000	/* Spread page cache over cpuset */
 #define PF_SPREAD_SLAB	0x02000000	/* Spread some slab caches over cpuset */
+#define PF_THREAD_BOUND 0x04000000	/* Thread bound to specific cpu */
 #define PF_MEMPOLICY	0x10000000	/* Non-default NUMA mempolicy */
 #define PF_MUTEX_TESTER	0x20000000	/* Thread belongs to the rt mutex tester */
 #define PF_FREEZER_SKIP	0x40000000	/* Freezer should not count it as freezeable */
diff --git a/kernel/cpuset.c b/kernel/cpuset.c
--- a/kernel/cpuset.c
+++ b/kernel/cpuset.c
@@ -1162,6 +1162,8 @@ static int cpuset_can_attach(struct cgroup_subsys *ss,
 
 	if (cpus_empty(cs->cpus_allowed) || nodes_empty(cs->mems_allowed))
 		return -ENOSPC;
+	if (tsk->flags & PF_THREAD_BOUND)
+		return -EINVAL;
 
 	return security_task_setscheduler(tsk, 0, NULL);
 }
diff --git a/kernel/kthread.c b/kernel/kthread.c
--- a/kernel/kthread.c
+++ b/kernel/kthread.c
@@ -180,6 +180,7 @@ void kthread_bind(struct task_struct *k, unsigned int cpu)
 	wait_task_inactive(k);
 	set_task_cpu(k, cpu);
 	k->cpus_allowed = cpumask_of_cpu(cpu);
+	k->flags |= PF_THREAD_BOUND;
 }
 EXPORT_SYMBOL(kthread_bind);
 
diff --git a/kernel/sched.c b/kernel/sched.c
--- a/kernel/sched.c
+++ b/kernel/sched.c
@@ -5345,6 +5345,12 @@ int set_cpus_allowed(struct task_struct *p, cpumask_t new_mask)
 		goto out;
 	}
 
+	if (unlikely((p->flags & PF_THREAD_BOUND) && p != current &&
+		     !cpus_equal(p->cpus_allowed, new_mask))) {
+		ret = -EINVAL;
+		goto out;
+	}
+
 	if (p->sched_class->set_cpus_allowed)
 		p->sched_class->set_cpus_allowed(p, &new_mask);
 	else {

  reply	other threads:[~2008-02-29  4:01 UTC|newest]

Thread overview: 94+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2008-02-27 22:21 Peter Zijlstra
2008-02-27 22:21 ` [RFC/PATCH 1/4] sched: remove isolcpus Peter Zijlstra
2008-02-27 23:57   ` Max Krasnyanskiy
2008-02-28 10:19     ` Peter Zijlstra
2008-02-28 19:36       ` Max Krasnyansky
2008-02-27 22:21 ` [RFC/PATCH 2/4] cpuset: system sets Peter Zijlstra
2008-02-27 23:39   ` Paul Jackson
2008-02-28  1:53     ` Max Krasnyanskiy
2008-02-27 23:52   ` Max Krasnyanskiy
2008-02-28  0:11     ` Paul Jackson
2008-02-28  0:29       ` Steven Rostedt
2008-02-28  1:45         ` Max Krasnyanskiy
2008-02-28  3:41           ` Steven Rostedt
2008-02-28  4:58             ` Max Krasnyansky
2008-02-27 22:21 ` [RFC/PATCH 3/4] genirq: system set irq affinities Peter Zijlstra
2008-02-28  0:10   ` Max Krasnyanskiy
2008-02-28 10:19     ` Peter Zijlstra
2008-02-27 22:21 ` [RFC/PATCH 4/4] kthread: system set kthread affinities Peter Zijlstra
2008-02-27 23:38 ` [RFC/PATCH 0/4] CPUSET driven CPU isolation Max Krasnyanskiy
2008-02-28 10:19   ` Peter Zijlstra
2008-02-28 17:33     ` Max Krasnyanskiy
2008-02-28  7:50 ` Ingo Molnar
2008-02-28  8:08   ` Paul Jackson
2008-02-28  9:08     ` Ingo Molnar
2008-02-28  9:17       ` Paul Jackson
2008-02-28  9:32         ` David Rientjes
2008-02-28 10:12           ` David Rientjes
2008-02-28 10:26             ` Peter Zijlstra
2008-02-28 17:37             ` Paul Jackson
2008-02-28 21:24               ` David Rientjes
2008-02-28 22:46                 ` Paul Jackson
2008-02-28 23:00                   ` David Rientjes
2008-02-29  0:16                     ` Paul Jackson
2008-02-29  1:05                       ` David Rientjes
2008-02-29  3:34                         ` Paul Jackson
2008-02-29  4:00                           ` David Rientjes [this message]
2008-02-29  6:53                             ` Paul Jackson
2008-02-28 10:46         ` Ingo Molnar
2008-02-28 17:47           ` Paul Jackson
2008-02-28 20:11           ` Max Krasnyansky
2008-02-28 20:13             ` Paul Jackson
2008-02-28 20:26               ` Max Krasnyansky
2008-02-28 20:27                 ` Paul Jackson
2008-02-28 20:45                   ` Max Krasnyansky
2008-02-28 20:23       ` Max Krasnyansky
2008-02-28 17:48   ` Max Krasnyanskiy
2008-02-29  8:31   ` Andrew Morton
2008-02-29  8:36     ` Andrew Morton
2008-02-29  9:10     ` Ingo Molnar
2008-02-29 18:06       ` Max Krasnyanskiy
2008-02-28 12:12 ` Mark Hounschell
2008-02-28 19:57   ` Max Krasnyansky
2008-02-29 18:55 ` [RFC/PATCH] cpuset: cpuset irq affinities Peter Zijlstra
2008-02-29 19:02   ` Ingo Molnar
2008-02-29 20:52     ` Max Krasnyanskiy
2008-02-29 21:03       ` Peter Zijlstra
2008-02-29 21:20         ` Max Krasnyanskiy
2008-03-03 11:57           ` Peter Zijlstra
2008-03-03 17:36             ` Paul Jackson
2008-03-03 17:57               ` Peter Zijlstra
2008-03-03 18:10                 ` Paul Jackson
2008-03-03 18:18                   ` Peter Zijlstra
2008-03-04  7:35                     ` Paul Jackson
2008-03-04 11:06                       ` Peter Zijlstra
2008-03-04 19:52                         ` Max Krasnyanskiy
2008-03-05  1:11                           ` Paul Jackson
2008-03-05  8:37                             ` Peter Zijlstra
2008-03-05  8:50                               ` Ingo Molnar
2008-03-05 12:35                                 ` Paul Jackson
2008-03-05 12:43                                   ` Ingo Molnar
2008-03-05 17:44                                     ` Paul Jackson
2008-03-05 19:17                               ` Max Krasnyansky
2008-03-06 13:47                               ` Paul Jackson
2008-03-06 15:21                                 ` Peter Zijlstra
2008-03-07  3:40                                   ` Paul Jackson
2008-03-07  6:39                                     ` Paul Jackson
2008-03-07  8:47                                       ` Paul Menage
2008-03-07 14:57                                         ` Paul Jackson
2008-03-03 18:41                   ` Paul Menage
2008-03-03 18:52                     ` Paul Jackson
2008-03-04  5:26                       ` Paul Menage
2008-03-04  6:15                         ` Paul Jackson
2008-03-04  6:21                           ` Paul Menage
2008-03-04  6:26                             ` Paul Jackson
2008-03-04  6:34                               ` Paul Menage
2008-03-04  6:51                                 ` Paul Jackson
2008-02-29 20:55   ` Paul Jackson
2008-02-29 21:14     ` Peter Zijlstra
2008-02-29 21:29       ` Ingo Molnar
2008-02-29 21:32       ` Ingo Molnar
2008-02-29 21:42       ` Max Krasnyanskiy
2008-02-29 22:00         ` Paul Jackson
2008-02-29 21:53       ` Paul Jackson
2008-03-02  5:18   ` Christoph Hellwig

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=alpine.DEB.1.00.0802281945060.31783@chino.kir.corp.google.com \
    --to=rientjes@google.com \
    --cc=a.p.zijlstra@chello.nl \
    --cc=linux-kernel@vger.kernel.org \
    --cc=maxk@qualcomm.com \
    --cc=mingo@elte.hu \
    --cc=oleg@tv-sign.ru \
    --cc=pj@sgi.com \
    --cc=rostedt@goodmis.org \
    --cc=tglx@linutronix.de \
    --subject='Re: [RFC/PATCH 0/4] CPUSET driven CPU isolation' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).