LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
* [PATCH v1] kthread/smpboot: Serialize kthread parking against wakeup
@ 2018-04-25  8:33 Gaurav Kohli
  2018-04-25 20:09 ` Peter Zijlstra
  2018-04-28  6:43 ` [lkp-robot] [kthread/smpboot] cad8e99675: inconsistent{IN-HARDIRQ-W}->{HARDIRQ-ON-W}usage kernel test robot
  0 siblings, 2 replies; 39+ messages in thread
From: Gaurav Kohli @ 2018-04-25  8:33 UTC (permalink / raw)
  To: peterz, tglx, mpe, mingo, bigeasy
  Cc: linux-kernel, linux-arm-msm, Gaurav Kohli, Neeraj Upadhyay

The control cpu thread which initiates hotplug calls kthread_park()
for hotplug thread and sets KTHREAD_SHOULD_PARK. After this control
thread wakes up the hotplug thread. There is a chance that wakeup
code sees the hotplug thread (running on AP core) in INTERRUPTIBLE
state, but sets its state to RUNNING after hotplug thread has entered
kthread_parkme() and changed its state to TASK_PARKED. This can result
in panic later on in kthread_unpark(), as it sees KTHREAD_IS_PARKED
flag set but fails to rebind the kthread, due to it being not in
TASK_PARKED state. Fix this, by serializing wakeup state change,
against state change before parking the kthread.

Below is the possible race:

Control thread				      Hotplug Thread

kthread_park()
set KTHREAD_SHOULD_PARK
					      smpboot_thread_fn
					      set_current_state(TASK_INTERRUPTIBLE);
					      kthread_parkme

wake_up_process()

raw_spin_lock_irqsave(&p->pi_lock, flags);
if (!(p->state & state)) -> this will fail
            goto out;

					      __kthread_parkme
					       __set_current_state(TASK_PARKED);

if (p->on_rq && ttwu_remote(p, wake_flags))
    ttwu_remote()
        p->state = TASK_RUNNING;
						schedule();

So to avoid this race, take pi_lock to serial state changes.

Suggested-by: Pavankumar Kondeti <pkondeti@codeaurora.org>
Co-developed-by: Neeraj Upadhyay <neeraju@codeaurora.org>
Signed-off-by: Neeraj Upadhyay <neeraju@codeaurora.org>
Signed-off-by: Gaurav Kohli <gkohli@codeaurora.org>
---

Changes since V1:
- Add comment to explain need of pi_lock.

diff --git a/kernel/smpboot.c b/kernel/smpboot.c
index 5043e74..c5c5184 100644
--- a/kernel/smpboot.c
+++ b/kernel/smpboot.c
@@ -122,7 +122,45 @@ static int smpboot_thread_fn(void *data)
 		}
 
 		if (kthread_should_park()) {
+			/*
+			 * Serialize against wakeup. If we take the lock first,
+			 * wakeup is skipped. If we run later, we observe,
+			 * TASK_RUNNING update from wakeup path, before moving
+			 * forward. This helps avoid the race, where wakeup
+			 * observes TASK_INTERRUPTIBLE, and also observes
+			 * the TASK_PARKED in kthread_parkme() before updating
+			 * task state to TASK_RUNNING. In this case, kthread
+			 * gets parked in TASK_RUNNING state. This results
+			 * in panic later on in kthread_unpark(), as it sees
+			 * KTHREAD_IS_PARKED flag set but fails to rebind the
+			 * kthread, due to it being not in TASK_PARKED state.
+			 *
+			 * Control thread                      Hotplug Thread
+			 *
+			 * kthread_park()
+			 *   set KTHREAD_SHOULD_PARK
+			 *                                smpboot_thread_fn()
+			 *                                set_current_state(
+			 *                                TASK_INTERRUPTIBLE);
+			 *                                kthread_parkme()
+			 *
+			 *   wake_up_process()
+			 *
+			 * raw_spin_lock_irqsave(&p->pi_lock, flags);
+			 * if (!(p->state & state))       __set_current_state(
+			 *            goto out;           TASK_RUNNING);
+			 *
+			 *                                __set_current_state(
+			 *                                TASK_PARKED);
+			 *
+			 * if (p->on_rq && ttwu_remote(p, wake_flags))
+			 *   ttwu_remote()
+			 *     p->state = TASK_RUNNING;
+			 *                                   schedule();
+			 */
+			raw_spin_lock(&current->pi_lock);
 			__set_current_state(TASK_RUNNING);
+			raw_spin_unlock(&current->pi_lock);
 			preempt_enable();
 			if (ht->park && td->status == HP_THREAD_ACTIVE) {
 				BUG_ON(td->cpu != smp_processor_id());
-- 
Qualcomm India Private Limited, on behalf of Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project.

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v1] kthread/smpboot: Serialize kthread parking against wakeup
  2018-04-25  8:33 [PATCH v1] kthread/smpboot: Serialize kthread parking against wakeup Gaurav Kohli
@ 2018-04-25 20:09 ` Peter Zijlstra
  2018-04-26  4:04   ` Kohli, Gaurav
  2018-04-26  8:41   ` Peter Zijlstra
  2018-04-28  6:43 ` [lkp-robot] [kthread/smpboot] cad8e99675: inconsistent{IN-HARDIRQ-W}->{HARDIRQ-ON-W}usage kernel test robot
  1 sibling, 2 replies; 39+ messages in thread
From: Peter Zijlstra @ 2018-04-25 20:09 UTC (permalink / raw)
  To: Gaurav Kohli
  Cc: tglx, mpe, mingo, bigeasy, linux-kernel, linux-arm-msm,
	Neeraj Upadhyay, Will Deacon

On Wed, Apr 25, 2018 at 02:03:19PM +0530, Gaurav Kohli wrote:
> diff --git a/kernel/smpboot.c b/kernel/smpboot.c
> index 5043e74..c5c5184 100644
> --- a/kernel/smpboot.c
> +++ b/kernel/smpboot.c
> @@ -122,7 +122,45 @@ static int smpboot_thread_fn(void *data)
>  		}
>  
>  		if (kthread_should_park()) {
> +			/*
> +			 * Serialize against wakeup.
			 *
			 * Prior wakeups must complete and later wakeups
			 * will observe TASK_RUNNING.
			 *
			 * This avoids the case where the TASK_RUNNING
			 * store from ttwu() competes with the
			 * TASK_PARKED store from kthread_parkme().
			 *
			 * If the TASK_PARKED store looses that
			 * competition, kthread_unpark() will go wobbly.
> +			 */
> +			raw_spin_lock(&current->pi_lock);
>  			__set_current_state(TASK_RUNNING);
> +			raw_spin_unlock(&current->pi_lock);
>  			preempt_enable();
>  			if (ht->park && td->status == HP_THREAD_ACTIVE) {
>  				BUG_ON(td->cpu != smp_processor_id());

Does that work for you?

But looking at this a bit more; don't we have the exact same problem
with the TASK_RUNNING store in the !ht->thread_should_run() case?
Suppose a ttwu() happens concurrently there, it can end up competing
against the TASK_INTERRUPTIBLE store, no?

Of course, that race is not fatal, we'll just end up going around the
loop once again I suppose. Maybe a comment there too?

			/*
			 * A similar race is possible here, but loosing
			 * the TASK_INTERRUPTIBLE store is harmless and
			 * will make us go around the loop once more.
			 */

And of course, I suspect we actually want to use TASK_IDLE, smpboot
threads don't want signals do they? But that probably ought to be a
separate patch.

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v1] kthread/smpboot: Serialize kthread parking against wakeup
  2018-04-25 20:09 ` Peter Zijlstra
@ 2018-04-26  4:04   ` Kohli, Gaurav
  2018-04-26  9:14     ` Peter Zijlstra
  2018-04-26  8:41   ` Peter Zijlstra
  1 sibling, 1 reply; 39+ messages in thread
From: Kohli, Gaurav @ 2018-04-26  4:04 UTC (permalink / raw)
  To: Peter Zijlstra
  Cc: tglx, mpe, mingo, bigeasy, linux-kernel, linux-arm-msm,
	Neeraj Upadhyay, Will Deacon

On 4/26/2018 1:39 AM, Peter Zijlstra wrote:

> On Wed, Apr 25, 2018 at 02:03:19PM +0530, Gaurav Kohli wrote:
>> diff --git a/kernel/smpboot.c b/kernel/smpboot.c
>> index 5043e74..c5c5184 100644
>> --- a/kernel/smpboot.c
>> +++ b/kernel/smpboot.c
>> @@ -122,7 +122,45 @@ static int smpboot_thread_fn(void *data)
>>   		}
>>   
>>   		if (kthread_should_park()) {
>> +			/*
>> +			 * Serialize against wakeup.
> 			 *
> 			 * Prior wakeups must complete and later wakeups
> 			 * will observe TASK_RUNNING.
> 			 *
> 			 * This avoids the case where the TASK_RUNNING
> 			 * store from ttwu() competes with the
> 			 * TASK_PARKED store from kthread_parkme().
> 			 *
> 			 * If the TASK_PARKED store looses that
> 			 * competition, kthread_unpark() will go wobbly.
>> +			 */
>> +			raw_spin_lock(&current->pi_lock);
>>   			__set_current_state(TASK_RUNNING);
>> +			raw_spin_unlock(&current->pi_lock);
>>   			preempt_enable();
>>   			if (ht->park && td->status == HP_THREAD_ACTIVE) {
>>   				BUG_ON(td->cpu != smp_processor_id());
> Does that work for you?

We have given patch for testing, usually it takes around 2-3 days for reproduction(we will update for the same).

>
> But looking at this a bit more; don't we have the exact same problem
> with the TASK_RUNNING store in the !ht->thread_should_run() case?
> Suppose a ttwu() happens concurrently there, it can end up competing
> against the TASK_INTERRUPTIBLE store, no?
>
> Of course, that race is not fatal, we'll just end up going around the
> loop once again I suppose. Maybe a comment there too?
>
> 			/*
> 			 * A similar race is possible here, but loosing
> 			 * the TASK_INTERRUPTIBLE store is harmless and
> 			 * will make us go around the loop once more.
> 			 */

Actually instead of race, i am seeing wakeup miss problem which is very rare, if we take case of hotplug thread

Controller                                           Hotplug

                                                              Loop start

                                                              set_current_state(TASK_INTERRUPTIBLE);

                                                              if (kthread_should_park()) { -> fails

Set Should_park

then wake_up

                                                             if (!ht->thread_should_run(td->cpu)) {

                                                             preempt_enable_no_resched();

                                                             schedule(); Again went to schedule(which is very rare to occur,not sure whether it hits)

                                           

>
> And of course, I suspect we actually want to use TASK_IDLE, smpboot
> threads don't want signals do they? But that probably ought to be a
> separate patch.

Yes I agree, we can control race from here as well,  Please suggest would below change be any help here:

  } else {

                         __set_current_state(TASK_RUNNING);

                         preempt_enable();

                         ht->thread_fn(td->cpu);

                        + set_current_state(TASK_INTERRUPTIBLE);

                        + schedule();

                 }

>
-- 
Qualcomm India Private Limited, on behalf of Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project.

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v1] kthread/smpboot: Serialize kthread parking against wakeup
  2018-04-25 20:09 ` Peter Zijlstra
  2018-04-26  4:04   ` Kohli, Gaurav
@ 2018-04-26  8:41   ` Peter Zijlstra
  2018-04-26  8:57     ` Peter Zijlstra
                       ` (2 more replies)
  1 sibling, 3 replies; 39+ messages in thread
From: Peter Zijlstra @ 2018-04-26  8:41 UTC (permalink / raw)
  To: Gaurav Kohli
  Cc: tglx, mpe, mingo, bigeasy, linux-kernel, linux-arm-msm,
	Neeraj Upadhyay, Will Deacon, Oleg Nesterov

On Wed, Apr 25, 2018 at 10:09:17PM +0200, Peter Zijlstra wrote:
> On Wed, Apr 25, 2018 at 02:03:19PM +0530, Gaurav Kohli wrote:
> > diff --git a/kernel/smpboot.c b/kernel/smpboot.c
> > index 5043e74..c5c5184 100644
> > --- a/kernel/smpboot.c
> > +++ b/kernel/smpboot.c
> > @@ -122,7 +122,45 @@ static int smpboot_thread_fn(void *data)
> >  		}
> >  
> >  		if (kthread_should_park()) {
> > +			/*
> > +			 * Serialize against wakeup.
> 			 *
> 			 * Prior wakeups must complete and later wakeups
> 			 * will observe TASK_RUNNING.
> 			 *
> 			 * This avoids the case where the TASK_RUNNING
> 			 * store from ttwu() competes with the
> 			 * TASK_PARKED store from kthread_parkme().
> 			 *
> 			 * If the TASK_PARKED store looses that
> 			 * competition, kthread_unpark() will go wobbly.
> > +			 */
> > +			raw_spin_lock(&current->pi_lock);
> >  			__set_current_state(TASK_RUNNING);
> > +			raw_spin_unlock(&current->pi_lock);
> >  			preempt_enable();
> >  			if (ht->park && td->status == HP_THREAD_ACTIVE) {
> >  				BUG_ON(td->cpu != smp_processor_id());
> 
> Does that work for you?
> 
> But looking at this a bit more; don't we have the exact same problem
> with the TASK_RUNNING store in the !ht->thread_should_run() case?
> Suppose a ttwu() happens concurrently there, it can end up competing
> against the TASK_INTERRUPTIBLE store, no?
> 
> Of course, that race is not fatal, we'll just end up going around the
> loop once again I suppose. Maybe a comment there too?
> 
> 			/*
> 			 * A similar race is possible here, but loosing
> 			 * the TASK_INTERRUPTIBLE store is harmless and
> 			 * will make us go around the loop once more.
> 			 */
> 

And with slightly more sleep I realize this is actually the normal
and expected pattern. The comment with __set_current_state() even
mentions this.

Also, I think the above patch is 'wrong'. It is not the TASK_RUNNING
store that is a problem it is the TASK_PARKED state that is special. And
if you look at do_task_dead() you'll see we do something very similar
for setting TASK_DEAD.

It is a problem specific to blocked states that do not follow the normal
wait pattern:

	for (;;) {
		set_current_state(STATE);
		if (cond)
			break;
		schedule();
	}
	__set_current_state(RUNNING);

The initial store or STATE can _always_ loose against a competing
RUNNING store from a previous wakeup, but the wait-loop and @cond test
will make it harmless.

The special states (DEAD,STOPPED,..) are different though, they
do not have a loop and expect to be honoured.

This had me looking at __kthread_park() and afaict we actually have a
condition, namely KTHREAD_SHOULD_PARK, which would suggest the following
change:

diff --git a/kernel/kthread.c b/kernel/kthread.c
index cd50e99202b0..4b6503c6a029 100644
--- a/kernel/kthread.c
+++ b/kernel/kthread.c
@@ -177,12 +177,13 @@ void *kthread_probe_data(struct task_struct *task)
 
 static void __kthread_parkme(struct kthread *self)
 {
-	__set_current_state(TASK_PARKED);
-	while (test_bit(KTHREAD_SHOULD_PARK, &self->flags)) {
+	for (;;) {
+		__set_task_state(TASK_PARKED);
+		if (!test_bit(KTHREAD_SHOULD_PARK, &self->flags))
+			break;
 		if (!test_and_set_bit(KTHREAD_IS_PARKED, &self->flags))
 			complete(&self->parked);
 		schedule();
-		__set_current_state(TASK_PARKED);
 	}
 	clear_bit(KTHREAD_IS_PARKED, &self->flags);
 	__set_current_state(TASK_RUNNING);


For the others, I think we want to do something like the below. I still
need to look at TASK_TRACED, which I suspect is also special, but ptrace
always hurts my brain.

Opinions?

diff --git a/include/linux/sched.h b/include/linux/sched.h
index b3d697f3b573..f4098435a882 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -110,19 +110,45 @@ struct task_group;
 					 (task->flags & PF_FROZEN) == 0 && \
 					 (task->state & TASK_NOLOAD) == 0)
 
+/*
+ * Special states are those that do not use the normal wait-loop pattern. See
+ * the comment with set_special_state().
+ */
+#define is_special_state(state)		\
+	((state) == TASK_DEAD ||	\
+	 (state) == TASK_STOPPED)
+
 #ifdef CONFIG_DEBUG_ATOMIC_SLEEP
 
+/*
+ * Assert we don't use the regular *set_current_state() helpers for special
+ * states. See the comment with set_special_state().
+ */
+#define assert_special_state(state)	WARN_ON_ONCE(is_special_state(state))
+
 #define __set_current_state(state_value)			\
 	do {							\
+		assert_special_state(state_value);		\
 		current->task_state_change = _THIS_IP_;		\
 		current->state = (state_value);			\
 	} while (0)
+
 #define set_current_state(state_value)				\
 	do {							\
+		assert_special_state(state_value);		\
 		current->task_state_change = _THIS_IP_;		\
 		smp_store_mb(current->state, (state_value));	\
 	} while (0)
 
+#define set_special_state(state_value)					\
+	do {								\
+		unsigned long flags; /* may shadow */			\
+		WARN_ON_ONCE(!is_special_state(state_value));		\
+		raw_spin_lock_irqsave(&current->pi_lock, flags);	\
+		current->task_state_change = _THIS_IP_;			\
+		current->state = (state_value);				\
+		raw_spin_unlock_irqrestore(&current->pi_lock, flags);	\
+	} while (0)
 #else
 /*
  * set_current_state() includes a barrier so that the write of current->state
@@ -154,12 +180,30 @@ struct task_group;
  * once it observes the TASK_UNINTERRUPTIBLE store the waking CPU can issue a
  * TASK_RUNNING store which can collide with __set_current_state(TASK_RUNNING).
  *
- * This is obviously fine, since they both store the exact same value.
+ * However, with slightly different timing the wakeup TASK_RUNNING store can
+ * also collide with the TASK_UNINTERRUPTIBLE store. Loosing that store is not
+ * a problem either because that will result in one extra go around the loop
+ * and our @cond test will save the day.
  *
  * Also see the comments of try_to_wake_up().
  */
 #define __set_current_state(state_value) do { current->state = (state_value); } while (0)
 #define set_current_state(state_value)	 smp_store_mb(current->state, (state_value))
+
+/*
+ * set_special_state() should be used for those states when the blocking task
+ * can not use the regular condition based wait-loop. In that case we must
+ * serialize against wakeups such that any possible in-flight TASK_RUNNING stores
+ * will not collide with out state change.
+ */
+#define set_special_state(state_value)					\
+	do {								\
+		unsigned long flags; /* may shadow */			\
+		raw_spin_lock_irqsave(&current->pi_lock, flags);	\
+		current->state = (state_value);				\
+		raw_spin_unlock_irqrestore(&current->pi_lock, flags);	\
+	} while (0)
+
 #endif
 
 /* Task command name length: */
diff --git a/include/linux/sched/signal.h b/include/linux/sched/signal.h
index a7ce74c74e49..113d1ad1ced7 100644
--- a/include/linux/sched/signal.h
+++ b/include/linux/sched/signal.h
@@ -280,7 +280,7 @@ static inline void kernel_signal_stop(void)
 {
 	spin_lock_irq(&current->sighand->siglock);
 	if (current->jobctl & JOBCTL_STOP_DEQUEUED)
-		__set_current_state(TASK_STOPPED);
+		set_special_state(TASK_STOPPED);
 	spin_unlock_irq(&current->sighand->siglock);
 
 	schedule();
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 5e10aaeebfcc..3898a8047c11 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -3498,23 +3498,8 @@ static void __sched notrace __schedule(bool preempt)
 
 void __noreturn do_task_dead(void)
 {
-	/*
-	 * The setting of TASK_RUNNING by try_to_wake_up() may be delayed
-	 * when the following two conditions become true.
-	 *   - There is race condition of mmap_sem (It is acquired by
-	 *     exit_mm()), and
-	 *   - SMI occurs before setting TASK_RUNINNG.
-	 *     (or hypervisor of virtual machine switches to other guest)
-	 *  As a result, we may become TASK_RUNNING after becoming TASK_DEAD
-	 *
-	 * To avoid it, we have to wait for releasing tsk->pi_lock which
-	 * is held by try_to_wake_up()
-	 */
-	raw_spin_lock_irq(&current->pi_lock);
-	raw_spin_unlock_irq(&current->pi_lock);
-
 	/* Causes final put_task_struct in finish_task_switch(): */
-	__set_current_state(TASK_DEAD);
+	set_special_state(TASK_DEAD);
 
 	/* Tell freezer to ignore us: */
 	current->flags |= PF_NOFREEZE;
diff --git a/kernel/signal.c b/kernel/signal.c
index d4ccea599692..c9cac52b1369 100644
--- a/kernel/signal.c
+++ b/kernel/signal.c
@@ -2176,7 +2176,7 @@ static bool do_signal_stop(int signr)
 		if (task_participate_group_stop(current))
 			notify = CLD_STOPPED;
 
-		__set_current_state(TASK_STOPPED);
+		set_special_state(TASK_STOPPED);
 		spin_unlock_irq(&current->sighand->siglock);
 
 		/*

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v1] kthread/smpboot: Serialize kthread parking against wakeup
  2018-04-26  8:41   ` Peter Zijlstra
@ 2018-04-26  8:57     ` Peter Zijlstra
  2018-04-26 15:53       ` Kohli, Gaurav
  2018-04-26 16:02     ` Andrea Parri
  2018-04-26 16:18     ` Oleg Nesterov
  2 siblings, 1 reply; 39+ messages in thread
From: Peter Zijlstra @ 2018-04-26  8:57 UTC (permalink / raw)
  To: Gaurav Kohli
  Cc: tglx, mpe, mingo, bigeasy, linux-kernel, linux-arm-msm,
	Neeraj Upadhyay, Will Deacon, Oleg Nesterov

On Thu, Apr 26, 2018 at 10:41:31AM +0200, Peter Zijlstra wrote:
> diff --git a/kernel/kthread.c b/kernel/kthread.c
> index cd50e99202b0..4b6503c6a029 100644
> --- a/kernel/kthread.c
> +++ b/kernel/kthread.c
> @@ -177,12 +177,13 @@ void *kthread_probe_data(struct task_struct *task)
>  
>  static void __kthread_parkme(struct kthread *self)
>  {
> -	__set_current_state(TASK_PARKED);
> -	while (test_bit(KTHREAD_SHOULD_PARK, &self->flags)) {
> +	for (;;) {
> +		__set_task_state(TASK_PARKED);

		set_current_state(TASK_PARKED);

of course..

> +		if (!test_bit(KTHREAD_SHOULD_PARK, &self->flags))
> +			break;
>  		if (!test_and_set_bit(KTHREAD_IS_PARKED, &self->flags))
>  			complete(&self->parked);
>  		schedule();
> -		__set_current_state(TASK_PARKED);
>  	}
>  	clear_bit(KTHREAD_IS_PARKED, &self->flags);
>  	__set_current_state(TASK_RUNNING);
> 

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v1] kthread/smpboot: Serialize kthread parking against wakeup
  2018-04-26  4:04   ` Kohli, Gaurav
@ 2018-04-26  9:14     ` Peter Zijlstra
  0 siblings, 0 replies; 39+ messages in thread
From: Peter Zijlstra @ 2018-04-26  9:14 UTC (permalink / raw)
  To: Kohli, Gaurav
  Cc: tglx, mpe, mingo, bigeasy, linux-kernel, linux-arm-msm,
	Neeraj Upadhyay, Will Deacon

On Thu, Apr 26, 2018 at 09:34:36AM +0530, Kohli, Gaurav wrote:
> On 4/26/2018 1:39 AM, Peter Zijlstra wrote:
> 
> > On Wed, Apr 25, 2018 at 02:03:19PM +0530, Gaurav Kohli wrote:
> > > diff --git a/kernel/smpboot.c b/kernel/smpboot.c
> > > index 5043e74..c5c5184 100644
> > > --- a/kernel/smpboot.c
> > > +++ b/kernel/smpboot.c
> > > @@ -122,7 +122,45 @@ static int smpboot_thread_fn(void *data)
> > >   		}
> > >   		if (kthread_should_park()) {
> > > +			/*
> > > +			 * Serialize against wakeup.
> > 			 *
> > 			 * Prior wakeups must complete and later wakeups
> > 			 * will observe TASK_RUNNING.
> > 			 *
> > 			 * This avoids the case where the TASK_RUNNING
> > 			 * store from ttwu() competes with the
> > 			 * TASK_PARKED store from kthread_parkme().
> > 			 *
> > 			 * If the TASK_PARKED store looses that
> > 			 * competition, kthread_unpark() will go wobbly.
> > > +			 */
> > > +			raw_spin_lock(&current->pi_lock);
> > >   			__set_current_state(TASK_RUNNING);
> > > +			raw_spin_unlock(&current->pi_lock);
> > >   			preempt_enable();
> > >   			if (ht->park && td->status == HP_THREAD_ACTIVE) {
> > >   				BUG_ON(td->cpu != smp_processor_id());
> > Does that work for you?
> 
> We have given patch for testing, usually it takes around 2-3 days for
> reproduction(we will update for the same).

I only changed the comment; surely your compiler doesn't generate
different code for that?

I was asking if the proposed comment was good with you; but see my more
recent email, that actually proposes a different fix.

> > 			/*
> > 			 * A similar race is possible here, but loosing
> > 			 * the TASK_INTERRUPTIBLE store is harmless and
> > 			 * will make us go around the loop once more.
> > 			 */
> 
> Actually instead of race, i am seeing wakeup miss problem which is
> very rare, if we take case of hotplug thread

Yes, triggering these issues is tricky, no doubt about that.

> > And of course, I suspect we actually want to use TASK_IDLE, smpboot
> > threads don't want signals do they? But that probably ought to be a
> > separate patch.
> 
> Yes I agree, we can control race from here as well,  Please suggest
> would below change be any help here:

That is not what I suggested. I said the thing should use TASK_IDLE
instead of TASK_INTERRUPTIBLE. Not change the location of it.

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v1] kthread/smpboot: Serialize kthread parking against wakeup
  2018-04-26  8:57     ` Peter Zijlstra
@ 2018-04-26 15:53       ` Kohli, Gaurav
  2018-04-30 11:17         ` Peter Zijlstra
  0 siblings, 1 reply; 39+ messages in thread
From: Kohli, Gaurav @ 2018-04-26 15:53 UTC (permalink / raw)
  To: Peter Zijlstra
  Cc: tglx, mpe, mingo, bigeasy, linux-kernel, linux-arm-msm,
	Neeraj Upadhyay, Will Deacon, Oleg Nesterov

On 4/26/2018 2:27 PM, Peter Zijlstra wrote:

> On Thu, Apr 26, 2018 at 10:41:31AM +0200, Peter Zijlstra wrote:
>> diff --git a/kernel/kthread.c b/kernel/kthread.c
>> index cd50e99202b0..4b6503c6a029 100644
>> --- a/kernel/kthread.c
>> +++ b/kernel/kthread.c
>> @@ -177,12 +177,13 @@ void *kthread_probe_data(struct task_struct *task)
>>   
>>   static void __kthread_parkme(struct kthread *self)
>>   {
>> -	__set_current_state(TASK_PARKED);
>> -	while (test_bit(KTHREAD_SHOULD_PARK, &self->flags)) {
>> +	for (;;) {
>> +		__set_task_state(TASK_PARKED);
> 		set_current_state(TASK_PARKED);
>
> of course..

Hi Peter,

Maybe i am missing something , but still that race can come as we don't put task_parked on special state.

Controller                                                                       Hotplug

                                                                                  Loop

                                                                                  Task_Interruptible

Set SHOULD_PARK

wakeup -> Proceeds

                                                                                   Set Running

                                                                                   kthread_parkme

                                                                                   SET TASK_PARKED

                                                                                   schedule

Set TASK_RUNNING

Can you please correct ME, if I misunderstood this.

>
>> +		if (!test_bit(KTHREAD_SHOULD_PARK, &self->flags))
>> +			break;
>>   		if (!test_and_set_bit(KTHREAD_IS_PARKED, &self->flags))
>>   			complete(&self->parked);
>>   		schedule();
>> -		__set_current_state(TASK_PARKED);
>>   	}
>>   	clear_bit(KTHREAD_IS_PARKED, &self->flags);
>>   	__set_current_state(TASK_RUNNING);
>>
>>
-- 
Qualcomm India Private Limited, on behalf of Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project.

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v1] kthread/smpboot: Serialize kthread parking against wakeup
  2018-04-26  8:41   ` Peter Zijlstra
  2018-04-26  8:57     ` Peter Zijlstra
@ 2018-04-26 16:02     ` Andrea Parri
  2018-04-26 16:18     ` Oleg Nesterov
  2 siblings, 0 replies; 39+ messages in thread
From: Andrea Parri @ 2018-04-26 16:02 UTC (permalink / raw)
  To: Peter Zijlstra
  Cc: Gaurav Kohli, tglx, mpe, mingo, bigeasy, linux-kernel,
	linux-arm-msm, Neeraj Upadhyay, Will Deacon, Oleg Nesterov

On Thu, Apr 26, 2018 at 10:41:31AM +0200, Peter Zijlstra wrote:

[...]

> +/*
> + * Special states are those that do not use the normal wait-loop pattern. See
> + * the comment with set_special_state().
> + */
> +#define is_special_state(state)		\
> +	((state) == TASK_DEAD ||	\
> +	 (state) == TASK_STOPPED)
> +
>  #ifdef CONFIG_DEBUG_ATOMIC_SLEEP
>  
> +/*
> + * Assert we don't use the regular *set_current_state() helpers for special
> + * states. See the comment with set_special_state().
> + */
> +#define assert_special_state(state)	WARN_ON_ONCE(is_special_state(state))

Nitpicking, this name suggests "Shout if the state is NOT special" to me:
maybe,

#define assert_special_state(state)	WARN_ON_ONCE(!is_special_state(state))
#define assert_regular_state(state)	WARN_ON_ONCE(is_special_state(state))

or just do with the WARN_ON_ONCE()s ?

  Andrea


> +
>  #define __set_current_state(state_value)			\
>  	do {							\
> +		assert_special_state(state_value);		\
>  		current->task_state_change = _THIS_IP_;		\
>  		current->state = (state_value);			\
>  	} while (0)
> +
>  #define set_current_state(state_value)				\
>  	do {							\
> +		assert_special_state(state_value);		\
>  		current->task_state_change = _THIS_IP_;		\
>  		smp_store_mb(current->state, (state_value));	\
>  	} while (0)
>  
> +#define set_special_state(state_value)					\
> +	do {								\
> +		unsigned long flags; /* may shadow */			\
> +		WARN_ON_ONCE(!is_special_state(state_value));		\
> +		raw_spin_lock_irqsave(&current->pi_lock, flags);	\
> +		current->task_state_change = _THIS_IP_;			\
> +		current->state = (state_value);				\
> +		raw_spin_unlock_irqrestore(&current->pi_lock, flags);	\
> +	} while (0)

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v1] kthread/smpboot: Serialize kthread parking against wakeup
  2018-04-26  8:41   ` Peter Zijlstra
  2018-04-26  8:57     ` Peter Zijlstra
  2018-04-26 16:02     ` Andrea Parri
@ 2018-04-26 16:18     ` Oleg Nesterov
  2018-04-30 11:20       ` Peter Zijlstra
  2 siblings, 1 reply; 39+ messages in thread
From: Oleg Nesterov @ 2018-04-26 16:18 UTC (permalink / raw)
  To: Peter Zijlstra
  Cc: Gaurav Kohli, tglx, mpe, mingo, bigeasy, linux-kernel,
	linux-arm-msm, Neeraj Upadhyay, Will Deacon

On 04/26, Peter Zijlstra wrote:
>
> For the others, I think we want to do something like the below. I still
> need to look at TASK_TRACED, which I suspect is also special,

Yes, and TASK_STOPPED.

ptrace_freeze_traced() and ptrace_unfreeze_traced() should be fine, but
ptrace_stop() wants set_special_state() too, I think.

> but ptrace always hurts my brain.

mine too ;)

> +/*
> + * set_special_state() should be used for those states when the blocking task
> + * can not use the regular condition based wait-loop. In that case we must
> + * serialize against wakeups such that any possible in-flight TASK_RUNNING stores
> + * will not collide with out state change.
> + */
> +#define set_special_state(state_value)					\
> +	do {								\
> +		unsigned long flags; /* may shadow */			\
> +		raw_spin_lock_irqsave(&current->pi_lock, flags);	\
> +		current->state = (state_value);				\
> +		raw_spin_unlock_irqrestore(&current->pi_lock, flags);	\
> +	} while (0)
> +

Agreed.

I thought that perhaps we can change ttwu_do_wakeup() cmpxchg() instead of
plain p->state = TASK_RUNNING, but this helper looks much more clear and simple.

Oleg.

^ permalink raw reply	[flat|nested] 39+ messages in thread

* [lkp-robot] [kthread/smpboot]  cad8e99675: inconsistent{IN-HARDIRQ-W}->{HARDIRQ-ON-W}usage
  2018-04-25  8:33 [PATCH v1] kthread/smpboot: Serialize kthread parking against wakeup Gaurav Kohli
  2018-04-25 20:09 ` Peter Zijlstra
@ 2018-04-28  6:43 ` kernel test robot
  1 sibling, 0 replies; 39+ messages in thread
From: kernel test robot @ 2018-04-28  6:43 UTC (permalink / raw)
  To: Gaurav Kohli
  Cc: peterz, tglx, mpe, mingo, bigeasy, linux-kernel, linux-arm-msm,
	Gaurav Kohli, Neeraj Upadhyay, lkp

[-- Attachment #1: Type: text/plain, Size: 7927 bytes --]


FYI, we noticed the following commit (built with gcc-6):

commit: cad8e9967526d263d9a4f04ca763b2d76c045750 ("kthread/smpboot: Serialize kthread parking against wakeup")
url: https://github.com/0day-ci/linux/commits/Gaurav-Kohli/kthread-smpboot-Serialize-kthread-parking-against-wakeup/20180426-185404


in testcase: boot

on test machine: qemu-system-x86_64 -enable-kvm -smp 2 -m 512M

caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):


+------------------------------------------------------------------+-----------+------------+
|                                                                  | v4.17-rc2 | cad8e99675 |
+------------------------------------------------------------------+-----------+------------+
| boot_successes                                                   | 60        | 0          |
| boot_failures                                                    | 34        | 10         |
| invoked_oom-killer:gfp_mask=0x                                   | 20        | 4          |
| Mem-Info                                                         | 20        | 4          |
| Kernel_panic-not_syncing:Out_of_memory_and_no_killable_processes | 11        |            |
| Out_of_memory:Kill_process                                       | 9         | 4          |
| RIP:__clear_user                                                 | 1         | 1          |
| WARNING:inconsistent_lock_state                                  | 6         | 10         |
| inconsistent{HARDIRQ-ON-W}->{IN-HARDIRQ-W}usage                  | 6         |            |
| calltrace:neigh_periodic_work                                    | 2         |            |
| RIP:e1000_watchdog                                               | 4         |            |
| BUG:sleeping_function_called_from_invalid_context_at_mm/slab.h   | 6         |            |
| calltrace:irq_exit                                               | 2         |            |
| IP-Config:Auto-configuration_of_network_failed                   | 8         | 4          |
| RIP:native_safe_halt                                             | 2         |            |
| inconsistent{IN-HARDIRQ-W}->{HARDIRQ-ON-W}usage                  | 0         | 10         |
+------------------------------------------------------------------+-----------+------------+



[   21.664248] WARNING: inconsistent lock state
[   21.664989] 4.17.0-rc2-00001-gcad8e99 #1 Not tainted
[   21.665824] --------------------------------
[   21.666510] inconsistent {IN-HARDIRQ-W} -> {HARDIRQ-ON-W} usage.
[   21.667591] cpuhp/0/14 [HC0[0]:SC0[0]:HE1:SE1] takes:
[   21.668441]         (ptrval) (&p->pi_lock){?.-.}, at: smpboot_thread_fn+0x183/0x42c
[   21.669715] {IN-HARDIRQ-W} state was registered at:
[   21.670739]   lock_acquire+0x99/0xc9
[   21.671341]   _raw_spin_lock_irqsave+0x45/0x59
[   21.672062]   try_to_wake_up+0x8f/0x528
[   21.672733]   wake_up_process+0x10/0x12
[   21.673434]   hrtimer_wakeup+0x27/0x2c
[   21.674068]   __hrtimer_run_queues+0x163/0x1d4
[   21.674785]   hrtimer_run_queues+0x175/0x1b4
[   21.675518]   run_local_timers+0x30/0x7f
[   21.676206]   update_process_times+0x22/0x4f
[   21.686136]   tick_periodic+0xc4/0xca
[   21.689746]   tick_handle_periodic+0x29/0x7b
[   21.690518]   smp_apic_timer_interrupt+0x9d/0xb1
[   21.691318]   apic_timer_interrupt+0xf/0x20
[   21.692022]   native_safe_halt+0x6/0x8
[   21.692635]   default_idle+0x13/0x1c
[   21.693253]   arch_cpu_idle+0xa/0xc
[   21.697213]   default_idle_call+0x35/0x38
[   21.697891]   do_idle+0x181/0x2bf
[   21.698467]   cpu_startup_entry+0x1a/0x1c
[   21.699156]   rest_init+0x22c/0x232
[   21.699752]   start_kernel+0x4fc/0x520
[   21.700309]   x86_64_start_reservations+0x24/0x26
[   21.701909]   x86_64_start_kernel+0x6f/0x72
[   21.702553]   secondary_startup_64+0xa5/0xb0
[   21.703294] irq event stamp: 1401
[   21.703834] hardirqs last  enabled at (1401): [<ffffffffbd3ef4f3>] _raw_spin_unlock_irq+0x27/0x4b
[   21.705223] hardirqs last disabled at (1400): [<ffffffffbd3e69e3>] __schedule+0x23b/0xccc
[   21.706490] softirqs last  enabled at (1206): [<ffffffffbd60028d>] __do_softirq+0x28d/0x2db
[   21.707466] softirqs last disabled at (1181): [<ffffffffbcab67cf>] irq_exit+0x67/0x100
[   21.708244] 
[   21.708244] other info that might help us debug this:
[   21.709009]  Possible unsafe locking scenario:
[   21.709009] 
[   21.709919]        CPU0
[   21.710300]        ----
[   21.710696]   lock(&p->pi_lock);
[   21.711199]   <Interrupt>
[   21.711614]     lock(&p->pi_lock);
[   21.712146] 
[   21.712146]  *** DEADLOCK ***
[   21.712146] 
[   21.713108] no locks held by cpuhp/0/14.
[   21.713773] 
[   21.713773] stack backtrace:
[   21.714502] CPU: 0 PID: 14 Comm: cpuhp/0 Not tainted 4.17.0-rc2-00001-gcad8e99 #1
[   21.715747] Call Trace:
[   21.716174]  dump_stack+0x9c/0xe9
[   21.716756]  print_usage_bug+0x321/0x330
[   21.717533]  mark_lock+0x4f5/0x779
[   21.718122]  ? finish_task_switch+0x274/0x325
[   21.718653]  ? print_shortest_lock_dependencies+0x1c6/0x1c6
[   21.719230]  __lock_acquire+0x4a0/0x888
[   21.719606]  lock_acquire+0x99/0xc9
[   21.719992]  ? smpboot_thread_fn+0x183/0x42c
[   21.720457]  _raw_spin_lock+0x2d/0x3c
[   21.720885]  ? smpboot_thread_fn+0x183/0x42c
[   21.721285]  smpboot_thread_fn+0x183/0x42c
[   21.721703]  ? sort_range+0x1d/0x1d
[   21.722105]  ? preempt_count_sub+0x13/0xb5
[   21.722542]  ? schedule+0xdd/0xe9
[   21.722934]  kthread+0x1db/0x1eb
[   21.723245]  ? sort_range+0x1d/0x1d
[   21.723620]  ? kthread_create_on_node+0xa1/0xa1
[   21.724137]  ret_from_fork+0x24/0x30
[   21.730948] numa_remove_cpu cpu 0 node 0: mask now 1
[   21.733638] CPU 0 is now offline
[   21.735209] debug: unmapping init [mem 0xffffffffbe187000-0xffffffffbe2affff]
[   21.736225] Write protecting the kernel read-only data: 20480k
[   21.739451] debug: unmapping init [mem 0xffff88000ec07000-0xffff88000edfffff]
[   21.741223] debug: unmapping init [mem 0xffff88000f2dc000-0xffff88000f3fffff]
[   21.798549] x86/mm: Checked W+X mappings: passed, no W+X pages found.
[   21.815648] random: init: uninitialized urandom read (12 bytes read)
[   21.876844] hwclock (101) used greatest stack depth: 27928 bytes left
[   21.886110] plymouthd (99) used greatest stack depth: 27656 bytes left
[   21.906815] sh (104) used greatest stack depth: 27640 bytes left
[   21.953005] mountall (108) used greatest stack depth: 26776 bytes left
[   21.976917] random: trinity: uninitialized urandom read (4 bytes read)
mountall: Event failed
[   22.102893] random: mountall: uninitialized urandom read (12 bytes read)
[   22.154174] init: Failed to create pty - disabling logging for job
[   22.155285] init: Temporary process spawn error: No such file or directory
[   22.191738] init: Failed to create pty - disabling logging for job
[   22.193058] init: Temporary process spawn error: No such file or directory
[   22.301682] init: Failed to create pty - disabling logging for job
[   22.302812] init: Temporary process spawn error: No such file or directory
[   22.307684] init: Failed to create pty - disabling logging for job
[   22.308793] init: Temporary process spawn error: No such file or directory
[   22.427189] init: Failed to create pty - disabling logging for job
[   22.428994] init: Temporary process spawn error: No such file or directory
[   22.439238] init: Failed to create pty - disabling logging for job
[   22.440420] init: Temporary process spawn error: No such file or directory
[   22.451555] init: Failed to create pty - disabling logging for job
[   22.452662] init: Temporary process spawn error: No such file or directory
[   22.477652] init: Failed to create pty - disabling logging for job


To reproduce:

        git clone https://github.com/intel/lkp-tests.git
        cd lkp-tests
        bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email



Thanks,
Xiaolong

[-- Attachment #2: config-4.17.0-rc2-00001-gcad8e99 --]
[-- Type: text/plain, Size: 94148 bytes --]

#
# Automatically generated file; DO NOT EDIT.
# Linux/x86_64 4.17.0-rc2 Kernel Configuration
#
CONFIG_64BIT=y
CONFIG_X86_64=y
CONFIG_X86=y
CONFIG_INSTRUCTION_DECODER=y
CONFIG_OUTPUT_FORMAT="elf64-x86-64"
CONFIG_ARCH_DEFCONFIG="arch/x86/configs/x86_64_defconfig"
CONFIG_LOCKDEP_SUPPORT=y
CONFIG_STACKTRACE_SUPPORT=y
CONFIG_MMU=y
CONFIG_ARCH_MMAP_RND_BITS_MIN=28
CONFIG_ARCH_MMAP_RND_BITS_MAX=32
CONFIG_ARCH_MMAP_RND_COMPAT_BITS_MIN=8
CONFIG_ARCH_MMAP_RND_COMPAT_BITS_MAX=16
CONFIG_NEED_DMA_MAP_STATE=y
CONFIG_NEED_SG_DMA_LENGTH=y
CONFIG_GENERIC_ISA_DMA=y
CONFIG_GENERIC_BUG=y
CONFIG_GENERIC_BUG_RELATIVE_POINTERS=y
CONFIG_GENERIC_HWEIGHT=y
CONFIG_ARCH_MAY_HAVE_PC_FDC=y
CONFIG_RWSEM_XCHGADD_ALGORITHM=y
CONFIG_GENERIC_CALIBRATE_DELAY=y
CONFIG_ARCH_HAS_CPU_RELAX=y
CONFIG_ARCH_HAS_CACHE_LINE_SIZE=y
CONFIG_HAVE_SETUP_PER_CPU_AREA=y
CONFIG_NEED_PER_CPU_EMBED_FIRST_CHUNK=y
CONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK=y
CONFIG_ARCH_HIBERNATION_POSSIBLE=y
CONFIG_ARCH_SUSPEND_POSSIBLE=y
CONFIG_ARCH_WANT_HUGE_PMD_SHARE=y
CONFIG_ARCH_WANT_GENERAL_HUGETLB=y
CONFIG_ZONE_DMA32=y
CONFIG_AUDIT_ARCH=y
CONFIG_ARCH_SUPPORTS_OPTIMIZED_INLINING=y
CONFIG_ARCH_SUPPORTS_DEBUG_PAGEALLOC=y
CONFIG_KASAN_SHADOW_OFFSET=0xdffffc0000000000
CONFIG_X86_64_SMP=y
CONFIG_ARCH_SUPPORTS_UPROBES=y
CONFIG_FIX_EARLYCON_MEM=y
CONFIG_PGTABLE_LEVELS=4
CONFIG_CONSTRUCTORS=y
CONFIG_IRQ_WORK=y
CONFIG_BUILDTIME_EXTABLE_SORT=y
CONFIG_THREAD_INFO_IN_TASK=y

#
# General setup
#
CONFIG_INIT_ENV_ARG_LIMIT=32
CONFIG_CROSS_COMPILE=""
# CONFIG_COMPILE_TEST is not set
CONFIG_LOCALVERSION=""
CONFIG_LOCALVERSION_AUTO=y
CONFIG_HAVE_KERNEL_GZIP=y
CONFIG_HAVE_KERNEL_BZIP2=y
CONFIG_HAVE_KERNEL_LZMA=y
CONFIG_HAVE_KERNEL_XZ=y
CONFIG_HAVE_KERNEL_LZO=y
CONFIG_HAVE_KERNEL_LZ4=y
# CONFIG_KERNEL_GZIP is not set
# CONFIG_KERNEL_BZIP2 is not set
CONFIG_KERNEL_LZMA=y
# CONFIG_KERNEL_XZ is not set
# CONFIG_KERNEL_LZO is not set
# CONFIG_KERNEL_LZ4 is not set
CONFIG_DEFAULT_HOSTNAME="(none)"
# CONFIG_SWAP is not set
# CONFIG_SYSVIPC is not set
# CONFIG_POSIX_MQUEUE is not set
# CONFIG_CROSS_MEMORY_ATTACH is not set
# CONFIG_USELIB is not set
# CONFIG_AUDIT is not set
CONFIG_HAVE_ARCH_AUDITSYSCALL=y

#
# IRQ subsystem
#
CONFIG_GENERIC_IRQ_PROBE=y
CONFIG_GENERIC_IRQ_SHOW=y
CONFIG_GENERIC_IRQ_EFFECTIVE_AFF_MASK=y
CONFIG_GENERIC_PENDING_IRQ=y
CONFIG_GENERIC_IRQ_MIGRATION=y
CONFIG_IRQ_DOMAIN=y
CONFIG_IRQ_DOMAIN_HIERARCHY=y
CONFIG_GENERIC_IRQ_MATRIX_ALLOCATOR=y
CONFIG_GENERIC_IRQ_RESERVATION_MODE=y
CONFIG_IRQ_FORCED_THREADING=y
CONFIG_SPARSE_IRQ=y
# CONFIG_GENERIC_IRQ_DEBUGFS is not set
CONFIG_CLOCKSOURCE_WATCHDOG=y
CONFIG_ARCH_CLOCKSOURCE_DATA=y
CONFIG_CLOCKSOURCE_VALIDATE_LAST_CYCLE=y
CONFIG_GENERIC_TIME_VSYSCALL=y
CONFIG_GENERIC_CLOCKEVENTS=y
CONFIG_GENERIC_CLOCKEVENTS_BROADCAST=y
CONFIG_GENERIC_CLOCKEVENTS_MIN_ADJUST=y
CONFIG_GENERIC_CMOS_UPDATE=y

#
# Timers subsystem
#
CONFIG_TICK_ONESHOT=y
CONFIG_NO_HZ_COMMON=y
# CONFIG_HZ_PERIODIC is not set
# CONFIG_NO_HZ_IDLE is not set
CONFIG_NO_HZ_FULL=y
# CONFIG_NO_HZ is not set
CONFIG_HIGH_RES_TIMERS=y

#
# CPU/Task time and stats accounting
#
CONFIG_VIRT_CPU_ACCOUNTING=y
CONFIG_VIRT_CPU_ACCOUNTING_GEN=y
CONFIG_IRQ_TIME_ACCOUNTING=y
# CONFIG_BSD_PROCESS_ACCT is not set
# CONFIG_TASKSTATS is not set
CONFIG_CPU_ISOLATION=y

#
# RCU Subsystem
#
CONFIG_PREEMPT_RCU=y
CONFIG_RCU_EXPERT=y
CONFIG_SRCU=y
CONFIG_TREE_SRCU=y
CONFIG_TASKS_RCU=y
CONFIG_RCU_STALL_COMMON=y
CONFIG_RCU_NEED_SEGCBLIST=y
CONFIG_CONTEXT_TRACKING=y
# CONFIG_CONTEXT_TRACKING_FORCE is not set
CONFIG_RCU_FANOUT=64
CONFIG_RCU_FANOUT_LEAF=16
# CONFIG_RCU_FAST_NO_HZ is not set
# CONFIG_RCU_BOOST is not set
CONFIG_RCU_NOCB_CPU=y
CONFIG_BUILD_BIN2C=y
CONFIG_IKCONFIG=y
CONFIG_IKCONFIG_PROC=y
CONFIG_LOG_BUF_SHIFT=20
CONFIG_LOG_CPU_MAX_BUF_SHIFT=12
CONFIG_PRINTK_SAFE_LOG_BUF_SHIFT=13
CONFIG_HAVE_UNSTABLE_SCHED_CLOCK=y
CONFIG_ARCH_SUPPORTS_NUMA_BALANCING=y
CONFIG_ARCH_WANT_BATCHED_UNMAP_TLB_FLUSH=y
CONFIG_ARCH_SUPPORTS_INT128=y
CONFIG_NUMA_BALANCING=y
# CONFIG_NUMA_BALANCING_DEFAULT_ENABLED is not set
CONFIG_CGROUPS=y
# CONFIG_MEMCG is not set
# CONFIG_BLK_CGROUP is not set
# CONFIG_CGROUP_SCHED is not set
# CONFIG_CGROUP_PIDS is not set
# CONFIG_CGROUP_RDMA is not set
# CONFIG_CGROUP_FREEZER is not set
# CONFIG_CGROUP_HUGETLB is not set
# CONFIG_CPUSETS is not set
# CONFIG_CGROUP_DEVICE is not set
# CONFIG_CGROUP_CPUACCT is not set
# CONFIG_CGROUP_PERF is not set
# CONFIG_CGROUP_DEBUG is not set
# CONFIG_NAMESPACES is not set
# CONFIG_SCHED_AUTOGROUP is not set
# CONFIG_SYSFS_DEPRECATED is not set
CONFIG_RELAY=y
CONFIG_BLK_DEV_INITRD=y
CONFIG_INITRAMFS_SOURCE=""
CONFIG_RD_GZIP=y
CONFIG_RD_BZIP2=y
CONFIG_RD_LZMA=y
# CONFIG_RD_XZ is not set
# CONFIG_RD_LZO is not set
# CONFIG_RD_LZ4 is not set
# CONFIG_CC_OPTIMIZE_FOR_PERFORMANCE is not set
CONFIG_CC_OPTIMIZE_FOR_SIZE=y
CONFIG_SYSCTL=y
CONFIG_ANON_INODES=y
CONFIG_SYSCTL_EXCEPTION_TRACE=y
CONFIG_HAVE_PCSPKR_PLATFORM=y
CONFIG_BPF=y
CONFIG_EXPERT=y
CONFIG_MULTIUSER=y
CONFIG_SGETMASK_SYSCALL=y
# CONFIG_SYSFS_SYSCALL is not set
# CONFIG_SYSCTL_SYSCALL is not set
CONFIG_FHANDLE=y
CONFIG_POSIX_TIMERS=y
CONFIG_PRINTK=y
CONFIG_PRINTK_NMI=y
CONFIG_BUG=y
CONFIG_PCSPKR_PLATFORM=y
# CONFIG_BASE_FULL is not set
CONFIG_FUTEX=y
CONFIG_FUTEX_PI=y
CONFIG_EPOLL=y
CONFIG_SIGNALFD=y
CONFIG_TIMERFD=y
CONFIG_EVENTFD=y
CONFIG_SHMEM=y
# CONFIG_AIO is not set
# CONFIG_ADVISE_SYSCALLS is not set
# CONFIG_MEMBARRIER is not set
CONFIG_CHECKPOINT_RESTORE=y
CONFIG_KALLSYMS=y
CONFIG_KALLSYMS_ALL=y
CONFIG_KALLSYMS_ABSOLUTE_PERCPU=y
CONFIG_KALLSYMS_BASE_RELATIVE=y
# CONFIG_BPF_SYSCALL is not set
CONFIG_USERFAULTFD=y
CONFIG_ARCH_HAS_MEMBARRIER_SYNC_CORE=y
CONFIG_EMBEDDED=y
CONFIG_HAVE_PERF_EVENTS=y
CONFIG_PERF_USE_VMALLOC=y
# CONFIG_PC104 is not set

#
# Kernel Performance Events And Counters
#
CONFIG_PERF_EVENTS=y
CONFIG_DEBUG_PERF_USE_VMALLOC=y
# CONFIG_VM_EVENT_COUNTERS is not set
# CONFIG_SLUB_DEBUG is not set
# CONFIG_COMPAT_BRK is not set
# CONFIG_SLAB is not set
CONFIG_SLUB=y
# CONFIG_SLOB is not set
CONFIG_SLAB_MERGE_DEFAULT=y
# CONFIG_SLAB_FREELIST_RANDOM is not set
CONFIG_SLAB_FREELIST_HARDENED=y
# CONFIG_SLUB_CPU_PARTIAL is not set
# CONFIG_PROFILING is not set
CONFIG_CRASH_CORE=y
CONFIG_KEXEC_CORE=y
CONFIG_HAVE_OPROFILE=y
CONFIG_OPROFILE_NMI_TIMER=y
CONFIG_KPROBES=y
CONFIG_JUMP_LABEL=y
# CONFIG_STATIC_KEYS_SELFTEST is not set
CONFIG_OPTPROBES=y
CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS=y
CONFIG_ARCH_USE_BUILTIN_BSWAP=y
CONFIG_KRETPROBES=y
CONFIG_HAVE_IOREMAP_PROT=y
CONFIG_HAVE_KPROBES=y
CONFIG_HAVE_KRETPROBES=y
CONFIG_HAVE_OPTPROBES=y
CONFIG_HAVE_KPROBES_ON_FTRACE=y
CONFIG_HAVE_FUNCTION_ERROR_INJECTION=y
CONFIG_HAVE_NMI=y
CONFIG_HAVE_ARCH_TRACEHOOK=y
CONFIG_HAVE_DMA_CONTIGUOUS=y
CONFIG_GENERIC_SMP_IDLE_THREAD=y
CONFIG_ARCH_HAS_FORTIFY_SOURCE=y
CONFIG_ARCH_HAS_SET_MEMORY=y
CONFIG_HAVE_ARCH_THREAD_STRUCT_WHITELIST=y
CONFIG_ARCH_WANTS_DYNAMIC_TASK_STRUCT=y
CONFIG_HAVE_REGS_AND_STACK_ACCESS_API=y
CONFIG_HAVE_CLK=y
CONFIG_HAVE_DMA_API_DEBUG=y
CONFIG_HAVE_HW_BREAKPOINT=y
CONFIG_HAVE_MIXED_BREAKPOINTS_REGS=y
CONFIG_HAVE_USER_RETURN_NOTIFIER=y
CONFIG_HAVE_PERF_EVENTS_NMI=y
CONFIG_HAVE_HARDLOCKUP_DETECTOR_PERF=y
CONFIG_HAVE_PERF_REGS=y
CONFIG_HAVE_PERF_USER_STACK_DUMP=y
CONFIG_HAVE_ARCH_JUMP_LABEL=y
CONFIG_HAVE_RCU_TABLE_FREE=y
CONFIG_ARCH_HAVE_NMI_SAFE_CMPXCHG=y
CONFIG_HAVE_ALIGNED_STRUCT_PAGE=y
CONFIG_HAVE_CMPXCHG_LOCAL=y
CONFIG_HAVE_CMPXCHG_DOUBLE=y
CONFIG_HAVE_ARCH_SECCOMP_FILTER=y
CONFIG_HAVE_GCC_PLUGINS=y
CONFIG_GCC_PLUGINS=y
# CONFIG_GCC_PLUGIN_CYC_COMPLEXITY is not set
CONFIG_GCC_PLUGIN_SANCOV=y
# CONFIG_GCC_PLUGIN_LATENT_ENTROPY is not set
# CONFIG_GCC_PLUGIN_STRUCTLEAK is not set
# CONFIG_GCC_PLUGIN_RANDSTRUCT is not set
CONFIG_HAVE_CC_STACKPROTECTOR=y
CONFIG_CC_STACKPROTECTOR_NONE=y
# CONFIG_CC_STACKPROTECTOR_REGULAR is not set
# CONFIG_CC_STACKPROTECTOR_STRONG is not set
# CONFIG_CC_STACKPROTECTOR_AUTO is not set
CONFIG_HAVE_ARCH_WITHIN_STACK_FRAMES=y
CONFIG_HAVE_CONTEXT_TRACKING=y
CONFIG_HAVE_VIRT_CPU_ACCOUNTING_GEN=y
CONFIG_HAVE_IRQ_TIME_ACCOUNTING=y
CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE=y
CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD=y
CONFIG_HAVE_ARCH_HUGE_VMAP=y
CONFIG_HAVE_ARCH_SOFT_DIRTY=y
CONFIG_HAVE_MOD_ARCH_SPECIFIC=y
CONFIG_MODULES_USE_ELF_RELA=y
CONFIG_HAVE_IRQ_EXIT_ON_IRQ_STACK=y
CONFIG_ARCH_HAS_ELF_RANDOMIZE=y
CONFIG_HAVE_ARCH_MMAP_RND_BITS=y
CONFIG_HAVE_EXIT_THREAD=y
CONFIG_ARCH_MMAP_RND_BITS=28
CONFIG_HAVE_ARCH_MMAP_RND_COMPAT_BITS=y
CONFIG_ARCH_MMAP_RND_COMPAT_BITS=8
CONFIG_HAVE_ARCH_COMPAT_MMAP_BASES=y
CONFIG_HAVE_COPY_THREAD_TLS=y
CONFIG_HAVE_STACK_VALIDATION=y
CONFIG_HAVE_ARCH_VMAP_STACK=y
CONFIG_ARCH_HAS_STRICT_KERNEL_RWX=y
CONFIG_STRICT_KERNEL_RWX=y
CONFIG_ARCH_HAS_STRICT_MODULE_RWX=y
CONFIG_STRICT_MODULE_RWX=y
CONFIG_ARCH_HAS_REFCOUNT=y
# CONFIG_REFCOUNT_FULL is not set

#
# GCOV-based kernel profiling
#
# CONFIG_GCOV_KERNEL is not set
CONFIG_ARCH_HAS_GCOV_PROFILE_ALL=y
CONFIG_RT_MUTEXES=y
CONFIG_BASE_SMALL=1
CONFIG_MODULES=y
# CONFIG_MODULE_FORCE_LOAD is not set
CONFIG_MODULE_UNLOAD=y
# CONFIG_MODULE_FORCE_UNLOAD is not set
# CONFIG_MODVERSIONS is not set
# CONFIG_MODULE_SRCVERSION_ALL is not set
# CONFIG_MODULE_SIG is not set
# CONFIG_MODULE_COMPRESS is not set
CONFIG_MODULES_TREE_LOOKUP=y
CONFIG_BLOCK=y
CONFIG_BLK_SCSI_REQUEST=y
CONFIG_BLK_DEV_BSG=y
CONFIG_BLK_DEV_BSGLIB=y
CONFIG_BLK_DEV_INTEGRITY=y
# CONFIG_BLK_DEV_ZONED is not set
# CONFIG_BLK_CMDLINE_PARSER is not set
# CONFIG_BLK_WBT is not set
CONFIG_BLK_DEBUG_FS=y
CONFIG_BLK_SED_OPAL=y

#
# Partition Types
#
# CONFIG_PARTITION_ADVANCED is not set
CONFIG_AMIGA_PARTITION=y
CONFIG_MSDOS_PARTITION=y
CONFIG_EFI_PARTITION=y
CONFIG_BLOCK_COMPAT=y
CONFIG_BLK_MQ_PCI=y
CONFIG_BLK_MQ_VIRTIO=y

#
# IO Schedulers
#
CONFIG_IOSCHED_NOOP=y
CONFIG_IOSCHED_DEADLINE=m
# CONFIG_IOSCHED_CFQ is not set
CONFIG_DEFAULT_NOOP=y
CONFIG_DEFAULT_IOSCHED="noop"
CONFIG_MQ_IOSCHED_DEADLINE=y
CONFIG_MQ_IOSCHED_KYBER=y
# CONFIG_IOSCHED_BFQ is not set
CONFIG_PADATA=y
CONFIG_ASN1=y
CONFIG_UNINLINE_SPIN_UNLOCK=y
CONFIG_ARCH_SUPPORTS_ATOMIC_RMW=y
CONFIG_MUTEX_SPIN_ON_OWNER=y
CONFIG_RWSEM_SPIN_ON_OWNER=y
CONFIG_LOCK_SPIN_ON_OWNER=y
CONFIG_ARCH_USE_QUEUED_SPINLOCKS=y
CONFIG_QUEUED_SPINLOCKS=y
CONFIG_ARCH_USE_QUEUED_RWLOCKS=y
CONFIG_QUEUED_RWLOCKS=y
CONFIG_ARCH_HAS_SYNC_CORE_BEFORE_USERMODE=y
CONFIG_ARCH_HAS_SYSCALL_WRAPPER=y
CONFIG_FREEZER=y

#
# Processor type and features
#
CONFIG_ZONE_DMA=y
CONFIG_SMP=y
CONFIG_X86_FEATURE_NAMES=y
# CONFIG_X86_X2APIC is not set
CONFIG_X86_MPPARSE=y
# CONFIG_GOLDFISH is not set
# CONFIG_RETPOLINE is not set
CONFIG_INTEL_RDT=y
# CONFIG_X86_EXTENDED_PLATFORM is not set
# CONFIG_X86_INTEL_LPSS is not set
# CONFIG_X86_AMD_PLATFORM_DEVICE is not set
# CONFIG_IOSF_MBI is not set
CONFIG_X86_SUPPORTS_MEMORY_FAILURE=y
# CONFIG_SCHED_OMIT_FRAME_POINTER is not set
CONFIG_HYPERVISOR_GUEST=y
CONFIG_PARAVIRT=y
# CONFIG_PARAVIRT_DEBUG is not set
# CONFIG_PARAVIRT_SPINLOCKS is not set
# CONFIG_XEN is not set
CONFIG_KVM_GUEST=y
# CONFIG_KVM_DEBUG_FS is not set
# CONFIG_PARAVIRT_TIME_ACCOUNTING is not set
CONFIG_PARAVIRT_CLOCK=y
# CONFIG_JAILHOUSE_GUEST is not set
CONFIG_NO_BOOTMEM=y
# CONFIG_MK8 is not set
# CONFIG_MPSC is not set
# CONFIG_MCORE2 is not set
# CONFIG_MATOM is not set
CONFIG_GENERIC_CPU=y
CONFIG_X86_INTERNODE_CACHE_SHIFT=6
CONFIG_X86_L1_CACHE_SHIFT=6
CONFIG_X86_TSC=y
CONFIG_X86_CMPXCHG64=y
CONFIG_X86_CMOV=y
CONFIG_X86_MINIMUM_CPU_FAMILY=64
CONFIG_X86_DEBUGCTLMSR=y
CONFIG_PROCESSOR_SELECT=y
CONFIG_CPU_SUP_INTEL=y
# CONFIG_CPU_SUP_AMD is not set
# CONFIG_CPU_SUP_CENTAUR is not set
CONFIG_HPET_TIMER=y
# CONFIG_DMI is not set
# CONFIG_CALGARY_IOMMU is not set
CONFIG_SWIOTLB=y
CONFIG_IOMMU_HELPER=y
# CONFIG_MAXSMP is not set
CONFIG_NR_CPUS_RANGE_BEGIN=2
CONFIG_NR_CPUS_RANGE_END=512
CONFIG_NR_CPUS_DEFAULT=64
CONFIG_NR_CPUS=64
# CONFIG_SCHED_SMT is not set
# CONFIG_SCHED_MC is not set
# CONFIG_PREEMPT_NONE is not set
# CONFIG_PREEMPT_VOLUNTARY is not set
CONFIG_PREEMPT=y
CONFIG_PREEMPT_COUNT=y
CONFIG_X86_LOCAL_APIC=y
CONFIG_X86_IO_APIC=y
CONFIG_X86_REROUTE_FOR_BROKEN_BOOT_IRQS=y
CONFIG_X86_MCE=y
# CONFIG_X86_MCELOG_LEGACY is not set
# CONFIG_X86_MCE_INTEL is not set
CONFIG_X86_MCE_INJECT=m

#
# Performance monitoring
#
CONFIG_PERF_EVENTS_INTEL_UNCORE=y
CONFIG_PERF_EVENTS_INTEL_RAPL=y
CONFIG_PERF_EVENTS_INTEL_CSTATE=y
CONFIG_X86_VSYSCALL_EMULATION=y
# CONFIG_I8K is not set
# CONFIG_MICROCODE is not set
CONFIG_X86_MSR=y
CONFIG_X86_CPUID=m
# CONFIG_X86_5LEVEL is not set
CONFIG_ARCH_PHYS_ADDR_T_64BIT=y
CONFIG_ARCH_DMA_ADDR_T_64BIT=y
CONFIG_ARCH_HAS_MEM_ENCRYPT=y
CONFIG_NUMA=y
CONFIG_AMD_NUMA=y
CONFIG_X86_64_ACPI_NUMA=y
CONFIG_NODES_SPAN_OTHER_NODES=y
CONFIG_NUMA_EMU=y
CONFIG_NODES_SHIFT=6
CONFIG_ARCH_SPARSEMEM_ENABLE=y
CONFIG_ARCH_SPARSEMEM_DEFAULT=y
CONFIG_ARCH_SELECT_MEMORY_MODEL=y
# CONFIG_ARCH_MEMORY_PROBE is not set
CONFIG_ILLEGAL_POINTER_VALUE=0xdead000000000000
CONFIG_SELECT_MEMORY_MODEL=y
CONFIG_SPARSEMEM_MANUAL=y
CONFIG_SPARSEMEM=y
CONFIG_NEED_MULTIPLE_NODES=y
CONFIG_HAVE_MEMORY_PRESENT=y
CONFIG_SPARSEMEM_EXTREME=y
CONFIG_SPARSEMEM_VMEMMAP_ENABLE=y
CONFIG_SPARSEMEM_ALLOC_MEM_MAP_TOGETHER=y
CONFIG_SPARSEMEM_VMEMMAP=y
CONFIG_HAVE_MEMBLOCK=y
CONFIG_HAVE_MEMBLOCK_NODE_MAP=y
CONFIG_HAVE_GENERIC_GUP=y
CONFIG_ARCH_DISCARD_MEMBLOCK=y
CONFIG_MEMORY_ISOLATION=y
CONFIG_HAVE_BOOTMEM_INFO_NODE=y
CONFIG_MEMORY_HOTPLUG=y
CONFIG_MEMORY_HOTPLUG_SPARSE=y
# CONFIG_MEMORY_HOTPLUG_DEFAULT_ONLINE is not set
CONFIG_MEMORY_HOTREMOVE=y
CONFIG_SPLIT_PTLOCK_CPUS=4
CONFIG_ARCH_ENABLE_SPLIT_PMD_PTLOCK=y
CONFIG_COMPACTION=y
CONFIG_MIGRATION=y
CONFIG_ARCH_ENABLE_HUGEPAGE_MIGRATION=y
CONFIG_PHYS_ADDR_T_64BIT=y
CONFIG_BOUNCE=y
CONFIG_VIRT_TO_BUS=y
# CONFIG_KSM is not set
CONFIG_DEFAULT_MMAP_MIN_ADDR=4096
CONFIG_ARCH_SUPPORTS_MEMORY_FAILURE=y
# CONFIG_MEMORY_FAILURE is not set
# CONFIG_TRANSPARENT_HUGEPAGE is not set
CONFIG_ARCH_WANTS_THP_SWAP=y
CONFIG_CLEANCACHE=y
CONFIG_CMA=y
CONFIG_CMA_DEBUG=y
# CONFIG_CMA_DEBUGFS is not set
CONFIG_CMA_AREAS=7
CONFIG_MEM_SOFT_DIRTY=y
CONFIG_ZPOOL=y
CONFIG_ZBUD=m
CONFIG_Z3FOLD=m
CONFIG_ZSMALLOC=y
# CONFIG_PGTABLE_MAPPING is not set
CONFIG_ZSMALLOC_STAT=y
CONFIG_GENERIC_EARLY_IOREMAP=y
CONFIG_DEFERRED_STRUCT_PAGE_INIT=y
CONFIG_IDLE_PAGE_TRACKING=y
CONFIG_ARCH_HAS_ZONE_DEVICE=y
CONFIG_ZONE_DEVICE=y
CONFIG_ARCH_HAS_HMM=y
CONFIG_MIGRATE_VMA_HELPER=y
CONFIG_HMM=y
# CONFIG_HMM_MIRROR is not set
CONFIG_DEVICE_PRIVATE=y
CONFIG_DEVICE_PUBLIC=y
CONFIG_PERCPU_STATS=y
# CONFIG_GUP_BENCHMARK is not set
CONFIG_X86_PMEM_LEGACY_DEVICE=y
CONFIG_X86_PMEM_LEGACY=m
CONFIG_X86_CHECK_BIOS_CORRUPTION=y
CONFIG_X86_BOOTPARAM_MEMORY_CORRUPTION_CHECK=y
CONFIG_X86_RESERVE_LOW=64
# CONFIG_MTRR is not set
# CONFIG_ARCH_RANDOM is not set
CONFIG_X86_SMAP=y
# CONFIG_X86_INTEL_UMIP is not set
# CONFIG_X86_INTEL_MPX is not set
# CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS is not set
# CONFIG_EFI is not set
# CONFIG_SECCOMP is not set
# CONFIG_HZ_100 is not set
# CONFIG_HZ_250 is not set
CONFIG_HZ_300=y
# CONFIG_HZ_1000 is not set
CONFIG_HZ=300
CONFIG_SCHED_HRTICK=y
CONFIG_KEXEC=y
# CONFIG_KEXEC_FILE is not set
# CONFIG_CRASH_DUMP is not set
CONFIG_PHYSICAL_START=0x1000000
CONFIG_RELOCATABLE=y
CONFIG_RANDOMIZE_BASE=y
CONFIG_X86_NEED_RELOCS=y
CONFIG_PHYSICAL_ALIGN=0x200000
CONFIG_DYNAMIC_MEMORY_LAYOUT=y
CONFIG_RANDOMIZE_MEMORY=y
CONFIG_RANDOMIZE_MEMORY_PHYSICAL_PADDING=0xa
CONFIG_HOTPLUG_CPU=y
CONFIG_BOOTPARAM_HOTPLUG_CPU0=y
CONFIG_DEBUG_HOTPLUG_CPU0=y
CONFIG_LEGACY_VSYSCALL_EMULATE=y
# CONFIG_LEGACY_VSYSCALL_NONE is not set
# CONFIG_CMDLINE_BOOL is not set
# CONFIG_MODIFY_LDT_SYSCALL is not set
CONFIG_HAVE_LIVEPATCH=y
CONFIG_ARCH_HAS_ADD_PAGES=y
CONFIG_ARCH_ENABLE_MEMORY_HOTPLUG=y
CONFIG_ARCH_ENABLE_MEMORY_HOTREMOVE=y
CONFIG_USE_PERCPU_NUMA_NODE_ID=y

#
# Power management and ACPI options
#
CONFIG_SUSPEND=y
CONFIG_SUSPEND_FREEZER=y
CONFIG_SUSPEND_SKIP_SYNC=y
CONFIG_PM_SLEEP=y
CONFIG_PM_SLEEP_SMP=y
CONFIG_PM_AUTOSLEEP=y
# CONFIG_PM_WAKELOCKS is not set
CONFIG_PM=y
# CONFIG_PM_DEBUG is not set
CONFIG_PM_CLK=y
# CONFIG_WQ_POWER_EFFICIENT_DEFAULT is not set
CONFIG_ACPI=y
CONFIG_ACPI_LEGACY_TABLES_LOOKUP=y
CONFIG_ARCH_MIGHT_HAVE_ACPI_PDC=y
CONFIG_ACPI_SYSTEM_POWER_STATES_SUPPORT=y
# CONFIG_ACPI_DEBUGGER is not set
CONFIG_ACPI_SPCR_TABLE=y
CONFIG_ACPI_LPIT=y
CONFIG_ACPI_SLEEP=y
# CONFIG_ACPI_PROCFS_POWER is not set
CONFIG_ACPI_REV_OVERRIDE_POSSIBLE=y
# CONFIG_ACPI_EC_DEBUGFS is not set
CONFIG_ACPI_AC=y
CONFIG_ACPI_BATTERY=y
CONFIG_ACPI_BUTTON=y
# CONFIG_ACPI_VIDEO is not set
CONFIG_ACPI_FAN=y
# CONFIG_ACPI_TAD is not set
# CONFIG_ACPI_DOCK is not set
CONFIG_ACPI_CPU_FREQ_PSS=y
CONFIG_ACPI_PROCESSOR_CSTATE=y
CONFIG_ACPI_PROCESSOR_IDLE=y
CONFIG_ACPI_PROCESSOR=y
CONFIG_ACPI_HOTPLUG_CPU=y
# CONFIG_ACPI_PROCESSOR_AGGREGATOR is not set
CONFIG_ACPI_THERMAL=y
CONFIG_ACPI_NUMA=y
CONFIG_ARCH_HAS_ACPI_TABLE_UPGRADE=y
CONFIG_ACPI_TABLE_UPGRADE=y
# CONFIG_ACPI_DEBUG is not set
# CONFIG_ACPI_PCI_SLOT is not set
CONFIG_ACPI_CONTAINER=y
# CONFIG_ACPI_HOTPLUG_MEMORY is not set
CONFIG_ACPI_HOTPLUG_IOAPIC=y
# CONFIG_ACPI_SBS is not set
# CONFIG_ACPI_HED is not set
# CONFIG_ACPI_CUSTOM_METHOD is not set
# CONFIG_ACPI_REDUCED_HARDWARE_ONLY is not set
# CONFIG_ACPI_NFIT is not set
CONFIG_HAVE_ACPI_APEI=y
CONFIG_HAVE_ACPI_APEI_NMI=y
# CONFIG_ACPI_APEI is not set
# CONFIG_DPTF_POWER is not set
# CONFIG_PMIC_OPREGION is not set
# CONFIG_ACPI_CONFIGFS is not set
CONFIG_X86_PM_TIMER=y
# CONFIG_SFI is not set

#
# CPU Frequency scaling
#
# CONFIG_CPU_FREQ is not set

#
# CPU Idle
#
CONFIG_CPU_IDLE=y
CONFIG_CPU_IDLE_GOV_LADDER=y
# CONFIG_CPU_IDLE_GOV_MENU is not set
CONFIG_INTEL_IDLE=y

#
# Bus options (PCI etc.)
#
CONFIG_PCI=y
CONFIG_PCI_DIRECT=y
CONFIG_PCI_MMCONFIG=y
CONFIG_PCI_DOMAINS=y
CONFIG_MMCONF_FAM10H=y
# CONFIG_PCI_CNB20LE_QUIRK is not set
# CONFIG_PCIEPORTBUS is not set
CONFIG_PCI_BUS_ADDR_T_64BIT=y
# CONFIG_PCI_MSI is not set
CONFIG_PCI_QUIRKS=y
# CONFIG_PCI_DEBUG is not set
# CONFIG_PCI_STUB is not set
CONFIG_PCI_LOCKLESS_CONFIG=y
# CONFIG_PCI_IOV is not set
# CONFIG_PCI_PRI is not set
# CONFIG_PCI_PASID is not set
CONFIG_PCI_LABEL=y
# CONFIG_HOTPLUG_PCI is not set

#
# Cadence PCIe controllers support
#
CONFIG_PCIE_CADENCE=y
# CONFIG_PCIE_CADENCE_HOST is not set
CONFIG_PCIE_CADENCE_EP=y

#
# DesignWare PCI Core Support
#

#
# PCI host controller drivers
#

#
# PCI Endpoint
#
CONFIG_PCI_ENDPOINT=y
# CONFIG_PCI_ENDPOINT_CONFIGFS is not set
CONFIG_PCI_EPF_TEST=y

#
# PCI switch controller drivers
#
# CONFIG_PCI_SW_SWITCHTEC is not set
# CONFIG_ISA_BUS is not set
CONFIG_ISA_DMA_API=y
CONFIG_PCCARD=y
CONFIG_PCMCIA=y
CONFIG_PCMCIA_LOAD_CIS=y
CONFIG_CARDBUS=y

#
# PC-card bridges
#
# CONFIG_YENTA is not set
# CONFIG_PD6729 is not set
# CONFIG_I82092 is not set
# CONFIG_RAPIDIO is not set
# CONFIG_X86_SYSFB is not set

#
# Executable file formats / Emulations
#
CONFIG_BINFMT_ELF=y
CONFIG_ELFCORE=y
CONFIG_BINFMT_SCRIPT=y
CONFIG_BINFMT_MISC=m
# CONFIG_COREDUMP is not set
# CONFIG_IA32_EMULATION is not set
CONFIG_X86_X32=y
CONFIG_COMPAT=y
CONFIG_COMPAT_FOR_U64_ALIGNMENT=y
CONFIG_X86_DEV_DMA_OPS=y
CONFIG_NET=y

#
# Networking options
#
# CONFIG_PACKET is not set
CONFIG_UNIX=y
# CONFIG_UNIX_DIAG is not set
# CONFIG_TLS is not set
CONFIG_XFRM=y
# CONFIG_XFRM_USER is not set
# CONFIG_XFRM_SUB_POLICY is not set
# CONFIG_XFRM_MIGRATE is not set
# CONFIG_XFRM_STATISTICS is not set
# CONFIG_NET_KEY is not set
CONFIG_INET=y
# CONFIG_IP_MULTICAST is not set
# CONFIG_IP_ADVANCED_ROUTER is not set
CONFIG_IP_PNP=y
CONFIG_IP_PNP_DHCP=y
# CONFIG_IP_PNP_BOOTP is not set
# CONFIG_IP_PNP_RARP is not set
# CONFIG_NET_IPIP is not set
# CONFIG_NET_IPGRE_DEMUX is not set
CONFIG_NET_IP_TUNNEL=y
# CONFIG_SYN_COOKIES is not set
# CONFIG_NET_IPVTI is not set
# CONFIG_NET_FOU is not set
# CONFIG_NET_FOU_IP_TUNNELS is not set
# CONFIG_INET_AH is not set
# CONFIG_INET_ESP is not set
# CONFIG_INET_IPCOMP is not set
CONFIG_INET_TUNNEL=y
CONFIG_INET_XFRM_MODE_TRANSPORT=y
CONFIG_INET_XFRM_MODE_TUNNEL=y
CONFIG_INET_XFRM_MODE_BEET=y
CONFIG_INET_DIAG=y
CONFIG_INET_TCP_DIAG=y
# CONFIG_INET_UDP_DIAG is not set
# CONFIG_INET_RAW_DIAG is not set
# CONFIG_INET_DIAG_DESTROY is not set
# CONFIG_TCP_CONG_ADVANCED is not set
CONFIG_TCP_CONG_CUBIC=y
CONFIG_DEFAULT_TCP_CONG="cubic"
# CONFIG_TCP_MD5SIG is not set
CONFIG_IPV6=y
# CONFIG_IPV6_ROUTER_PREF is not set
# CONFIG_IPV6_OPTIMISTIC_DAD is not set
# CONFIG_INET6_AH is not set
# CONFIG_INET6_ESP is not set
# CONFIG_INET6_IPCOMP is not set
# CONFIG_IPV6_MIP6 is not set
CONFIG_INET6_XFRM_MODE_TRANSPORT=y
CONFIG_INET6_XFRM_MODE_TUNNEL=y
CONFIG_INET6_XFRM_MODE_BEET=y
# CONFIG_INET6_XFRM_MODE_ROUTEOPTIMIZATION is not set
# CONFIG_IPV6_VTI is not set
CONFIG_IPV6_SIT=y
# CONFIG_IPV6_SIT_6RD is not set
CONFIG_IPV6_NDISC_NODETYPE=y
# CONFIG_IPV6_TUNNEL is not set
# CONFIG_IPV6_MULTIPLE_TABLES is not set
# CONFIG_IPV6_MROUTE is not set
# CONFIG_IPV6_SEG6_LWTUNNEL is not set
# CONFIG_IPV6_SEG6_HMAC is not set
# CONFIG_NETWORK_SECMARK is not set
CONFIG_NET_PTP_CLASSIFY=y
# CONFIG_NETWORK_PHY_TIMESTAMPING is not set
# CONFIG_NETFILTER is not set
# CONFIG_IP_DCCP is not set
# CONFIG_IP_SCTP is not set
# CONFIG_RDS is not set
# CONFIG_TIPC is not set
# CONFIG_ATM is not set
# CONFIG_L2TP is not set
# CONFIG_BRIDGE is not set
CONFIG_HAVE_NET_DSA=y
# CONFIG_NET_DSA is not set
# CONFIG_VLAN_8021Q is not set
# CONFIG_DECNET is not set
# CONFIG_LLC2 is not set
# CONFIG_ATALK is not set
# CONFIG_X25 is not set
# CONFIG_LAPB is not set
# CONFIG_PHONET is not set
# CONFIG_6LOWPAN is not set
# CONFIG_IEEE802154 is not set
# CONFIG_NET_SCHED is not set
# CONFIG_DCB is not set
CONFIG_DNS_RESOLVER=m
# CONFIG_BATMAN_ADV is not set
# CONFIG_OPENVSWITCH is not set
# CONFIG_VSOCKETS is not set
# CONFIG_NETLINK_DIAG is not set
# CONFIG_MPLS is not set
# CONFIG_NET_NSH is not set
# CONFIG_HSR is not set
# CONFIG_NET_SWITCHDEV is not set
# CONFIG_NET_L3_MASTER_DEV is not set
# CONFIG_NET_NCSI is not set
CONFIG_RPS=y
CONFIG_RFS_ACCEL=y
CONFIG_XPS=y
# CONFIG_CGROUP_NET_PRIO is not set
# CONFIG_CGROUP_NET_CLASSID is not set
CONFIG_NET_RX_BUSY_POLL=y
CONFIG_BQL=y
# CONFIG_BPF_JIT is not set
CONFIG_NET_FLOW_LIMIT=y

#
# Network testing
#
# CONFIG_NET_PKTGEN is not set
# CONFIG_HAMRADIO is not set
# CONFIG_CAN is not set
# CONFIG_BT is not set
# CONFIG_AF_RXRPC is not set
# CONFIG_AF_KCM is not set
CONFIG_WIRELESS=y
# CONFIG_CFG80211 is not set

#
# CFG80211 needs to be enabled for MAC80211
#
CONFIG_MAC80211_STA_HASH_MAX_SIZE=0
# CONFIG_WIMAX is not set
# CONFIG_RFKILL is not set
# CONFIG_NET_9P is not set
# CONFIG_CAIF is not set
# CONFIG_CEPH_LIB is not set
# CONFIG_NFC is not set
# CONFIG_PSAMPLE is not set
# CONFIG_NET_IFE is not set
# CONFIG_LWTUNNEL is not set
CONFIG_DST_CACHE=y
CONFIG_GRO_CELLS=y
# CONFIG_NET_DEVLINK is not set
CONFIG_MAY_USE_DEVLINK=y
CONFIG_HAVE_EBPF_JIT=y

#
# Device Drivers
#

#
# Generic Driver Options
#
# CONFIG_UEVENT_HELPER is not set
CONFIG_DEVTMPFS=y
# CONFIG_DEVTMPFS_MOUNT is not set
CONFIG_STANDALONE=y
# CONFIG_PREVENT_FIRMWARE_BUILD is not set
CONFIG_FW_LOADER=y
CONFIG_EXTRA_FIRMWARE=""
CONFIG_FW_LOADER_USER_HELPER=y
CONFIG_FW_LOADER_USER_HELPER_FALLBACK=y
CONFIG_WANT_DEV_COREDUMP=y
CONFIG_ALLOW_DEV_COREDUMP=y
CONFIG_DEV_COREDUMP=y
# CONFIG_DEBUG_DRIVER is not set
# CONFIG_DEBUG_DEVRES is not set
# CONFIG_DEBUG_TEST_DRIVER_REMOVE is not set
CONFIG_TEST_ASYNC_DRIVER_PROBE=m
CONFIG_GENERIC_CPU_AUTOPROBE=y
CONFIG_GENERIC_CPU_VULNERABILITIES=y
CONFIG_REGMAP=y
CONFIG_REGMAP_I2C=m
CONFIG_REGMAP_SPI=y
CONFIG_REGMAP_W1=m
CONFIG_REGMAP_MMIO=y
CONFIG_REGMAP_IRQ=y
CONFIG_DMA_SHARED_BUFFER=y
CONFIG_DMA_FENCE_TRACE=y
# CONFIG_DMA_CMA is not set

#
# Bus devices
#
CONFIG_SIMPLE_PM_BUS=m
# CONFIG_CONNECTOR is not set
CONFIG_MTD=y
# CONFIG_MTD_TESTS is not set
CONFIG_MTD_REDBOOT_PARTS=m
CONFIG_MTD_REDBOOT_DIRECTORY_BLOCK=-1
CONFIG_MTD_REDBOOT_PARTS_UNALLOCATED=y
# CONFIG_MTD_REDBOOT_PARTS_READONLY is not set
CONFIG_MTD_CMDLINE_PARTS=y
# CONFIG_MTD_OF_PARTS is not set
CONFIG_MTD_AR7_PARTS=y

#
# Partition parsers
#

#
# User Modules And Translation Layers
#
CONFIG_MTD_BLKDEVS=y
CONFIG_MTD_BLOCK=y
CONFIG_FTL=y
# CONFIG_NFTL is not set
CONFIG_INFTL=m
# CONFIG_RFD_FTL is not set
# CONFIG_SSFDC is not set
CONFIG_SM_FTL=y
CONFIG_MTD_OOPS=y
CONFIG_MTD_PARTITIONED_MASTER=y

#
# RAM/ROM/Flash chip drivers
#
# CONFIG_MTD_CFI is not set
CONFIG_MTD_JEDECPROBE=y
CONFIG_MTD_GEN_PROBE=y
# CONFIG_MTD_CFI_ADV_OPTIONS is not set
CONFIG_MTD_MAP_BANK_WIDTH_1=y
CONFIG_MTD_MAP_BANK_WIDTH_2=y
CONFIG_MTD_MAP_BANK_WIDTH_4=y
CONFIG_MTD_CFI_I1=y
CONFIG_MTD_CFI_I2=y
CONFIG_MTD_CFI_INTELEXT=m
CONFIG_MTD_CFI_AMDSTD=y
# CONFIG_MTD_CFI_STAA is not set
CONFIG_MTD_CFI_UTIL=y
CONFIG_MTD_RAM=m
CONFIG_MTD_ROM=m
CONFIG_MTD_ABSENT=y

#
# Mapping drivers for chip access
#
CONFIG_MTD_COMPLEX_MAPPINGS=y
CONFIG_MTD_PHYSMAP=y
CONFIG_MTD_PHYSMAP_COMPAT=y
CONFIG_MTD_PHYSMAP_START=0x8000000
CONFIG_MTD_PHYSMAP_LEN=0
CONFIG_MTD_PHYSMAP_BANKWIDTH=2
CONFIG_MTD_PHYSMAP_OF=y
CONFIG_MTD_PHYSMAP_OF_VERSATILE=y
# CONFIG_MTD_PHYSMAP_OF_GEMINI is not set
# CONFIG_MTD_SBC_GXX is not set
# CONFIG_MTD_AMD76XROM is not set
CONFIG_MTD_ICHXROM=y
# CONFIG_MTD_ESB2ROM is not set
# CONFIG_MTD_CK804XROM is not set
# CONFIG_MTD_SCB2_FLASH is not set
# CONFIG_MTD_NETtel is not set
CONFIG_MTD_L440GX=y
# CONFIG_MTD_PCI is not set
CONFIG_MTD_PCMCIA=y
# CONFIG_MTD_PCMCIA_ANONYMOUS is not set
# CONFIG_MTD_INTEL_VR_NOR is not set
# CONFIG_MTD_PLATRAM is not set
# CONFIG_MTD_LATCH_ADDR is not set

#
# Self-contained MTD device drivers
#
# CONFIG_MTD_PMC551 is not set
CONFIG_MTD_DATAFLASH=m
CONFIG_MTD_DATAFLASH_WRITE_VERIFY=y
CONFIG_MTD_DATAFLASH_OTP=y
CONFIG_MTD_MCHP23K256=m
CONFIG_MTD_SST25L=m
CONFIG_MTD_SLRAM=m
CONFIG_MTD_PHRAM=m
CONFIG_MTD_MTDRAM=y
CONFIG_MTDRAM_TOTAL_SIZE=4096
CONFIG_MTDRAM_ERASE_SIZE=128
CONFIG_MTD_BLOCK2MTD=y

#
# Disk-On-Chip Device Drivers
#
CONFIG_MTD_DOCG3=y
CONFIG_BCH_CONST_M=14
CONFIG_BCH_CONST_T=4
CONFIG_MTD_ONENAND=y
CONFIG_MTD_ONENAND_VERIFY_WRITE=y
CONFIG_MTD_ONENAND_GENERIC=y
CONFIG_MTD_ONENAND_OTP=y
# CONFIG_MTD_ONENAND_2X_PROGRAM is not set
CONFIG_MTD_NAND_ECC=y
CONFIG_MTD_NAND_ECC_SMC=y
CONFIG_MTD_NAND=y
CONFIG_MTD_NAND_BCH=y
CONFIG_MTD_NAND_ECC_BCH=y
# CONFIG_MTD_NAND_DENALI_PCI is not set
# CONFIG_MTD_NAND_DENALI_DT is not set
# CONFIG_MTD_NAND_RICOH is not set
# CONFIG_MTD_NAND_DISKONCHIP is not set
CONFIG_MTD_NAND_DOCG4=m
# CONFIG_MTD_NAND_CAFE is not set
CONFIG_MTD_NAND_NANDSIM=y
CONFIG_MTD_NAND_PLATFORM=y

#
# LPDDR & LPDDR2 PCM memory drivers
#
CONFIG_MTD_LPDDR=y
CONFIG_MTD_QINFO_PROBE=y
# CONFIG_MTD_SPI_NOR is not set
CONFIG_MTD_UBI=y
CONFIG_MTD_UBI_WL_THRESHOLD=4096
CONFIG_MTD_UBI_BEB_LIMIT=20
# CONFIG_MTD_UBI_FASTMAP is not set
# CONFIG_MTD_UBI_GLUEBI is not set
CONFIG_MTD_UBI_BLOCK=y
CONFIG_DTC=y
CONFIG_OF=y
CONFIG_OF_UNITTEST=y
CONFIG_OF_FLATTREE=y
CONFIG_OF_EARLY_FLATTREE=y
CONFIG_OF_KOBJ=y
CONFIG_OF_DYNAMIC=y
CONFIG_OF_ADDRESS=y
CONFIG_OF_IRQ=y
CONFIG_OF_NET=y
CONFIG_OF_RESOLVE=y
# CONFIG_OF_OVERLAY is not set
CONFIG_ARCH_MIGHT_HAVE_PC_PARPORT=y
CONFIG_PARPORT=y
CONFIG_PARPORT_PC=y
# CONFIG_PARPORT_SERIAL is not set
CONFIG_PARPORT_PC_FIFO=y
CONFIG_PARPORT_PC_SUPERIO=y
# CONFIG_PARPORT_PC_PCMCIA is not set
# CONFIG_PARPORT_AX88796 is not set
CONFIG_PARPORT_1284=y
CONFIG_PNP=y
CONFIG_PNP_DEBUG_MESSAGES=y

#
# Protocols
#
CONFIG_PNPACPI=y
CONFIG_BLK_DEV=y
# CONFIG_BLK_DEV_NULL_BLK is not set
CONFIG_BLK_DEV_FD=y
CONFIG_CDROM=y
# CONFIG_PARIDE is not set
# CONFIG_BLK_DEV_PCIESSD_MTIP32XX is not set
# CONFIG_ZRAM is not set
# CONFIG_BLK_DEV_DAC960 is not set
# CONFIG_BLK_DEV_UMEM is not set
CONFIG_BLK_DEV_LOOP=y
CONFIG_BLK_DEV_LOOP_MIN_COUNT=8
CONFIG_BLK_DEV_CRYPTOLOOP=m
# CONFIG_BLK_DEV_DRBD is not set
# CONFIG_BLK_DEV_NBD is not set
# CONFIG_BLK_DEV_SKD is not set
# CONFIG_BLK_DEV_SX8 is not set
CONFIG_BLK_DEV_RAM=m
CONFIG_BLK_DEV_RAM_COUNT=16
CONFIG_BLK_DEV_RAM_SIZE=4096
CONFIG_CDROM_PKTCDVD=y
CONFIG_CDROM_PKTCDVD_BUFFERS=8
# CONFIG_CDROM_PKTCDVD_WCACHE is not set
# CONFIG_ATA_OVER_ETH is not set
# CONFIG_VIRTIO_BLK is not set
# CONFIG_BLK_DEV_RBD is not set
# CONFIG_BLK_DEV_RSXX is not set

#
# NVME Support
#
CONFIG_NVME_CORE=y
# CONFIG_BLK_DEV_NVME is not set
# CONFIG_NVME_MULTIPATH is not set
CONFIG_NVME_FABRICS=y
CONFIG_NVME_FC=y
CONFIG_NVME_TARGET=m
CONFIG_NVME_TARGET_LOOP=m
CONFIG_NVME_TARGET_FC=m
CONFIG_NVME_TARGET_FCLOOP=m

#
# Misc devices
#
CONFIG_SENSORS_LIS3LV02D=m
# CONFIG_AD525X_DPOT is not set
CONFIG_DUMMY_IRQ=y
# CONFIG_IBM_ASM is not set
# CONFIG_PHANTOM is not set
# CONFIG_SGI_IOC4 is not set
# CONFIG_TIFM_CORE is not set
CONFIG_ICS932S401=m
CONFIG_ENCLOSURE_SERVICES=m
# CONFIG_HP_ILO is not set
CONFIG_APDS9802ALS=m
CONFIG_ISL29003=m
CONFIG_ISL29020=m
CONFIG_SENSORS_TSL2550=m
CONFIG_SENSORS_BH1770=m
# CONFIG_SENSORS_APDS990X is not set
CONFIG_HMC6352=m
CONFIG_DS1682=m
CONFIG_USB_SWITCH_FSA9480=m
CONFIG_LATTICE_ECP3_CONFIG=m
# CONFIG_SRAM is not set
# CONFIG_PCI_ENDPOINT_TEST is not set
CONFIG_C2PORT=m
# CONFIG_C2PORT_DURAMAR_2150 is not set

#
# EEPROM support
#
# CONFIG_EEPROM_AT24 is not set
CONFIG_EEPROM_AT25=m
# CONFIG_EEPROM_LEGACY is not set
CONFIG_EEPROM_MAX6875=m
CONFIG_EEPROM_93CX6=y
CONFIG_EEPROM_93XX46=y
CONFIG_EEPROM_IDT_89HPESX=m
# CONFIG_CB710_CORE is not set

#
# Texas Instruments shared transport line discipline
#
CONFIG_SENSORS_LIS3_I2C=m

#
# Altera FPGA firmware download module (requires I2C)
#
# CONFIG_ALTERA_STAPL is not set
# CONFIG_INTEL_MEI is not set
# CONFIG_INTEL_MEI_ME is not set
# CONFIG_INTEL_MEI_TXE is not set
# CONFIG_VMWARE_VMCI is not set

#
# Intel MIC & related support
#

#
# Intel MIC Bus Driver
#
# CONFIG_INTEL_MIC_BUS is not set

#
# SCIF Bus Driver
#
# CONFIG_SCIF_BUS is not set

#
# VOP Bus Driver
#
# CONFIG_VOP_BUS is not set

#
# Intel MIC Host Driver
#

#
# Intel MIC Card Driver
#

#
# SCIF Driver
#

#
# Intel MIC Coprocessor State Management (COSM) Drivers
#

#
# VOP Driver
#
# CONFIG_GENWQE is not set
CONFIG_ECHO=y
# CONFIG_MISC_RTSX_PCI is not set
CONFIG_HAVE_IDE=y
CONFIG_IDE=m

#
# Please see Documentation/ide/ide.txt for help/info on IDE drives
#
CONFIG_IDE_XFER_MODE=y
CONFIG_IDE_TIMINGS=y
CONFIG_IDE_ATAPI=y
CONFIG_BLK_DEV_IDE_SATA=y
CONFIG_IDE_GD=m
CONFIG_IDE_GD_ATA=y
# CONFIG_IDE_GD_ATAPI is not set
CONFIG_BLK_DEV_IDECS=m
# CONFIG_BLK_DEV_DELKIN is not set
CONFIG_BLK_DEV_IDECD=m
CONFIG_BLK_DEV_IDECD_VERBOSE_ERRORS=y
CONFIG_BLK_DEV_IDETAPE=m
# CONFIG_BLK_DEV_IDEACPI is not set
CONFIG_IDE_TASK_IOCTL=y
# CONFIG_IDE_PROC_FS is not set

#
# IDE chipset support/bugfixes
#
CONFIG_IDE_GENERIC=m
CONFIG_BLK_DEV_PLATFORM=m
CONFIG_BLK_DEV_CMD640=m
# CONFIG_BLK_DEV_CMD640_ENHANCED is not set
# CONFIG_BLK_DEV_IDEPNP is not set

#
# PCI IDE chipsets support
#
# CONFIG_BLK_DEV_GENERIC is not set
# CONFIG_BLK_DEV_OPTI621 is not set
# CONFIG_BLK_DEV_RZ1000 is not set
# CONFIG_BLK_DEV_AEC62XX is not set
# CONFIG_BLK_DEV_ALI15X3 is not set
# CONFIG_BLK_DEV_AMD74XX is not set
# CONFIG_BLK_DEV_ATIIXP is not set
# CONFIG_BLK_DEV_CMD64X is not set
# CONFIG_BLK_DEV_TRIFLEX is not set
# CONFIG_BLK_DEV_HPT366 is not set
# CONFIG_BLK_DEV_JMICRON is not set
# CONFIG_BLK_DEV_PIIX is not set
# CONFIG_BLK_DEV_IT8172 is not set
# CONFIG_BLK_DEV_IT8213 is not set
# CONFIG_BLK_DEV_IT821X is not set
# CONFIG_BLK_DEV_NS87415 is not set
# CONFIG_BLK_DEV_PDC202XX_OLD is not set
# CONFIG_BLK_DEV_PDC202XX_NEW is not set
# CONFIG_BLK_DEV_SVWKS is not set
# CONFIG_BLK_DEV_SIIMAGE is not set
# CONFIG_BLK_DEV_SIS5513 is not set
# CONFIG_BLK_DEV_SLC90E66 is not set
# CONFIG_BLK_DEV_TRM290 is not set
# CONFIG_BLK_DEV_VIA82CXXX is not set
# CONFIG_BLK_DEV_TC86C001 is not set

#
# SCSI device support
#
CONFIG_SCSI_MOD=m
CONFIG_RAID_ATTRS=m
CONFIG_SCSI=m
CONFIG_SCSI_DMA=y
# CONFIG_SCSI_MQ_DEFAULT is not set
CONFIG_SCSI_PROC_FS=y

#
# SCSI support type (disk, tape, CD-ROM)
#
# CONFIG_BLK_DEV_SD is not set
CONFIG_CHR_DEV_ST=m
# CONFIG_CHR_DEV_OSST is not set
CONFIG_BLK_DEV_SR=m
CONFIG_BLK_DEV_SR_VENDOR=y
# CONFIG_CHR_DEV_SG is not set
CONFIG_CHR_DEV_SCH=m
CONFIG_SCSI_ENCLOSURE=m
# CONFIG_SCSI_CONSTANTS is not set
CONFIG_SCSI_LOGGING=y
CONFIG_SCSI_SCAN_ASYNC=y

#
# SCSI Transports
#
# CONFIG_SCSI_SPI_ATTRS is not set
# CONFIG_SCSI_FC_ATTRS is not set
# CONFIG_SCSI_ISCSI_ATTRS is not set
CONFIG_SCSI_SAS_ATTRS=m
CONFIG_SCSI_SAS_LIBSAS=m
# CONFIG_SCSI_SAS_ATA is not set
# CONFIG_SCSI_SAS_HOST_SMP is not set
# CONFIG_SCSI_SRP_ATTRS is not set
# CONFIG_SCSI_LOWLEVEL is not set
# CONFIG_SCSI_LOWLEVEL_PCMCIA is not set
# CONFIG_SCSI_DH is not set
# CONFIG_SCSI_OSD_INITIATOR is not set
CONFIG_ATA=m
CONFIG_ATA_VERBOSE_ERROR=y
CONFIG_ATA_ACPI=y
# CONFIG_SATA_ZPODD is not set
CONFIG_SATA_PMP=y

#
# Controllers with non-SFF native interface
#
# CONFIG_SATA_AHCI is not set
CONFIG_SATA_AHCI_PLATFORM=m
CONFIG_AHCI_CEVA=m
CONFIG_AHCI_QORIQ=m
# CONFIG_SATA_INIC162X is not set
# CONFIG_SATA_ACARD_AHCI is not set
# CONFIG_SATA_SIL24 is not set
# CONFIG_ATA_SFF is not set
# CONFIG_MD is not set
CONFIG_TARGET_CORE=m
CONFIG_TCM_IBLOCK=m
# CONFIG_TCM_FILEIO is not set
CONFIG_TCM_PSCSI=m
# CONFIG_TCM_USER2 is not set
# CONFIG_LOOPBACK_TARGET is not set
# CONFIG_ISCSI_TARGET is not set
# CONFIG_FUSION is not set

#
# IEEE 1394 (FireWire) support
#
# CONFIG_FIREWIRE is not set
# CONFIG_FIREWIRE_NOSY is not set
# CONFIG_MACINTOSH_DRIVERS is not set
CONFIG_NETDEVICES=y
CONFIG_NET_CORE=y
# CONFIG_BONDING is not set
# CONFIG_DUMMY is not set
# CONFIG_EQUALIZER is not set
# CONFIG_NET_FC is not set
# CONFIG_NET_TEAM is not set
# CONFIG_MACVLAN is not set
# CONFIG_VXLAN is not set
# CONFIG_MACSEC is not set
# CONFIG_NETCONSOLE is not set
# CONFIG_TUN is not set
# CONFIG_TUN_VNET_CROSS_LE is not set
# CONFIG_VETH is not set
# CONFIG_VIRTIO_NET is not set
# CONFIG_NLMON is not set
# CONFIG_ARCNET is not set

#
# CAIF transport drivers
#

#
# Distributed Switch Architecture drivers
#
CONFIG_ETHERNET=y
CONFIG_MDIO=m
CONFIG_NET_VENDOR_3COM=y
# CONFIG_PCMCIA_3C574 is not set
# CONFIG_PCMCIA_3C589 is not set
# CONFIG_VORTEX is not set
# CONFIG_TYPHOON is not set
CONFIG_NET_VENDOR_ADAPTEC=y
# CONFIG_ADAPTEC_STARFIRE is not set
CONFIG_NET_VENDOR_AGERE=y
# CONFIG_ET131X is not set
CONFIG_NET_VENDOR_ALACRITECH=y
# CONFIG_SLICOSS is not set
CONFIG_NET_VENDOR_ALTEON=y
# CONFIG_ACENIC is not set
# CONFIG_ALTERA_TSE is not set
CONFIG_NET_VENDOR_AMAZON=y
CONFIG_NET_VENDOR_AMD=y
# CONFIG_AMD8111_ETH is not set
# CONFIG_PCNET32 is not set
# CONFIG_PCMCIA_NMCLAN is not set
# CONFIG_AMD_XGBE is not set
CONFIG_NET_VENDOR_AQUANTIA=y
# CONFIG_AQTION is not set
CONFIG_NET_VENDOR_ARC=y
CONFIG_NET_VENDOR_ATHEROS=y
# CONFIG_ATL2 is not set
# CONFIG_ATL1 is not set
# CONFIG_ATL1E is not set
# CONFIG_ATL1C is not set
# CONFIG_ALX is not set
# CONFIG_NET_VENDOR_AURORA is not set
CONFIG_NET_CADENCE=y
# CONFIG_MACB is not set
CONFIG_NET_VENDOR_BROADCOM=y
# CONFIG_B44 is not set
# CONFIG_BCMGENET is not set
# CONFIG_BNX2 is not set
# CONFIG_CNIC is not set
# CONFIG_TIGON3 is not set
# CONFIG_BNX2X is not set
# CONFIG_SYSTEMPORT is not set
# CONFIG_BNXT is not set
CONFIG_NET_VENDOR_BROCADE=y
# CONFIG_BNA is not set
CONFIG_NET_VENDOR_CAVIUM=y
# CONFIG_THUNDER_NIC_PF is not set
# CONFIG_THUNDER_NIC_VF is not set
# CONFIG_THUNDER_NIC_BGX is not set
# CONFIG_THUNDER_NIC_RGX is not set
CONFIG_CAVIUM_PTP=y
# CONFIG_LIQUIDIO is not set
CONFIG_NET_VENDOR_CHELSIO=y
# CONFIG_CHELSIO_T1 is not set
# CONFIG_CHELSIO_T3 is not set
# CONFIG_CHELSIO_T4 is not set
# CONFIG_CHELSIO_T4VF is not set
CONFIG_NET_VENDOR_CISCO=y
# CONFIG_ENIC is not set
CONFIG_NET_VENDOR_CORTINA=y
# CONFIG_GEMINI_ETHERNET is not set
# CONFIG_CX_ECAT is not set
# CONFIG_DNET is not set
CONFIG_NET_VENDOR_DEC=y
# CONFIG_NET_TULIP is not set
CONFIG_NET_VENDOR_DLINK=y
# CONFIG_DL2K is not set
# CONFIG_SUNDANCE is not set
CONFIG_NET_VENDOR_EMULEX=y
# CONFIG_BE2NET is not set
CONFIG_NET_VENDOR_EZCHIP=y
# CONFIG_EZCHIP_NPS_MANAGEMENT_ENET is not set
CONFIG_NET_VENDOR_EXAR=y
# CONFIG_S2IO is not set
# CONFIG_VXGE is not set
CONFIG_NET_VENDOR_FUJITSU=y
# CONFIG_PCMCIA_FMVJ18X is not set
CONFIG_NET_VENDOR_HP=y
# CONFIG_HP100 is not set
CONFIG_NET_VENDOR_HUAWEI=y
CONFIG_NET_VENDOR_INTEL=y
# CONFIG_E100 is not set
CONFIG_E1000=y
CONFIG_E1000E=m
CONFIG_E1000E_HWTS=y
CONFIG_IGB=m
# CONFIG_IGBVF is not set
# CONFIG_IXGB is not set
CONFIG_IXGBE=m
# CONFIG_I40E is not set
CONFIG_NET_VENDOR_I825XX=y
# CONFIG_JME is not set
CONFIG_NET_VENDOR_MARVELL=y
# CONFIG_MVMDIO is not set
# CONFIG_SKGE is not set
# CONFIG_SKY2 is not set
CONFIG_NET_VENDOR_MELLANOX=y
# CONFIG_MLX4_EN is not set
# CONFIG_MLX5_CORE is not set
# CONFIG_MLXSW_CORE is not set
# CONFIG_MLXFW is not set
CONFIG_NET_VENDOR_MICREL=y
# CONFIG_KS8842 is not set
# CONFIG_KS8851 is not set
# CONFIG_KS8851_MLL is not set
# CONFIG_KSZ884X_PCI is not set
CONFIG_NET_VENDOR_MICROCHIP=y
# CONFIG_ENC28J60 is not set
# CONFIG_ENCX24J600 is not set
# CONFIG_LAN743X is not set
CONFIG_NET_VENDOR_MYRI=y
# CONFIG_MYRI10GE is not set
# CONFIG_FEALNX is not set
CONFIG_NET_VENDOR_NATSEMI=y
# CONFIG_NATSEMI is not set
# CONFIG_NS83820 is not set
CONFIG_NET_VENDOR_NETRONOME=y
CONFIG_NET_VENDOR_NI=y
CONFIG_NET_VENDOR_8390=y
# CONFIG_PCMCIA_AXNET is not set
# CONFIG_NE2K_PCI is not set
# CONFIG_PCMCIA_PCNET is not set
CONFIG_NET_VENDOR_NVIDIA=y
# CONFIG_FORCEDETH is not set
CONFIG_NET_VENDOR_OKI=y
# CONFIG_ETHOC is not set
CONFIG_NET_PACKET_ENGINE=y
# CONFIG_HAMACHI is not set
# CONFIG_YELLOWFIN is not set
CONFIG_NET_VENDOR_QLOGIC=y
# CONFIG_QLA3XXX is not set
# CONFIG_QLCNIC is not set
# CONFIG_QLGE is not set
# CONFIG_NETXEN_NIC is not set
# CONFIG_QED is not set
CONFIG_NET_VENDOR_QUALCOMM=y
# CONFIG_QCA7000_SPI is not set
# CONFIG_QCA7000_UART is not set
# CONFIG_QCOM_EMAC is not set
# CONFIG_RMNET is not set
CONFIG_NET_VENDOR_REALTEK=y
# CONFIG_ATP is not set
# CONFIG_8139CP is not set
# CONFIG_8139TOO is not set
# CONFIG_R8169 is not set
CONFIG_NET_VENDOR_RENESAS=y
CONFIG_NET_VENDOR_RDC=y
# CONFIG_R6040 is not set
CONFIG_NET_VENDOR_ROCKER=y
CONFIG_NET_VENDOR_SAMSUNG=y
# CONFIG_SXGBE_ETH is not set
CONFIG_NET_VENDOR_SEEQ=y
CONFIG_NET_VENDOR_SILAN=y
# CONFIG_SC92031 is not set
CONFIG_NET_VENDOR_SIS=y
# CONFIG_SIS900 is not set
# CONFIG_SIS190 is not set
CONFIG_NET_VENDOR_SOLARFLARE=y
# CONFIG_SFC is not set
# CONFIG_SFC_FALCON is not set
CONFIG_NET_VENDOR_SMSC=y
# CONFIG_PCMCIA_SMC91C92 is not set
# CONFIG_EPIC100 is not set
# CONFIG_SMSC911X is not set
# CONFIG_SMSC9420 is not set
CONFIG_NET_VENDOR_SOCIONEXT=y
CONFIG_NET_VENDOR_STMICRO=y
# CONFIG_STMMAC_ETH is not set
CONFIG_NET_VENDOR_SUN=y
# CONFIG_HAPPYMEAL is not set
# CONFIG_SUNGEM is not set
# CONFIG_CASSINI is not set
# CONFIG_NIU is not set
CONFIG_NET_VENDOR_TEHUTI=y
# CONFIG_TEHUTI is not set
CONFIG_NET_VENDOR_TI=y
# CONFIG_TI_CPSW_ALE is not set
# CONFIG_TLAN is not set
CONFIG_NET_VENDOR_VIA=y
# CONFIG_VIA_RHINE is not set
# CONFIG_VIA_VELOCITY is not set
CONFIG_NET_VENDOR_WIZNET=y
# CONFIG_WIZNET_W5100 is not set
# CONFIG_WIZNET_W5300 is not set
CONFIG_NET_VENDOR_XIRCOM=y
# CONFIG_PCMCIA_XIRC2PS is not set
CONFIG_NET_VENDOR_SYNOPSYS=y
# CONFIG_DWC_XLGMAC is not set
# CONFIG_FDDI is not set
# CONFIG_HIPPI is not set
# CONFIG_NET_SB1000 is not set
# CONFIG_MDIO_DEVICE is not set
# CONFIG_PHYLIB is not set
# CONFIG_MICREL_KS8995MA is not set
# CONFIG_PLIP is not set
# CONFIG_PPP is not set
# CONFIG_SLIP is not set

#
# Host-side USB support is needed for USB Network Adapter support
#
CONFIG_WLAN=y
# CONFIG_WIRELESS_WDS is not set
CONFIG_WLAN_VENDOR_ADMTEK=y
CONFIG_WLAN_VENDOR_ATH=y
# CONFIG_ATH_DEBUG is not set
# CONFIG_ATH5K_PCI is not set
CONFIG_WLAN_VENDOR_ATMEL=y
CONFIG_WLAN_VENDOR_BROADCOM=y
CONFIG_WLAN_VENDOR_CISCO=y
CONFIG_WLAN_VENDOR_INTEL=y
CONFIG_WLAN_VENDOR_INTERSIL=y
# CONFIG_HOSTAP is not set
# CONFIG_PRISM54 is not set
CONFIG_WLAN_VENDOR_MARVELL=y
CONFIG_WLAN_VENDOR_MEDIATEK=y
CONFIG_WLAN_VENDOR_RALINK=y
CONFIG_WLAN_VENDOR_REALTEK=y
CONFIG_WLAN_VENDOR_RSI=y
CONFIG_WLAN_VENDOR_ST=y
CONFIG_WLAN_VENDOR_TI=y
CONFIG_WLAN_VENDOR_ZYDAS=y
CONFIG_WLAN_VENDOR_QUANTENNA=y
# CONFIG_PCMCIA_RAYCS is not set

#
# Enable WiMAX (Networking options) to see the WiMAX drivers
#
# CONFIG_WAN is not set
# CONFIG_VMXNET3 is not set
# CONFIG_FUJITSU_ES is not set
# CONFIG_NETDEVSIM is not set
# CONFIG_ISDN is not set
# CONFIG_NVM is not set

#
# Input device support
#
CONFIG_INPUT=y
# CONFIG_INPUT_LEDS is not set
CONFIG_INPUT_FF_MEMLESS=m
CONFIG_INPUT_POLLDEV=m
# CONFIG_INPUT_SPARSEKMAP is not set
# CONFIG_INPUT_MATRIXKMAP is not set

#
# Userland interfaces
#
CONFIG_INPUT_MOUSEDEV=m
CONFIG_INPUT_MOUSEDEV_PSAUX=y
CONFIG_INPUT_MOUSEDEV_SCREEN_X=1024
CONFIG_INPUT_MOUSEDEV_SCREEN_Y=768
# CONFIG_INPUT_JOYDEV is not set
# CONFIG_INPUT_EVDEV is not set
CONFIG_INPUT_EVBUG=m

#
# Input Device Drivers
#
CONFIG_INPUT_KEYBOARD=y
# CONFIG_KEYBOARD_ADP5588 is not set
# CONFIG_KEYBOARD_ADP5589 is not set
CONFIG_KEYBOARD_ATKBD=y
# CONFIG_KEYBOARD_QT1070 is not set
# CONFIG_KEYBOARD_QT2160 is not set
# CONFIG_KEYBOARD_DLINK_DIR685 is not set
# CONFIG_KEYBOARD_LKKBD is not set
# CONFIG_KEYBOARD_TCA6416 is not set
# CONFIG_KEYBOARD_TCA8418 is not set
# CONFIG_KEYBOARD_LM8323 is not set
# CONFIG_KEYBOARD_LM8333 is not set
# CONFIG_KEYBOARD_MAX7359 is not set
# CONFIG_KEYBOARD_MCS is not set
# CONFIG_KEYBOARD_MPR121 is not set
# CONFIG_KEYBOARD_NEWTON is not set
# CONFIG_KEYBOARD_OPENCORES is not set
# CONFIG_KEYBOARD_SAMSUNG is not set
# CONFIG_KEYBOARD_STOWAWAY is not set
# CONFIG_KEYBOARD_SUNKBD is not set
# CONFIG_KEYBOARD_OMAP4 is not set
# CONFIG_KEYBOARD_TM2_TOUCHKEY is not set
# CONFIG_KEYBOARD_XTKBD is not set
# CONFIG_KEYBOARD_CAP11XX is not set
# CONFIG_KEYBOARD_BCM is not set
# CONFIG_INPUT_MOUSE is not set
CONFIG_INPUT_JOYSTICK=y
CONFIG_JOYSTICK_ANALOG=m
# CONFIG_JOYSTICK_A3D is not set
CONFIG_JOYSTICK_ADI=m
# CONFIG_JOYSTICK_COBRA is not set
# CONFIG_JOYSTICK_GF2K is not set
# CONFIG_JOYSTICK_GRIP is not set
# CONFIG_JOYSTICK_GRIP_MP is not set
# CONFIG_JOYSTICK_GUILLEMOT is not set
# CONFIG_JOYSTICK_INTERACT is not set
# CONFIG_JOYSTICK_SIDEWINDER is not set
CONFIG_JOYSTICK_TMDC=m
CONFIG_JOYSTICK_IFORCE=m
CONFIG_JOYSTICK_IFORCE_232=y
CONFIG_JOYSTICK_WARRIOR=m
CONFIG_JOYSTICK_MAGELLAN=m
# CONFIG_JOYSTICK_SPACEORB is not set
CONFIG_JOYSTICK_SPACEBALL=m
# CONFIG_JOYSTICK_STINGER is not set
# CONFIG_JOYSTICK_TWIDJOY is not set
CONFIG_JOYSTICK_ZHENHUA=m
CONFIG_JOYSTICK_DB9=m
# CONFIG_JOYSTICK_GAMECON is not set
# CONFIG_JOYSTICK_TURBOGRAFX is not set
CONFIG_JOYSTICK_AS5011=m
CONFIG_JOYSTICK_JOYDUMP=m
# CONFIG_JOYSTICK_XPAD is not set
CONFIG_JOYSTICK_WALKERA0701=m
CONFIG_JOYSTICK_PSXPAD_SPI=m
CONFIG_JOYSTICK_PSXPAD_SPI_FF=y
# CONFIG_JOYSTICK_PXRC is not set
# CONFIG_INPUT_TABLET is not set
# CONFIG_INPUT_TOUCHSCREEN is not set
CONFIG_INPUT_MISC=y
CONFIG_INPUT_AD714X=m
CONFIG_INPUT_AD714X_I2C=m
CONFIG_INPUT_AD714X_SPI=m
CONFIG_INPUT_ATMEL_CAPTOUCH=m
CONFIG_INPUT_BMA150=m
CONFIG_INPUT_E3X0_BUTTON=m
CONFIG_INPUT_PCSPKR=m
CONFIG_INPUT_MAX77693_HAPTIC=m
CONFIG_INPUT_MC13783_PWRBUTTON=m
CONFIG_INPUT_MMA8450=m
CONFIG_INPUT_APANEL=m
CONFIG_INPUT_CPCAP_PWRBUTTON=m
# CONFIG_INPUT_ATLAS_BTNS is not set
# CONFIG_INPUT_ATI_REMOTE2 is not set
# CONFIG_INPUT_KEYSPAN_REMOTE is not set
CONFIG_INPUT_KXTJ9=m
CONFIG_INPUT_KXTJ9_POLLED_MODE=y
# CONFIG_INPUT_POWERMATE is not set
# CONFIG_INPUT_YEALINK is not set
# CONFIG_INPUT_CM109 is not set
CONFIG_INPUT_REGULATOR_HAPTIC=m
# CONFIG_INPUT_RETU_PWRBUTTON is not set
CONFIG_INPUT_TPS65218_PWRBUTTON=m
CONFIG_INPUT_UINPUT=m
CONFIG_INPUT_PCF50633_PMU=m
CONFIG_INPUT_PCF8574=m
CONFIG_INPUT_PWM_BEEPER=m
CONFIG_INPUT_PWM_VIBRA=m
# CONFIG_INPUT_DA9063_ONKEY is not set
CONFIG_INPUT_ADXL34X=m
# CONFIG_INPUT_ADXL34X_I2C is not set
# CONFIG_INPUT_ADXL34X_SPI is not set
CONFIG_INPUT_CMA3000=m
# CONFIG_INPUT_CMA3000_I2C is not set
CONFIG_INPUT_IDEAPAD_SLIDEBAR=m
CONFIG_INPUT_DRV2665_HAPTICS=m
CONFIG_INPUT_DRV2667_HAPTICS=m
CONFIG_INPUT_RAVE_SP_PWRBUTTON=m
CONFIG_RMI4_CORE=m
CONFIG_RMI4_I2C=m
CONFIG_RMI4_SPI=m
CONFIG_RMI4_SMB=m
CONFIG_RMI4_F03=y
CONFIG_RMI4_F03_SERIO=m
CONFIG_RMI4_2D_SENSOR=y
CONFIG_RMI4_F11=y
CONFIG_RMI4_F12=y
CONFIG_RMI4_F30=y
CONFIG_RMI4_F34=y
CONFIG_RMI4_F55=y

#
# Hardware I/O ports
#
CONFIG_SERIO=y
CONFIG_ARCH_MIGHT_HAVE_PC_SERIO=y
CONFIG_SERIO_I8042=y
CONFIG_SERIO_SERPORT=y
CONFIG_SERIO_CT82C710=m
# CONFIG_SERIO_PARKBD is not set
# CONFIG_SERIO_PCIPS2 is not set
CONFIG_SERIO_LIBPS2=y
CONFIG_SERIO_RAW=m
CONFIG_SERIO_ALTERA_PS2=m
CONFIG_SERIO_PS2MULT=m
CONFIG_SERIO_ARC_PS2=m
# CONFIG_SERIO_APBPS2 is not set
CONFIG_USERIO=m
CONFIG_GAMEPORT=m
# CONFIG_GAMEPORT_NS558 is not set
# CONFIG_GAMEPORT_L4 is not set
# CONFIG_GAMEPORT_EMU10K1 is not set
# CONFIG_GAMEPORT_FM801 is not set

#
# Character devices
#
CONFIG_TTY=y
# CONFIG_VT is not set
CONFIG_UNIX98_PTYS=y
CONFIG_LEGACY_PTYS=y
CONFIG_LEGACY_PTY_COUNT=256
# CONFIG_SERIAL_NONSTANDARD is not set
# CONFIG_NOZOMI is not set
# CONFIG_N_GSM is not set
# CONFIG_TRACE_SINK is not set
CONFIG_DEVMEM=y
CONFIG_DEVKMEM=y

#
# Serial drivers
#
CONFIG_SERIAL_EARLYCON=y
CONFIG_SERIAL_8250=y
CONFIG_SERIAL_8250_DEPRECATED_OPTIONS=y
CONFIG_SERIAL_8250_PNP=y
# CONFIG_SERIAL_8250_FINTEK is not set
CONFIG_SERIAL_8250_CONSOLE=y
CONFIG_SERIAL_8250_DMA=y
CONFIG_SERIAL_8250_PCI=y
CONFIG_SERIAL_8250_EXAR=y
# CONFIG_SERIAL_8250_CS is not set
CONFIG_SERIAL_8250_NR_UARTS=4
CONFIG_SERIAL_8250_RUNTIME_UARTS=4
# CONFIG_SERIAL_8250_EXTENDED is not set
# CONFIG_SERIAL_8250_ASPEED_VUART is not set
# CONFIG_SERIAL_8250_DW is not set
# CONFIG_SERIAL_8250_RT288X is not set
CONFIG_SERIAL_8250_LPSS=y
CONFIG_SERIAL_8250_MID=y
# CONFIG_SERIAL_8250_MOXA is not set
# CONFIG_SERIAL_OF_PLATFORM is not set

#
# Non-8250 serial port support
#
# CONFIG_SERIAL_MAX3100 is not set
# CONFIG_SERIAL_MAX310X is not set
# CONFIG_SERIAL_UARTLITE is not set
CONFIG_SERIAL_CORE=y
CONFIG_SERIAL_CORE_CONSOLE=y
# CONFIG_SERIAL_JSM is not set
# CONFIG_SERIAL_SCCNXP is not set
# CONFIG_SERIAL_SC16IS7XX is not set
# CONFIG_SERIAL_ALTERA_JTAGUART is not set
# CONFIG_SERIAL_ALTERA_UART is not set
# CONFIG_SERIAL_XILINX_PS_UART is not set
# CONFIG_SERIAL_ARC is not set
# CONFIG_SERIAL_RP2 is not set
# CONFIG_SERIAL_FSL_LPUART is not set
# CONFIG_SERIAL_CONEXANT_DIGICOLOR is not set
CONFIG_SERIAL_DEV_BUS=y
CONFIG_SERIAL_DEV_CTRL_TTYPORT=y
# CONFIG_TTY_PRINTK is not set
# CONFIG_PRINTER is not set
CONFIG_PPDEV=m
# CONFIG_VIRTIO_CONSOLE is not set
# CONFIG_IPMI_HANDLER is not set
# CONFIG_HW_RANDOM is not set
# CONFIG_NVRAM is not set
# CONFIG_R3964 is not set
# CONFIG_APPLICOM is not set

#
# PCMCIA character devices
#
# CONFIG_SYNCLINK_CS is not set
# CONFIG_CARDMAN_4000 is not set
# CONFIG_CARDMAN_4040 is not set
# CONFIG_SCR24X is not set
# CONFIG_IPWIRELESS is not set
# CONFIG_MWAVE is not set
CONFIG_RAW_DRIVER=y
CONFIG_MAX_RAW_DEVS=256
# CONFIG_HPET is not set
CONFIG_HANGCHECK_TIMER=y
# CONFIG_TCG_TPM is not set
# CONFIG_TELCLOCK is not set
CONFIG_DEVPORT=y
# CONFIG_XILLYBUS is not set

#
# I2C support
#
CONFIG_I2C=m
CONFIG_I2C_BOARDINFO=y
CONFIG_I2C_COMPAT=y
# CONFIG_I2C_CHARDEV is not set
CONFIG_I2C_MUX=m

#
# Multiplexer I2C Chip support
#
CONFIG_I2C_MUX_GPMUX=m
# CONFIG_I2C_MUX_LTC4306 is not set
CONFIG_I2C_MUX_PCA9541=m
# CONFIG_I2C_MUX_REG is not set
CONFIG_I2C_MUX_MLXCPLD=m
CONFIG_I2C_HELPER_AUTO=y
CONFIG_I2C_SMBUS=m
CONFIG_I2C_ALGOBIT=m
CONFIG_I2C_ALGOPCA=m

#
# I2C Hardware Bus support
#

#
# PC SMBus host controller drivers
#
# CONFIG_I2C_ALI1535 is not set
# CONFIG_I2C_ALI1563 is not set
# CONFIG_I2C_ALI15X3 is not set
# CONFIG_I2C_AMD756 is not set
# CONFIG_I2C_AMD8111 is not set
# CONFIG_I2C_I801 is not set
# CONFIG_I2C_ISCH is not set
# CONFIG_I2C_ISMT is not set
# CONFIG_I2C_PIIX4 is not set
# CONFIG_I2C_NFORCE2 is not set
# CONFIG_I2C_SIS5595 is not set
# CONFIG_I2C_SIS630 is not set
# CONFIG_I2C_SIS96X is not set
# CONFIG_I2C_VIA is not set
# CONFIG_I2C_VIAPRO is not set

#
# ACPI drivers
#
# CONFIG_I2C_SCMI is not set

#
# I2C system bus drivers (mostly embedded / system-on-chip)
#
# CONFIG_I2C_DESIGNWARE_PLATFORM is not set
# CONFIG_I2C_DESIGNWARE_PCI is not set
# CONFIG_I2C_EMEV2 is not set
CONFIG_I2C_KEMPLD=m
CONFIG_I2C_OCORES=m
CONFIG_I2C_PCA_PLATFORM=m
# CONFIG_I2C_RK3X is not set
CONFIG_I2C_SIMTEC=m
CONFIG_I2C_XILINX=m

#
# External I2C/SMBus adapter drivers
#
CONFIG_I2C_PARPORT=m
CONFIG_I2C_PARPORT_LIGHT=m
# CONFIG_I2C_TAOS_EVM is not set

#
# Other I2C/SMBus bus drivers
#
CONFIG_I2C_MLXCPLD=m
# CONFIG_I2C_STUB is not set
CONFIG_I2C_SLAVE=y
CONFIG_I2C_SLAVE_EEPROM=m
# CONFIG_I2C_DEBUG_CORE is not set
# CONFIG_I2C_DEBUG_ALGO is not set
# CONFIG_I2C_DEBUG_BUS is not set
CONFIG_SPI=y
# CONFIG_SPI_DEBUG is not set
CONFIG_SPI_MASTER=y

#
# SPI Master Controller Drivers
#
CONFIG_SPI_ALTERA=y
CONFIG_SPI_AXI_SPI_ENGINE=y
CONFIG_SPI_BITBANG=y
CONFIG_SPI_BUTTERFLY=m
# CONFIG_SPI_CADENCE is not set
CONFIG_SPI_DESIGNWARE=m
# CONFIG_SPI_DW_PCI is not set
# CONFIG_SPI_DW_MMIO is not set
CONFIG_SPI_LM70_LLP=y
# CONFIG_SPI_FSL_SPI is not set
# CONFIG_SPI_PXA2XX is not set
# CONFIG_SPI_ROCKCHIP is not set
CONFIG_SPI_SC18IS602=m
# CONFIG_SPI_XCOMM is not set
# CONFIG_SPI_XILINX is not set
# CONFIG_SPI_ZYNQMP_GQSPI is not set

#
# SPI Protocol Masters
#
CONFIG_SPI_SPIDEV=y
CONFIG_SPI_LOOPBACK_TEST=m
CONFIG_SPI_TLE62X0=m
CONFIG_SPI_SLAVE=y
# CONFIG_SPI_SLAVE_TIME is not set
# CONFIG_SPI_SLAVE_SYSTEM_CONTROL is not set
CONFIG_SPMI=m
# CONFIG_HSI is not set
CONFIG_PPS=y
# CONFIG_PPS_DEBUG is not set

#
# PPS clients support
#
# CONFIG_PPS_CLIENT_KTIMER is not set
# CONFIG_PPS_CLIENT_LDISC is not set
# CONFIG_PPS_CLIENT_PARPORT is not set
# CONFIG_PPS_CLIENT_GPIO is not set

#
# PPS generators support
#

#
# PTP clock support
#
CONFIG_PTP_1588_CLOCK=y

#
# Enable PHYLIB and NETWORK_PHY_TIMESTAMPING to see the additional clocks.
#
CONFIG_PTP_1588_CLOCK_KVM=y
# CONFIG_PINCTRL is not set
# CONFIG_GPIOLIB is not set
CONFIG_W1=y

#
# 1-wire Bus Masters
#
# CONFIG_W1_MASTER_MATROX is not set
# CONFIG_W1_MASTER_DS2482 is not set
CONFIG_W1_MASTER_DS1WM=y

#
# 1-wire Slaves
#
# CONFIG_W1_SLAVE_THERM is not set
CONFIG_W1_SLAVE_SMEM=m
CONFIG_W1_SLAVE_DS2405=m
CONFIG_W1_SLAVE_DS2408=m
# CONFIG_W1_SLAVE_DS2408_READBACK is not set
CONFIG_W1_SLAVE_DS2413=m
CONFIG_W1_SLAVE_DS2406=m
CONFIG_W1_SLAVE_DS2423=y
CONFIG_W1_SLAVE_DS2805=y
# CONFIG_W1_SLAVE_DS2431 is not set
# CONFIG_W1_SLAVE_DS2433 is not set
CONFIG_W1_SLAVE_DS2438=m
CONFIG_W1_SLAVE_DS2760=m
CONFIG_W1_SLAVE_DS2780=y
CONFIG_W1_SLAVE_DS2781=y
# CONFIG_W1_SLAVE_DS28E04 is not set
CONFIG_W1_SLAVE_DS28E17=m
# CONFIG_POWER_AVS is not set
CONFIG_POWER_RESET=y
# CONFIG_POWER_RESET_RESTART is not set
# CONFIG_POWER_RESET_SYSCON is not set
CONFIG_POWER_RESET_SYSCON_POWEROFF=y
CONFIG_REBOOT_MODE=y
CONFIG_SYSCON_REBOOT_MODE=y
CONFIG_POWER_SUPPLY=y
CONFIG_POWER_SUPPLY_DEBUG=y
# CONFIG_PDA_POWER is not set
# CONFIG_TEST_POWER is not set
# CONFIG_BATTERY_ACT8945A is not set
CONFIG_BATTERY_DS2760=m
CONFIG_BATTERY_DS2780=y
CONFIG_BATTERY_DS2781=y
CONFIG_BATTERY_DS2782=m
# CONFIG_BATTERY_SBS is not set
CONFIG_CHARGER_SBS=m
CONFIG_BATTERY_BQ27XXX=m
CONFIG_BATTERY_BQ27XXX_I2C=m
CONFIG_BATTERY_BQ27XXX_HDQ=m
# CONFIG_BATTERY_BQ27XXX_DT_UPDATES_NVM is not set
CONFIG_BATTERY_DA9150=m
CONFIG_BATTERY_MAX17040=m
CONFIG_BATTERY_MAX17042=m
CONFIG_BATTERY_MAX1721X=m
CONFIG_CHARGER_PCF50633=m
CONFIG_CHARGER_MAX8903=y
# CONFIG_CHARGER_LP8727 is not set
# CONFIG_CHARGER_MANAGER is not set
CONFIG_CHARGER_MAX14577=m
# CONFIG_CHARGER_DETECTOR_MAX14656 is not set
CONFIG_CHARGER_MAX77693=m
CONFIG_CHARGER_BQ2415X=m
CONFIG_CHARGER_SMB347=m
# CONFIG_CHARGER_TPS65217 is not set
# CONFIG_BATTERY_GAUGE_LTC2941 is not set
# CONFIG_HWMON is not set
CONFIG_THERMAL=y
# CONFIG_THERMAL_STATISTICS is not set
CONFIG_THERMAL_EMERGENCY_POWEROFF_DELAY_MS=0
CONFIG_THERMAL_OF=y
# CONFIG_THERMAL_WRITABLE_TRIPS is not set
CONFIG_THERMAL_DEFAULT_GOV_STEP_WISE=y
# CONFIG_THERMAL_DEFAULT_GOV_FAIR_SHARE is not set
# CONFIG_THERMAL_DEFAULT_GOV_USER_SPACE is not set
# CONFIG_THERMAL_DEFAULT_GOV_POWER_ALLOCATOR is not set
# CONFIG_THERMAL_GOV_FAIR_SHARE is not set
CONFIG_THERMAL_GOV_STEP_WISE=y
# CONFIG_THERMAL_GOV_BANG_BANG is not set
# CONFIG_THERMAL_GOV_USER_SPACE is not set
# CONFIG_THERMAL_GOV_POWER_ALLOCATOR is not set
# CONFIG_CLOCK_THERMAL is not set
# CONFIG_DEVFREQ_THERMAL is not set
# CONFIG_THERMAL_EMULATION is not set
# CONFIG_QORIQ_THERMAL is not set
# CONFIG_DA9062_THERMAL is not set
# CONFIG_INTEL_POWERCLAMP is not set
# CONFIG_INTEL_SOC_DTS_THERMAL is not set

#
# ACPI INT340X thermal drivers
#
# CONFIG_INT340X_THERMAL is not set
# CONFIG_INTEL_PCH_THERMAL is not set
# CONFIG_WATCHDOG is not set
CONFIG_SSB_POSSIBLE=y
CONFIG_SSB=y
CONFIG_SSB_SPROM=y
CONFIG_SSB_PCIHOST_POSSIBLE=y
CONFIG_SSB_PCIHOST=y
CONFIG_SSB_PCMCIAHOST_POSSIBLE=y
CONFIG_SSB_PCMCIAHOST=y
# CONFIG_SSB_SILENT is not set
CONFIG_SSB_DEBUG=y
CONFIG_SSB_DRIVER_PCICORE_POSSIBLE=y
# CONFIG_SSB_DRIVER_PCICORE is not set
CONFIG_BCMA_POSSIBLE=y
CONFIG_BCMA=m
CONFIG_BCMA_HOST_PCI_POSSIBLE=y
CONFIG_BCMA_HOST_PCI=y
CONFIG_BCMA_HOST_SOC=y
CONFIG_BCMA_DRIVER_PCI=y
# CONFIG_BCMA_SFLASH is not set
CONFIG_BCMA_DRIVER_GMAC_CMN=y
# CONFIG_BCMA_DEBUG is not set

#
# Multifunction device drivers
#
CONFIG_MFD_CORE=y
CONFIG_MFD_ACT8945A=m
# CONFIG_MFD_ATMEL_FLEXCOM is not set
CONFIG_MFD_ATMEL_HLCDC=m
# CONFIG_MFD_BCM590XX is not set
CONFIG_MFD_BD9571MWV=m
# CONFIG_MFD_AXP20X_I2C is not set
# CONFIG_MFD_CROS_EC is not set
# CONFIG_MFD_DA9052_SPI is not set
CONFIG_MFD_DA9062=m
CONFIG_MFD_DA9063=m
CONFIG_MFD_DA9150=m
CONFIG_MFD_MC13XXX=m
# CONFIG_MFD_MC13XXX_SPI is not set
CONFIG_MFD_MC13XXX_I2C=m
# CONFIG_MFD_HI6421_PMIC is not set
CONFIG_HTC_PASIC3=m
# CONFIG_MFD_INTEL_QUARK_I2C_GPIO is not set
# CONFIG_LPC_ICH is not set
# CONFIG_LPC_SCH is not set
# CONFIG_MFD_INTEL_LPSS_ACPI is not set
# CONFIG_MFD_INTEL_LPSS_PCI is not set
# CONFIG_MFD_JANZ_CMODIO is not set
CONFIG_MFD_KEMPLD=m
# CONFIG_MFD_88PM800 is not set
# CONFIG_MFD_88PM805 is not set
CONFIG_MFD_MAX14577=m
# CONFIG_MFD_MAX77686 is not set
CONFIG_MFD_MAX77693=m
CONFIG_MFD_MAX8907=m
# CONFIG_MFD_MT6397 is not set
CONFIG_MFD_MENF21BMC=m
# CONFIG_EZX_PCAP is not set
CONFIG_MFD_CPCAP=m
CONFIG_MFD_RETU=m
CONFIG_MFD_PCF50633=m
CONFIG_PCF50633_ADC=m
CONFIG_PCF50633_GPIO=m
# CONFIG_MFD_RDC321X is not set
# CONFIG_MFD_RT5033 is not set
# CONFIG_MFD_RK808 is not set
CONFIG_MFD_RN5T618=m
CONFIG_MFD_SI476X_CORE=m
# CONFIG_MFD_SM501 is not set
CONFIG_MFD_SKY81452=m
# CONFIG_ABX500_CORE is not set
# CONFIG_MFD_STMPE is not set
CONFIG_MFD_SYSCON=y
CONFIG_MFD_TI_AM335X_TSCADC=m
# CONFIG_MFD_LP3943 is not set
CONFIG_MFD_TI_LMU=m
# CONFIG_TPS6105X is not set
CONFIG_TPS6507X=m
CONFIG_MFD_TPS65086=m
CONFIG_MFD_TPS65217=m
# CONFIG_MFD_TI_LP873X is not set
CONFIG_MFD_TI_LP87565=m
CONFIG_MFD_TPS65218=m
CONFIG_MFD_TPS65912=m
# CONFIG_MFD_TPS65912_I2C is not set
CONFIG_MFD_TPS65912_SPI=m
# CONFIG_MFD_WL1273_CORE is not set
# CONFIG_MFD_LM3533 is not set
# CONFIG_MFD_VX855 is not set
CONFIG_MFD_ARIZONA=y
# CONFIG_MFD_ARIZONA_I2C is not set
CONFIG_MFD_ARIZONA_SPI=m
CONFIG_MFD_CS47L24=y
# CONFIG_MFD_WM5102 is not set
# CONFIG_MFD_WM5110 is not set
# CONFIG_MFD_WM8997 is not set
CONFIG_MFD_WM8998=y
# CONFIG_MFD_WM831X_SPI is not set
CONFIG_MFD_WM8994=m
CONFIG_RAVE_SP_CORE=y
CONFIG_REGULATOR=y
# CONFIG_REGULATOR_DEBUG is not set
CONFIG_REGULATOR_FIXED_VOLTAGE=y
CONFIG_REGULATOR_VIRTUAL_CONSUMER=m
CONFIG_REGULATOR_USERSPACE_CONSUMER=m
# CONFIG_REGULATOR_88PG86X is not set
CONFIG_REGULATOR_ACT8865=m
CONFIG_REGULATOR_ACT8945A=m
CONFIG_REGULATOR_AD5398=m
CONFIG_REGULATOR_ANATOP=m
CONFIG_REGULATOR_BD9571MWV=m
CONFIG_REGULATOR_CPCAP=m
# CONFIG_REGULATOR_DA9062 is not set
# CONFIG_REGULATOR_DA9063 is not set
CONFIG_REGULATOR_DA9210=m
CONFIG_REGULATOR_DA9211=m
# CONFIG_REGULATOR_FAN53555 is not set
CONFIG_REGULATOR_ISL9305=m
CONFIG_REGULATOR_ISL6271A=m
# CONFIG_REGULATOR_LM363X is not set
# CONFIG_REGULATOR_LP3971 is not set
CONFIG_REGULATOR_LP3972=m
# CONFIG_REGULATOR_LP872X is not set
# CONFIG_REGULATOR_LP8755 is not set
CONFIG_REGULATOR_LP87565=m
CONFIG_REGULATOR_LTC3589=m
# CONFIG_REGULATOR_LTC3676 is not set
# CONFIG_REGULATOR_MAX14577 is not set
CONFIG_REGULATOR_MAX1586=m
CONFIG_REGULATOR_MAX8649=m
# CONFIG_REGULATOR_MAX8660 is not set
CONFIG_REGULATOR_MAX8907=m
CONFIG_REGULATOR_MAX8952=m
# CONFIG_REGULATOR_MAX8973 is not set
# CONFIG_REGULATOR_MAX77693 is not set
CONFIG_REGULATOR_MC13XXX_CORE=m
# CONFIG_REGULATOR_MC13783 is not set
CONFIG_REGULATOR_MC13892=m
CONFIG_REGULATOR_MT6311=m
CONFIG_REGULATOR_PCF50633=m
# CONFIG_REGULATOR_PFUZE100 is not set
CONFIG_REGULATOR_PV88060=m
CONFIG_REGULATOR_PV88080=m
CONFIG_REGULATOR_PV88090=m
CONFIG_REGULATOR_PWM=y
CONFIG_REGULATOR_QCOM_SPMI=m
CONFIG_REGULATOR_RN5T618=m
# CONFIG_REGULATOR_SKY81452 is not set
# CONFIG_REGULATOR_TPS51632 is not set
# CONFIG_REGULATOR_TPS62360 is not set
CONFIG_REGULATOR_TPS65023=m
CONFIG_REGULATOR_TPS6507X=m
CONFIG_REGULATOR_TPS65086=m
# CONFIG_REGULATOR_TPS65217 is not set
CONFIG_REGULATOR_TPS65218=m
CONFIG_REGULATOR_TPS6524X=y
CONFIG_REGULATOR_TPS65912=m
CONFIG_REGULATOR_VCTRL=m
CONFIG_REGULATOR_WM8994=m
CONFIG_CEC_CORE=m
CONFIG_CEC_NOTIFIER=y
CONFIG_RC_CORE=m
# CONFIG_RC_MAP is not set
CONFIG_LIRC=y
# CONFIG_RC_DECODERS is not set
# CONFIG_RC_DEVICES is not set
CONFIG_MEDIA_SUPPORT=m

#
# Multimedia core support
#
# CONFIG_MEDIA_CAMERA_SUPPORT is not set
# CONFIG_MEDIA_ANALOG_TV_SUPPORT is not set
# CONFIG_MEDIA_DIGITAL_TV_SUPPORT is not set
# CONFIG_MEDIA_RADIO_SUPPORT is not set
# CONFIG_MEDIA_SDR_SUPPORT is not set
CONFIG_MEDIA_CEC_SUPPORT=y
CONFIG_MEDIA_CEC_RC=y
CONFIG_VIDEO_ADV_DEBUG=y
CONFIG_VIDEO_FIXED_MINOR_RANGES=y

#
# Media drivers
#
# CONFIG_MEDIA_PCI_SUPPORT is not set
CONFIG_CEC_PLATFORM_DRIVERS=y
# CONFIG_CEC_GPIO is not set

#
# Supported MMC/SDIO adapters
#

#
# Media ancillary drivers (tuners, sensors, i2c, spi, frontends)
#

#
# Media SPI Adapters
#

#
# Customise DVB Frontends
#

#
# Tools to develop new frontends
#

#
# Graphics support
#
# CONFIG_AGP is not set
CONFIG_VGA_ARB=y
CONFIG_VGA_ARB_MAX_GPUS=16
# CONFIG_VGA_SWITCHEROO is not set
CONFIG_DRM=m
CONFIG_DRM_MIPI_DSI=y
CONFIG_DRM_DP_AUX_CHARDEV=y
CONFIG_DRM_DEBUG_MM_SELFTEST=m
CONFIG_DRM_KMS_HELPER=m
CONFIG_DRM_KMS_FB_HELPER=y
# CONFIG_DRM_FBDEV_EMULATION is not set
# CONFIG_DRM_LOAD_EDID_FIRMWARE is not set
CONFIG_DRM_TTM=m
CONFIG_DRM_GEM_CMA_HELPER=y
CONFIG_DRM_KMS_CMA_HELPER=y
CONFIG_DRM_VM=y

#
# I2C encoder or helper chips
#
CONFIG_DRM_I2C_CH7006=m
# CONFIG_DRM_I2C_SIL164 is not set
# CONFIG_DRM_I2C_NXP_TDA998X is not set
# CONFIG_DRM_RADEON is not set
# CONFIG_DRM_AMDGPU is not set

#
# ACP (Audio CoProcessor) Configuration
#

#
# AMD Library routines
#
# CONFIG_DRM_NOUVEAU is not set
# CONFIG_DRM_I915 is not set
CONFIG_DRM_VGEM=m
# CONFIG_DRM_VMWGFX is not set
# CONFIG_DRM_GMA500 is not set
# CONFIG_DRM_UDL is not set
# CONFIG_DRM_AST is not set
# CONFIG_DRM_MGAG200 is not set
# CONFIG_DRM_CIRRUS_QEMU is not set
CONFIG_DRM_RCAR_DW_HDMI=m
# CONFIG_DRM_RCAR_LVDS is not set
# CONFIG_DRM_QXL is not set
# CONFIG_DRM_BOCHS is not set
CONFIG_DRM_VIRTIO_GPU=m
CONFIG_DRM_PANEL=y

#
# Display Panels
#
# CONFIG_DRM_PANEL_ARM_VERSATILE is not set
# CONFIG_DRM_PANEL_LVDS is not set
CONFIG_DRM_PANEL_SIMPLE=m
CONFIG_DRM_PANEL_ILITEK_IL9322=m
CONFIG_DRM_PANEL_INNOLUX_P079ZCA=m
CONFIG_DRM_PANEL_JDI_LT070ME05000=m
# CONFIG_DRM_PANEL_SAMSUNG_LD9040 is not set
# CONFIG_DRM_PANEL_LG_LG4573 is not set
CONFIG_DRM_PANEL_ORISETECH_OTM8009A=m
# CONFIG_DRM_PANEL_PANASONIC_VVX10F034N00 is not set
CONFIG_DRM_PANEL_RASPBERRYPI_TOUCHSCREEN=m
# CONFIG_DRM_PANEL_RAYDIUM_RM68200 is not set
# CONFIG_DRM_PANEL_SAMSUNG_S6E3HA2 is not set
CONFIG_DRM_PANEL_SAMSUNG_S6E63J0X03=m
CONFIG_DRM_PANEL_SAMSUNG_S6E8AA0=m
CONFIG_DRM_PANEL_SEIKO_43WVF1G=m
CONFIG_DRM_PANEL_SHARP_LQ101R1SX01=m
# CONFIG_DRM_PANEL_SHARP_LS043T1LE01 is not set
# CONFIG_DRM_PANEL_SITRONIX_ST7789V is not set
CONFIG_DRM_BRIDGE=y
CONFIG_DRM_PANEL_BRIDGE=y

#
# Display Interface Bridges
#
CONFIG_DRM_ANALOGIX_ANX78XX=m
CONFIG_DRM_DUMB_VGA_DAC=m
CONFIG_DRM_LVDS_ENCODER=m
# CONFIG_DRM_MEGACHIPS_STDPXXXX_GE_B850V3_FW is not set
# CONFIG_DRM_NXP_PTN3460 is not set
CONFIG_DRM_PARADE_PS8622=m
CONFIG_DRM_SIL_SII8620=m
CONFIG_DRM_SII902X=m
CONFIG_DRM_SII9234=m
CONFIG_DRM_TOSHIBA_TC358767=m
CONFIG_DRM_TI_TFP410=m
# CONFIG_DRM_I2C_ADV7511 is not set
CONFIG_DRM_DW_HDMI=m
CONFIG_DRM_DW_HDMI_CEC=m
CONFIG_DRM_ARCPGU=m
# CONFIG_DRM_HISI_HIBMC is not set
# CONFIG_DRM_MXSFB is not set
CONFIG_DRM_TINYDRM=m
CONFIG_TINYDRM_MIPI_DBI=m
# CONFIG_TINYDRM_ILI9225 is not set
CONFIG_TINYDRM_MI0283QT=m
# CONFIG_TINYDRM_REPAPER is not set
# CONFIG_TINYDRM_ST7586 is not set
CONFIG_TINYDRM_ST7735R=m
CONFIG_DRM_LEGACY=y
# CONFIG_DRM_TDFX is not set
# CONFIG_DRM_R128 is not set
# CONFIG_DRM_MGA is not set
# CONFIG_DRM_VIA is not set
# CONFIG_DRM_SAVAGE is not set
CONFIG_DRM_PANEL_ORIENTATION_QUIRKS=m
CONFIG_DRM_LIB_RANDOM=y

#
# Frame buffer Devices
#
CONFIG_FB=m
# CONFIG_FIRMWARE_EDID is not set
CONFIG_FB_CMDLINE=y
CONFIG_FB_NOTIFY=y
CONFIG_FB_CFB_FILLRECT=m
CONFIG_FB_CFB_COPYAREA=m
CONFIG_FB_CFB_IMAGEBLIT=m
CONFIG_FB_SYS_FILLRECT=m
CONFIG_FB_SYS_COPYAREA=m
CONFIG_FB_SYS_IMAGEBLIT=m
# CONFIG_FB_FOREIGN_ENDIAN is not set
CONFIG_FB_SYS_FOPS=m
CONFIG_FB_DEFERRED_IO=y
CONFIG_FB_MODE_HELPERS=y
CONFIG_FB_TILEBLITTING=y

#
# Frame buffer hardware drivers
#
# CONFIG_FB_CIRRUS is not set
# CONFIG_FB_PM2 is not set
# CONFIG_FB_CYBER2000 is not set
CONFIG_FB_ARC=m
# CONFIG_FB_VGA16 is not set
# CONFIG_FB_N411 is not set
# CONFIG_FB_HGA is not set
CONFIG_FB_OPENCORES=m
# CONFIG_FB_S1D13XXX is not set
# CONFIG_FB_NVIDIA is not set
# CONFIG_FB_RIVA is not set
# CONFIG_FB_I740 is not set
# CONFIG_FB_LE80578 is not set
# CONFIG_FB_MATROX is not set
# CONFIG_FB_RADEON is not set
# CONFIG_FB_ATY128 is not set
# CONFIG_FB_ATY is not set
# CONFIG_FB_S3 is not set
# CONFIG_FB_SAVAGE is not set
# CONFIG_FB_SIS is not set
# CONFIG_FB_NEOMAGIC is not set
# CONFIG_FB_KYRO is not set
# CONFIG_FB_3DFX is not set
# CONFIG_FB_VOODOO1 is not set
# CONFIG_FB_VT8623 is not set
# CONFIG_FB_TRIDENT is not set
# CONFIG_FB_ARK is not set
# CONFIG_FB_PM3 is not set
# CONFIG_FB_CARMINE is not set
# CONFIG_FB_IBM_GXT4500 is not set
CONFIG_FB_VIRTUAL=m
CONFIG_FB_METRONOME=m
# CONFIG_FB_MB862XX is not set
CONFIG_FB_BROADSHEET=m
CONFIG_FB_AUO_K190X=m
CONFIG_FB_AUO_K1900=m
CONFIG_FB_AUO_K1901=m
# CONFIG_FB_SM712 is not set
CONFIG_BACKLIGHT_LCD_SUPPORT=y
# CONFIG_LCD_CLASS_DEVICE is not set
CONFIG_BACKLIGHT_CLASS_DEVICE=y
# CONFIG_BACKLIGHT_GENERIC is not set
# CONFIG_BACKLIGHT_PWM is not set
# CONFIG_BACKLIGHT_APPLE is not set
# CONFIG_BACKLIGHT_PM8941_WLED is not set
# CONFIG_BACKLIGHT_SAHARA is not set
CONFIG_BACKLIGHT_ADP8860=m
CONFIG_BACKLIGHT_ADP8870=m
CONFIG_BACKLIGHT_PCF50633=m
CONFIG_BACKLIGHT_LM3630A=m
# CONFIG_BACKLIGHT_LM3639 is not set
# CONFIG_BACKLIGHT_LP855X is not set
# CONFIG_BACKLIGHT_SKY81452 is not set
# CONFIG_BACKLIGHT_TPS65217 is not set
CONFIG_BACKLIGHT_LV5207LP=m
# CONFIG_BACKLIGHT_BD6107 is not set
CONFIG_BACKLIGHT_ARCXCNN=m
CONFIG_VIDEOMODE_HELPERS=y
CONFIG_HDMI=y
CONFIG_LOGO=y
CONFIG_LOGO_LINUX_MONO=y
CONFIG_LOGO_LINUX_VGA16=y
CONFIG_LOGO_LINUX_CLUT224=y
# CONFIG_SOUND is not set

#
# HID support
#
CONFIG_HID=m
# CONFIG_HID_BATTERY_STRENGTH is not set
CONFIG_HIDRAW=y
CONFIG_UHID=m
# CONFIG_HID_GENERIC is not set

#
# Special HID drivers
#
# CONFIG_HID_A4TECH is not set
CONFIG_HID_ACRUX=m
# CONFIG_HID_ACRUX_FF is not set
# CONFIG_HID_APPLE is not set
CONFIG_HID_ASUS=m
CONFIG_HID_AUREAL=m
# CONFIG_HID_BELKIN is not set
CONFIG_HID_CHERRY=m
CONFIG_HID_CHICONY=m
# CONFIG_HID_CMEDIA is not set
CONFIG_HID_CYPRESS=m
CONFIG_HID_DRAGONRISE=m
CONFIG_DRAGONRISE_FF=y
CONFIG_HID_EMS_FF=m
# CONFIG_HID_ELECOM is not set
# CONFIG_HID_EZKEY is not set
CONFIG_HID_GEMBIRD=m
CONFIG_HID_GFRM=m
CONFIG_HID_KEYTOUCH=m
CONFIG_HID_KYE=m
# CONFIG_HID_WALTOP is not set
CONFIG_HID_GYRATION=m
CONFIG_HID_ICADE=m
# CONFIG_HID_ITE is not set
CONFIG_HID_JABRA=m
CONFIG_HID_TWINHAN=m
CONFIG_HID_KENSINGTON=m
CONFIG_HID_LCPOWER=m
CONFIG_HID_LED=m
CONFIG_HID_LENOVO=m
CONFIG_HID_LOGITECH=m
# CONFIG_HID_LOGITECH_DJ is not set
# CONFIG_HID_LOGITECH_HIDPP is not set
CONFIG_LOGITECH_FF=y
CONFIG_LOGIRUMBLEPAD2_FF=y
# CONFIG_LOGIG940_FF is not set
# CONFIG_LOGIWHEELS_FF is not set
CONFIG_HID_MAGICMOUSE=m
CONFIG_HID_MAYFLASH=m
CONFIG_HID_MICROSOFT=m
CONFIG_HID_MONTEREY=m
CONFIG_HID_MULTITOUCH=m
CONFIG_HID_NTI=m
CONFIG_HID_ORTEK=m
# CONFIG_HID_PANTHERLORD is not set
# CONFIG_HID_PETALYNX is not set
# CONFIG_HID_PICOLCD is not set
# CONFIG_HID_PLANTRONICS is not set
CONFIG_HID_PRIMAX=m
CONFIG_HID_SAITEK=m
# CONFIG_HID_SAMSUNG is not set
CONFIG_HID_SPEEDLINK=m
CONFIG_HID_STEELSERIES=m
CONFIG_HID_SUNPLUS=m
CONFIG_HID_RMI=m
# CONFIG_HID_GREENASIA is not set
CONFIG_HID_SMARTJOYPLUS=m
CONFIG_SMARTJOYPLUS_FF=y
CONFIG_HID_TIVO=m
CONFIG_HID_TOPSEED=m
# CONFIG_HID_THINGM is not set
CONFIG_HID_THRUSTMASTER=m
CONFIG_THRUSTMASTER_FF=y
CONFIG_HID_UDRAW_PS3=m
# CONFIG_HID_WIIMOTE is not set
CONFIG_HID_XINMO=m
CONFIG_HID_ZEROPLUS=m
# CONFIG_ZEROPLUS_FF is not set
# CONFIG_HID_ZYDACRON is not set
CONFIG_HID_SENSOR_HUB=m
CONFIG_HID_SENSOR_CUSTOM_SENSOR=m
# CONFIG_HID_ALPS is not set

#
# I2C HID support
#
# CONFIG_I2C_HID is not set

#
# Intel ISH HID support
#
# CONFIG_INTEL_ISH_HID is not set
CONFIG_USB_OHCI_LITTLE_ENDIAN=y
CONFIG_USB_SUPPORT=y
CONFIG_USB_ARCH_HAS_HCD=y
# CONFIG_USB is not set
CONFIG_USB_PCI=y

#
# USB port drivers
#

#
# USB Physical Layer drivers
#
# CONFIG_NOP_USB_XCEIV is not set
# CONFIG_TAHVO_USB is not set
# CONFIG_USB_GADGET is not set
# CONFIG_TYPEC is not set
# CONFIG_USB_ULPI_BUS is not set
CONFIG_UWB=y
# CONFIG_UWB_WHCI is not set
CONFIG_MMC=m
CONFIG_PWRSEQ_EMMC=m
# CONFIG_PWRSEQ_SIMPLE is not set
# CONFIG_MMC_BLOCK is not set
# CONFIG_SDIO_UART is not set
CONFIG_MMC_TEST=m

#
# MMC/SD/SDIO Host Controller Drivers
#
CONFIG_MMC_DEBUG=y
# CONFIG_MMC_SDHCI is not set
CONFIG_MMC_WBSD=m
# CONFIG_MMC_TIFM_SD is not set
CONFIG_MMC_SPI=m
# CONFIG_MMC_SDRICOH_CS is not set
# CONFIG_MMC_CB710 is not set
# CONFIG_MMC_VIA_SDMMC is not set
# CONFIG_MMC_USDHI6ROL0 is not set
CONFIG_MMC_CQHCI=m
# CONFIG_MMC_TOSHIBA_PCI is not set
CONFIG_MMC_MTK=m
CONFIG_MEMSTICK=m
# CONFIG_MEMSTICK_DEBUG is not set

#
# MemoryStick drivers
#
# CONFIG_MEMSTICK_UNSAFE_RESUME is not set
CONFIG_MSPRO_BLOCK=m
CONFIG_MS_BLOCK=m

#
# MemoryStick Host Controller Drivers
#
# CONFIG_MEMSTICK_TIFM_MS is not set
# CONFIG_MEMSTICK_JMICRON_38X is not set
# CONFIG_MEMSTICK_R592 is not set
CONFIG_NEW_LEDS=y
CONFIG_LEDS_CLASS=m
# CONFIG_LEDS_CLASS_FLASH is not set
# CONFIG_LEDS_BRIGHTNESS_HW_CHANGED is not set

#
# LED drivers
#
# CONFIG_LEDS_BCM6328 is not set
# CONFIG_LEDS_BCM6358 is not set
# CONFIG_LEDS_CPCAP is not set
CONFIG_LEDS_LM3530=m
CONFIG_LEDS_LM3642=m
CONFIG_LEDS_LM3692X=m
CONFIG_LEDS_PCA9532=m
CONFIG_LEDS_LP3944=m
CONFIG_LEDS_LP55XX_COMMON=m
CONFIG_LEDS_LP5521=m
# CONFIG_LEDS_LP5523 is not set
# CONFIG_LEDS_LP5562 is not set
# CONFIG_LEDS_LP8501 is not set
CONFIG_LEDS_LP8860=m
CONFIG_LEDS_PCA955X=m
CONFIG_LEDS_PCA963X=m
CONFIG_LEDS_DAC124S085=m
# CONFIG_LEDS_PWM is not set
CONFIG_LEDS_REGULATOR=m
# CONFIG_LEDS_BD2802 is not set
# CONFIG_LEDS_MC13783 is not set
CONFIG_LEDS_TCA6507=m
CONFIG_LEDS_TLC591XX=m
CONFIG_LEDS_LM355x=m
# CONFIG_LEDS_MENF21BMC is not set
# CONFIG_LEDS_IS31FL319X is not set
CONFIG_LEDS_IS31FL32XX=m

#
# LED driver for blink(1) USB RGB LED is under Special HID drivers (HID_THINGM)
#
# CONFIG_LEDS_BLINKM is not set
# CONFIG_LEDS_MLXREG is not set
CONFIG_LEDS_USER=m
# CONFIG_LEDS_NIC78BX is not set

#
# LED Triggers
#
# CONFIG_LEDS_TRIGGERS is not set
# CONFIG_ACCESSIBILITY is not set
# CONFIG_INFINIBAND is not set
CONFIG_EDAC_ATOMIC_SCRUB=y
CONFIG_EDAC_SUPPORT=y
CONFIG_RTC_LIB=y
CONFIG_RTC_MC146818_LIB=y
CONFIG_RTC_CLASS=y
# CONFIG_RTC_HCTOSYS is not set
CONFIG_RTC_SYSTOHC=y
CONFIG_RTC_SYSTOHC_DEVICE="rtc0"
# CONFIG_RTC_DEBUG is not set
# CONFIG_RTC_NVMEM is not set

#
# RTC interfaces
#
CONFIG_RTC_INTF_SYSFS=y
# CONFIG_RTC_INTF_PROC is not set
# CONFIG_RTC_INTF_DEV is not set
CONFIG_RTC_DRV_TEST=y

#
# I2C RTC drivers
#
CONFIG_RTC_DRV_ABB5ZES3=m
# CONFIG_RTC_DRV_ABX80X is not set
CONFIG_RTC_DRV_DS1307=m
CONFIG_RTC_DRV_DS1307_CENTURY=y
CONFIG_RTC_DRV_DS1374=m
# CONFIG_RTC_DRV_DS1374_WDT is not set
CONFIG_RTC_DRV_DS1672=m
CONFIG_RTC_DRV_HYM8563=m
CONFIG_RTC_DRV_MAX6900=m
CONFIG_RTC_DRV_MAX8907=m
CONFIG_RTC_DRV_RS5C372=m
# CONFIG_RTC_DRV_ISL1208 is not set
CONFIG_RTC_DRV_ISL12022=m
# CONFIG_RTC_DRV_ISL12026 is not set
CONFIG_RTC_DRV_X1205=m
CONFIG_RTC_DRV_PCF8523=m
CONFIG_RTC_DRV_PCF85063=m
CONFIG_RTC_DRV_PCF85363=m
CONFIG_RTC_DRV_PCF8563=m
# CONFIG_RTC_DRV_PCF8583 is not set
# CONFIG_RTC_DRV_M41T80 is not set
CONFIG_RTC_DRV_BQ32K=m
CONFIG_RTC_DRV_S35390A=m
CONFIG_RTC_DRV_FM3130=m
# CONFIG_RTC_DRV_RX8010 is not set
CONFIG_RTC_DRV_RX8581=m
# CONFIG_RTC_DRV_RX8025 is not set
CONFIG_RTC_DRV_EM3027=m
CONFIG_RTC_DRV_RV8803=m

#
# SPI RTC drivers
#
# CONFIG_RTC_DRV_M41T93 is not set
CONFIG_RTC_DRV_M41T94=y
# CONFIG_RTC_DRV_DS1302 is not set
CONFIG_RTC_DRV_DS1305=m
# CONFIG_RTC_DRV_DS1343 is not set
CONFIG_RTC_DRV_DS1347=y
CONFIG_RTC_DRV_DS1390=m
CONFIG_RTC_DRV_MAX6916=y
# CONFIG_RTC_DRV_R9701 is not set
CONFIG_RTC_DRV_RX4581=m
CONFIG_RTC_DRV_RX6110=m
# CONFIG_RTC_DRV_RS5C348 is not set
CONFIG_RTC_DRV_MAX6902=m
CONFIG_RTC_DRV_PCF2123=m
CONFIG_RTC_DRV_MCP795=m
CONFIG_RTC_I2C_AND_SPI=m

#
# SPI and I2C RTC drivers
#
CONFIG_RTC_DRV_DS3232=m
# CONFIG_RTC_DRV_PCF2127 is not set
CONFIG_RTC_DRV_RV3029C2=m

#
# Platform RTC drivers
#
# CONFIG_RTC_DRV_CMOS is not set
CONFIG_RTC_DRV_DS1286=m
# CONFIG_RTC_DRV_DS1511 is not set
CONFIG_RTC_DRV_DS1553=m
# CONFIG_RTC_DRV_DS1685_FAMILY is not set
CONFIG_RTC_DRV_DS1742=y
CONFIG_RTC_DRV_DS2404=y
CONFIG_RTC_DRV_DA9063=m
# CONFIG_RTC_DRV_STK17TA8 is not set
CONFIG_RTC_DRV_M48T86=m
# CONFIG_RTC_DRV_M48T35 is not set
# CONFIG_RTC_DRV_M48T59 is not set
# CONFIG_RTC_DRV_MSM6242 is not set
# CONFIG_RTC_DRV_BQ4802 is not set
CONFIG_RTC_DRV_RP5C01=y
CONFIG_RTC_DRV_V3020=y
# CONFIG_RTC_DRV_PCF50633 is not set
CONFIG_RTC_DRV_ZYNQMP=y

#
# on-CPU RTC drivers
#
# CONFIG_RTC_DRV_FTRTC010 is not set
CONFIG_RTC_DRV_MC13XXX=m
CONFIG_RTC_DRV_SNVS=y
# CONFIG_RTC_DRV_R7301 is not set
CONFIG_RTC_DRV_CPCAP=m

#
# HID Sensor RTC drivers
#
CONFIG_DMADEVICES=y
CONFIG_DMADEVICES_DEBUG=y
CONFIG_DMADEVICES_VDEBUG=y

#
# DMA Devices
#
CONFIG_DMA_ENGINE=y
CONFIG_DMA_VIRTUAL_CHANNELS=y
CONFIG_DMA_ACPI=y
CONFIG_DMA_OF=y
# CONFIG_ALTERA_MSGDMA is not set
# CONFIG_DW_AXI_DMAC is not set
CONFIG_FSL_EDMA=m
CONFIG_INTEL_IDMA64=y
# CONFIG_INTEL_IOATDMA is not set
# CONFIG_QCOM_HIDMA_MGMT is not set
CONFIG_QCOM_HIDMA=y
CONFIG_DW_DMAC_CORE=y
# CONFIG_DW_DMAC is not set
# CONFIG_DW_DMAC_PCI is not set
CONFIG_HSU_DMA=y

#
# DMA Clients
#
CONFIG_ASYNC_TX_DMA=y
CONFIG_DMATEST=y
CONFIG_DMA_ENGINE_RAID=y

#
# DMABUF options
#
CONFIG_SYNC_FILE=y
# CONFIG_SW_SYNC is not set
CONFIG_AUXDISPLAY=y
CONFIG_CHARLCD=m
CONFIG_KS0108=y
CONFIG_KS0108_PORT=0x378
CONFIG_KS0108_DELAY=2
# CONFIG_CFAG12864B is not set
CONFIG_IMG_ASCII_LCD=m
# CONFIG_HT16K33 is not set
CONFIG_PANEL=m
CONFIG_PANEL_PARPORT=0
CONFIG_PANEL_PROFILE=5
CONFIG_PANEL_CHANGE_MESSAGE=y
CONFIG_PANEL_BOOT_MESSAGE=""
CONFIG_UIO=y
# CONFIG_UIO_CIF is not set
CONFIG_UIO_PDRV_GENIRQ=y
# CONFIG_UIO_DMEM_GENIRQ is not set
# CONFIG_UIO_AEC is not set
# CONFIG_UIO_SERCOS3 is not set
# CONFIG_UIO_PCI_GENERIC is not set
# CONFIG_UIO_NETX is not set
CONFIG_UIO_PRUSS=m
# CONFIG_UIO_MF624 is not set
CONFIG_VIRT_DRIVERS=y
# CONFIG_VBOXGUEST is not set
CONFIG_VIRTIO=y
# CONFIG_VIRTIO_MENU is not set

#
# Microsoft Hyper-V guest support
#
# CONFIG_HYPERV is not set
# CONFIG_STAGING is not set
# CONFIG_X86_PLATFORM_DEVICES is not set
CONFIG_PMC_ATOM=y
# CONFIG_CHROME_PLATFORMS is not set
# CONFIG_MELLANOX_PLATFORM is not set
CONFIG_CLKDEV_LOOKUP=y
CONFIG_HAVE_CLK_PREPARE=y
CONFIG_COMMON_CLK=y

#
# Common Clock Framework
#
# CONFIG_CLK_HSDK is not set
# CONFIG_COMMON_CLK_SI5351 is not set
# CONFIG_COMMON_CLK_SI514 is not set
# CONFIG_COMMON_CLK_SI544 is not set
# CONFIG_COMMON_CLK_SI570 is not set
# CONFIG_COMMON_CLK_CDCE706 is not set
# CONFIG_COMMON_CLK_CDCE925 is not set
# CONFIG_COMMON_CLK_CS2000_CP is not set
# CONFIG_COMMON_CLK_PWM is not set
# CONFIG_COMMON_CLK_VC5 is not set
# CONFIG_HWSPINLOCK is not set

#
# Clock Source drivers
#
CONFIG_CLKEVT_I8253=y
CONFIG_I8253_LOCK=y
CONFIG_CLKBLD_I8253=y
# CONFIG_MAILBOX is not set
CONFIG_IOMMU_SUPPORT=y

#
# Generic IOMMU Pagetable Support
#
# CONFIG_AMD_IOMMU is not set

#
# Remoteproc drivers
#
CONFIG_REMOTEPROC=y

#
# Rpmsg drivers
#
CONFIG_RPMSG=m
# CONFIG_RPMSG_CHAR is not set
CONFIG_RPMSG_VIRTIO=m
CONFIG_SOUNDWIRE=y

#
# SoundWire Devices
#
# CONFIG_SOUNDWIRE_INTEL is not set

#
# SOC (System On Chip) specific Drivers
#

#
# Amlogic SoC drivers
#

#
# Broadcom SoC drivers
#

#
# i.MX SoC drivers
#

#
# Qualcomm SoC drivers
#
CONFIG_SOC_TI=y

#
# Xilinx SoC drivers
#
# CONFIG_XILINX_VCU is not set
CONFIG_PM_DEVFREQ=y

#
# DEVFREQ Governors
#
CONFIG_DEVFREQ_GOV_SIMPLE_ONDEMAND=y
# CONFIG_DEVFREQ_GOV_PERFORMANCE is not set
# CONFIG_DEVFREQ_GOV_POWERSAVE is not set
CONFIG_DEVFREQ_GOV_USERSPACE=y
CONFIG_DEVFREQ_GOV_PASSIVE=y

#
# DEVFREQ Drivers
#
CONFIG_PM_DEVFREQ_EVENT=y
CONFIG_EXTCON=m

#
# Extcon Device Drivers
#
CONFIG_EXTCON_MAX14577=m
# CONFIG_EXTCON_MAX77693 is not set
CONFIG_EXTCON_RT8973A=m
# CONFIG_EXTCON_SM5502 is not set
# CONFIG_MEMORY is not set
# CONFIG_IIO is not set
# CONFIG_NTB is not set
# CONFIG_VME_BUS is not set
CONFIG_PWM=y
CONFIG_PWM_SYSFS=y
# CONFIG_PWM_ATMEL_HLCDC_PWM is not set
# CONFIG_PWM_FSL_FTM is not set
# CONFIG_PWM_LPSS_PCI is not set
# CONFIG_PWM_LPSS_PLATFORM is not set
CONFIG_PWM_PCA9685=m

#
# IRQ chip support
#
CONFIG_IRQCHIP=y
CONFIG_ARM_GIC_MAX_NR=1
CONFIG_IPACK_BUS=m
# CONFIG_BOARD_TPCI200 is not set
# CONFIG_SERIAL_IPOCTAL is not set
CONFIG_RESET_CONTROLLER=y
CONFIG_RESET_TI_SYSCON=m
CONFIG_FMC=y
# CONFIG_FMC_FAKEDEV is not set
# CONFIG_FMC_TRIVIAL is not set
CONFIG_FMC_WRITE_EEPROM=y
CONFIG_FMC_CHARDEV=y

#
# PHY Subsystem
#
CONFIG_GENERIC_PHY=y
# CONFIG_BCM_KONA_USB2_PHY is not set
CONFIG_PHY_PXA_28NM_HSIC=y
CONFIG_PHY_PXA_28NM_USB2=m
# CONFIG_PHY_MAPPHONE_MDM6600 is not set
# CONFIG_POWERCAP is not set
# CONFIG_MCB is not set

#
# Performance monitor support
#
# CONFIG_RAS is not set
# CONFIG_THUNDERBOLT is not set

#
# Android
#
# CONFIG_ANDROID is not set
CONFIG_LIBNVDIMM=m
# CONFIG_BLK_DEV_PMEM is not set
CONFIG_ND_BLK=m
# CONFIG_BTT is not set
# CONFIG_NVDIMM_PFN is not set
# CONFIG_OF_PMEM is not set
CONFIG_DAX=y
CONFIG_NVMEM=y

#
# HW tracing support
#
# CONFIG_STM is not set
CONFIG_INTEL_TH=y
# CONFIG_INTEL_TH_PCI is not set
# CONFIG_INTEL_TH_ACPI is not set
CONFIG_INTEL_TH_GTH=y
# CONFIG_INTEL_TH_MSU is not set
CONFIG_INTEL_TH_PTI=y
# CONFIG_INTEL_TH_DEBUG is not set
CONFIG_FPGA=y
CONFIG_ALTERA_PR_IP_CORE=m
CONFIG_ALTERA_PR_IP_CORE_PLAT=m
CONFIG_FPGA_MGR_ALTERA_PS_SPI=m
# CONFIG_FPGA_MGR_ALTERA_CVP is not set
# CONFIG_FPGA_MGR_XILINX_SPI is not set
CONFIG_FPGA_MGR_ICE40_SPI=y
CONFIG_FPGA_BRIDGE=m
CONFIG_XILINX_PR_DECOUPLER=m
CONFIG_FPGA_REGION=m
# CONFIG_OF_FPGA_REGION is not set
CONFIG_FSI=y
# CONFIG_FSI_MASTER_HUB is not set
# CONFIG_FSI_SCOM is not set
CONFIG_MULTIPLEXER=m

#
# Multiplexer drivers
#
CONFIG_MUX_ADG792A=m
CONFIG_MUX_MMIO=m
CONFIG_PM_OPP=y
# CONFIG_UNISYS_VISORBUS is not set
# CONFIG_SIOX is not set
CONFIG_SLIMBUS=y
# CONFIG_SLIM_QCOM_CTRL is not set

#
# Firmware Drivers
#
CONFIG_EDD=m
CONFIG_EDD_OFF=y
# CONFIG_FIRMWARE_MEMMAP is not set
# CONFIG_DELL_RBU is not set
CONFIG_DCDBAS=y
# CONFIG_ISCSI_IBFT_FIND is not set
CONFIG_FW_CFG_SYSFS=m
CONFIG_FW_CFG_SYSFS_CMDLINE=y
# CONFIG_GOOGLE_FIRMWARE is not set

#
# Tegra firmware driver
#

#
# File systems
#
CONFIG_DCACHE_WORD_ACCESS=y
CONFIG_FS_IOMAP=y
CONFIG_EXT2_FS=m
# CONFIG_EXT2_FS_XATTR is not set
# CONFIG_EXT3_FS is not set
CONFIG_EXT4_FS=y
# CONFIG_EXT4_FS_POSIX_ACL is not set
CONFIG_EXT4_FS_SECURITY=y
# CONFIG_EXT4_ENCRYPTION is not set
CONFIG_EXT4_DEBUG=y
CONFIG_JBD2=y
CONFIG_JBD2_DEBUG=y
CONFIG_FS_MBCACHE=y
# CONFIG_REISERFS_FS is not set
# CONFIG_JFS_FS is not set
CONFIG_XFS_FS=m
# CONFIG_XFS_QUOTA is not set
# CONFIG_XFS_POSIX_ACL is not set
CONFIG_XFS_RT=y
# CONFIG_XFS_ONLINE_SCRUB is not set
CONFIG_XFS_WARN=y
# CONFIG_XFS_DEBUG is not set
# CONFIG_GFS2_FS is not set
# CONFIG_OCFS2_FS is not set
# CONFIG_BTRFS_FS is not set
CONFIG_NILFS2_FS=y
CONFIG_F2FS_FS=m
CONFIG_F2FS_STAT_FS=y
CONFIG_F2FS_FS_XATTR=y
CONFIG_F2FS_FS_POSIX_ACL=y
# CONFIG_F2FS_FS_SECURITY is not set
CONFIG_F2FS_CHECK_FS=y
CONFIG_F2FS_FS_ENCRYPTION=y
CONFIG_F2FS_FAULT_INJECTION=y
CONFIG_FS_DAX=y
CONFIG_FS_POSIX_ACL=y
CONFIG_EXPORTFS=y
CONFIG_EXPORTFS_BLOCK_OPS=y
CONFIG_FILE_LOCKING=y
# CONFIG_MANDATORY_FILE_LOCKING is not set
CONFIG_FS_ENCRYPTION=m
CONFIG_FSNOTIFY=y
# CONFIG_DNOTIFY is not set
CONFIG_INOTIFY_USER=y
# CONFIG_FANOTIFY is not set
# CONFIG_QUOTA is not set
CONFIG_AUTOFS4_FS=m
CONFIG_FUSE_FS=y
CONFIG_CUSE=m
CONFIG_OVERLAY_FS=m
# CONFIG_OVERLAY_FS_REDIRECT_DIR is not set
# CONFIG_OVERLAY_FS_REDIRECT_ALWAYS_FOLLOW is not set
# CONFIG_OVERLAY_FS_INDEX is not set
CONFIG_OVERLAY_FS_XINO_AUTO=y

#
# Caches
#
CONFIG_FSCACHE=m
# CONFIG_FSCACHE_STATS is not set
# CONFIG_FSCACHE_HISTOGRAM is not set
CONFIG_FSCACHE_DEBUG=y
# CONFIG_FSCACHE_OBJECT_LIST is not set
CONFIG_CACHEFILES=m
CONFIG_CACHEFILES_DEBUG=y
CONFIG_CACHEFILES_HISTOGRAM=y

#
# CD-ROM/DVD Filesystems
#
CONFIG_ISO9660_FS=y
CONFIG_JOLIET=y
CONFIG_ZISOFS=y
CONFIG_UDF_FS=y
CONFIG_UDF_NLS=y

#
# DOS/FAT/NT Filesystems
#
CONFIG_FAT_FS=y
CONFIG_MSDOS_FS=y
CONFIG_VFAT_FS=m
CONFIG_FAT_DEFAULT_CODEPAGE=437
CONFIG_FAT_DEFAULT_IOCHARSET="iso8859-1"
CONFIG_FAT_DEFAULT_UTF8=y
CONFIG_NTFS_FS=m
# CONFIG_NTFS_DEBUG is not set
# CONFIG_NTFS_RW is not set

#
# Pseudo filesystems
#
CONFIG_PROC_FS=y
# CONFIG_PROC_KCORE is not set
CONFIG_PROC_SYSCTL=y
CONFIG_PROC_PAGE_MONITOR=y
CONFIG_PROC_CHILDREN=y
CONFIG_KERNFS=y
CONFIG_SYSFS=y
CONFIG_TMPFS=y
# CONFIG_TMPFS_POSIX_ACL is not set
# CONFIG_TMPFS_XATTR is not set
CONFIG_HUGETLBFS=y
CONFIG_HUGETLB_PAGE=y
CONFIG_ARCH_HAS_GIGANTIC_PAGE=y
CONFIG_CONFIGFS_FS=m
CONFIG_MISC_FILESYSTEMS=y
CONFIG_ORANGEFS_FS=m
# CONFIG_ADFS_FS is not set
CONFIG_AFFS_FS=y
CONFIG_ECRYPT_FS=m
# CONFIG_ECRYPT_FS_MESSAGING is not set
CONFIG_HFS_FS=y
CONFIG_HFSPLUS_FS=y
# CONFIG_HFSPLUS_FS_POSIX_ACL is not set
CONFIG_BEFS_FS=y
# CONFIG_BEFS_DEBUG is not set
CONFIG_BFS_FS=y
CONFIG_EFS_FS=m
CONFIG_JFFS2_FS=y
CONFIG_JFFS2_FS_DEBUG=0
CONFIG_JFFS2_FS_WRITEBUFFER=y
CONFIG_JFFS2_FS_WBUF_VERIFY=y
# CONFIG_JFFS2_SUMMARY is not set
CONFIG_JFFS2_FS_XATTR=y
# CONFIG_JFFS2_FS_POSIX_ACL is not set
# CONFIG_JFFS2_FS_SECURITY is not set
# CONFIG_JFFS2_COMPRESSION_OPTIONS is not set
CONFIG_JFFS2_ZLIB=y
CONFIG_JFFS2_RTIME=y
CONFIG_UBIFS_FS=m
# CONFIG_UBIFS_FS_ADVANCED_COMPR is not set
CONFIG_UBIFS_FS_LZO=y
CONFIG_UBIFS_FS_ZLIB=y
CONFIG_UBIFS_ATIME_SUPPORT=y
# CONFIG_UBIFS_FS_ENCRYPTION is not set
CONFIG_UBIFS_FS_SECURITY=y
# CONFIG_CRAMFS is not set
# CONFIG_SQUASHFS is not set
CONFIG_VXFS_FS=y
# CONFIG_MINIX_FS is not set
CONFIG_OMFS_FS=m
CONFIG_HPFS_FS=m
CONFIG_QNX4FS_FS=y
CONFIG_QNX6FS_FS=y
# CONFIG_QNX6FS_DEBUG is not set
CONFIG_ROMFS_FS=y
# CONFIG_ROMFS_BACKED_BY_BLOCK is not set
CONFIG_ROMFS_BACKED_BY_MTD=y
# CONFIG_ROMFS_BACKED_BY_BOTH is not set
CONFIG_ROMFS_ON_MTD=y
CONFIG_PSTORE=m
CONFIG_PSTORE_DEFLATE_COMPRESS=m
CONFIG_PSTORE_LZO_COMPRESS=m
CONFIG_PSTORE_LZ4_COMPRESS=m
CONFIG_PSTORE_LZ4HC_COMPRESS=m
# CONFIG_PSTORE_842_COMPRESS is not set
CONFIG_PSTORE_COMPRESS=y
CONFIG_PSTORE_DEFLATE_COMPRESS_DEFAULT=y
# CONFIG_PSTORE_LZO_COMPRESS_DEFAULT is not set
# CONFIG_PSTORE_LZ4_COMPRESS_DEFAULT is not set
# CONFIG_PSTORE_LZ4HC_COMPRESS_DEFAULT is not set
CONFIG_PSTORE_COMPRESS_DEFAULT="deflate"
# CONFIG_PSTORE_CONSOLE is not set
# CONFIG_PSTORE_PMSG is not set
CONFIG_PSTORE_RAM=m
# CONFIG_SYSV_FS is not set
CONFIG_UFS_FS=m
# CONFIG_UFS_FS_WRITE is not set
# CONFIG_UFS_DEBUG is not set
CONFIG_NETWORK_FILESYSTEMS=y
CONFIG_NFS_FS=y
CONFIG_NFS_V2=y
CONFIG_NFS_V3=y
# CONFIG_NFS_V3_ACL is not set
CONFIG_NFS_V4=m
# CONFIG_NFS_SWAP is not set
# CONFIG_NFS_V4_1 is not set
# CONFIG_ROOT_NFS is not set
# CONFIG_NFS_USE_LEGACY_DNS is not set
CONFIG_NFS_USE_KERNEL_DNS=y
# CONFIG_NFSD is not set
CONFIG_GRACE_PERIOD=y
CONFIG_LOCKD=y
CONFIG_LOCKD_V4=y
CONFIG_NFS_COMMON=y
CONFIG_SUNRPC=y
CONFIG_SUNRPC_GSS=m
CONFIG_RPCSEC_GSS_KRB5=m
# CONFIG_SUNRPC_DEBUG is not set
# CONFIG_CEPH_FS is not set
CONFIG_CIFS=m
# CONFIG_CIFS_STATS is not set
# CONFIG_CIFS_WEAK_PW_HASH is not set
# CONFIG_CIFS_UPCALL is not set
# CONFIG_CIFS_XATTR is not set
CONFIG_CIFS_DEBUG=y
# CONFIG_CIFS_DEBUG2 is not set
# CONFIG_CIFS_DEBUG_DUMP_KEYS is not set
# CONFIG_CIFS_DFS_UPCALL is not set
# CONFIG_CIFS_SMB311 is not set
# CONFIG_CIFS_FSCACHE is not set
# CONFIG_CODA_FS is not set
# CONFIG_AFS_FS is not set
CONFIG_NLS=y
CONFIG_NLS_DEFAULT="iso8859-1"
# CONFIG_NLS_CODEPAGE_437 is not set
# CONFIG_NLS_CODEPAGE_737 is not set
# CONFIG_NLS_CODEPAGE_775 is not set
# CONFIG_NLS_CODEPAGE_850 is not set
# CONFIG_NLS_CODEPAGE_852 is not set
# CONFIG_NLS_CODEPAGE_855 is not set
CONFIG_NLS_CODEPAGE_857=y
CONFIG_NLS_CODEPAGE_860=m
# CONFIG_NLS_CODEPAGE_861 is not set
CONFIG_NLS_CODEPAGE_862=m
CONFIG_NLS_CODEPAGE_863=y
CONFIG_NLS_CODEPAGE_864=y
# CONFIG_NLS_CODEPAGE_865 is not set
CONFIG_NLS_CODEPAGE_866=y
CONFIG_NLS_CODEPAGE_869=y
CONFIG_NLS_CODEPAGE_936=y
# CONFIG_NLS_CODEPAGE_950 is not set
CONFIG_NLS_CODEPAGE_932=y
CONFIG_NLS_CODEPAGE_949=m
# CONFIG_NLS_CODEPAGE_874 is not set
CONFIG_NLS_ISO8859_8=m
CONFIG_NLS_CODEPAGE_1250=m
CONFIG_NLS_CODEPAGE_1251=m
# CONFIG_NLS_ASCII is not set
CONFIG_NLS_ISO8859_1=y
CONFIG_NLS_ISO8859_2=m
CONFIG_NLS_ISO8859_3=y
CONFIG_NLS_ISO8859_4=y
CONFIG_NLS_ISO8859_5=m
CONFIG_NLS_ISO8859_6=y
CONFIG_NLS_ISO8859_7=y
# CONFIG_NLS_ISO8859_9 is not set
CONFIG_NLS_ISO8859_13=m
# CONFIG_NLS_ISO8859_14 is not set
CONFIG_NLS_ISO8859_15=y
# CONFIG_NLS_KOI8_R is not set
CONFIG_NLS_KOI8_U=m
# CONFIG_NLS_MAC_ROMAN is not set
# CONFIG_NLS_MAC_CELTIC is not set
CONFIG_NLS_MAC_CENTEURO=y
CONFIG_NLS_MAC_CROATIAN=y
CONFIG_NLS_MAC_CYRILLIC=m
CONFIG_NLS_MAC_GAELIC=y
CONFIG_NLS_MAC_GREEK=y
CONFIG_NLS_MAC_ICELAND=y
CONFIG_NLS_MAC_INUIT=m
CONFIG_NLS_MAC_ROMANIAN=m
CONFIG_NLS_MAC_TURKISH=y
CONFIG_NLS_UTF8=y
# CONFIG_DLM is not set

#
# Kernel hacking
#
CONFIG_TRACE_IRQFLAGS_SUPPORT=y

#
# printk and dmesg options
#
CONFIG_PRINTK_TIME=y
CONFIG_CONSOLE_LOGLEVEL_DEFAULT=7
CONFIG_MESSAGE_LOGLEVEL_DEFAULT=4
CONFIG_BOOT_PRINTK_DELAY=y
CONFIG_DYNAMIC_DEBUG=y

#
# Compile-time checks and compiler options
#
CONFIG_DEBUG_INFO=y
CONFIG_DEBUG_INFO_REDUCED=y
# CONFIG_DEBUG_INFO_SPLIT is not set
# CONFIG_DEBUG_INFO_DWARF4 is not set
# CONFIG_GDB_SCRIPTS is not set
CONFIG_ENABLE_WARN_DEPRECATED=y
# CONFIG_ENABLE_MUST_CHECK is not set
CONFIG_FRAME_WARN=8192
# CONFIG_STRIP_ASM_SYMS is not set
# CONFIG_READABLE_ASM is not set
CONFIG_UNUSED_SYMBOLS=y
# CONFIG_PAGE_OWNER is not set
CONFIG_DEBUG_FS=y
CONFIG_HEADERS_CHECK=y
CONFIG_DEBUG_SECTION_MISMATCH=y
# CONFIG_SECTION_MISMATCH_WARN_ONLY is not set
CONFIG_FRAME_POINTER=y
# CONFIG_STACK_VALIDATION is not set
CONFIG_DEBUG_FORCE_WEAK_PER_CPU=y
CONFIG_MAGIC_SYSRQ=y
CONFIG_MAGIC_SYSRQ_DEFAULT_ENABLE=0x1
CONFIG_MAGIC_SYSRQ_SERIAL=y
CONFIG_DEBUG_KERNEL=y

#
# Memory Debugging
#
CONFIG_PAGE_EXTENSION=y
CONFIG_DEBUG_PAGEALLOC=y
CONFIG_DEBUG_PAGEALLOC_ENABLE_DEFAULT=y
CONFIG_PAGE_POISONING=y
# CONFIG_PAGE_POISONING_NO_SANITY is not set
# CONFIG_PAGE_POISONING_ZERO is not set
# CONFIG_DEBUG_RODATA_TEST is not set
# CONFIG_DEBUG_OBJECTS is not set
# CONFIG_SLUB_STATS is not set
CONFIG_HAVE_DEBUG_KMEMLEAK=y
# CONFIG_DEBUG_KMEMLEAK is not set
CONFIG_DEBUG_STACK_USAGE=y
CONFIG_DEBUG_VM=y
CONFIG_DEBUG_VM_VMACACHE=y
# CONFIG_DEBUG_VM_RB is not set
CONFIG_DEBUG_VM_PGFLAGS=y
CONFIG_ARCH_HAS_DEBUG_VIRTUAL=y
# CONFIG_DEBUG_VIRTUAL is not set
CONFIG_DEBUG_MEMORY_INIT=y
# CONFIG_MEMORY_NOTIFIER_ERROR_INJECT is not set
CONFIG_DEBUG_PER_CPU_MAPS=y
CONFIG_HAVE_DEBUG_STACKOVERFLOW=y
CONFIG_DEBUG_STACKOVERFLOW=y
CONFIG_HAVE_ARCH_KASAN=y
CONFIG_KASAN=y
# CONFIG_KASAN_EXTRA is not set
CONFIG_KASAN_OUTLINE=y
# CONFIG_KASAN_INLINE is not set
CONFIG_TEST_KASAN=m
CONFIG_ARCH_HAS_KCOV=y
CONFIG_KCOV=y
# CONFIG_KCOV_ENABLE_COMPARISONS is not set
# CONFIG_KCOV_INSTRUMENT_ALL is not set
CONFIG_DEBUG_SHIRQ=y

#
# Debug Lockups and Hangs
#
CONFIG_LOCKUP_DETECTOR=y
CONFIG_SOFTLOCKUP_DETECTOR=y
# CONFIG_BOOTPARAM_SOFTLOCKUP_PANIC is not set
CONFIG_BOOTPARAM_SOFTLOCKUP_PANIC_VALUE=0
CONFIG_HARDLOCKUP_DETECTOR_PERF=y
CONFIG_HARDLOCKUP_CHECK_TIMESTAMP=y
CONFIG_HARDLOCKUP_DETECTOR=y
CONFIG_BOOTPARAM_HARDLOCKUP_PANIC=y
CONFIG_BOOTPARAM_HARDLOCKUP_PANIC_VALUE=1
# CONFIG_DETECT_HUNG_TASK is not set
CONFIG_WQ_WATCHDOG=y
# CONFIG_PANIC_ON_OOPS is not set
CONFIG_PANIC_ON_OOPS_VALUE=0
CONFIG_PANIC_TIMEOUT=0
CONFIG_SCHED_DEBUG=y
CONFIG_SCHED_INFO=y
CONFIG_SCHEDSTATS=y
CONFIG_SCHED_STACK_END_CHECK=y
# CONFIG_DEBUG_TIMEKEEPING is not set
CONFIG_DEBUG_PREEMPT=y

#
# Lock Debugging (spinlocks, mutexes, etc...)
#
CONFIG_LOCK_DEBUGGING_SUPPORT=y
CONFIG_PROVE_LOCKING=y
# CONFIG_LOCK_STAT is not set
CONFIG_DEBUG_RT_MUTEXES=y
CONFIG_DEBUG_SPINLOCK=y
CONFIG_DEBUG_MUTEXES=y
CONFIG_DEBUG_WW_MUTEX_SLOWPATH=y
CONFIG_DEBUG_RWSEMS=y
CONFIG_DEBUG_LOCK_ALLOC=y
CONFIG_LOCKDEP=y
# CONFIG_DEBUG_LOCKDEP is not set
CONFIG_DEBUG_ATOMIC_SLEEP=y
# CONFIG_DEBUG_LOCKING_API_SELFTESTS is not set
# CONFIG_LOCK_TORTURE_TEST is not set
CONFIG_WW_MUTEX_SELFTEST=y
CONFIG_TRACE_IRQFLAGS=y
CONFIG_STACKTRACE=y
# CONFIG_WARN_ALL_UNSEEDED_RANDOM is not set
# CONFIG_DEBUG_KOBJECT is not set
CONFIG_DEBUG_BUGVERBOSE=y
CONFIG_DEBUG_LIST=y
CONFIG_DEBUG_PI_LIST=y
# CONFIG_DEBUG_SG is not set
# CONFIG_DEBUG_NOTIFIERS is not set
CONFIG_DEBUG_CREDENTIALS=y

#
# RCU Debugging
#
CONFIG_PROVE_RCU=y
CONFIG_TORTURE_TEST=m
CONFIG_RCU_PERF_TEST=m
CONFIG_RCU_TORTURE_TEST=m
CONFIG_RCU_CPU_STALL_TIMEOUT=21
# CONFIG_RCU_TRACE is not set
CONFIG_RCU_EQS_DEBUG=y
CONFIG_DEBUG_WQ_FORCE_RR_CPU=y
# CONFIG_DEBUG_BLOCK_EXT_DEVT is not set
# CONFIG_CPU_HOTPLUG_STATE_CONTROL is not set
CONFIG_NOTIFIER_ERROR_INJECTION=m
CONFIG_PM_NOTIFIER_ERROR_INJECT=m
CONFIG_OF_RECONFIG_NOTIFIER_ERROR_INJECT=m
# CONFIG_NETDEV_NOTIFIER_ERROR_INJECT is not set
# CONFIG_FAULT_INJECTION is not set
CONFIG_FUNCTION_ERROR_INJECTION=y
CONFIG_LATENCYTOP=y
CONFIG_USER_STACKTRACE_SUPPORT=y
CONFIG_HAVE_FUNCTION_TRACER=y
CONFIG_HAVE_FUNCTION_GRAPH_TRACER=y
CONFIG_HAVE_DYNAMIC_FTRACE=y
CONFIG_HAVE_DYNAMIC_FTRACE_WITH_REGS=y
CONFIG_HAVE_FTRACE_MCOUNT_RECORD=y
CONFIG_HAVE_SYSCALL_TRACEPOINTS=y
CONFIG_HAVE_FENTRY=y
CONFIG_HAVE_C_RECORDMCOUNT=y
CONFIG_TRACING_SUPPORT=y
# CONFIG_FTRACE is not set
# CONFIG_PROVIDE_OHCI1394_DMA_INIT is not set
# CONFIG_DMA_API_DEBUG is not set
# CONFIG_RUNTIME_TESTING_MENU is not set
CONFIG_MEMTEST=y
# CONFIG_BUG_ON_DATA_CORRUPTION is not set
# CONFIG_SAMPLES is not set
CONFIG_HAVE_ARCH_KGDB=y
# CONFIG_KGDB is not set
CONFIG_ARCH_HAS_UBSAN_SANITIZE_ALL=y
CONFIG_UBSAN=y
# CONFIG_UBSAN_SANITIZE_ALL is not set
# CONFIG_UBSAN_ALIGNMENT is not set
# CONFIG_UBSAN_NULL is not set
CONFIG_TEST_UBSAN=m
CONFIG_ARCH_HAS_DEVMEM_IS_ALLOWED=y
# CONFIG_STRICT_DEVMEM is not set
CONFIG_X86_VERBOSE_BOOTUP=y
# CONFIG_EARLY_PRINTK is not set
CONFIG_X86_PTDUMP_CORE=y
CONFIG_X86_PTDUMP=m
CONFIG_DEBUG_WX=y
# CONFIG_DOUBLEFAULT is not set
CONFIG_DEBUG_TLBFLUSH=y
CONFIG_HAVE_MMIOTRACE_SUPPORT=y
# CONFIG_X86_DECODER_SELFTEST is not set
CONFIG_IO_DELAY_TYPE_0X80=0
CONFIG_IO_DELAY_TYPE_0XED=1
CONFIG_IO_DELAY_TYPE_UDELAY=2
CONFIG_IO_DELAY_TYPE_NONE=3
# CONFIG_IO_DELAY_0X80 is not set
CONFIG_IO_DELAY_0XED=y
# CONFIG_IO_DELAY_UDELAY is not set
# CONFIG_IO_DELAY_NONE is not set
CONFIG_DEFAULT_IO_DELAY_TYPE=1
# CONFIG_DEBUG_BOOT_PARAMS is not set
# CONFIG_CPA_DEBUG is not set
# CONFIG_OPTIMIZE_INLINING is not set
# CONFIG_DEBUG_ENTRY is not set
CONFIG_DEBUG_NMI_SELFTEST=y
# CONFIG_X86_DEBUG_FPU is not set
# CONFIG_PUNIT_ATOM_DEBUG is not set
# CONFIG_UNWINDER_ORC is not set
CONFIG_UNWINDER_FRAME_POINTER=y

#
# Security options
#
CONFIG_KEYS=y
CONFIG_KEYS_COMPAT=y
CONFIG_PERSISTENT_KEYRINGS=y
# CONFIG_BIG_KEYS is not set
CONFIG_ENCRYPTED_KEYS=m
# CONFIG_KEY_DH_OPERATIONS is not set
CONFIG_SECURITY_DMESG_RESTRICT=y
# CONFIG_SECURITY is not set
# CONFIG_SECURITYFS is not set
# CONFIG_PAGE_TABLE_ISOLATION is not set
CONFIG_HAVE_HARDENED_USERCOPY_ALLOCATOR=y
CONFIG_HARDENED_USERCOPY=y
# CONFIG_HARDENED_USERCOPY_FALLBACK is not set
# CONFIG_HARDENED_USERCOPY_PAGESPAN is not set
CONFIG_FORTIFY_SOURCE=y
CONFIG_STATIC_USERMODEHELPER=y
CONFIG_STATIC_USERMODEHELPER_PATH="/sbin/usermode-helper"
CONFIG_DEFAULT_SECURITY_DAC=y
CONFIG_DEFAULT_SECURITY=""
CONFIG_CRYPTO=y

#
# Crypto core or helper
#
CONFIG_CRYPTO_ALGAPI=y
CONFIG_CRYPTO_ALGAPI2=y
CONFIG_CRYPTO_AEAD=y
CONFIG_CRYPTO_AEAD2=y
CONFIG_CRYPTO_BLKCIPHER=y
CONFIG_CRYPTO_BLKCIPHER2=y
CONFIG_CRYPTO_HASH=y
CONFIG_CRYPTO_HASH2=y
CONFIG_CRYPTO_RNG=m
CONFIG_CRYPTO_RNG2=y
CONFIG_CRYPTO_RNG_DEFAULT=m
CONFIG_CRYPTO_AKCIPHER2=y
CONFIG_CRYPTO_AKCIPHER=y
CONFIG_CRYPTO_KPP2=y
CONFIG_CRYPTO_KPP=y
CONFIG_CRYPTO_ACOMP2=y
CONFIG_CRYPTO_RSA=y
CONFIG_CRYPTO_DH=y
CONFIG_CRYPTO_ECDH=m
CONFIG_CRYPTO_MANAGER=y
CONFIG_CRYPTO_MANAGER2=y
# CONFIG_CRYPTO_USER is not set
CONFIG_CRYPTO_MANAGER_DISABLE_TESTS=y
CONFIG_CRYPTO_GF128MUL=m
CONFIG_CRYPTO_NULL=m
CONFIG_CRYPTO_NULL2=y
CONFIG_CRYPTO_PCRYPT=m
CONFIG_CRYPTO_WORKQUEUE=y
CONFIG_CRYPTO_CRYPTD=y
CONFIG_CRYPTO_MCRYPTD=y
CONFIG_CRYPTO_AUTHENC=m
CONFIG_CRYPTO_TEST=m
CONFIG_CRYPTO_SIMD=y
CONFIG_CRYPTO_GLUE_HELPER_X86=y

#
# Authenticated Encryption with Associated Data
#
CONFIG_CRYPTO_CCM=m
CONFIG_CRYPTO_GCM=m
CONFIG_CRYPTO_CHACHA20POLY1305=y
CONFIG_CRYPTO_SEQIV=m
# CONFIG_CRYPTO_ECHAINIV is not set

#
# Block modes
#
CONFIG_CRYPTO_CBC=y
CONFIG_CRYPTO_CFB=m
CONFIG_CRYPTO_CTR=m
CONFIG_CRYPTO_CTS=m
CONFIG_CRYPTO_ECB=y
# CONFIG_CRYPTO_LRW is not set
CONFIG_CRYPTO_PCBC=m
CONFIG_CRYPTO_XTS=y
CONFIG_CRYPTO_KEYWRAP=y

#
# Hash modes
#
CONFIG_CRYPTO_CMAC=m
CONFIG_CRYPTO_HMAC=m
CONFIG_CRYPTO_XCBC=m
CONFIG_CRYPTO_VMAC=y

#
# Digest
#
CONFIG_CRYPTO_CRC32C=y
# CONFIG_CRYPTO_CRC32C_INTEL is not set
CONFIG_CRYPTO_CRC32=y
CONFIG_CRYPTO_CRC32_PCLMUL=y
CONFIG_CRYPTO_CRCT10DIF=y
# CONFIG_CRYPTO_CRCT10DIF_PCLMUL is not set
CONFIG_CRYPTO_GHASH=m
CONFIG_CRYPTO_POLY1305=y
CONFIG_CRYPTO_POLY1305_X86_64=y
CONFIG_CRYPTO_MD4=y
CONFIG_CRYPTO_MD5=m
# CONFIG_CRYPTO_MICHAEL_MIC is not set
CONFIG_CRYPTO_RMD128=y
# CONFIG_CRYPTO_RMD160 is not set
# CONFIG_CRYPTO_RMD256 is not set
# CONFIG_CRYPTO_RMD320 is not set
CONFIG_CRYPTO_SHA1=y
CONFIG_CRYPTO_SHA1_SSSE3=m
CONFIG_CRYPTO_SHA256_SSSE3=y
CONFIG_CRYPTO_SHA512_SSSE3=m
CONFIG_CRYPTO_SHA1_MB=m
CONFIG_CRYPTO_SHA256_MB=m
CONFIG_CRYPTO_SHA512_MB=m
CONFIG_CRYPTO_SHA256=y
CONFIG_CRYPTO_SHA512=m
# CONFIG_CRYPTO_SHA3 is not set
CONFIG_CRYPTO_SM3=m
CONFIG_CRYPTO_TGR192=m
CONFIG_CRYPTO_WP512=m
# CONFIG_CRYPTO_GHASH_CLMUL_NI_INTEL is not set

#
# Ciphers
#
CONFIG_CRYPTO_AES=y
CONFIG_CRYPTO_AES_TI=y
CONFIG_CRYPTO_AES_X86_64=y
CONFIG_CRYPTO_AES_NI_INTEL=y
# CONFIG_CRYPTO_ANUBIS is not set
CONFIG_CRYPTO_ARC4=y
CONFIG_CRYPTO_BLOWFISH=y
CONFIG_CRYPTO_BLOWFISH_COMMON=y
CONFIG_CRYPTO_BLOWFISH_X86_64=y
# CONFIG_CRYPTO_CAMELLIA is not set
CONFIG_CRYPTO_CAMELLIA_X86_64=m
CONFIG_CRYPTO_CAMELLIA_AESNI_AVX_X86_64=m
# CONFIG_CRYPTO_CAMELLIA_AESNI_AVX2_X86_64 is not set
CONFIG_CRYPTO_CAST_COMMON=y
CONFIG_CRYPTO_CAST5=y
CONFIG_CRYPTO_CAST5_AVX_X86_64=y
CONFIG_CRYPTO_CAST6=y
# CONFIG_CRYPTO_CAST6_AVX_X86_64 is not set
CONFIG_CRYPTO_DES=y
CONFIG_CRYPTO_DES3_EDE_X86_64=y
CONFIG_CRYPTO_FCRYPT=m
CONFIG_CRYPTO_KHAZAD=m
CONFIG_CRYPTO_SALSA20=y
CONFIG_CRYPTO_SALSA20_X86_64=y
CONFIG_CRYPTO_CHACHA20=y
CONFIG_CRYPTO_CHACHA20_X86_64=m
CONFIG_CRYPTO_SEED=y
CONFIG_CRYPTO_SERPENT=y
CONFIG_CRYPTO_SERPENT_SSE2_X86_64=m
CONFIG_CRYPTO_SERPENT_AVX_X86_64=y
# CONFIG_CRYPTO_SERPENT_AVX2_X86_64 is not set
# CONFIG_CRYPTO_SM4 is not set
# CONFIG_CRYPTO_SPECK is not set
CONFIG_CRYPTO_TEA=m
CONFIG_CRYPTO_TWOFISH=y
CONFIG_CRYPTO_TWOFISH_COMMON=y
CONFIG_CRYPTO_TWOFISH_X86_64=m
# CONFIG_CRYPTO_TWOFISH_X86_64_3WAY is not set
# CONFIG_CRYPTO_TWOFISH_AVX_X86_64 is not set

#
# Compression
#
CONFIG_CRYPTO_DEFLATE=m
CONFIG_CRYPTO_LZO=m
# CONFIG_CRYPTO_842 is not set
CONFIG_CRYPTO_LZ4=m
CONFIG_CRYPTO_LZ4HC=m

#
# Random Number Generation
#
CONFIG_CRYPTO_ANSI_CPRNG=m
CONFIG_CRYPTO_DRBG_MENU=m
CONFIG_CRYPTO_DRBG_HMAC=y
# CONFIG_CRYPTO_DRBG_HASH is not set
CONFIG_CRYPTO_DRBG_CTR=y
CONFIG_CRYPTO_DRBG=m
CONFIG_CRYPTO_JITTERENTROPY=m
# CONFIG_CRYPTO_USER_API_HASH is not set
# CONFIG_CRYPTO_USER_API_SKCIPHER is not set
# CONFIG_CRYPTO_USER_API_RNG is not set
# CONFIG_CRYPTO_USER_API_AEAD is not set
CONFIG_CRYPTO_HASH_INFO=y
# CONFIG_CRYPTO_HW is not set
CONFIG_ASYMMETRIC_KEY_TYPE=y
CONFIG_ASYMMETRIC_PUBLIC_KEY_SUBTYPE=y
CONFIG_X509_CERTIFICATE_PARSER=y
CONFIG_PKCS7_MESSAGE_PARSER=y

#
# Certificates for signature checking
#
CONFIG_SYSTEM_TRUSTED_KEYRING=y
CONFIG_SYSTEM_TRUSTED_KEYS=""
CONFIG_SYSTEM_EXTRA_CERTIFICATE=y
CONFIG_SYSTEM_EXTRA_CERTIFICATE_SIZE=4096
# CONFIG_SECONDARY_TRUSTED_KEYRING is not set
# CONFIG_SYSTEM_BLACKLIST_KEYRING is not set
CONFIG_HAVE_KVM=y
# CONFIG_VIRTUALIZATION is not set

#
# Library routines
#
CONFIG_BITREVERSE=y
CONFIG_RATIONAL=y
CONFIG_GENERIC_STRNCPY_FROM_USER=y
CONFIG_GENERIC_STRNLEN_USER=y
CONFIG_GENERIC_NET_UTILS=y
CONFIG_GENERIC_FIND_FIRST_BIT=y
CONFIG_GENERIC_PCI_IOMAP=y
CONFIG_GENERIC_IOMAP=y
CONFIG_ARCH_USE_CMPXCHG_LOCKREF=y
CONFIG_ARCH_HAS_FAST_MULTIPLIER=y
CONFIG_CRC_CCITT=y
CONFIG_CRC16=y
CONFIG_CRC_T10DIF=y
CONFIG_CRC_ITU_T=y
CONFIG_CRC32=y
# CONFIG_CRC32_SELFTEST is not set
# CONFIG_CRC32_SLICEBY8 is not set
CONFIG_CRC32_SLICEBY4=y
# CONFIG_CRC32_SARWATE is not set
# CONFIG_CRC32_BIT is not set
CONFIG_CRC4=y
CONFIG_CRC7=m
CONFIG_LIBCRC32C=m
# CONFIG_CRC8 is not set
# CONFIG_RANDOM32_SELFTEST is not set
CONFIG_ZLIB_INFLATE=y
CONFIG_ZLIB_DEFLATE=y
CONFIG_LZO_COMPRESS=m
CONFIG_LZO_DECOMPRESS=m
CONFIG_LZ4_COMPRESS=m
CONFIG_LZ4HC_COMPRESS=m
CONFIG_LZ4_DECOMPRESS=m
CONFIG_XZ_DEC=m
# CONFIG_XZ_DEC_X86 is not set
CONFIG_XZ_DEC_POWERPC=y
CONFIG_XZ_DEC_IA64=y
# CONFIG_XZ_DEC_ARM is not set
CONFIG_XZ_DEC_ARMTHUMB=y
CONFIG_XZ_DEC_SPARC=y
CONFIG_XZ_DEC_BCJ=y
CONFIG_XZ_DEC_TEST=m
CONFIG_DECOMPRESS_GZIP=y
CONFIG_DECOMPRESS_BZIP2=y
CONFIG_DECOMPRESS_LZMA=y
CONFIG_GENERIC_ALLOCATOR=y
CONFIG_REED_SOLOMON=m
CONFIG_REED_SOLOMON_ENC8=y
CONFIG_REED_SOLOMON_DEC8=y
CONFIG_BCH=y
CONFIG_BCH_CONST_PARAMS=y
CONFIG_RADIX_TREE_MULTIORDER=y
CONFIG_ASSOCIATIVE_ARRAY=y
CONFIG_HAS_IOMEM=y
CONFIG_HAS_IOPORT_MAP=y
CONFIG_HAS_DMA=y
CONFIG_SGL_ALLOC=y
CONFIG_DMA_DIRECT_OPS=y
CONFIG_CHECK_SIGNATURE=y
# CONFIG_CPUMASK_OFFSTACK is not set
CONFIG_CPU_RMAP=y
CONFIG_DQL=y
CONFIG_GLOB=y
CONFIG_GLOB_SELFTEST=y
CONFIG_NLATTR=y
CONFIG_CLZ_TAB=y
# CONFIG_CORDIC is not set
# CONFIG_DDR is not set
CONFIG_IRQ_POLL=y
CONFIG_MPILIB=y
CONFIG_LIBFDT=y
CONFIG_OID_REGISTRY=y
CONFIG_SG_POOL=y
CONFIG_ARCH_HAS_SG_CHAIN=y
CONFIG_ARCH_HAS_PMEM_API=y
CONFIG_ARCH_HAS_UACCESS_FLUSHCACHE=y
CONFIG_STACKDEPOT=y
CONFIG_SBITMAP=y
CONFIG_PRIME_NUMBERS=m
CONFIG_STRING_SELFTEST=y

[-- Attachment #3: job-script --]
[-- Type: text/plain, Size: 4073 bytes --]

#!/bin/sh

export_top_env()
{
	export suite='boot'
	export testcase='boot'
	export timeout='10m'
	export job_origin='/lkp/lkp/src/jobs/boot.yaml'
	export queue='bisect'
	export testbox='vm-ivb41-quantal-x86_64-30'
	export tbox_group='vm-ivb41-quantal-x86_64'
	export branch='linux-devel/devel-spot-201804261954'
	export commit='cad8e9967526d263d9a4f04ca763b2d76c045750'
	export kconfig='x86_64-randconfig-s1-04262148'
	export submit_id='5ae20e490b9a93398a661233'
	export job_file='/lkp/scheduled/vm-ivb41-quantal-x86_64-30/boot-1-quantal-core-x86_64.cgz-cad8e9967526d263d9a4f04ca763b2d76c045750-20180427-80266-1hb8lge-1.yaml'
	export id='36a19937402b261429c4e10eddb71d4661592e3c'
	export model='qemu-system-x86_64 -enable-kvm'
	export nr_vm=32
	export nr_cpu=2
	export memory='512M'
	export rootfs='quantal-core-x86_64.cgz'
	export need_kconfig='CONFIG_KVM_GUEST=y'
	export compiler='gcc-6'
	export enqueue_time='2018-04-27 01:37:13 +0800'
	export _id='5ae2120a0b9a93398a661234'
	export _rt='/result/boot/1/vm-ivb41-quantal-x86_64/quantal-core-x86_64.cgz/x86_64-randconfig-s1-04262148/gcc-6/cad8e9967526d263d9a4f04ca763b2d76c045750'
	export user='lkp'
	export result_root='/result/boot/1/vm-ivb41-quantal-x86_64/quantal-core-x86_64.cgz/x86_64-randconfig-s1-04262148/gcc-6/cad8e9967526d263d9a4f04ca763b2d76c045750/0'
	export LKP_SERVER='inn'
	export max_uptime=600
	export initrd='/osimage/quantal/quantal-core-x86_64.cgz'
	export bootloader_append='root=/dev/ram0
user=lkp
job=/lkp/scheduled/vm-ivb41-quantal-x86_64-30/boot-1-quantal-core-x86_64.cgz-cad8e9967526d263d9a4f04ca763b2d76c045750-20180427-80266-1hb8lge-1.yaml
ARCH=x86_64
kconfig=x86_64-randconfig-s1-04262148
branch=linux-devel/devel-spot-201804261954
commit=cad8e9967526d263d9a4f04ca763b2d76c045750
BOOT_IMAGE=/pkg/linux/x86_64-randconfig-s1-04262148/gcc-6/cad8e9967526d263d9a4f04ca763b2d76c045750/vmlinuz-4.17.0-rc2-00001-gcad8e99
max_uptime=600
RESULT_ROOT=/result/boot/1/vm-ivb41-quantal-x86_64/quantal-core-x86_64.cgz/x86_64-randconfig-s1-04262148/gcc-6/cad8e9967526d263d9a4f04ca763b2d76c045750/0
LKP_SERVER=inn
debug
apic=debug
sysrq_always_enabled
rcupdate.rcu_cpu_stall_timeout=100
net.ifnames=0
printk.devkmsg=on
panic=-1
softlockup_panic=1
nmi_watchdog=panic
oops=panic
load_ramdisk=2
prompt_ramdisk=0
drbd.minor_count=8
systemd.log_level=err
ignore_loglevel
console=tty0
earlyprintk=ttyS0,115200
console=ttyS0,115200
vga=normal
rw'
	export modules_initrd='/pkg/linux/x86_64-randconfig-s1-04262148/gcc-6/cad8e9967526d263d9a4f04ca763b2d76c045750/modules.cgz'
	export lkp_initrd='/lkp/lkp/lkp-x86_64.cgz'
	export site='inn'
	export LKP_CGI_PORT=80
	export LKP_CIFS_PORT=139
	export kernel='/pkg/linux/x86_64-randconfig-s1-04262148/gcc-6/cad8e9967526d263d9a4f04ca763b2d76c045750/vmlinuz-4.17.0-rc2-00001-gcad8e99'
	export dequeue_time='2018-04-27 01:53:25 +0800'
	export job_initrd='/lkp/scheduled/vm-ivb41-quantal-x86_64-30/boot-1-quantal-core-x86_64.cgz-cad8e9967526d263d9a4f04ca763b2d76c045750-20180427-80266-1hb8lge-1.cgz'

	[ -n "$LKP_SRC" ] ||
	export LKP_SRC=/lkp/${user:-lkp}/src
}

run_job()
{
	echo $$ > $TMP/run-job.pid

	. $LKP_SRC/lib/http.sh
	. $LKP_SRC/lib/job.sh
	. $LKP_SRC/lib/env.sh

	export_top_env

	run_monitor $LKP_SRC/monitors/one-shot/wrapper boot-slabinfo
	run_monitor $LKP_SRC/monitors/one-shot/wrapper boot-meminfo
	run_monitor $LKP_SRC/monitors/one-shot/wrapper memmap
	run_monitor $LKP_SRC/monitors/no-stdout/wrapper boot-time
	run_monitor $LKP_SRC/monitors/wrapper kmsg
	run_monitor $LKP_SRC/monitors/wrapper oom-killer
	run_monitor $LKP_SRC/monitors/plain/watchdog

	run_test $LKP_SRC/tests/wrapper sleep 1
}

extract_stats()
{
	$LKP_SRC/stats/wrapper boot-slabinfo
	$LKP_SRC/stats/wrapper boot-meminfo
	$LKP_SRC/stats/wrapper memmap
	$LKP_SRC/stats/wrapper boot-memory
	$LKP_SRC/stats/wrapper boot-time
	$LKP_SRC/stats/wrapper kernel-size
	$LKP_SRC/stats/wrapper kmsg

	$LKP_SRC/stats/wrapper time sleep.time
	$LKP_SRC/stats/wrapper time
	$LKP_SRC/stats/wrapper dmesg
	$LKP_SRC/stats/wrapper kmsg
	$LKP_SRC/stats/wrapper stderr
	$LKP_SRC/stats/wrapper last_state
}

"$@"

[-- Attachment #4: dmesg.xz --]
[-- Type: application/x-xz, Size: 16768 bytes --]

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v1] kthread/smpboot: Serialize kthread parking against wakeup
  2018-04-26 15:53       ` Kohli, Gaurav
@ 2018-04-30 11:17         ` Peter Zijlstra
  2018-05-01  7:50           ` Kohli, Gaurav
  0 siblings, 1 reply; 39+ messages in thread
From: Peter Zijlstra @ 2018-04-30 11:17 UTC (permalink / raw)
  To: Kohli, Gaurav
  Cc: tglx, mpe, mingo, bigeasy, linux-kernel, linux-arm-msm,
	Neeraj Upadhyay, Will Deacon, Oleg Nesterov

On Thu, Apr 26, 2018 at 09:23:25PM +0530, Kohli, Gaurav wrote:
> On 4/26/2018 2:27 PM, Peter Zijlstra wrote:
> 
> > On Thu, Apr 26, 2018 at 10:41:31AM +0200, Peter Zijlstra wrote:
> > > diff --git a/kernel/kthread.c b/kernel/kthread.c
> > > index cd50e99202b0..4b6503c6a029 100644
> > > --- a/kernel/kthread.c
> > > +++ b/kernel/kthread.c
> > > @@ -177,12 +177,13 @@ void *kthread_probe_data(struct task_struct *task)
> > >   static void __kthread_parkme(struct kthread *self)
> > >   {
> > > -	__set_current_state(TASK_PARKED);
> > > -	while (test_bit(KTHREAD_SHOULD_PARK, &self->flags)) {
> > > +	for (;;) {
> > > +		__set_task_state(TASK_PARKED);
> > 		set_current_state(TASK_PARKED);
> > 
> > of course..
> 
> Hi Peter,
> 
> Maybe i am missing something , but still that race can come as we don't put task_parked on special state.
> 
> Controller                                                                       Hotplug
> 
>                                                                                  Loop
> 
>                                                                                  Task_Interruptible
> 
> Set SHOULD_PARK
> 
> wakeup -> Proceeds
> 
>                                                                                   Set Running
> 
>                                                                                   kthread_parkme
> 
>                                                                                   SET TASK_PARKED
> 
>                                                                                   schedule
> 
> Set TASK_RUNNING
> 
> Can you please correct ME, if I misunderstood this.

If that could happen, all wait-loops would be broken. However,
AFAICT that cannot happen, because ttwu_remote() and schedule()
serialize on rq->lock. See:


A						B

for (;;) {
	set_current_state(UNINTERRUPTIBLE);

						cond1 = true;
						wake_up_process(A)
						  lock(A->pi_lock)
						  smp_mb__after_spinlock()
						  if (A->state & TASK_NORMAL)
						    A->on_rq && ttwu_remote()
	if (cond1) // true
		break;
	schedule();
}
__set_current_state(RUNNING);

for (;;) {
	set_current_state(UNINTERRUPTIBLE);
	if (cond2)
		break;

	schedule();
	  lock(rq->lock)
	  smp_mb__after_spinlock();
	  deactivate_task(A);
	  <sched-out>
	  unlock(rq->lock);
						      rq = __task_rq_lock(A)
						      if (A->on_rq) // false
						        A->state = TASK_RUNNING;
						      __task_rq_unlock(rq)


Either A's schedule() must observe RUNNING (not shown) or B must
observe !A->on_rq (shown) and not issue the store.

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v1] kthread/smpboot: Serialize kthread parking against wakeup
  2018-04-26 16:18     ` Oleg Nesterov
@ 2018-04-30 11:20       ` Peter Zijlstra
  2018-04-30 11:56         ` Peter Zijlstra
  0 siblings, 1 reply; 39+ messages in thread
From: Peter Zijlstra @ 2018-04-30 11:20 UTC (permalink / raw)
  To: Oleg Nesterov
  Cc: Gaurav Kohli, tglx, mpe, mingo, bigeasy, linux-kernel,
	linux-arm-msm, Neeraj Upadhyay, Will Deacon

On Thu, Apr 26, 2018 at 06:18:20PM +0200, Oleg Nesterov wrote:
> On 04/26, Peter Zijlstra wrote:
> >
> > For the others, I think we want to do something like the below. I still
> > need to look at TASK_TRACED, which I suspect is also special,
> 
> Yes, and TASK_STOPPED.

I already did that one, right ;-)

> 
> ptrace_freeze_traced() and ptrace_unfreeze_traced() should be fine, but
> ptrace_stop() wants set_special_state() too, I think.

Thanks for going through that, I'll update the patch.

> > +/*
> > + * set_special_state() should be used for those states when the blocking task
> > + * can not use the regular condition based wait-loop. In that case we must
> > + * serialize against wakeups such that any possible in-flight TASK_RUNNING stores
> > + * will not collide with out state change.
> > + */
> > +#define set_special_state(state_value)					\
> > +	do {								\
> > +		unsigned long flags; /* may shadow */			\
> > +		raw_spin_lock_irqsave(&current->pi_lock, flags);	\
> > +		current->state = (state_value);				\
> > +		raw_spin_unlock_irqrestore(&current->pi_lock, flags);	\
> > +	} while (0)
> > +
> 
> Agreed.
> 
> I thought that perhaps we can change ttwu_do_wakeup() cmpxchg() instead of
> plain p->state = TASK_RUNNING, but this helper looks much more clear and simple.

Aside from cmpxchg() slowing down the common fast path, that also
would not work on architectures where cmpxchg() is not in fact atomic vs
regular stores (and yes, sadly we still have those, PARISC and SPARC32
come to mind).

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v1] kthread/smpboot: Serialize kthread parking against wakeup
  2018-04-30 11:20       ` Peter Zijlstra
@ 2018-04-30 11:56         ` Peter Zijlstra
  0 siblings, 0 replies; 39+ messages in thread
From: Peter Zijlstra @ 2018-04-30 11:56 UTC (permalink / raw)
  To: Oleg Nesterov
  Cc: Gaurav Kohli, tglx, mpe, mingo, bigeasy, linux-kernel,
	linux-arm-msm, Neeraj Upadhyay, Will Deacon

On Mon, Apr 30, 2018 at 01:20:29PM +0200, Peter Zijlstra wrote:
> On Thu, Apr 26, 2018 at 06:18:20PM +0200, Oleg Nesterov wrote:
> > On 04/26, Peter Zijlstra wrote:
> > >
> > > For the others, I think we want to do something like the below. I still
> > > need to look at TASK_TRACED, which I suspect is also special,
> > 
> > ptrace_freeze_traced() and ptrace_unfreeze_traced() should be fine, but
> > ptrace_stop() wants set_special_state() too, I think.
> 
> Thanks for going through that, I'll update the patch.

Something like so then... I'll go write me some Changelog.


diff --git a/include/linux/sched.h b/include/linux/sched.h
index b3d697f3b573..51f46704e566 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -112,17 +112,38 @@ struct task_group;
 
 #ifdef CONFIG_DEBUG_ATOMIC_SLEEP
 
+/*
+ * Special states are those that do not use the normal wait-loop pattern. See
+ * the comment with set_special_state().
+ */
+#define is_special_task_state(state)	\
+	((state) == TASK_DEAD    ||	\
+	 (state) == TASK_STOPPED ||	\
+	 (state) == TASK_TRACED)
+
 #define __set_current_state(state_value)			\
 	do {							\
+		WARN_ON_ONCE(is_special_task_state(state));	\
 		current->task_state_change = _THIS_IP_;		\
 		current->state = (state_value);			\
 	} while (0)
+
 #define set_current_state(state_value)				\
 	do {							\
+		WARN_ON_ONCE(is_special_task_state(state));	\
 		current->task_state_change = _THIS_IP_;		\
 		smp_store_mb(current->state, (state_value));	\
 	} while (0)
 
+#define set_special_state(state_value)					\
+	do {								\
+		unsigned long flags; /* may shadow */			\
+		WARN_ON_ONCE(!is_special_task_state(state_value));	\
+		raw_spin_lock_irqsave(&current->pi_lock, flags);	\
+		current->task_state_change = _THIS_IP_;			\
+		current->state = (state_value);				\
+		raw_spin_unlock_irqrestore(&current->pi_lock, flags);	\
+	} while (0)
 #else
 /*
  * set_current_state() includes a barrier so that the write of current->state
@@ -144,8 +165,8 @@ struct task_group;
  *
  * The above is typically ordered against the wakeup, which does:
  *
- *	need_sleep = false;
- *	wake_up_state(p, TASK_UNINTERRUPTIBLE);
+ *   need_sleep = false;
+ *   wake_up_state(p, TASK_UNINTERRUPTIBLE);
  *
  * Where wake_up_state() (and all other wakeup primitives) imply enough
  * barriers to order the store of the variable against wakeup.
@@ -154,12 +175,33 @@ struct task_group;
  * once it observes the TASK_UNINTERRUPTIBLE store the waking CPU can issue a
  * TASK_RUNNING store which can collide with __set_current_state(TASK_RUNNING).
  *
- * This is obviously fine, since they both store the exact same value.
+ * However, with slightly different timing the wakeup TASK_RUNNING store can
+ * also collide with the TASK_UNINTERRUPTIBLE store. Loosing that store is not
+ * a problem either because that will result in one extra go around the loop
+ * and our @cond test will save the day.
  *
  * Also see the comments of try_to_wake_up().
  */
-#define __set_current_state(state_value) do { current->state = (state_value); } while (0)
-#define set_current_state(state_value)	 smp_store_mb(current->state, (state_value))
+#define __set_current_state(state_value)				\
+	current->state = (state_value)
+
+#define set_current_state(state_value)					\
+	smp_store_mb(current->state, (state_value))
+
+/*
+ * set_special_state() should be used for those states when the blocking task
+ * can not use the regular condition based wait-loop. In that case we must
+ * serialize against wakeups such that any possible in-flight TASK_RUNNING stores
+ * will not collide with our state change.
+ */
+#define set_special_state(state_value)					\
+	do {								\
+		unsigned long flags; /* may shadow */			\
+		raw_spin_lock_irqsave(&current->pi_lock, flags);	\
+		current->state = (state_value);				\
+		raw_spin_unlock_irqrestore(&current->pi_lock, flags);	\
+	} while (0)
+
 #endif
 
 /* Task command name length: */
diff --git a/include/linux/sched/signal.h b/include/linux/sched/signal.h
index a7ce74c74e49..113d1ad1ced7 100644
--- a/include/linux/sched/signal.h
+++ b/include/linux/sched/signal.h
@@ -280,7 +280,7 @@ static inline void kernel_signal_stop(void)
 {
 	spin_lock_irq(&current->sighand->siglock);
 	if (current->jobctl & JOBCTL_STOP_DEQUEUED)
-		__set_current_state(TASK_STOPPED);
+		set_special_state(TASK_STOPPED);
 	spin_unlock_irq(&current->sighand->siglock);
 
 	schedule();
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 5e10aaeebfcc..3898a8047c11 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -3498,23 +3498,8 @@ static void __sched notrace __schedule(bool preempt)
 
 void __noreturn do_task_dead(void)
 {
-	/*
-	 * The setting of TASK_RUNNING by try_to_wake_up() may be delayed
-	 * when the following two conditions become true.
-	 *   - There is race condition of mmap_sem (It is acquired by
-	 *     exit_mm()), and
-	 *   - SMI occurs before setting TASK_RUNINNG.
-	 *     (or hypervisor of virtual machine switches to other guest)
-	 *  As a result, we may become TASK_RUNNING after becoming TASK_DEAD
-	 *
-	 * To avoid it, we have to wait for releasing tsk->pi_lock which
-	 * is held by try_to_wake_up()
-	 */
-	raw_spin_lock_irq(&current->pi_lock);
-	raw_spin_unlock_irq(&current->pi_lock);
-
 	/* Causes final put_task_struct in finish_task_switch(): */
-	__set_current_state(TASK_DEAD);
+	set_special_state(TASK_DEAD);
 
 	/* Tell freezer to ignore us: */
 	current->flags |= PF_NOFREEZE;
diff --git a/kernel/signal.c b/kernel/signal.c
index d4ccea599692..564ee214e80a 100644
--- a/kernel/signal.c
+++ b/kernel/signal.c
@@ -1968,7 +1968,7 @@ static void ptrace_stop(int exit_code, int why, int clear_code, siginfo_t *info)
 	 * atomic with respect to siglock and should be done after the arch
 	 * hook as siglock is released and regrabbed across it.
 	 */
-	set_current_state(TASK_TRACED);
+	set_special_state(TASK_TRACED);
 
 	current->last_siginfo = info;
 	current->exit_code = exit_code;
@@ -2176,7 +2176,7 @@ static bool do_signal_stop(int signr)
 		if (task_participate_group_stop(current))
 			notify = CLD_STOPPED;
 
-		__set_current_state(TASK_STOPPED);
+		set_special_state(TASK_STOPPED);
 		spin_unlock_irq(&current->sighand->siglock);
 
 		/*

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v1] kthread/smpboot: Serialize kthread parking against wakeup
  2018-04-30 11:17         ` Peter Zijlstra
@ 2018-05-01  7:50           ` Kohli, Gaurav
  2018-05-01 10:18             ` Peter Zijlstra
  0 siblings, 1 reply; 39+ messages in thread
From: Kohli, Gaurav @ 2018-05-01  7:50 UTC (permalink / raw)
  To: Peter Zijlstra
  Cc: tglx, mpe, mingo, bigeasy, linux-kernel, linux-arm-msm,
	Neeraj Upadhyay, Will Deacon, Oleg Nesterov

sorry for spam, Adding list

On 4/30/2018 4:47 PM, Peter Zijlstra wrote:
> On Thu, Apr 26, 2018 at 09:23:25PM +0530, Kohli, Gaurav wrote:
>> On 4/26/2018 2:27 PM, Peter Zijlstra wrote:
>>
>>> On Thu, Apr 26, 2018 at 10:41:31AM +0200, Peter Zijlstra wrote:
>>>> diff --git a/kernel/kthread.c b/kernel/kthread.c
>>>> index cd50e99202b0..4b6503c6a029 100644
>>>> --- a/kernel/kthread.c
>>>> +++ b/kernel/kthread.c
>>>> @@ -177,12 +177,13 @@ void *kthread_probe_data(struct task_struct *task)
>>>>    static void __kthread_parkme(struct kthread *self)
>>>>    {
>>>> -	__set_current_state(TASK_PARKED);
>>>> -	while (test_bit(KTHREAD_SHOULD_PARK, &self->flags)) {
>>>> +	for (;;) {
>>>> +		__set_task_state(TASK_PARKED);
>>> 		set_current_state(TASK_PARKED);
>>>
>>> of course..
>>
>> Hi Peter,
>>
>> Maybe i am missing something , but still that race can come as we don't put task_parked on special state.
>>
>> Controller                                                                       Hotplug
>>
>>                                                                                   Loop
>>
>>                                                                                   Task_Interruptible
>>
>> Set SHOULD_PARK
>>
>> wakeup -> Proceeds
>>
>>                                                                                    Set Running
>>
>>                                                                                    kthread_parkme
>>
>>                                                                                    SET TASK_PARKED
>>
>>                                                                                    schedule
>>
>> Set TASK_RUNNING
>>
>> Can you please correct ME, if I misunderstood this.
> 
> If that could happen, all wait-loops would be broken. However,
> AFAICT that cannot happen, because ttwu_remote() and schedule()
> serialize on rq->lock. See:
> 
> 
> A						B
> 
> for (;;) {
> 	set_current_state(UNINTERRUPTIBLE);
> 
> 						cond1 = true;
> 						wake_up_process(A)
> 						  lock(A->pi_lock)
> 						  smp_mb__after_spinlock()
> 						  if (A->state & TASK_NORMAL)
> 						    A->on_rq && ttwu_remote()
> 	if (cond1) // true
> 		break;
> 	schedule();
> }
> __set_current_state(RUNNING);
> 

Hi Peter,

Sorry for the late reply and i was on leave.

Thanks for the new patches, We will apply and test for issue reproduction.

But In our older case, where we have seen failure below is the wake up 
path and ftraces, Wakeup occured  and completed before schedule call only.

So final state of CPUHP is running not parked. I have also pasted debug 
ftraces that we got during issue reproduction.

Here wakeup for cpuhp is below:

takedown_cpu-> kthread_park-> wake_up_process


  39,034,311,742,395  apps (10240)        Trace Printk 
cpuhp/0  (16)  [000]  39015.625000: <debug> __kthread_parkme state=512 
task=ffffffcc7458e680 flags: 0x5 -> state 5 -> state is parked inside 
parkme function

39,034,311,846,510  apps (10240)        Trace Printk 
cpuhp/0  (16)  [000]  39015.625000: <debug> before schedule 
__kthread_parkme state=0 task=ffffffcc7458e680 flags: 0xd  ->  just 
before schedule call, state is running

tatic void __kthread_parkme(struct kthread *self)

{

         __set_current_state(TASK_PARKED);

         while (test_bit(KTHREAD_SHOULD_PARK, &self->flags)) {

                 if (!test_and_set_bit(KTHREAD_IS_PARKED, &self->flags))

                         complete(&self->parked);

                 schedule();

                 __set_current_state(TASK_PARKED);

         }

         clear_bit(KTHREAD_IS_PARKED, &self->flags);

         __set_current_state(TASK_RUNNING);

}

So my point is here also, if it is reschedule then it can set 
TASK_PARKED, but it seems after takedown_cpu call this thread never get 
a chance to run, So final state is TASK_RUNNING.

In our current fix also can't we observe same scenario where final state 
is TASK_RUNNING.

Regards

Gaurav

> for (;;) {
> 	set_current_state(UNINTERRUPTIBLE);
> 	if (cond2)
> 		break;
> 
> 	schedule();
> 	  lock(rq->lock)
> 	  smp_mb__after_spinlock();
> 	  deactivate_task(A);
> 	  <sched-out>
> 	  unlock(rq->lock);
> 						      rq = __task_rq_lock(A)
> 						      if (A->on_rq) // false
> 						        A->state = TASK_RUNNING;
> 						      __task_rq_unlock(rq)
> 
> 
> Either A's schedule() must observe RUNNING (not shown) or B must
> observe !A->on_rq (shown) and not issue the store.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-arm-msm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

-- 
Qualcomm India Private Limited, on behalf of Qualcomm Innovation Center, 
Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project.

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v1] kthread/smpboot: Serialize kthread parking against wakeup
  2018-05-01  7:50           ` Kohli, Gaurav
@ 2018-05-01 10:18             ` Peter Zijlstra
  2018-05-01 10:40               ` Peter Zijlstra
                                 ` (2 more replies)
  0 siblings, 3 replies; 39+ messages in thread
From: Peter Zijlstra @ 2018-05-01 10:18 UTC (permalink / raw)
  To: Kohli, Gaurav
  Cc: tglx, mpe, mingo, bigeasy, linux-kernel, linux-arm-msm,
	Neeraj Upadhyay, Will Deacon, Oleg Nesterov

On Tue, May 01, 2018 at 01:20:26PM +0530, Kohli, Gaurav wrote:
> But In our older case, where we have seen failure below is the wake up path
> and ftraces, Wakeup occured  and completed before schedule call only.
> 
> So final state of CPUHP is running not parked. I have also pasted debug
> ftraces that we got during issue reproduction.
> 
> Here wakeup for cpuhp is below:
> 
> takedown_cpu-> kthread_park-> wake_up_process
> 
> 
>  39,034,311,742,395  apps (10240)        Trace Printk cpuhp/0  (16)  [000]
> 39015.625000: <debug> __kthread_parkme state=512 task=ffffffcc7458e680
> flags: 0x5 -> state 5 -> state is parked inside parkme function
> 
> 39,034,311,846,510  apps (10240)        Trace Printk cpuhp/0  (16)  [000]
> 39015.625000: <debug> before schedule __kthread_parkme state=0
> task=ffffffcc7458e680 flags: 0xd  ->  just before schedule call, state is
> running
> 
> tatic void __kthread_parkme(struct kthread *self)
> 
> {
> 
>         __set_current_state(TASK_PARKED);
> 
>         while (test_bit(KTHREAD_SHOULD_PARK, &self->flags)) {
> 
>                 if (!test_and_set_bit(KTHREAD_IS_PARKED, &self->flags))
> 
>                         complete(&self->parked);
> 
>                 schedule();
> 
>                 __set_current_state(TASK_PARKED);
> 
>         }
> 
>         clear_bit(KTHREAD_IS_PARKED, &self->flags);
> 
>         __set_current_state(TASK_RUNNING);
> 
> }
> 
> So my point is here also, if it is reschedule then it can set TASK_PARKED,
> but it seems after takedown_cpu call this thread never get a chance to run,
> So final state is TASK_RUNNING.
> 
> In our current fix also can't we observe same scenario where final state is
> TASK_RUNNING.

I'm not sure I understand your concern. Loosing the TASK_PARKED store
with the above code is obviously bad. But with the loop as proposed I
don't see a problem.

takedown_cpu() can proceed beyond smpboot_park_threads() and kill the
CPU before any of the threads are parked -- per having the complete()
before hitting schedule().

And, afaict, that is harmless. When we go offline, sched_cpu_dying() ->
migrate_tasks() will migrate any still runnable threads off the cpu.
But because at this point the thread must be in the PARKED wait-loop, it
will hit schedule() and go to sleep eventually.

Also note that kthread_unpark() does __kthread_bind() to rebind the
threads.

Aaaah... I think I've spotted a problem there. We clear SHOULD_PARK
before we rebind, so if the thread lost the first PARKED store,
does the completion, gets migrated, cycles through the loop and now
observes !SHOULD_PARK and bails the wait-loop, then __kthread_bind()
will forever wait.

Is that what you had in mind?

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v1] kthread/smpboot: Serialize kthread parking against wakeup
  2018-05-01 10:18             ` Peter Zijlstra
@ 2018-05-01 10:40               ` Peter Zijlstra
  2018-05-01 10:40               ` Kohli, Gaurav
  2018-05-01 10:44               ` Peter Zijlstra
  2 siblings, 0 replies; 39+ messages in thread
From: Peter Zijlstra @ 2018-05-01 10:40 UTC (permalink / raw)
  To: Kohli, Gaurav
  Cc: tglx, mpe, mingo, bigeasy, linux-kernel, linux-arm-msm,
	Neeraj Upadhyay, Will Deacon, Oleg Nesterov

On Tue, May 01, 2018 at 12:18:45PM +0200, Peter Zijlstra wrote:
> Aaaah... I think I've spotted a problem there. We clear SHOULD_PARK
> before we rebind, so if the thread lost the first PARKED store,
> does the completion, gets migrated, cycles through the loop and now
> observes !SHOULD_PARK and bails the wait-loop, then __kthread_bind()
> will forever wait.
> 
> Is that what you had in mind?

Another possible problem is concurrent thread_park(), if both observe
!IS_PARKED, we'll end up with 2 threads waiting on the completion, but
we only do a single complete().

Of course, this might not be a suppored use of the API, and I don't
think this will ever actually happen. But the whole !test_bit(IS_PARKED)
thing seems to suggest it is fine calling this on an already parked
thread.

Confusing stuff that should be cleared up in any case.

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v1] kthread/smpboot: Serialize kthread parking against wakeup
  2018-05-01 10:18             ` Peter Zijlstra
  2018-05-01 10:40               ` Peter Zijlstra
@ 2018-05-01 10:40               ` Kohli, Gaurav
  2018-05-01 11:31                 ` Peter Zijlstra
  2018-05-01 10:44               ` Peter Zijlstra
  2 siblings, 1 reply; 39+ messages in thread
From: Kohli, Gaurav @ 2018-05-01 10:40 UTC (permalink / raw)
  To: Peter Zijlstra
  Cc: tglx, mpe, mingo, bigeasy, linux-kernel, linux-arm-msm,
	Neeraj Upadhyay, Will Deacon, Oleg Nesterov



On 5/1/2018 3:48 PM, Peter Zijlstra wrote:
> On Tue, May 01, 2018 at 01:20:26PM +0530, Kohli, Gaurav wrote:
>> But In our older case, where we have seen failure below is the wake up path
>> and ftraces, Wakeup occured  and completed before schedule call only.
>>
>> So final state of CPUHP is running not parked. I have also pasted debug
>> ftraces that we got during issue reproduction.
>>
>> Here wakeup for cpuhp is below:
>>
>> takedown_cpu-> kthread_park-> wake_up_process
>>
>>
>>   39,034,311,742,395  apps (10240)        Trace Printk cpuhp/0  (16)  [000]
>> 39015.625000: <debug> __kthread_parkme state=512 task=ffffffcc7458e680
>> flags: 0x5 -> state 5 -> state is parked inside parkme function
>>
>> 39,034,311,846,510  apps (10240)        Trace Printk cpuhp/0  (16)  [000]
>> 39015.625000: <debug> before schedule __kthread_parkme state=0
>> task=ffffffcc7458e680 flags: 0xd  ->  just before schedule call, state is
>> running
>>
>> tatic void __kthread_parkme(struct kthread *self)
>>
>> {
>>
>>          __set_current_state(TASK_PARKED);
>>
>>          while (test_bit(KTHREAD_SHOULD_PARK, &self->flags)) {
>>
>>                  if (!test_and_set_bit(KTHREAD_IS_PARKED, &self->flags))
>>
>>                          complete(&self->parked);
>>
>>                  schedule();
>>
>>                  __set_current_state(TASK_PARKED);
>>
>>          }
>>
>>          clear_bit(KTHREAD_IS_PARKED, &self->flags);
>>
>>          __set_current_state(TASK_RUNNING);
>>
>> }
>>
>> So my point is here also, if it is reschedule then it can set TASK_PARKED,
>> but it seems after takedown_cpu call this thread never get a chance to run,
>> So final state is TASK_RUNNING.
>>
>> In our current fix also can't we observe same scenario where final state is
>> TASK_RUNNING.
> 
> I'm not sure I understand your concern. Loosing the TASK_PARKED store
> with the above code is obviously bad. But with the loop as proposed I
> don't see a problem.

Yes with loop, it will reset TASK_PARKED but that is not happening in 
the dumps we have seen. Here before schedule state is RUNNING and cpuhp
got migrate to some core but never get a chance to run so state is running.
> 
> takedown_cpu() can proceed beyond smpboot_park_threads() and kill the
> CPU before any of the threads are parked -- per having the complete()
> before hitting schedule().
> 
> And, afaict, that is harmless. When we go offline, sched_cpu_dying() ->
> migrate_tasks() will migrate any still runnable threads off the cpu.
> But because at this point the thread must be in the PARKED wait-loop, it
> will hit schedule() and go to sleep eventually.
> 
> Also note that kthread_unpark() does __kthread_bind() to rebind the
> threads.
> 
> Aaaah... I think I've spotted a problem there. We clear SHOULD_PARK
> before we rebind, so if the thread lost the first PARKED store,
> does the completion, gets migrated, cycles through the loop and now
> observes !SHOULD_PARK and bails the wait-loop, then __kthread_bind()
> will forever wait.
> 

So during next unpark
__kthread_unpark -> __kthread_bind -> wait_task_inactive (this got 
failed, as current state is running so failed on below call:


  while (task_running(rq, p)) {
                         if (match_state && unlikely(p->state != 
match_state))
                                 return 0;
                         cpu_relax();
                 }

and gives warning:


         if (!wait_task_inactive(p, state)) {
                 WARN_ON(1);
                 return; -> return from here, and further binding call 
fail which is after this code.
         }


finally it is giving bug_on here as we failed to rebind hotplug to our core:
                         }
                         kthread_parkme();
                         /* We might have been woken for stop */
                         continue;
                 }

                 BUG_ON(td->cpu != smp_processor_id()); panic occured.


So it seems we always have to be in PARKED state only , not miss any 
single instance.
> Is that what you had in mind?
> --
> To unsubscribe from this list: send the line "unsubscribe linux-arm-msm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

-- 
Qualcomm India Private Limited, on behalf of Qualcomm Innovation Center, 
Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project.

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v1] kthread/smpboot: Serialize kthread parking against wakeup
  2018-05-01 10:18             ` Peter Zijlstra
  2018-05-01 10:40               ` Peter Zijlstra
  2018-05-01 10:40               ` Kohli, Gaurav
@ 2018-05-01 10:44               ` Peter Zijlstra
  2 siblings, 0 replies; 39+ messages in thread
From: Peter Zijlstra @ 2018-05-01 10:44 UTC (permalink / raw)
  To: Kohli, Gaurav
  Cc: tglx, mpe, mingo, bigeasy, linux-kernel, linux-arm-msm,
	Neeraj Upadhyay, Will Deacon, Oleg Nesterov

On Tue, May 01, 2018 at 12:18:45PM +0200, Peter Zijlstra wrote:
> Aaaah... I think I've spotted a problem there. We clear SHOULD_PARK
> before we rebind, so if the thread lost the first PARKED store,
> does the completion, gets migrated, cycles through the loop and now
> observes !SHOULD_PARK and bails the wait-loop, then __kthread_bind()
> will forever wait.


Something like so perhaps...

--- a/kernel/kthread.c
+++ b/kernel/kthread.c
@@ -451,6 +451,21 @@ void kthread_unpark(struct task_struct *
 {
 	struct kthread *kthread = to_kthread(k);
 
+	if (test_bit(KTHREAD_IS_PARKED)) {
+		/*
+		 * Newly created kthread was parked when the CPU was offline.
+		 * The binding was lost and we need to set it again.
+		 */
+		if (test_bit(KTHREAD_IS_PER_CPU, &kthread->flags))
+			__kthread_bind(k, kthread->cpu, TASK_PARKED);
+	}
+
+	/*
+	 * Ensures the IS_PARKED load precedes the !SHOULD_PARK store.
+	 * matched by the smp_mb() from test_and_set_bit() in __kthread_parkme().
+	 */
+	smp_mb__before_atomic();
+
 	clear_bit(KTHREAD_SHOULD_PARK, &kthread->flags);
 	/*
 	 * We clear the IS_PARKED bit here as we don't wait
@@ -458,15 +473,8 @@ void kthread_unpark(struct task_struct *
 	 * park before that happens we'd see the IS_PARKED bit
 	 * which might be about to be cleared.
 	 */
-	if (test_and_clear_bit(KTHREAD_IS_PARKED, &kthread->flags)) {
-		/*
-		 * Newly created kthread was parked when the CPU was offline.
-		 * The binding was lost and we need to set it again.
-		 */
-		if (test_bit(KTHREAD_IS_PER_CPU, &kthread->flags))
-			__kthread_bind(k, kthread->cpu, TASK_PARKED);
+	if (test_and_clear_bit(KTHREAD_IS_PARKED, &kthread->flags))
 		wake_up_state(k, TASK_PARKED);
-	}
 }
 EXPORT_SYMBOL_GPL(kthread_unpark);
 

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v1] kthread/smpboot: Serialize kthread parking against wakeup
  2018-05-01 10:40               ` Kohli, Gaurav
@ 2018-05-01 11:31                 ` Peter Zijlstra
  2018-05-01 11:46                   ` Kohli, Gaurav
  0 siblings, 1 reply; 39+ messages in thread
From: Peter Zijlstra @ 2018-05-01 11:31 UTC (permalink / raw)
  To: Kohli, Gaurav
  Cc: tglx, mpe, mingo, bigeasy, linux-kernel, linux-arm-msm,
	Neeraj Upadhyay, Will Deacon, Oleg Nesterov

On Tue, May 01, 2018 at 04:10:53PM +0530, Kohli, Gaurav wrote:
> Yes with loop, it will reset TASK_PARKED but that is not happening in the
> dumps we have seen.

But was that with or without the fixed wait-loop? I don't care about
stuff you might have seen with the current code, that is clearly broken.

> > takedown_cpu() can proceed beyond smpboot_park_threads() and kill the
> > CPU before any of the threads are parked -- per having the complete()
> > before hitting schedule().
> > 
> > And, afaict, that is harmless. When we go offline, sched_cpu_dying() ->
> > migrate_tasks() will migrate any still runnable threads off the cpu.
> > But because at this point the thread must be in the PARKED wait-loop, it
> > will hit schedule() and go to sleep eventually.
> > 
> > Also note that kthread_unpark() does __kthread_bind() to rebind the
> > threads.
> > 
> > Aaaah... I think I've spotted a problem there. We clear SHOULD_PARK
> > before we rebind, so if the thread lost the first PARKED store,
> > does the completion, gets migrated, cycles through the loop and now
> > observes !SHOULD_PARK and bails the wait-loop, then __kthread_bind()
> > will forever wait.
> > 
> 
> So during next unpark
> __kthread_unpark -> __kthread_bind -> wait_task_inactive (this got failed,
> as current state is running so failed on below call:

Aah, yes, I seem to have mis-remembered how wait_task_inactive() works.
And it is indeed still a problem..

Let me ponder what the best solution is, it's a bit of a mess.

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v1] kthread/smpboot: Serialize kthread parking against wakeup
  2018-05-01 11:31                 ` Peter Zijlstra
@ 2018-05-01 11:46                   ` Kohli, Gaurav
  2018-05-01 13:19                     ` Peter Zijlstra
  0 siblings, 1 reply; 39+ messages in thread
From: Kohli, Gaurav @ 2018-05-01 11:46 UTC (permalink / raw)
  To: Peter Zijlstra
  Cc: tglx, mpe, mingo, bigeasy, linux-kernel, linux-arm-msm,
	Neeraj Upadhyay, Will Deacon, Oleg Nesterov



On 5/1/2018 5:01 PM, Peter Zijlstra wrote:
> On Tue, May 01, 2018 at 04:10:53PM +0530, Kohli, Gaurav wrote:
>> Yes with loop, it will reset TASK_PARKED but that is not happening in the
>> dumps we have seen.
> 
> But was that with or without the fixed wait-loop? I don't care about
> stuff you might have seen with the current code, that is clearly broken.
> 
>>> takedown_cpu() can proceed beyond smpboot_park_threads() and kill the
>>> CPU before any of the threads are parked -- per having the complete()
>>> before hitting schedule().
>>>
>>> And, afaict, that is harmless. When we go offline, sched_cpu_dying() ->
>>> migrate_tasks() will migrate any still runnable threads off the cpu.
>>> But because at this point the thread must be in the PARKED wait-loop, it
>>> will hit schedule() and go to sleep eventually.
>>>
>>> Also note that kthread_unpark() does __kthread_bind() to rebind the
>>> threads.
>>>
>>> Aaaah... I think I've spotted a problem there. We clear SHOULD_PARK
>>> before we rebind, so if the thread lost the first PARKED store,
>>> does the completion, gets migrated, cycles through the loop and now
>>> observes !SHOULD_PARK and bails the wait-loop, then __kthread_bind()
>>> will forever wait.
>>>
>>
>> So during next unpark
>> __kthread_unpark -> __kthread_bind -> wait_task_inactive (this got failed,
>> as current state is running so failed on below call:
> 
> Aah, yes, I seem to have mis-remembered how wait_task_inactive() works.
> And it is indeed still a problem..
> 
> Let me ponder what the best solution is, it's a bit of a mess.
> 

Sure , Thanks a lot.
-- 
Qualcomm India Private Limited, on behalf of Qualcomm Innovation Center, 
Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project.

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v1] kthread/smpboot: Serialize kthread parking against wakeup
  2018-05-01 11:46                   ` Kohli, Gaurav
@ 2018-05-01 13:19                     ` Peter Zijlstra
  2018-05-02  5:15                       ` Kohli, Gaurav
  0 siblings, 1 reply; 39+ messages in thread
From: Peter Zijlstra @ 2018-05-01 13:19 UTC (permalink / raw)
  To: Kohli, Gaurav
  Cc: tglx, mpe, mingo, bigeasy, linux-kernel, linux-arm-msm,
	Neeraj Upadhyay, Will Deacon, Oleg Nesterov

> On 5/1/2018 5:01 PM, Peter Zijlstra wrote:

> > Let me ponder what the best solution is, it's a bit of a mess.

So there's:

 - TASK_PARKED, which we can make a special state; this solves the
   problem because then wait_task_inactive() is guaranteed to see
   TASK_PARKED and DTRT.

 - complete(&kthread->parked), which we can do inside schedule(); this
   solves the problem because then kthread_park() will not return early
   and the task really is blocked.

and I'm fairly sure I thought of a 3rd way to cure things, but now that
I'm writing things down I cannot seem to remember :/ -- could be we muck
with wait_task_inactive().

In any case, I hate all of them, but I think the completion one is the
least horrible because it gives the strongest guarantees and cleans up
most. But promoting TASK_PARKED to special certainly is the earier
patch.

The below boots, but that's about all I did with it. Opinions?

---

--- a/include/linux/kthread.h
+++ b/include/linux/kthread.h
@@ -62,6 +62,7 @@ void *kthread_probe_data(struct task_str
 int kthread_park(struct task_struct *k);
 void kthread_unpark(struct task_struct *k);
 void kthread_parkme(void);
+void kthread_park_complete(struct task_struct *k);
 
 int kthreadd(void *unused);
 extern struct task_struct *kthreadd_task;
--- a/kernel/kthread.c
+++ b/kernel/kthread.c
@@ -55,7 +55,6 @@ enum KTHREAD_BITS {
 	KTHREAD_IS_PER_CPU = 0,
 	KTHREAD_SHOULD_STOP,
 	KTHREAD_SHOULD_PARK,
-	KTHREAD_IS_PARKED,
 };
 
 static inline void set_kthread_struct(void *kthread)
@@ -177,14 +176,12 @@ void *kthread_probe_data(struct task_str
 
 static void __kthread_parkme(struct kthread *self)
 {
-	__set_current_state(TASK_PARKED);
-	while (test_bit(KTHREAD_SHOULD_PARK, &self->flags)) {
-		if (!test_and_set_bit(KTHREAD_IS_PARKED, &self->flags))
-			complete(&self->parked);
+	for (;;) {
+		set_current_state(TASK_PARKED);
+		if (!test_bit(KTHREAD_SHOULD_PARK, &self->flags))
+			break;
 		schedule();
-		__set_current_state(TASK_PARKED);
 	}
-	clear_bit(KTHREAD_IS_PARKED, &self->flags);
 	__set_current_state(TASK_RUNNING);
 }
 
@@ -194,6 +191,11 @@ void kthread_parkme(void)
 }
 EXPORT_SYMBOL_GPL(kthread_parkme);
 
+void kthread_park_complete(struct task_struct *k)
+{
+	complete(&to_kthread(k)->parked);
+}
+
 static int kthread(void *_create)
 {
 	/* Copy data: it's on kthread's stack */
@@ -450,22 +452,15 @@ void kthread_unpark(struct task_struct *
 {
 	struct kthread *kthread = to_kthread(k);
 
-	clear_bit(KTHREAD_SHOULD_PARK, &kthread->flags);
 	/*
-	 * We clear the IS_PARKED bit here as we don't wait
-	 * until the task has left the park code. So if we'd
-	 * park before that happens we'd see the IS_PARKED bit
-	 * which might be about to be cleared.
+	 * Newly created kthread was parked when the CPU was offline.
+	 * The binding was lost and we need to set it again.
 	 */
-	if (test_and_clear_bit(KTHREAD_IS_PARKED, &kthread->flags)) {
-		/*
-		 * Newly created kthread was parked when the CPU was offline.
-		 * The binding was lost and we need to set it again.
-		 */
-		if (test_bit(KTHREAD_IS_PER_CPU, &kthread->flags))
-			__kthread_bind(k, kthread->cpu, TASK_PARKED);
-		wake_up_state(k, TASK_PARKED);
-	}
+	if (test_bit(KTHREAD_IS_PER_CPU, &kthread->flags))
+		__kthread_bind(k, kthread->cpu, TASK_PARKED);
+
+	clear_bit(KTHREAD_SHOULD_PARK, &kthread->flags);
+	wake_up_state(k, TASK_PARKED);
 }
 EXPORT_SYMBOL_GPL(kthread_unpark);
 
@@ -488,12 +483,13 @@ int kthread_park(struct task_struct *k)
 	if (WARN_ON(k->flags & PF_EXITING))
 		return -ENOSYS;
 
-	if (!test_bit(KTHREAD_IS_PARKED, &kthread->flags)) {
-		set_bit(KTHREAD_SHOULD_PARK, &kthread->flags);
-		if (k != current) {
-			wake_up_process(k);
-			wait_for_completion(&kthread->parked);
-		}
+	if (WARN_ON_ONCE(test_bit(KTHREAD_SHOULD_PARK, &kthread->flags)))
+		return -EBUSY;
+
+	set_bit(KTHREAD_SHOULD_PARK, &kthread->flags);
+	if (k != current) {
+		wake_up_process(k);
+		wait_for_completion(&kthread->parked);
 	}
 
 	return 0;
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -7,6 +7,8 @@
  */
 #include "sched.h"
 
+#include <linux/kthread.h>
+
 #include <asm/switch_to.h>
 #include <asm/tlb.h>
 
@@ -2718,20 +2720,28 @@ static struct rq *finish_task_switch(str
 		membarrier_mm_sync_core_before_usermode(mm);
 		mmdrop(mm);
 	}
-	if (unlikely(prev_state == TASK_DEAD)) {
-		if (prev->sched_class->task_dead)
-			prev->sched_class->task_dead(prev);
+	if (unlikely(prev_state & (TASK_DEAD|TASK_PARKED))) {
+		switch (prev_state) {
+		case TASK_DEAD:
+			if (prev->sched_class->task_dead)
+				prev->sched_class->task_dead(prev);
+
+			/*
+			 * Remove function-return probe instances associated with this
+			 * task and put them back on the free list.
+			 */
+			kprobe_flush_task(prev);
 
-		/*
-		 * Remove function-return probe instances associated with this
-		 * task and put them back on the free list.
-		 */
-		kprobe_flush_task(prev);
+			/* Task is done with its stack. */
+			put_task_stack(prev);
 
-		/* Task is done with its stack. */
-		put_task_stack(prev);
+			put_task_struct(prev);
+			break;
 
-		put_task_struct(prev);
+		case TASK_PARKED:
+			kthread_park_complete(prev);
+			break;
+		}
 	}
 
 	tick_nohz_task_switch();

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v1] kthread/smpboot: Serialize kthread parking against wakeup
  2018-05-01 13:19                     ` Peter Zijlstra
@ 2018-05-02  5:15                       ` Kohli, Gaurav
  2018-05-02  8:20                         ` Peter Zijlstra
  0 siblings, 1 reply; 39+ messages in thread
From: Kohli, Gaurav @ 2018-05-02  5:15 UTC (permalink / raw)
  To: Peter Zijlstra
  Cc: tglx, mpe, mingo, bigeasy, linux-kernel, linux-arm-msm,
	Neeraj Upadhyay, Will Deacon, Oleg Nesterov



On 5/1/2018 6:49 PM, Peter Zijlstra wrote:
>> On 5/1/2018 5:01 PM, Peter Zijlstra wrote:
> 
>>> Let me ponder what the best solution is, it's a bit of a mess.
> 
> So there's:
> 
>   - TASK_PARKED, which we can make a special state; this solves the
>     problem because then wait_task_inactive() is guaranteed to see
>     TASK_PARKED and DTRT.

Yes this should work, as it takes pi-lock and blocking kthread_parkme.
> 
>   - complete(&kthread->parked), which we can do inside schedule(); this
>     solves the problem because then kthread_park() will not return early
>     and the task really is blocked.

I think complete will not help, as problem is like below :

Control Thread                                CPUHP thread
					
					      cpuhp_thread_fun
					      Wake control thread
					      complete(&st->done);

takedown_cpu
kthread_park
set_bit(KTHREAD_SHOULD_PARK

					     Here cpuhp is looping,
					//success case
					     Generally when issue is not
					     coming
					     it schedule out by below :
                                            ht->thread_should_run(td->cpu
					      scheduler
					//failure case
					before schedule
					loop check
					(kthread_should_park()
				         enter here as PARKED set

wake_up_process(k)
					__kthread_parkme
					 complete(&self->parked);
SETS RUNNING
		                         schedule			
wait_for_completion(&kthread->parked);


So even we protect complete, it may see TASK_RUNNING as final state.
That's why we took pi-lock , So wake_up_process either see TASK_RUNNING
so return or it will exit early before parkme call.

Please correct me , if i misunderstood it. With pi-lock approach we are 
not seeing issue. SO you first solution should fix this(put parked in 
special state).

Regards
Gaurav
					

> 
> and I'm fairly sure I thought of a 3rd way to cure things, but now that
> I'm writing things down I cannot seem to remember :/ -- could be we muck
> with wait_task_inactive().
> 
> In any case, I hate all of them, but I think the completion one is the
> least horrible because it gives the strongest guarantees and cleans up
> most. But promoting TASK_PARKED to special certainly is the earier
> patch.
> 
> The below boots, but that's about all I did with it. Opinions?
> 
> ---
> 
> --- a/include/linux/kthread.h
> +++ b/include/linux/kthread.h
> @@ -62,6 +62,7 @@ void *kthread_probe_data(struct task_str
>   int kthread_park(struct task_struct *k);
>   void kthread_unpark(struct task_struct *k);
>   void kthread_parkme(void);
> +void kthread_park_complete(struct task_struct *k);
>   
>   int kthreadd(void *unused);
>   extern struct task_struct *kthreadd_task;
> --- a/kernel/kthread.c
> +++ b/kernel/kthread.c
> @@ -55,7 +55,6 @@ enum KTHREAD_BITS {
>   	KTHREAD_IS_PER_CPU = 0,
>   	KTHREAD_SHOULD_STOP,
>   	KTHREAD_SHOULD_PARK,
> -	KTHREAD_IS_PARKED,
>   };
>   
>   static inline void set_kthread_struct(void *kthread)
> @@ -177,14 +176,12 @@ void *kthread_probe_data(struct task_str
>   
>   static void __kthread_parkme(struct kthread *self)
>   {
> -	__set_current_state(TASK_PARKED);
> -	while (test_bit(KTHREAD_SHOULD_PARK, &self->flags)) {
> -		if (!test_and_set_bit(KTHREAD_IS_PARKED, &self->flags))
> -			complete(&self->parked);
> +	for (;;) {
> +		set_current_state(TASK_PARKED);
> +		if (!test_bit(KTHREAD_SHOULD_PARK, &self->flags))
> +			break;
>   		schedule();
> -		__set_current_state(TASK_PARKED);
>   	}
> -	clear_bit(KTHREAD_IS_PARKED, &self->flags);
>   	__set_current_state(TASK_RUNNING);
>   }
>   
> @@ -194,6 +191,11 @@ void kthread_parkme(void)
>   }
>   EXPORT_SYMBOL_GPL(kthread_parkme);
>   
> +void kthread_park_complete(struct task_struct *k)
> +{
> +	complete(&to_kthread(k)->parked);
> +}
> +
>   static int kthread(void *_create)
>   {
>   	/* Copy data: it's on kthread's stack */
> @@ -450,22 +452,15 @@ void kthread_unpark(struct task_struct *
>   {
>   	struct kthread *kthread = to_kthread(k);
>   
> -	clear_bit(KTHREAD_SHOULD_PARK, &kthread->flags);
>   	/*
> -	 * We clear the IS_PARKED bit here as we don't wait
> -	 * until the task has left the park code. So if we'd
> -	 * park before that happens we'd see the IS_PARKED bit
> -	 * which might be about to be cleared.
> +	 * Newly created kthread was parked when the CPU was offline.
> +	 * The binding was lost and we need to set it again.
>   	 */
> -	if (test_and_clear_bit(KTHREAD_IS_PARKED, &kthread->flags)) {
> -		/*
> -		 * Newly created kthread was parked when the CPU was offline.
> -		 * The binding was lost and we need to set it again.
> -		 */
> -		if (test_bit(KTHREAD_IS_PER_CPU, &kthread->flags))
> -			__kthread_bind(k, kthread->cpu, TASK_PARKED);
> -		wake_up_state(k, TASK_PARKED);
> -	}
> +	if (test_bit(KTHREAD_IS_PER_CPU, &kthread->flags))
> +		__kthread_bind(k, kthread->cpu, TASK_PARKED);
> +
> +	clear_bit(KTHREAD_SHOULD_PARK, &kthread->flags);
> +	wake_up_state(k, TASK_PARKED);
>   }
>   EXPORT_SYMBOL_GPL(kthread_unpark);
>   
> @@ -488,12 +483,13 @@ int kthread_park(struct task_struct *k)
>   	if (WARN_ON(k->flags & PF_EXITING))
>   		return -ENOSYS;
>   
> -	if (!test_bit(KTHREAD_IS_PARKED, &kthread->flags)) {
> -		set_bit(KTHREAD_SHOULD_PARK, &kthread->flags);
> -		if (k != current) {
> -			wake_up_process(k);
> -			wait_for_completion(&kthread->parked);
> -		}
> +	if (WARN_ON_ONCE(test_bit(KTHREAD_SHOULD_PARK, &kthread->flags)))
> +		return -EBUSY;
> +
> +	set_bit(KTHREAD_SHOULD_PARK, &kthread->flags);
> +	if (k != current) {
> +		wake_up_process(k);
> +		wait_for_completion(&kthread->parked);
>   	}
>   
>   	return 0;
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -7,6 +7,8 @@
>    */
>   #include "sched.h"
>   
> +#include <linux/kthread.h>
> +
>   #include <asm/switch_to.h>
>   #include <asm/tlb.h>
>   
> @@ -2718,20 +2720,28 @@ static struct rq *finish_task_switch(str
>   		membarrier_mm_sync_core_before_usermode(mm);
>   		mmdrop(mm);
>   	}
> -	if (unlikely(prev_state == TASK_DEAD)) {
> -		if (prev->sched_class->task_dead)
> -			prev->sched_class->task_dead(prev);
> +	if (unlikely(prev_state & (TASK_DEAD|TASK_PARKED))) {
> +		switch (prev_state) {
> +		case TASK_DEAD:
> +			if (prev->sched_class->task_dead)
> +				prev->sched_class->task_dead(prev);
> +
> +			/*
> +			 * Remove function-return probe instances associated with this
> +			 * task and put them back on the free list.
> +			 */
> +			kprobe_flush_task(prev);
>   
> -		/*
> -		 * Remove function-return probe instances associated with this
> -		 * task and put them back on the free list.
> -		 */
> -		kprobe_flush_task(prev);
> +			/* Task is done with its stack. */
> +			put_task_stack(prev);
>   
> -		/* Task is done with its stack. */
> -		put_task_stack(prev);
> +			put_task_struct(prev);
> +			break;
>   
> -		put_task_struct(prev);
> +		case TASK_PARKED:
> +			kthread_park_complete(prev);
> +			break;
> +		}
>   	}
>   
>   	tick_nohz_task_switch();
> 

-- 
Qualcomm India Private Limited, on behalf of Qualcomm Innovation Center, 
Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project.

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v1] kthread/smpboot: Serialize kthread parking against wakeup
  2018-05-02  5:15                       ` Kohli, Gaurav
@ 2018-05-02  8:20                         ` Peter Zijlstra
  2018-05-02 10:13                           ` Kohli, Gaurav
  0 siblings, 1 reply; 39+ messages in thread
From: Peter Zijlstra @ 2018-05-02  8:20 UTC (permalink / raw)
  To: Kohli, Gaurav
  Cc: tglx, mpe, mingo, bigeasy, linux-kernel, linux-arm-msm,
	Neeraj Upadhyay, Will Deacon, Oleg Nesterov

On Wed, May 02, 2018 at 10:45:52AM +0530, Kohli, Gaurav wrote:
> On 5/1/2018 6:49 PM, Peter Zijlstra wrote:

> >   - complete(&kthread->parked), which we can do inside schedule(); this
> >     solves the problem because then kthread_park() will not return early
> >     and the task really is blocked.
> 
> I think complete will not help, as problem is like below :
> 
> Control Thread                                CPUHP thread
> 					
> 					      cpuhp_thread_fun
> 					      Wake control thread
> 					      complete(&st->done);
> 
> takedown_cpu
> kthread_park
> set_bit(KTHREAD_SHOULD_PARK
> 
> 					     Here cpuhp is looping,
> 					//success case
> 					     Generally when issue is not
> 					     coming
> 					     it schedule out by below :
>                                            ht->thread_should_run(td->cpu
> 					      scheduler
> 					//failure case
> 					before schedule
> 					loop check
> 					(kthread_should_park()
> 				         enter here as PARKED set
> 
> wake_up_process(k)

If k has TASK_PARKED, then wake_up_process() which uses TASK_NORMAL will
no-op, because:

	TASK_PARKED & TASK_NORMAL == 0

> 					__kthread_parkme
> 					 complete(&self->parked);
> SETS RUNNING
> 		                         schedule			

But suppose, you do get that store, and we get to schedule with
TASK_RUNNING, then schedule will no-op and we'll go around the loop and
not complete.

See also: lkml.kernel.org/r/20180430111744.GE4082@hirez.programming.kicks-ass.net

Either TASK_RUNNING gets set before we do schedule() and we go around
again, re-set TASK_PARKED, resched the condition and re-call schedule(),
or we schedule() first and ttwu() will not issue the TASK_RUNNING store.

In either case, we'll eventually hit schedule() with TASK_PARKED. Then,
and only then will the complete() happen.

> wait_for_completion(&kthread->parked);

The point is, we'll only ever complete ^ that completion when we've
scheduled out the task in TASK_PARKED state. If the task didn't get
parked, no completion.


And that is the reason I like this approach above the others. It
guarantees the task really is parked when we ask for it. We don't have
to deal with the task still running and getting migrated to another CPU
nonsense.

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v1] kthread/smpboot: Serialize kthread parking against wakeup
  2018-05-02  8:20                         ` Peter Zijlstra
@ 2018-05-02 10:13                           ` Kohli, Gaurav
  2018-05-07 11:09                             ` Kohli, Gaurav
  0 siblings, 1 reply; 39+ messages in thread
From: Kohli, Gaurav @ 2018-05-02 10:13 UTC (permalink / raw)
  To: Peter Zijlstra
  Cc: tglx, mpe, mingo, bigeasy, linux-kernel, linux-arm-msm,
	Neeraj Upadhyay, Will Deacon, Oleg Nesterov



On 5/2/2018 1:50 PM, Peter Zijlstra wrote:
> On Wed, May 02, 2018 at 10:45:52AM +0530, Kohli, Gaurav wrote:
>> On 5/1/2018 6:49 PM, Peter Zijlstra wrote:
> 
>>>    - complete(&kthread->parked), which we can do inside schedule(); this
>>>      solves the problem because then kthread_park() will not return early
>>>      and the task really is blocked.
>>
>> I think complete will not help, as problem is like below :
>>
>> Control Thread                                CPUHP thread
>> 					
>> 					      cpuhp_thread_fun
>> 					      Wake control thread
>> 					      complete(&st->done);
>>
>> takedown_cpu
>> kthread_park
>> set_bit(KTHREAD_SHOULD_PARK
>>
>> 					     Here cpuhp is looping,
>> 					//success case
>> 					     Generally when issue is not
>> 					     coming
>> 					     it schedule out by below :
>>                                             ht->thread_should_run(td->cpu
>> 					      scheduler
>> 					//failure case
>> 					before schedule
>> 					loop check
>> 					(kthread_should_park()
>> 				         enter here as PARKED set
>>
>> wake_up_process(k)
> 
> If k has TASK_PARKED, then wake_up_process() which uses TASK_NORMAL will
> no-op, because:
> 
> 	TASK_PARKED & TASK_NORMAL == 0
> 
>> 					__kthread_parkme
>> 					 complete(&self->parked);
>> SETS RUNNING
>> 		                         schedule			
> 
> But suppose, you do get that store, and we get to schedule with
> TASK_RUNNING, then schedule will no-op and we'll go around the loop and
> not complete.
> 
> See also: lkml.kernel.org/r/20180430111744.GE4082@hirez.programming.kicks-ass.net
> 
> Either TASK_RUNNING gets set before we do schedule() and we go around
> again, re-set TASK_PARKED, resched the condition and re-call schedule(),
> or we schedule() first and ttwu() will not issue the TASK_RUNNING store.
> 
> In either case, we'll eventually hit schedule() with TASK_PARKED. Then,
> and only then will the complete() happen.
> 
>> wait_for_completion(&kthread->parked);
> 
> The point is, we'll only ever complete ^ that completion when we've
> scheduled out the task in TASK_PARKED state. If the task didn't get
> parked, no completion.

Thanks for the detailed explanation, yes in all cases unpark will 
observe parked state only.
> 
> 
> And that is the reason I like this approach above the others. It
> guarantees the task really is parked when we ask for it. We don't have
> to deal with the task still running and getting migrated to another CPU
> nonsense.
> 

-- 
Qualcomm India Private Limited, on behalf of Qualcomm Innovation Center, 
Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project.

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v1] kthread/smpboot: Serialize kthread parking against wakeup
  2018-05-02 10:13                           ` Kohli, Gaurav
@ 2018-05-07 11:09                             ` Kohli, Gaurav
  2018-05-07 11:23                               ` Kohli, Gaurav
  0 siblings, 1 reply; 39+ messages in thread
From: Kohli, Gaurav @ 2018-05-07 11:09 UTC (permalink / raw)
  To: Peter Zijlstra
  Cc: tglx, mpe, mingo, bigeasy, linux-kernel, linux-arm-msm,
	Neeraj Upadhyay, Will Deacon, Oleg Nesterov



On 5/2/2018 3:43 PM, Kohli, Gaurav wrote:
> 
> 
> On 5/2/2018 1:50 PM, Peter Zijlstra wrote:
>> On Wed, May 02, 2018 at 10:45:52AM +0530, Kohli, Gaurav wrote:
>>> On 5/1/2018 6:49 PM, Peter Zijlstra wrote:
>>
>>>>    - complete(&kthread->parked), which we can do inside schedule(); 
>>>> this
>>>>      solves the problem because then kthread_park() will not return 
>>>> early
>>>>      and the task really is blocked.
>>>
>>> I think complete will not help, as problem is like below :
>>>
>>> Control Thread                                CPUHP thread
>>>
>>>                           cpuhp_thread_fun
>>>                           Wake control thread
>>>                           complete(&st->done);
>>>
>>> takedown_cpu
>>> kthread_park
>>> set_bit(KTHREAD_SHOULD_PARK
>>>
>>>                          Here cpuhp is looping,
>>>                     //success case
>>>                          Generally when issue is not
>>>                          coming
>>>                          it schedule out by below :
>>>                                             
>>> ht->thread_should_run(td->cpu
>>>                           scheduler
>>>                     //failure case
>>>                     before schedule
>>>                     loop check
>>>                     (kthread_should_park()
>>>                          enter here as PARKED set
>>>
>>> wake_up_process(k)
>>
>> If k has TASK_PARKED, then wake_up_process() which uses TASK_NORMAL will
>> no-op, because:
>>
>>     TASK_PARKED & TASK_NORMAL == 0
>>
>>>                     __kthread_parkme
>>>                      complete(&self->parked);
>>> SETS RUNNING
>>>                                  schedule
>>
>> But suppose, you do get that store, and we get to schedule with
>> TASK_RUNNING, then schedule will no-op and we'll go around the loop and
>> not complete.
>>
>> See also: 
>> lkml.kernel.org/r/20180430111744.GE4082@hirez.programming.kicks-ass.net
>>
>> Either TASK_RUNNING gets set before we do schedule() and we go around
>> again, re-set TASK_PARKED, resched the condition and re-call schedule(),
>> or we schedule() first and ttwu() will not issue the TASK_RUNNING store.
>>
>> In either case, we'll eventually hit schedule() with TASK_PARKED. Then,
>> and only then will the complete() happen.
>>
>>> wait_for_completion(&kthread->parked);
>>
>> The point is, we'll only ever complete ^ that completion when we've
>> scheduled out the task in TASK_PARKED state. If the task didn't get
>> parked, no completion.
> 
> Thanks for the detailed explanation, yes in all cases unpark will 
> observe parked state only.
>>
>>
>> And that is the reason I like this approach above the others. It
>> guarantees the task really is parked when we ask for it. We don't have
>> to deal with the task still running and getting migrated to another CPU
>> nonsense.
>>
> 

HI Peter,

We have tested with new patch and still seeing same issue, in this dumps 
we don't have debug traces, but seems there still exist race from code 
review , Can you please check it once:

Controller Thread                                   CPUHP Thread
takedown_cpu
kthread_park
kthread_parkme
Set KTHREAD_SHOULD_PARK
						smpboot_thread_fn
						set Task interruptible
						
						
wake_up_process

                                                 Kthread_parkme
                                                 SET TASK_PARKED
                                                 schedule
			                        raw_spin_lock(&rq->lock)
					
                                                 context_switch

						finish_lock_switch



                                                Case TASK_PARKED
                                                kthread_park_complete


SET TASK_INTERRUPTIBLE

	
And also seeing the same warning during unpark of cpuhp from controller:
  if (!wait_task_inactive(p, state)) {
                 WARN_ON(1);
                 return;
         }
325.065893] [<ffffff8920ed0200>] kthread_unpark+0x80/0xd8
[  325.065902] [<ffffff8920eab754>] bringup_cpu+0xa0/0x12c
[  325.065910] [<ffffff8920eaae90>] cpuhp_invoke_callback+0xb4/0x5c8
[  325.065917] [<ffffff8920eabd98>] cpuhp_up_callbacks+0x3c/0x154
[  325.065924] [<ffffff8920ead220>] _cpu_up+0x134/0x208
[  325.065931] [<ffffff8920ead45c>] do_cpu_up+0x168/0x1a0
[  325.065938] [<ffffff8920ead4b8>] cpu_up+0x24/0x30
[  325.065948] [<ffffff89215b1408>] cpu_subsys_online+0x20/0x2c
[  325.065956] [<ffffff89215aac64>] device_online+0x70/0xb4
[  325.065962] [<ffffff89215aad78>] online_store+0xd0/0xdc
[  325.065971] [<ffffff89215a7424>] dev_attr_store+0x40/0x54
[  325.065982] [<ffffff89210d8a98>] sysfs_kf_write+0x5c/0x74
[  325.065988] [<ffffff89210d7b9c>] kernfs_fop_write+0xcc/0x1ec
[  325.065999] [<ffffff8921049288>] vfs_write+0xb4/0x1d0
[  325.066006] [<ffffff892104a858>] SyS_write+0x60/0xc0
[  325.066014] [<ffffff8920e83770>] el0_svc_naked+0x24/0x28


And after this same crash occured:
[  325.521307] [<ffffff8920ed4aac>] smpboot_thread_fn+0x26c/0x2c8
[  325.527295] [<ffffff8920ecfb24>] kthread+0xf4/0x108

I will put more debug ftraces to check what is going on exactly.
			
Regards
Gaurav

							


-- 
Qualcomm India Private Limited, on behalf of Qualcomm Innovation Center, 
Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project.

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v1] kthread/smpboot: Serialize kthread parking against wakeup
  2018-05-07 11:09                             ` Kohli, Gaurav
@ 2018-05-07 11:23                               ` Kohli, Gaurav
  2018-06-05 11:13                                 ` Kohli, Gaurav
  0 siblings, 1 reply; 39+ messages in thread
From: Kohli, Gaurav @ 2018-05-07 11:23 UTC (permalink / raw)
  To: Peter Zijlstra
  Cc: tglx, mpe, mingo, bigeasy, linux-kernel, linux-arm-msm,
	Neeraj Upadhyay, Will Deacon, Oleg Nesterov

Corrected the formatting, Sorry for spam.


> 
> HI Peter,
> 
> We have tested with new patch and still seeing same issue, in this dumps 
> we don't have debug traces, but seems there still exist race from code 
> review , Can you please check it once:
> 
> Controller Thread                               CPUHP Thread
> takedown_cpu
> kthread_park
> kthread_parkme
> Set KTHREAD_SHOULD_PARK
>                                                 smpboot_thread_fn
>                                                 set Task interruptible
> 
> 
> wake_up_process
> 
>                                                 Kthread_parkme
>                                                 SET TASK_PARKED
>                                                 schedule
>                                                 raw_spin_lock(&rq->lock)
> 
>                                                 context_switch
> 
>                                                 finish_lock_switch
> 
> 
> 
>                                                 Case TASK_PARKED
>                                                 kthread_park_complete
> 
> 
> SET TASK_INTERRUPTIBLE
> 
> 
> And also seeing the same warning during unpark of cpuhp from controller:
>   if (!wait_task_inactive(p, state)) {
>                  WARN_ON(1);
>                  return;
>          }
> 325.065893] [<ffffff8920ed0200>] kthread_unpark+0x80/0xd8
> [  325.065902] [<ffffff8920eab754>] bringup_cpu+0xa0/0x12c
> [  325.065910] [<ffffff8920eaae90>] cpuhp_invoke_callback+0xb4/0x5c8
> [  325.065917] [<ffffff8920eabd98>] cpuhp_up_callbacks+0x3c/0x154
> [  325.065924] [<ffffff8920ead220>] _cpu_up+0x134/0x208
> [  325.065931] [<ffffff8920ead45c>] do_cpu_up+0x168/0x1a0
> [  325.065938] [<ffffff8920ead4b8>] cpu_up+0x24/0x30
> [  325.065948] [<ffffff89215b1408>] cpu_subsys_online+0x20/0x2c
> [  325.065956] [<ffffff89215aac64>] device_online+0x70/0xb4
> [  325.065962] [<ffffff89215aad78>] online_store+0xd0/0xdc
> [  325.065971] [<ffffff89215a7424>] dev_attr_store+0x40/0x54
> [  325.065982] [<ffffff89210d8a98>] sysfs_kf_write+0x5c/0x74
> [  325.065988] [<ffffff89210d7b9c>] kernfs_fop_write+0xcc/0x1ec
> [  325.065999] [<ffffff8921049288>] vfs_write+0xb4/0x1d0
> [  325.066006] [<ffffff892104a858>] SyS_write+0x60/0xc0
> [  325.066014] [<ffffff8920e83770>] el0_svc_naked+0x24/0x28
> 
> 
> And after this same crash occured:
> [  325.521307] [<ffffff8920ed4aac>] smpboot_thread_fn+0x26c/0x2c8
> [  325.527295] [<ffffff8920ecfb24>] kthread+0xf4/0x108
> 
> I will put more debug ftraces to check what is going on exactly.
> 
> Regards
> Gaurav
> 
> 
> 
> 

-- 
Qualcomm India Private Limited, on behalf of Qualcomm Innovation Center, 
Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project.

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v1] kthread/smpboot: Serialize kthread parking against wakeup
  2018-05-07 11:23                               ` Kohli, Gaurav
@ 2018-06-05 11:13                                 ` Kohli, Gaurav
  2018-06-05 15:08                                   ` Oleg Nesterov
  0 siblings, 1 reply; 39+ messages in thread
From: Kohli, Gaurav @ 2018-06-05 11:13 UTC (permalink / raw)
  To: Peter Zijlstra
  Cc: tglx, mpe, mingo, bigeasy, linux-kernel, linux-arm-msm,
	Neeraj Upadhyay, Will Deacon, Oleg Nesterov

Hi Peter,

As last mentioned on mail, we are still seeing issue with the latest 
approach and below is the susceptible race as mentioned earlier..
controller Thread                               CPUHP Thread
takedown_cpu
kthread_park
kthread_parkme
Set KTHREAD_SHOULD_PARK
                                                 smpboot_thread_fn
                                                 set Task interruptible


wake_up_process
  if (!(p->state & state))
                 goto out;

                                                 Kthread_parkme
                                                 SET TASK_PARKED
                                                 schedule
                                                 raw_spin_lock(&rq->lock)
ttwu_remote
waiting for __task_rq_lock
                                                 context_switch

                                                 finish_lock_switch



                                                 Case TASK_PARKED
                                                 kthread_park_complete


SET Running


So it seems issue is still their with the latest mentioned fix
kthread, sched/wait: Fix kthread_parkme() completion issue.

Regards
Gaurav

On 5/7/2018 4:53 PM, Kohli, Gaurav wrote:
> Corrected the formatting, Sorry for spam.
> 
> 
>>
>> HI Peter,
>>
>> We have tested with new patch and still seeing same issue, in this 
>> dumps we don't have debug traces, but seems there still exist race 
>> from code review , Can you please check it once:
>>
>> Controller Thread                               CPUHP Thread
>> takedown_cpu
>> kthread_park
>> kthread_parkme
>> Set KTHREAD_SHOULD_PARK
>>                                                 smpboot_thread_fn
>>                                                 set Task interruptible
>>
>>
>> wake_up_process
>>
>>                                                 Kthread_parkme
>>                                                 SET TASK_PARKED
>>                                                 schedule
>>                                                 raw_spin_lock(&rq->lock)
>>
>>                                                 context_switch
>>
>>                                                 finish_lock_switch
>>
>>
>>
>>                                                 Case TASK_PARKED
>>                                                 kthread_park_complete
>>
>>
>> SET TASK_INTERRUPTIBLE
>>
>>
>> And also seeing the same warning during unpark of cpuhp from controller:
>>   if (!wait_task_inactive(p, state)) {
>>                  WARN_ON(1);
>>                  return;
>>          }
>> 325.065893] [<ffffff8920ed0200>] kthread_unpark+0x80/0xd8
>> [  325.065902] [<ffffff8920eab754>] bringup_cpu+0xa0/0x12c
>> [  325.065910] [<ffffff8920eaae90>] cpuhp_invoke_callback+0xb4/0x5c8
>> [  325.065917] [<ffffff8920eabd98>] cpuhp_up_callbacks+0x3c/0x154
>> [  325.065924] [<ffffff8920ead220>] _cpu_up+0x134/0x208
>> [  325.065931] [<ffffff8920ead45c>] do_cpu_up+0x168/0x1a0
>> [  325.065938] [<ffffff8920ead4b8>] cpu_up+0x24/0x30
>> [  325.065948] [<ffffff89215b1408>] cpu_subsys_online+0x20/0x2c
>> [  325.065956] [<ffffff89215aac64>] device_online+0x70/0xb4
>> [  325.065962] [<ffffff89215aad78>] online_store+0xd0/0xdc
>> [  325.065971] [<ffffff89215a7424>] dev_attr_store+0x40/0x54
>> [  325.065982] [<ffffff89210d8a98>] sysfs_kf_write+0x5c/0x74
>> [  325.065988] [<ffffff89210d7b9c>] kernfs_fop_write+0xcc/0x1ec
>> [  325.065999] [<ffffff8921049288>] vfs_write+0xb4/0x1d0
>> [  325.066006] [<ffffff892104a858>] SyS_write+0x60/0xc0
>> [  325.066014] [<ffffff8920e83770>] el0_svc_naked+0x24/0x28
>>
>>
>> And after this same crash occured:
>> [  325.521307] [<ffffff8920ed4aac>] smpboot_thread_fn+0x26c/0x2c8
>> [  325.527295] [<ffffff8920ecfb24>] kthread+0xf4/0x108
>>
>> I will put more debug ftraces to check what is going on exactly.
>>
>> Regards
>> Gaurav
>>
>>
>>
>>
> 

-- 
Qualcomm India Private Limited, on behalf of Qualcomm Innovation Center, 
Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project.

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v1] kthread/smpboot: Serialize kthread parking against wakeup
  2018-06-05 11:13                                 ` Kohli, Gaurav
@ 2018-06-05 15:08                                   ` Oleg Nesterov
  2018-06-05 15:22                                     ` Peter Zijlstra
  0 siblings, 1 reply; 39+ messages in thread
From: Oleg Nesterov @ 2018-06-05 15:08 UTC (permalink / raw)
  To: Kohli, Gaurav
  Cc: Peter Zijlstra, tglx, mpe, mingo, bigeasy, linux-kernel,
	linux-arm-msm, Neeraj Upadhyay, Will Deacon

I have to admit that I didn't try to follow this discussion, somehow I thought
that the plan was to use set_special_state(PARKED)...

On 06/05, Kohli, Gaurav wrote:
>
> As last mentioned on mail, we are still seeing issue with the latest
> approach and below is the susceptible race as mentioned earlier..
> controller Thread                               CPUHP Thread
> takedown_cpu
> kthread_park
> kthread_parkme
> Set KTHREAD_SHOULD_PARK
>                                                 smpboot_thread_fn
>                                                 set Task interruptible
>
>
> wake_up_process
>  if (!(p->state & state))
>                 goto out;
>
>                                                 Kthread_parkme
>                                                 SET TASK_PARKED
>                                                 schedule
>                                                 raw_spin_lock(&rq->lock)
> ttwu_remote
> waiting for __task_rq_lock
>                                                 context_switch
>
>                                                 finish_lock_switch
>
>
>
>                                                 Case TASK_PARKED
>                                                 kthread_park_complete
>
>
> SET Running

I think you are right.

And, now that I look at 85f1abe0019fcb3ea10df7029056cf42702283a8
("kthread, sched/wait: Fix kthread_parkme() completion issue") I see this note
int the changelog:

	The alternative is to promote TASK_PARKED to a special state, this
	guarantees wait_task_inactive() cannot observe a 'stale' TASK_RUNNING
	and we'll end up doing the right thing, but this preserves the whole
	icky business of potentially migating the still runnable thing.

OK, but __kthread_parkme() can be preempted before it calls schedule(), so the
caller still can be migrated? Plus kthread_park_complete() can be called twice.

No?

Oleg.

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v1] kthread/smpboot: Serialize kthread parking against wakeup
  2018-06-05 15:08                                   ` Oleg Nesterov
@ 2018-06-05 15:22                                     ` Peter Zijlstra
  2018-06-05 15:40                                       ` Peter Zijlstra
  0 siblings, 1 reply; 39+ messages in thread
From: Peter Zijlstra @ 2018-06-05 15:22 UTC (permalink / raw)
  To: Oleg Nesterov
  Cc: Kohli, Gaurav, tglx, mpe, mingo, bigeasy, linux-kernel,
	linux-arm-msm, Neeraj Upadhyay, Will Deacon

On Tue, Jun 05, 2018 at 05:08:41PM +0200, Oleg Nesterov wrote:

> On 06/05, Kohli, Gaurav wrote:
> >
> > As last mentioned on mail, we are still seeing issue with the latest
> > approach and below is the susceptible race as mentioned earlier..
> > controller Thread                               CPUHP Thread
> > takedown_cpu
> > kthread_park
> > kthread_parkme
> > Set KTHREAD_SHOULD_PARK
> >                                                 smpboot_thread_fn
> >                                                 set Task interruptible
> >
> >
> > wake_up_process
> >  if (!(p->state & state))
> >                 goto out;
> >
> >                                                 Kthread_parkme
> >                                                 SET TASK_PARKED
> >                                                 schedule
> >                                                 raw_spin_lock(&rq->lock)
> > ttwu_remote
> > waiting for __task_rq_lock
> >                                                 context_switch
> >
> >                                                 finish_lock_switch
> >
> >
> >
> >                                                 Case TASK_PARKED
> >                                                 kthread_park_complete
> >
> >
> > SET Running
> 
> I think you are right.
> 
> And, now that I look at 85f1abe0019fcb3ea10df7029056cf42702283a8
> ("kthread, sched/wait: Fix kthread_parkme() completion issue") I see this note
> int the changelog:
> 
> 	The alternative is to promote TASK_PARKED to a special state, this
> 	guarantees wait_task_inactive() cannot observe a 'stale' TASK_RUNNING
> 	and we'll end up doing the right thing, but this preserves the whole
> 	icky business of potentially migating the still runnable thing.
> 
> OK, but __kthread_parkme() can be preempted before it calls schedule(), so the
> caller still can be migrated? Plus kthread_park_complete() can be called twice.

Argh... I forgot TASK_DEAD does the whole thing with preempt_disable().
Let me stare at that a bit.

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v1] kthread/smpboot: Serialize kthread parking against wakeup
  2018-06-05 15:22                                     ` Peter Zijlstra
@ 2018-06-05 15:40                                       ` Peter Zijlstra
  2018-06-05 16:35                                         ` Oleg Nesterov
  0 siblings, 1 reply; 39+ messages in thread
From: Peter Zijlstra @ 2018-06-05 15:40 UTC (permalink / raw)
  To: Oleg Nesterov
  Cc: Kohli, Gaurav, tglx, mpe, mingo, bigeasy, linux-kernel,
	linux-arm-msm, Neeraj Upadhyay, Will Deacon

On Tue, Jun 05, 2018 at 05:22:12PM +0200, Peter Zijlstra wrote:

> > OK, but __kthread_parkme() can be preempted before it calls schedule(), so the
> > caller still can be migrated? Plus kthread_park_complete() can be called twice.
> 
> Argh... I forgot TASK_DEAD does the whole thing with preempt_disable().
> Let me stare at that a bit.

This should ensure we only ever complete when we read PARKED, right?

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 8d59b259af4a..e513b4600796 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -2641,7 +2641,7 @@ prepare_task_switch(struct rq *rq, struct task_struct *prev,
  * past. prev == current is still correct but we need to recalculate this_rq
  * because prev may have moved to another CPU.
  */
-static struct rq *finish_task_switch(struct task_struct *prev)
+static struct rq *finish_task_switch(struct task_struct *prev, bool preempt)
 	__releases(rq->lock)
 {
 	struct rq *rq = this_rq();
@@ -2674,7 +2674,7 @@ static struct rq *finish_task_switch(struct task_struct *prev)
 	 *
 	 * We must observe prev->state before clearing prev->on_cpu (in
 	 * finish_task), otherwise a concurrent wakeup can get prev
-	 * running on another CPU and we could rave with its RUNNING -> DEAD
+	 * running on another CPU and we could race with its RUNNING -> DEAD
 	 * transition, resulting in a double drop.
 	 */
 	prev_state = prev->state;
@@ -2720,7 +2720,8 @@ static struct rq *finish_task_switch(struct task_struct *prev)
 			break;
 
 		case TASK_PARKED:
-			kthread_park_complete(prev);
+			if (!preempt)
+				kthread_park_complete(prev);
 			break;
 		}
 	}
@@ -2784,7 +2785,7 @@ asmlinkage __visible void schedule_tail(struct task_struct *prev)
 	 * PREEMPT_COUNT kernels).
 	 */
 
-	rq = finish_task_switch(prev);
+	rq = finish_task_switch(prev, false);
 	balance_callback(rq);
 	preempt_enable();
 
@@ -2797,7 +2798,7 @@ asmlinkage __visible void schedule_tail(struct task_struct *prev)
  */
 static __always_inline struct rq *
 context_switch(struct rq *rq, struct task_struct *prev,
-	       struct task_struct *next, struct rq_flags *rf)
+	       struct task_struct *next, bool preempt, struct rq_flags *rf)
 {
 	struct mm_struct *mm, *oldmm;
 
@@ -2839,7 +2840,7 @@ context_switch(struct rq *rq, struct task_struct *prev,
 	switch_to(prev, next, prev);
 	barrier();
 
-	return finish_task_switch(prev);
+	return finish_task_switch(prev, preempt);
 }
 
 /*
@@ -3478,7 +3479,7 @@ static void __sched notrace __schedule(bool preempt)
 		trace_sched_switch(preempt, prev, next);
 
 		/* Also unlocks the rq: */
-		rq = context_switch(rq, prev, next, &rf);
+		rq = context_switch(rq, prev, next, preempt, &rf);
 	} else {
 		rq->clock_update_flags &= ~(RQCF_ACT_SKIP|RQCF_REQ_SKIP);
 		rq_unlock_irq(rq, &rf);
@@ -3487,6 +3488,7 @@ static void __sched notrace __schedule(bool preempt)
 	balance_callback(rq);
 }
 
+/* called with preemption disabled */
 void __noreturn do_task_dead(void)
 {
 	/* Causes final put_task_struct in finish_task_switch(): */

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v1] kthread/smpboot: Serialize kthread parking against wakeup
  2018-06-05 15:40                                       ` Peter Zijlstra
@ 2018-06-05 16:35                                         ` Oleg Nesterov
  2018-06-05 18:21                                           ` Kohli, Gaurav
  2018-06-05 20:13                                           ` Peter Zijlstra
  0 siblings, 2 replies; 39+ messages in thread
From: Oleg Nesterov @ 2018-06-05 16:35 UTC (permalink / raw)
  To: Peter Zijlstra
  Cc: Kohli, Gaurav, tglx, mpe, mingo, bigeasy, linux-kernel,
	linux-arm-msm, Neeraj Upadhyay, Will Deacon

On 06/05, Peter Zijlstra wrote:
>
> On Tue, Jun 05, 2018 at 05:22:12PM +0200, Peter Zijlstra wrote:
> 
> > > OK, but __kthread_parkme() can be preempted before it calls schedule(), so the
> > > caller still can be migrated? Plus kthread_park_complete() can be called twice.
> > 
> > Argh... I forgot TASK_DEAD does the whole thing with preempt_disable().
> > Let me stare at that a bit.
> 
> This should ensure we only ever complete when we read PARKED, right?
> 
> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> index 8d59b259af4a..e513b4600796 100644
> --- a/kernel/sched/core.c
> +++ b/kernel/sched/core.c
> @@ -2641,7 +2641,7 @@ prepare_task_switch(struct rq *rq, struct task_struct *prev,
>   * past. prev == current is still correct but we need to recalculate this_rq
>   * because prev may have moved to another CPU.
>   */
> -static struct rq *finish_task_switch(struct task_struct *prev)
> +static struct rq *finish_task_switch(struct task_struct *prev, bool preempt)
>  	__releases(rq->lock)
>  {
>  	struct rq *rq = this_rq();
> @@ -2674,7 +2674,7 @@ static struct rq *finish_task_switch(struct task_struct *prev)
>  	 *
>  	 * We must observe prev->state before clearing prev->on_cpu (in
>  	 * finish_task), otherwise a concurrent wakeup can get prev
> -	 * running on another CPU and we could rave with its RUNNING -> DEAD
> +	 * running on another CPU and we could race with its RUNNING -> DEAD
>  	 * transition, resulting in a double drop.
>  	 */
>  	prev_state = prev->state;
> @@ -2720,7 +2720,8 @@ static struct rq *finish_task_switch(struct task_struct *prev)
>  			break;
>
>  		case TASK_PARKED:
> -			kthread_park_complete(prev);
> +			if (!preempt)
> +				kthread_park_complete(prev);


Yes, but this won't fix the race decribed by Kohli...

Plus this complicates the schedule() paths for the very special case, and to me
it seems that all this kthread_park/unpark logic needs some serious cleanups...

Not that I can suggest something better right now.

Oleg.

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v1] kthread/smpboot: Serialize kthread parking against wakeup
  2018-06-05 16:35                                         ` Oleg Nesterov
@ 2018-06-05 18:21                                           ` Kohli, Gaurav
  2018-06-05 20:13                                           ` Peter Zijlstra
  1 sibling, 0 replies; 39+ messages in thread
From: Kohli, Gaurav @ 2018-06-05 18:21 UTC (permalink / raw)
  To: Oleg Nesterov, Peter Zijlstra
  Cc: tglx, mpe, mingo, bigeasy, linux-kernel, linux-arm-msm,
	Neeraj Upadhyay, Will Deacon

Hi,

Just for info , the patch that I have shared earlier with pi_lock 
approach has been tested since last one month and no issue has been 
observed,

https://lkml.org/lkml/2018/4/25/189

Can we take this if it looks good?

Regards
Gaurav

On 6/5/2018 10:05 PM, Oleg Nesterov wrote:
> On 06/05, Peter Zijlstra wrote:
>>
>> On Tue, Jun 05, 2018 at 05:22:12PM +0200, Peter Zijlstra wrote:
>>
>>>> OK, but __kthread_parkme() can be preempted before it calls schedule(), so the
>>>> caller still can be migrated? Plus kthread_park_complete() can be called twice.
>>>
>>> Argh... I forgot TASK_DEAD does the whole thing with preempt_disable().
>>> Let me stare at that a bit.
>>
>> This should ensure we only ever complete when we read PARKED, right?
>>
>> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
>> index 8d59b259af4a..e513b4600796 100644
>> --- a/kernel/sched/core.c
>> +++ b/kernel/sched/core.c
>> @@ -2641,7 +2641,7 @@ prepare_task_switch(struct rq *rq, struct task_struct *prev,
>>    * past. prev == current is still correct but we need to recalculate this_rq
>>    * because prev may have moved to another CPU.
>>    */
>> -static struct rq *finish_task_switch(struct task_struct *prev)
>> +static struct rq *finish_task_switch(struct task_struct *prev, bool preempt)
>>   	__releases(rq->lock)
>>   {
>>   	struct rq *rq = this_rq();
>> @@ -2674,7 +2674,7 @@ static struct rq *finish_task_switch(struct task_struct *prev)
>>   	 *
>>   	 * We must observe prev->state before clearing prev->on_cpu (in
>>   	 * finish_task), otherwise a concurrent wakeup can get prev
>> -	 * running on another CPU and we could rave with its RUNNING -> DEAD
>> +	 * running on another CPU and we could race with its RUNNING -> DEAD
>>   	 * transition, resulting in a double drop.
>>   	 */
>>   	prev_state = prev->state;
>> @@ -2720,7 +2720,8 @@ static struct rq *finish_task_switch(struct task_struct *prev)
>>   			break;
>>
>>   		case TASK_PARKED:
>> -			kthread_park_complete(prev);
>> +			if (!preempt)
>> +				kthread_park_complete(prev);
> 
> 
> Yes, but this won't fix the race decribed by Kohli...
> 
> Plus this complicates the schedule() paths for the very special case, and to me
> it seems that all this kthread_park/unpark logic needs some serious cleanups...
> 
> Not that I can suggest something better right now.
> 
> Oleg.
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-arm-msm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

-- 
Qualcomm India Private Limited, on behalf of Qualcomm Innovation Center, 
Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project.

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v1] kthread/smpboot: Serialize kthread parking against wakeup
  2018-06-05 16:35                                         ` Oleg Nesterov
  2018-06-05 18:21                                           ` Kohli, Gaurav
@ 2018-06-05 20:13                                           ` Peter Zijlstra
  2018-06-06 13:51                                             ` Oleg Nesterov
  1 sibling, 1 reply; 39+ messages in thread
From: Peter Zijlstra @ 2018-06-05 20:13 UTC (permalink / raw)
  To: Oleg Nesterov
  Cc: Kohli, Gaurav, tglx, mpe, mingo, bigeasy, linux-kernel,
	linux-arm-msm, Neeraj Upadhyay, Will Deacon

On Tue, Jun 05, 2018 at 06:35:16PM +0200, Oleg Nesterov wrote:
> Yes, but this won't fix the race decribed by Kohli...

Clearly I'm not going strong today... and yes, looking at that again I
didn't address that.

> Plus this complicates the schedule() paths for the very special case

I checked, we already spilled @preempt onto the stack, context_switch()
is inlined so the extra argument is a no-op, if finish_task_switch()
also gets inlined (possible, could force it) the only additional cost is
the @preempt load from stack in the slow path. If it doesn't get inlined
we get the additional function call overhead, which should be minimal.

But yes, I wasn't a fan either.

> and to me it seems that all this kthread_park/unpark logic needs some
> serious cleanups...

I really hated how even with TASK_PARKED special the smpboot thread
could still get migrated, which is why I moved the completion. Do you
have any saner suggestions?

Humm, I suppose we could do wait_task_inactive() after
wait_for_completion().  I absolutely abhor wait_task_inactive(), but I
remember us both failing to fix that at least twice :/

Also, I think we still need TASK_PARKED as a special state for that.

How's the below?

---
 include/linux/kthread.h |  1 -
 include/linux/sched.h   |  2 +-
 kernel/kthread.c        | 32 +++++++++++++++++++++++++-------
 kernel/sched/core.c     | 31 +++++++++++--------------------
 4 files changed, 37 insertions(+), 29 deletions(-)

diff --git a/include/linux/kthread.h b/include/linux/kthread.h
index 2803264c512f..c1961761311d 100644
--- a/include/linux/kthread.h
+++ b/include/linux/kthread.h
@@ -62,7 +62,6 @@ void *kthread_probe_data(struct task_struct *k);
 int kthread_park(struct task_struct *k);
 void kthread_unpark(struct task_struct *k);
 void kthread_parkme(void);
-void kthread_park_complete(struct task_struct *k);
 
 int kthreadd(void *unused);
 extern struct task_struct *kthreadd_task;
diff --git a/include/linux/sched.h b/include/linux/sched.h
index 14e4f9c12337..4e32c1cc7794 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -117,7 +117,7 @@ struct task_group;
  * the comment with set_special_state().
  */
 #define is_special_task_state(state)				\
-	((state) & (__TASK_STOPPED | __TASK_TRACED | TASK_DEAD))
+	((state) & (__TASK_STOPPED | __TASK_TRACED | TASK_PARKED | TASK_DEAD))
 
 #define __set_current_state(state_value)			\
 	do {							\
diff --git a/kernel/kthread.c b/kernel/kthread.c
index 481951bf091d..8f66a3dc767a 100644
--- a/kernel/kthread.c
+++ b/kernel/kthread.c
@@ -177,12 +177,24 @@ void *kthread_probe_data(struct task_struct *task)
 static void __kthread_parkme(struct kthread *self)
 {
 	for (;;) {
-		set_current_state(TASK_PARKED);
+		/*
+		 * TASK_PARKED is a special state; we must serialize against
+		 * possible pending wakeups to avoid store-store collisions on
+		 * task->state.
+		 *
+		 * Such a collision might possibly result in the task state
+		 * changin from TASK_PARKED and us failing the
+		 * wait_task_inactive() in kthread_park().
+		 */
+		set_special_state(TASK_PARKED);
 		if (!test_bit(KTHREAD_SHOULD_PARK, &self->flags))
 			break;
+
+		complete_all(&self->parked);
 		schedule();
 	}
 	__set_current_state(TASK_RUNNING);
+	reinit_completion(&self->parked);
 }
 
 void kthread_parkme(void)
@@ -191,11 +203,6 @@ void kthread_parkme(void)
 }
 EXPORT_SYMBOL_GPL(kthread_parkme);
 
-void kthread_park_complete(struct task_struct *k)
-{
-	complete_all(&to_kthread(k)->parked);
-}
-
 static int kthread(void *_create)
 {
 	/* Copy data: it's on kthread's stack */
@@ -459,8 +466,10 @@ void kthread_unpark(struct task_struct *k)
 	if (test_bit(KTHREAD_IS_PER_CPU, &kthread->flags))
 		__kthread_bind(k, kthread->cpu, TASK_PARKED);
 
-	reinit_completion(&kthread->parked);
 	clear_bit(KTHREAD_SHOULD_PARK, &kthread->flags);
+	/*
+	 * __kthread_parkme() will either see !SHOULD_PARK or get the wakeup.
+	 */
 	wake_up_state(k, TASK_PARKED);
 }
 EXPORT_SYMBOL_GPL(kthread_unpark);
@@ -487,7 +496,16 @@ int kthread_park(struct task_struct *k)
 	set_bit(KTHREAD_SHOULD_PARK, &kthread->flags);
 	if (k != current) {
 		wake_up_process(k);
+		/*
+		 * Wait for __kthread_parkme() to complete(), this means we
+		 * _will_ have TASK_PARKED and are about to call schedule().
+		 */
 		wait_for_completion(&kthread->parked);
+		/*
+		 * Now wait for that schedule() to complete and the task to
+		 * get scheduled out.
+		 */
+		WARN_ON_ONCE(!wait_task_inactive(p, TASK_PARKED));
 	}
 
 	return 0;
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 8d59b259af4a..cf72c4eed7da 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -7,7 +7,6 @@
  */
 #include "sched.h"
 
-#include <linux/kthread.h>
 #include <linux/nospec.h>
 
 #include <asm/switch_to.h>
@@ -2701,28 +2700,20 @@ static struct rq *finish_task_switch(struct task_struct *prev)
 		membarrier_mm_sync_core_before_usermode(mm);
 		mmdrop(mm);
 	}
-	if (unlikely(prev_state & (TASK_DEAD|TASK_PARKED))) {
-		switch (prev_state) {
-		case TASK_DEAD:
-			if (prev->sched_class->task_dead)
-				prev->sched_class->task_dead(prev);
+	if (unlikely(prev_state == TASK_DEAD)) {
+		if (prev->sched_class->task_dead)
+			prev->sched_class->task_dead(prev);
 
-			/*
-			 * Remove function-return probe instances associated with this
-			 * task and put them back on the free list.
-			 */
-			kprobe_flush_task(prev);
-
-			/* Task is done with its stack. */
-			put_task_stack(prev);
+		/*
+		 * Remove function-return probe instances associated with this
+		 * task and put them back on the free list.
+		 */
+		kprobe_flush_task(prev);
 
-			put_task_struct(prev);
-			break;
+		/* Task is done with its stack. */
+		put_task_stack(prev);
 
-		case TASK_PARKED:
-			kthread_park_complete(prev);
-			break;
-		}
+		put_task_struct(prev);
 	}
 
 	tick_nohz_task_switch();

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v1] kthread/smpboot: Serialize kthread parking against wakeup
  2018-06-05 20:13                                           ` Peter Zijlstra
@ 2018-06-06 13:51                                             ` Oleg Nesterov
  2018-06-06 15:03                                               ` Peter Zijlstra
                                                                 ` (3 more replies)
  0 siblings, 4 replies; 39+ messages in thread
From: Oleg Nesterov @ 2018-06-06 13:51 UTC (permalink / raw)
  To: Peter Zijlstra
  Cc: Kohli, Gaurav, tglx, mpe, mingo, bigeasy, linux-kernel,
	linux-arm-msm, Neeraj Upadhyay, Will Deacon

On 06/05, Peter Zijlstra wrote:
>
> Also, I think we still need TASK_PARKED as a special state for that.

I think it would be nice to kill the TASK_PARKED state altogether. But I don't
know how. I'll try to look at this code later, but I am not sure I will find a
way to cleanup it...


> --- a/kernel/kthread.c
> +++ b/kernel/kthread.c
> @@ -177,12 +177,24 @@ void *kthread_probe_data(struct task_struct *task)
>  static void __kthread_parkme(struct kthread *self)
>  {
>  	for (;;) {
> -		set_current_state(TASK_PARKED);
> +		/*
> +		 * TASK_PARKED is a special state; we must serialize against
> +		 * possible pending wakeups to avoid store-store collisions on
> +		 * task->state.
> +		 *
> +		 * Such a collision might possibly result in the task state
> +		 * changin from TASK_PARKED and us failing the
> +		 * wait_task_inactive() in kthread_park().
> +		 */
> +		set_special_state(TASK_PARKED);

Agreed,

>  		if (!test_bit(KTHREAD_SHOULD_PARK, &self->flags))
>  			break;
> +
> +		complete_all(&self->parked);
>  		schedule();
>  	}
>  	__set_current_state(TASK_RUNNING);
> +	reinit_completion(&self->parked);

But how can we know that all the callers of kthread_park() have already returned
from wait_for_completion() ?

Oh. The very fact that __kthread_parkme() does complete_all() proves that we need
some serious cleanups. In particular, I think that kthread_park() on a parked kthread
must not be possible.

Just look at this code. It looks as if __kthread_parkme() can race with _unpark()
and thus we need this wait-event-like loop.

But if it can race with _unpark() then kthread_park() can block forever.


For the start, can't we change kthread_park()

	-	set_bit(KTHREAD_SHOULD_PARK, &kthread->flags);
	+ 	if (test_and_set_bit(...))
	+		return -EAGAIN;

and s/complete_all/complete/ in __kthread_parkme() ?

IIUC, this will only affect smpboot_update_cpumask_percpu_thread() which can hit
an already parked thread, but it doesn't need to wait.

And it seems that smpboot_update_cpumask_percpu_thread() in turn needs some cleanups.
Hmm. and its single user: kernel/watchdog.c.

And speaking of watchdog.c, can't we simply kill the "watchdog/%u" threads? This is
off-topic, but can't watchdog_timer_fn() use stop_one_cpu_nowait(watchdog) ?

And I really think we should unexport kthread_park/unpark(), only smpboot_thread_fn()
should use them. kthread() should not play with __kthread_parkme(). And even
KTHREAD_SHOULD_PARK must die, I mean it should live in struct smp_hotplug_thread,
not in struct kthread.

OK, this is off-topic too.

In short, I think this patch is fine but I didn't read it carefully, will try tomorrow.

And, let me repeat, can't we avoid complete_all() ?

Oleg.

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v1] kthread/smpboot: Serialize kthread parking against wakeup
  2018-06-06 13:51                                             ` Oleg Nesterov
@ 2018-06-06 15:03                                               ` Peter Zijlstra
  2018-06-06 15:04                                               ` Peter Zijlstra
                                                                 ` (2 subsequent siblings)
  3 siblings, 0 replies; 39+ messages in thread
From: Peter Zijlstra @ 2018-06-06 15:03 UTC (permalink / raw)
  To: Oleg Nesterov
  Cc: Kohli, Gaurav, tglx, mpe, mingo, bigeasy, linux-kernel,
	linux-arm-msm, Neeraj Upadhyay, Will Deacon

On Wed, Jun 06, 2018 at 03:51:16PM +0200, Oleg Nesterov wrote:
> In particular, I think that kthread_park() on a parked kthread
> must not be possible.

It happens though; I put in a WARN and someone triggered it -- although
I could not reproduce :/

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v1] kthread/smpboot: Serialize kthread parking against wakeup
  2018-06-06 13:51                                             ` Oleg Nesterov
  2018-06-06 15:03                                               ` Peter Zijlstra
@ 2018-06-06 15:04                                               ` Peter Zijlstra
  2018-06-06 15:22                                               ` Peter Zijlstra
  2018-06-06 18:59                                               ` Peter Zijlstra
  3 siblings, 0 replies; 39+ messages in thread
From: Peter Zijlstra @ 2018-06-06 15:04 UTC (permalink / raw)
  To: Oleg Nesterov
  Cc: Kohli, Gaurav, tglx, mpe, mingo, bigeasy, linux-kernel,
	linux-arm-msm, Neeraj Upadhyay, Will Deacon

On Wed, Jun 06, 2018 at 03:51:16PM +0200, Oleg Nesterov wrote:
> And I really think we should unexport kthread_park/unpark(), only smpboot_thread_fn()
> should use them. kthread() should not play with __kthread_parkme(). And even
> KTHREAD_SHOULD_PARK must die, I mean it should live in struct smp_hotplug_thread,
> not in struct kthread.

I tend to agree; however we need to fix drm and md first, because they
already use them :-(

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v1] kthread/smpboot: Serialize kthread parking against wakeup
  2018-06-06 13:51                                             ` Oleg Nesterov
  2018-06-06 15:03                                               ` Peter Zijlstra
  2018-06-06 15:04                                               ` Peter Zijlstra
@ 2018-06-06 15:22                                               ` Peter Zijlstra
  2018-06-06 18:59                                               ` Peter Zijlstra
  3 siblings, 0 replies; 39+ messages in thread
From: Peter Zijlstra @ 2018-06-06 15:22 UTC (permalink / raw)
  To: Oleg Nesterov
  Cc: Kohli, Gaurav, tglx, mpe, mingo, bigeasy, linux-kernel,
	linux-arm-msm, Neeraj Upadhyay, Will Deacon

On Wed, Jun 06, 2018 at 03:51:16PM +0200, Oleg Nesterov wrote:
> On 06/05, Peter Zijlstra wrote:
> >
> > Also, I think we still need TASK_PARKED as a special state for that.
> 
> I think it would be nice to kill the TASK_PARKED state altogether. But I don't
> know how. I'll try to look at this code later, but I am not sure I will find a
> way to cleanup it...

So the nice thing about having TASK_PARKED is that it guarantees no
spurious wakeups. You know the thread will not wake up while it's CPU is
gone.

We could possibly re-purpose TASK_STOPPED (because kernel threads don't
do that) but that seems dodgy at best.

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v1] kthread/smpboot: Serialize kthread parking against wakeup
  2018-06-06 13:51                                             ` Oleg Nesterov
                                                                 ` (2 preceding siblings ...)
  2018-06-06 15:22                                               ` Peter Zijlstra
@ 2018-06-06 18:59                                               ` Peter Zijlstra
  2018-06-07  8:30                                                 ` Kohli, Gaurav
  3 siblings, 1 reply; 39+ messages in thread
From: Peter Zijlstra @ 2018-06-06 18:59 UTC (permalink / raw)
  To: Oleg Nesterov
  Cc: Kohli, Gaurav, tglx, mpe, mingo, bigeasy, linux-kernel,
	linux-arm-msm, Neeraj Upadhyay, Will Deacon

On Wed, Jun 06, 2018 at 03:51:16PM +0200, Oleg Nesterov wrote:
> IIUC, this will only affect smpboot_update_cpumask_percpu_thread() which can hit
> an already parked thread, but it doesn't need to wait.
> 
> And it seems that smpboot_update_cpumask_percpu_thread() in turn needs some cleanups.
> Hmm. and its single user: kernel/watchdog.c.
> 
> And speaking of watchdog.c, can't we simply kill the "watchdog/%u" threads? This is
> off-topic, but can't watchdog_timer_fn() use stop_one_cpu_nowait(watchdog) ?
> 
> And I really think we should unexport kthread_park/unpark(), only smpboot_thread_fn()
> should use them. kthread() should not play with __kthread_parkme(). And even
> KTHREAD_SHOULD_PARK must die, I mean it should live in struct smp_hotplug_thread,
> not in struct kthread.
> 
> OK, this is off-topic too.

> And, let me repeat, can't we avoid complete_all() ?

Yes, or at least if that watchdog crap is the only user.

I have most of the patch reworking watchdog.c to use stop_one_cpu*(),
and that cleans up lots -- of course, I've not tested it yet, so it
could also be breaking lots :-)

^ permalink raw reply	[flat|nested] 39+ messages in thread

* Re: [PATCH v1] kthread/smpboot: Serialize kthread parking against wakeup
  2018-06-06 18:59                                               ` Peter Zijlstra
@ 2018-06-07  8:30                                                 ` Kohli, Gaurav
  0 siblings, 0 replies; 39+ messages in thread
From: Kohli, Gaurav @ 2018-06-07  8:30 UTC (permalink / raw)
  To: Peter Zijlstra, Oleg Nesterov
  Cc: tglx, mpe, mingo, bigeasy, linux-kernel, linux-arm-msm,
	Neeraj Upadhyay, Will Deacon

HI ,

In the latest patch mentioned, k should be their instead of p:

-WARN_ON_ONCE(!wait_task_inactive(p, TASK_PARKED))
+WARN_ON_ONCE(!wait_task_inactive(k, TASK_PARKED))

Regards
Gaurav

On 6/7/2018 12:29 AM, Peter Zijlstra wrote:
> On Wed, Jun 06, 2018 at 03:51:16PM +0200, Oleg Nesterov wrote:
>> IIUC, this will only affect smpboot_update_cpumask_percpu_thread() which can hit
>> an already parked thread, but it doesn't need to wait.
>>
>> And it seems that smpboot_update_cpumask_percpu_thread() in turn needs some cleanups.
>> Hmm. and its single user: kernel/watchdog.c.
>>
>> And speaking of watchdog.c, can't we simply kill the "watchdog/%u" threads? This is
>> off-topic, but can't watchdog_timer_fn() use stop_one_cpu_nowait(watchdog) ?
>>
>> And I really think we should unexport kthread_park/unpark(), only smpboot_thread_fn()
>> should use them. kthread() should not play with __kthread_parkme(). And even
>> KTHREAD_SHOULD_PARK must die, I mean it should live in struct smp_hotplug_thread,
>> not in struct kthread.
>>
>> OK, this is off-topic too.
> 
>> And, let me repeat, can't we avoid complete_all() ?
> 
> Yes, or at least if that watchdog crap is the only user.
> 
> I have most of the patch reworking watchdog.c to use stop_one_cpu*(),
> and that cleans up lots -- of course, I've not tested it yet, so it
> could also be breaking lots :-)
> 

-- 
Qualcomm India Private Limited, on behalf of Qualcomm Innovation Center, 
Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project.

^ permalink raw reply	[flat|nested] 39+ messages in thread

end of thread, other threads:[~2018-06-07  8:31 UTC | newest]

Thread overview: 39+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-04-25  8:33 [PATCH v1] kthread/smpboot: Serialize kthread parking against wakeup Gaurav Kohli
2018-04-25 20:09 ` Peter Zijlstra
2018-04-26  4:04   ` Kohli, Gaurav
2018-04-26  9:14     ` Peter Zijlstra
2018-04-26  8:41   ` Peter Zijlstra
2018-04-26  8:57     ` Peter Zijlstra
2018-04-26 15:53       ` Kohli, Gaurav
2018-04-30 11:17         ` Peter Zijlstra
2018-05-01  7:50           ` Kohli, Gaurav
2018-05-01 10:18             ` Peter Zijlstra
2018-05-01 10:40               ` Peter Zijlstra
2018-05-01 10:40               ` Kohli, Gaurav
2018-05-01 11:31                 ` Peter Zijlstra
2018-05-01 11:46                   ` Kohli, Gaurav
2018-05-01 13:19                     ` Peter Zijlstra
2018-05-02  5:15                       ` Kohli, Gaurav
2018-05-02  8:20                         ` Peter Zijlstra
2018-05-02 10:13                           ` Kohli, Gaurav
2018-05-07 11:09                             ` Kohli, Gaurav
2018-05-07 11:23                               ` Kohli, Gaurav
2018-06-05 11:13                                 ` Kohli, Gaurav
2018-06-05 15:08                                   ` Oleg Nesterov
2018-06-05 15:22                                     ` Peter Zijlstra
2018-06-05 15:40                                       ` Peter Zijlstra
2018-06-05 16:35                                         ` Oleg Nesterov
2018-06-05 18:21                                           ` Kohli, Gaurav
2018-06-05 20:13                                           ` Peter Zijlstra
2018-06-06 13:51                                             ` Oleg Nesterov
2018-06-06 15:03                                               ` Peter Zijlstra
2018-06-06 15:04                                               ` Peter Zijlstra
2018-06-06 15:22                                               ` Peter Zijlstra
2018-06-06 18:59                                               ` Peter Zijlstra
2018-06-07  8:30                                                 ` Kohli, Gaurav
2018-05-01 10:44               ` Peter Zijlstra
2018-04-26 16:02     ` Andrea Parri
2018-04-26 16:18     ` Oleg Nesterov
2018-04-30 11:20       ` Peter Zijlstra
2018-04-30 11:56         ` Peter Zijlstra
2018-04-28  6:43 ` [lkp-robot] [kthread/smpboot] cad8e99675: inconsistent{IN-HARDIRQ-W}->{HARDIRQ-ON-W}usage kernel test robot

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).