LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: tip-bot for Mike Galbraith <tipbot@zytor.com>
To: linux-tip-commits@vger.kernel.org
Cc: linux-kernel@vger.kernel.org, efault@gmx.de, mingo@kernel.org,
	lenb@kernel.org, tglx@linutronix.de, jwboyer@redhat.com,
	peterz@infradead.org, bitbucket@online.de,
	torvalds@linux-foundation.org, hpa@zytor.com, ibmalone@gmail.com,
	bp@alien8.de, len.brown@intel.com
Subject: [tip:sched/core] sched/idle/x86: Optimize unnecessary mwait_idle( ) resched IPIs
Date: Mon, 16 Mar 2015 05:11:25 -0700	[thread overview]
Message-ID: <tip-f8e617f4582995f7c25ef25b4167213120ad122b@git.kernel.org> (raw)
In-Reply-To: <1390061684.5566.4.camel@marge.simpson.net>

Commit-ID:  f8e617f4582995f7c25ef25b4167213120ad122b
Gitweb:     http://git.kernel.org/tip/f8e617f4582995f7c25ef25b4167213120ad122b
Author:     Mike Galbraith <bitbucket@online.de>
AuthorDate: Sat, 18 Jan 2014 17:14:44 +0100
Committer:  Ingo Molnar <mingo@kernel.org>
CommitDate: Mon, 16 Mar 2015 11:14:22 +0100

sched/idle/x86: Optimize unnecessary mwait_idle() resched IPIs

To fully take advantage of MWAIT, apparently the CLFLUSH instruction needs
another quirk on certain CPUs: proper barriers around it on certain machines.

On a Q6600 SMP system, pipe-test scheduling performance, cross core,
improves significantly:

  3.8.13                   487.2 KHz    1.000
  3.13.0-master            415.5 KHz     .852
  3.13.0-master+           415.2 KHz     .852     + restore mwait_idle
  3.13.0-master++          488.5 KHz    1.002     + restore mwait_idle + IPI fix

Since X86_BUG_CLFLUSH_MONITOR is already a quirk, don't create a separate
quirk for the extra smp_mb()s.

Signed-off-by: Mike Galbraith <bitbucket@online.de>
Cc: <stable@vger.kernel.org> # 3.10+
Cc: Borislav Petkov <bp@alien8.de>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Ian Malone <ibmalone@gmail.com>
Cc: Josh Boyer <jwboyer@redhat.com>
Cc: Len Brown <len.brown@intel.com>
Cc: Len Brown <lenb@kernel.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Link: http://lkml.kernel.org/r/1390061684.5566.4.camel@marge.simpson.net
[ Ported to recent kernel, added comments about the quirk. ]
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
 arch/x86/kernel/process.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/arch/x86/kernel/process.c b/arch/x86/kernel/process.c
index da06f74..6ad8a63 100644
--- a/arch/x86/kernel/process.c
+++ b/arch/x86/kernel/process.c
@@ -428,18 +428,22 @@ static int prefer_mwait_c1_over_halt(const struct cpuinfo_x86 *c)
 
 static void mwait_idle(void)
 {
-	if (!need_resched()) {
-		if (this_cpu_has(X86_BUG_CLFLUSH_MONITOR))
+	if (!current_set_polling_and_test()) {
+		if (this_cpu_has(X86_BUG_CLFLUSH_MONITOR)) {
+			smp_mb(); /* quirk */
 			clflush((void *)&current_thread_info()->flags);
+			smp_mb(); /* quirk */
+		}
 
 		__monitor((void *)&current_thread_info()->flags, 0, 0);
-		smp_mb();
 		if (!need_resched())
 			__sti_mwait(0, 0);
 		else
 			local_irq_enable();
-	} else
+	} else {
 		local_irq_enable();
+	}
+	__current_clr_polling();
 }
 
 void select_idle_routine(const struct cpuinfo_x86 *c)

  parent reply	other threads:[~2015-03-16 12:12 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <1ae2a8af42281d6b9888ac8c76bab7bd2f431d44.1389763084.git.len.brown@intel.com>
2014-01-15  5:37 ` [PATCH REGRESSION FIX] x86 idle: restore mwait_idle() Len Brown
2014-01-15  9:28   ` Mike Galbraith
2014-01-16 22:00   ` Andy Lutomirski
2014-01-17  4:20     ` Mike Galbraith
2014-01-18  9:33       ` Mike Galbraith
2014-01-18 16:14         ` Mike Galbraith
2015-03-14 23:44           ` Ian Malone
2015-03-15  4:53             ` Mike Galbraith
2015-03-16 23:32               ` Ian Malone
2015-03-17  8:22                 ` Ingo Molnar
2015-03-17  8:33                   ` Mike Galbraith
2015-03-18  1:55                   ` Ian Malone
2015-03-16 12:11           ` tip-bot for Mike Galbraith [this message]
2015-03-16 12:11   ` [tip:sched/core] sched/idle/x86: Restore mwait_idle() to fix boot hangs, to improve power savings and to improve performance tip-bot for Len Brown

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=tip-f8e617f4582995f7c25ef25b4167213120ad122b@git.kernel.org \
    --to=tipbot@zytor.com \
    --cc=bitbucket@online.de \
    --cc=bp@alien8.de \
    --cc=efault@gmx.de \
    --cc=hpa@zytor.com \
    --cc=ibmalone@gmail.com \
    --cc=jwboyer@redhat.com \
    --cc=len.brown@intel.com \
    --cc=lenb@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-tip-commits@vger.kernel.org \
    --cc=mingo@kernel.org \
    --cc=peterz@infradead.org \
    --cc=tglx@linutronix.de \
    --cc=torvalds@linux-foundation.org \
    --subject='Re: [tip:sched/core] sched/idle/x86: Optimize unnecessary mwait_idle( ) resched IPIs' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).