LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Vitaly Kuznetsov <vkuznets@redhat.com>
To: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: "K. Y. Srinivasan" <kys@microsoft.com>,
	devel@linuxdriverproject.org,
	Haiyang Zhang <haiyangz@microsoft.com>,
	linux-kernel@vger.kernel.org, Dexuan Cui <decui@microsoft.com>
Subject: Re: [PATCH v3] Drivers: hv: vmbus: prevent cpu offlining on newer hypervisors
Date: Mon, 26 Jan 2015 11:38:54 +0100	[thread overview]
Message-ID: <87k309g729.fsf@vitty.brq.redhat.com> (raw)
In-Reply-To: <20150125133659.GA21552@kroah.com> (Greg Kroah-Hartman's message of "Sun, 25 Jan 2015 21:36:59 +0800")

[-- Attachment #1: Type: text/plain, Size: 2056 bytes --]

Greg Kroah-Hartman <gregkh@linuxfoundation.org> writes:

> On Mon, Jan 12, 2015 at 05:50:11PM +0100, Vitaly Kuznetsov wrote:
>> When an SMP Hyper-V guest is running on top of 2012R2 Server and secondary
>> cpus are sent offline (with echo 0 > /sys/devices/system/cpu/cpu$cpu/online)
>> the system freeze is observed. This happens due to the fact that on newer
>> hypervisors (Win8, WS2012R2, ...) vmbus channel handlers are distributed
>> across all cpus (see init_vp_index() function in drivers/hv/channel_mgmt.c)
>> and on cpu offlining nobody reassigns them to CPU0. Prevent cpu offlining
>> when vmbus is loaded until the issue is fixed host-side.
>> 
>> This patch also disables hibernation but it is OK as it is also broken (MCE
>> error is hit on resume). Suspend still works.
>> 
>> Tested with WS2008R2 and WS2012R2.
>> 
>> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
>> Signed-off-by: K. Y. Srinivasan <kys@microsoft.com>
>> ---
>> Changes since v2:
>> - repair the build when vmbus is builded as a module [Greg KH] by saving
>>   current cpu_disable pointer to previous_cpu_disable and restoring it on
>>   unload;
>> - return -ENOSYS (same as native_cpu_disable when !CONFIG_HOTPLUG_CPU) instead
>>   of -1 in hyperv_cpu_disable().
>> 
>> Changes since v1:
>> - introduce hv_cpu_hotplug_quirk() function to not spread #ifdefs [Greg KH];
>> - add pr_notice() message "hv_vmbus: CPU offlining is not supported by
>>   hypervisor".
>> ---
>>  drivers/hv/vmbus_drv.c | 36 ++++++++++++++++++++++++++++++++++++
>>  1 file changed, 36 insertions(+)
>
> Doesn't apply to my char-misc-test branch at all :(

Another mid-air collision with K.Y's "Drivers: hv: vmbus: Implement a
clockevent device", please use the attached version. No functional
changes are required, I just fixed the merge conflict (includes).

Othere than that (and sorry for meddling), would it it be better if
you switch to 'pull requests' workflow with K.Y? There is a lot of
ongoing work in hyperv nowdays and such collisions seem otherwise
inevitable ...

-- 
  Vitaly


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Drivers-hv-vmbus-prevent-cpu-offlining-on-newer-hype.patch --]
[-- Type: text/x-patch, Size: 2781 bytes --]

>From 95f9b0ff3f73c3bbe8aa9c525414097c2c26a3ef Mon Sep 17 00:00:00 2001
From: Vitaly Kuznetsov <vkuznets@redhat.com>
Date: Mon, 12 Jan 2015 17:50:11 +0100
Subject: [PATCH] Drivers: hv: vmbus: prevent cpu offlining on newer
 hypervisors

When an SMP Hyper-V guest is running on top of 2012R2 Server and secondary
cpus are sent offline (with echo 0 > /sys/devices/system/cpu/cpu$cpu/online)
the system freeze is observed. This happens due to the fact that on newer
hypervisors (Win8, WS2012R2, ...) vmbus channel handlers are distributed
across all cpus (see init_vp_index() function in drivers/hv/channel_mgmt.c)
and on cpu offlining nobody reassigns them to CPU0. Prevent cpu offlining
when vmbus is loaded until the issue is fixed host-side.

This patch also disables hibernation but it is OK as it is also broken (MCE
error is hit on resume). Suspend still works.

Tested with WS2008R2 and WS2012R2.

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Signed-off-by: K. Y. Srinivasan <kys@microsoft.com>
---
 drivers/hv/vmbus_drv.c | 36 ++++++++++++++++++++++++++++++++++++
 1 file changed, 36 insertions(+)

diff --git a/drivers/hv/vmbus_drv.c b/drivers/hv/vmbus_drv.c
index 35e3f42..90c3400 100644
--- a/drivers/hv/vmbus_drv.c
+++ b/drivers/hv/vmbus_drv.c
@@ -33,6 +33,7 @@
 #include <linux/hyperv.h>
 #include <linux/kernel_stat.h>
 #include <linux/clockchips.h>
+#include <linux/cpu.h>
 #include <asm/hyperv.h>
 #include <asm/hypervisor.h>
 #include <asm/mshyperv.h>
@@ -704,6 +705,39 @@ static void vmbus_isr(void)
 	}
 }
 
+#ifdef CONFIG_HOTPLUG_CPU
+static int hyperv_cpu_disable(void)
+{
+	return -ENOSYS;
+}
+
+static void hv_cpu_hotplug_quirk(bool vmbus_loaded)
+{
+	static void *previous_cpu_disable;
+
+	/*
+	 * Offlining a CPU when running on newer hypervisors (WS2012R2, Win8,
+	 * ...) is not supported at this moment as channel interrupts are
+	 * distributed across all of them.
+	 */
+
+	if ((vmbus_proto_version == VERSION_WS2008) ||
+	    (vmbus_proto_version == VERSION_WIN7))
+		return;
+
+	if (vmbus_loaded) {
+		previous_cpu_disable = smp_ops.cpu_disable;
+		smp_ops.cpu_disable = hyperv_cpu_disable;
+		pr_notice("CPU offlining is not supported by hypervisor\n");
+	} else if (previous_cpu_disable)
+		smp_ops.cpu_disable = previous_cpu_disable;
+}
+#else
+static void hv_cpu_hotplug_quirk(bool vmbus_loaded)
+{
+}
+#endif
+
 /*
  * vmbus_bus_init -Main vmbus driver initialization routine.
  *
@@ -744,6 +778,7 @@ static int vmbus_bus_init(int irq)
 	if (ret)
 		goto err_alloc;
 
+	hv_cpu_hotplug_quirk(true);
 	vmbus_request_offers();
 
 	return 0;
@@ -997,6 +1032,7 @@ static void __exit vmbus_exit(void)
 	bus_unregister(&hv_bus);
 	hv_cleanup();
 	acpi_bus_unregister_driver(&vmbus_acpi_driver);
+	hv_cpu_hotplug_quirk(false);
 }
 
 
-- 
1.9.3


  reply	other threads:[~2015-01-26 10:39 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2014-11-26 13:52 [PATCH] " Vitaly Kuznetsov
2014-11-27  3:02 ` Greg Kroah-Hartman
2014-11-27  3:24   ` Dexuan Cui
2014-11-27  9:52     ` Vitaly Kuznetsov
2014-12-01 10:52 ` [PATCH v2] " Vitaly Kuznetsov
2014-12-01 11:12   ` Dexuan Cui
2015-01-09 20:57   ` Greg Kroah-Hartman
2015-01-12 16:50   ` [PATCH v3] " Vitaly Kuznetsov
2015-01-12 18:54     ` KY Srinivasan
2015-01-25 13:36     ` Greg Kroah-Hartman
2015-01-26 10:38       ` Vitaly Kuznetsov [this message]
2015-01-26 16:11         ` KY Srinivasan
2015-01-26 22:54         ` Greg Kroah-Hartman
2015-01-26 23:53           ` KY Srinivasan

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87k309g729.fsf@vitty.brq.redhat.com \
    --to=vkuznets@redhat.com \
    --cc=decui@microsoft.com \
    --cc=devel@linuxdriverproject.org \
    --cc=gregkh@linuxfoundation.org \
    --cc=haiyangz@microsoft.com \
    --cc=kys@microsoft.com \
    --cc=linux-kernel@vger.kernel.org \
    --subject='Re: [PATCH v3] Drivers: hv: vmbus: prevent cpu offlining on newer hypervisors' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).