LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Brendan Cully <brendan@cs.ubc.ca>
To: Ian Campbell <ijc@hellion.org.uk>
Cc: Alan Stern <stern@rowland.harvard.edu>,
	"Rafael J. Wysocki" <rjw@sisk.pl>,
	linux-pm@lists.linux-foundation.org,
	xen-devel@lists.xensource.com, "SUZUKI,
	Kazuhiro" <kaz@jp.fujitsu.com>,
	LKML <linux-kernel@vger.kernel.org>
Subject: Re: [linux-pm] [PATCH 0/2] Fix hangup after creating checkpoint on Xen.
Date: Wed, 9 Feb 2011 15:16:59 -0800	[thread overview]
Message-ID: <20110209231659.GB249@zanzibar.kublai.com> (raw)
In-Reply-To: <1297186554.9388.248.camel@zakaz.uk.xensource.com>

On Tuesday, 08 February 2011 at 17:35, Ian Campbell wrote:
> On Tue, 2011-02-08 at 11:46 -0500, Alan Stern wrote:
> > On Tue, 8 Feb 2011, Ian Campbell wrote:
> > 
> > > The problem is that currently we have:
> > > 
> > >         dpm_suspend_start(PMSG_SUSPEND);
> > >         
> > >                 dpm_suspend_noirq(PMSG_SUSPEND);
> > >                         
> > >                         sysdev_suspend(PMSG_SUSPEND);
> > >                         /* suspend hypercall */
> > >                         sysdev_resume();
> > >                 
> > >                 dpm_resume_noirq(PMSG_RESUME);
> > >         
> > >         dpm_resume_end(PMSG_RESUME);
> > > 
> > > However the suspend hypercall can return a value indicating that the
> > > suspend didn't actually happen (e.g. was cancelled). This is used e.g.
> > > when checkpointing guests, because in that case you want the original
> > > guest to continue. When the suspend didn't happen the drivers need to
> > > recover differently from if it did.
> > 
> > That is odd, and it is quite different from the intended design of the 
> > PM core.  Drivers are supposed to put their devices into a known 
> > suspended state; then afterwards they put the devices back into an 
> > operational state.  What happens while the devices are in the suspended 
> > state isn't supposed to matter -- the system transition can fail, but 
> > devices get treated exactly the same way as if it succeeded.
> > 
> > Why do your drivers need to recover differently based on the success of 
> > the hypercall?
> 
> checkpointing isn't really my area but AIUI you don't want to do a full
> device teardown and reconnect like you would with a proper suspend
> because of the time that takes which prevents you from doing continuous
> rolling checkpoints at granularity which people want to implement
> various disaster recovery schemes.
> 
> Hopefully one of the Xen checkpointing folks will chime in and explain
> why this is not possible to achieve at the individual driver level (or,
> even better, with a patch which does it that way ;-)).

As Ian says, Xen has suspend_cancel because while the normal full
suspend/resume path works, it is much slower, and the work done during
resume is redundant. I don't remember the numbers offhand, but when we
added suspend_cancel I think we could do full suspend/resume in under
100us, which was maybe a couple of orders of magnitude faster than
full resume (largely due to slow xenstore handshaking on resume,
IIRC). It made a big difference for our Remus HA project, which was
checkpointing tens of times per second.

I'd like to keep the fast resume option, and expect that it can be
contained entirely in Xen-specific code. I'll try to get someone to
look into it here.

I think fast resume is somewhat orthogonal to the problem of hanging
on resume, which just sounds like a xen-specific bug in the slow
path.

  reply	other threads:[~2011-02-09 23:23 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <20110207.180718.885783824287845425.kaz@jp.fujitsu.com>
2011-02-07  9:35 ` Rafael J. Wysocki
2011-02-08 11:22   ` Ian Campbell
2011-02-08 16:46     ` [linux-pm] " Alan Stern
2011-02-08 17:35       ` Ian Campbell
2011-02-09 23:16         ` Brendan Cully [this message]
2011-02-09 23:42           ` Alan Stern
2011-02-10 11:40             ` [Xen-devel] " Ian Campbell
2011-02-10 16:00               ` Alan Stern
2011-02-10 16:26                 ` Rafael J. Wysocki
2011-02-10 16:34                 ` Ian Campbell
2011-02-10 17:01                   ` Rafael J. Wysocki
2011-02-10 18:56                   ` Alan Stern
2011-02-10 11:31           ` Ian Campbell
2011-02-10 17:53             ` Brendan Cully

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20110209231659.GB249@zanzibar.kublai.com \
    --to=brendan@cs.ubc.ca \
    --cc=ijc@hellion.org.uk \
    --cc=kaz@jp.fujitsu.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-pm@lists.linux-foundation.org \
    --cc=rjw@sisk.pl \
    --cc=stern@rowland.harvard.edu \
    --cc=xen-devel@lists.xensource.com \
    --subject='Re: [linux-pm] [PATCH 0/2] Fix hangup after creating checkpoint on Xen.' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).