LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
* [regression] nfs4: 30 second delay during umount of remote fs on system shutdown
@ 2008-02-28 17:29 Frans Pop
  2008-02-28 21:46 ` Trond Myklebust
  0 siblings, 1 reply; 5+ messages in thread
From: Frans Pop @ 2008-02-28 17:29 UTC (permalink / raw)
  To: linux-nfs; +Cc: linux-kernel

With 2.6.25-rc2 and -rc3 I noticed a pause of 30 seconds when my system 
(Debian unstable, x86_64) is being shutdown. The cause is the unmounting of 
an nfs4 remote file system.
The pause does not happen with 2.6.24. I'm unsure about 2.6.25-rc1 as I was 
concentrating on other issues when I was testing that.

I can reproduce the same pause on an active system by stopping portmap 
before doing an umount of the filesystem.

The shutdown sequence on my box includes:
K20nfs-common
[...]
S20sendsigs
[...]
S31umountnfs.sh
S32portmap

So it seems that things should be OK (S32portmap is after S31umountnfs.sh), 
but in fact portmap has already been killed during S20sendsigs!

I have verified that portmap is also not running during S31umountnfs.sh with 
2.6.24, so apparently there has been a change in the kernel that has made 
umount look for portmap and cause the 30 second delay if it is not running.

Is this a kernel regression?

Cheers,
FJP

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [regression] nfs4: 30 second delay during umount of remote fs on system shutdown
  2008-02-28 17:29 [regression] nfs4: 30 second delay during umount of remote fs on system shutdown Frans Pop
@ 2008-02-28 21:46 ` Trond Myklebust
  2008-02-29  1:03   ` Frans Pop
  0 siblings, 1 reply; 5+ messages in thread
From: Trond Myklebust @ 2008-02-28 21:46 UTC (permalink / raw)
  To: Frans Pop; +Cc: linux-nfs, linux-kernel


On Thu, 2008-02-28 at 18:29 +0100, Frans Pop wrote:

> I have verified that portmap is also not running during S31umountnfs.sh with 
> 2.6.24, so apparently there has been a change in the kernel that has made 
> umount look for portmap and cause the 30 second delay if it is not running.

You are probably seeing the effect of lockd going down: it always
attempts to unregister from the portmapper (and no, this is _not_ new
behaviour).

If debian is killing the portmapper while lockd is still up, then that
is a definite debian bug...

Trond


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [regression] nfs4: 30 second delay during umount of remote fs on system shutdown
  2008-02-28 21:46 ` Trond Myklebust
@ 2008-02-29  1:03   ` Frans Pop
  2008-02-29  1:09     ` Jeff Layton
  2008-02-29  4:55     ` Trond Myklebust
  0 siblings, 2 replies; 5+ messages in thread
From: Frans Pop @ 2008-02-29  1:03 UTC (permalink / raw)
  To: Trond Myklebust; +Cc: linux-nfs, linux-kernel

On Thursday 28 February 2008, Trond Myklebust wrote:
> On Thu, 2008-02-28 at 18:29 +0100, Frans Pop wrote:
> > I have verified that portmap is also not running during S31umountnfs.sh
> > with 2.6.24, so apparently there has been a change in the kernel that
> > has made umount look for portmap and cause the 30 second delay if it is
> > not running.
>
> You are probably seeing the effect of lockd going down: it always
> attempts to unregister from the portmapper (and no, this is _not_ new
> behaviour).

It is _very much_ new behavior because as soon as I boot (and shut down) 
with a 2.6.24 kernel the problem completely disappears. Userland is *100% 
identical* between my tests.

> If debian is killing the portmapper while lockd is still up, then that
> is a definite debian bug...

I have no idea about that. AFAICT lockd is not even running (at least, 
there's no process named lockd, even when the system is running normally.
Isn't lockd an nfs3 thing? I'm using nfs4 here.

Cheers,
FJP

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [regression] nfs4: 30 second delay during umount of remote fs on system shutdown
  2008-02-29  1:03   ` Frans Pop
@ 2008-02-29  1:09     ` Jeff Layton
  2008-02-29  4:55     ` Trond Myklebust
  1 sibling, 0 replies; 5+ messages in thread
From: Jeff Layton @ 2008-02-29  1:09 UTC (permalink / raw)
  To: Frans Pop; +Cc: Trond Myklebust, linux-nfs, linux-kernel

On Fri, 29 Feb 2008 02:03:05 +0100
Frans Pop <elendil@planet.nl> wrote:

> On Thursday 28 February 2008, Trond Myklebust wrote:
> > On Thu, 2008-02-28 at 18:29 +0100, Frans Pop wrote:
> > > I have verified that portmap is also not running during S31umountnfs.sh
> > > with 2.6.24, so apparently there has been a change in the kernel that
> > > has made umount look for portmap and cause the 30 second delay if it is
> > > not running.
> >
> > You are probably seeing the effect of lockd going down: it always
> > attempts to unregister from the portmapper (and no, this is _not_ new
> > behaviour).
> 
> It is _very much_ new behavior because as soon as I boot (and shut down) 
> with a 2.6.24 kernel the problem completely disappears. Userland is *100% 
> identical* between my tests.
> 
> > If debian is killing the portmapper while lockd is still up, then that
> > is a definite debian bug...
> 
> I have no idea about that. AFAICT lockd is not even running (at least, 
> there's no process named lockd, even when the system is running normally.
> Isn't lockd an nfs3 thing? I'm using nfs4 here.
> 

It may be the NFSv4 callback thread going down then. It also tries to
unregister with the portmapper and recent patches fixed the reference
counting so that it actually tears things down instead of just leaking
memory and sockets.

I'm not sure what we can do about that other than try to fix it so that
it doesn't try to unregister with the portmapper. It really doesn't need
to since it no longer registers with it.

-- 
Jeff Layton <jlayton@redhat.com>

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [regression] nfs4: 30 second delay during umount of remote fs on system shutdown
  2008-02-29  1:03   ` Frans Pop
  2008-02-29  1:09     ` Jeff Layton
@ 2008-02-29  4:55     ` Trond Myklebust
  1 sibling, 0 replies; 5+ messages in thread
From: Trond Myklebust @ 2008-02-29  4:55 UTC (permalink / raw)
  To: Frans Pop; +Cc: linux-nfs, linux-kernel


On Fri, 2008-02-29 at 02:03 +0100, Frans Pop wrote:
> On Thursday 28 February 2008, Trond Myklebust wrote:
> > On Thu, 2008-02-28 at 18:29 +0100, Frans Pop wrote:
> > > I have verified that portmap is also not running during S31umountnfs.sh
> > > with 2.6.24, so apparently there has been a change in the kernel that
> > > has made umount look for portmap and cause the 30 second delay if it is
> > > not running.
> >
> > You are probably seeing the effect of lockd going down: it always
> > attempts to unregister from the portmapper (and no, this is _not_ new
> > behaviour).
> 
> It is _very much_ new behavior because as soon as I boot (and shut down) 
> with a 2.6.24 kernel the problem completely disappears. Userland is *100% 
> identical* between my tests.

The "new behaviour" is that we fixed a bug whereby the lockd code was
failing to flush signals before attempting the rpc unregister. The sad
fact is that userland appears to have been 100% broken for some time.
The difference is that it was able to get away with it earlier...

> > If debian is killing the portmapper while lockd is still up, then that
> > is a definite debian bug...
> 
> I have no idea about that. AFAICT lockd is not even running (at least, 
> there's no process named lockd, even when the system is running normally.
> Isn't lockd an nfs3 thing? I'm using nfs4 here.

NFSv4 has a delegation recall server to kill. Same difference...

Trond


^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2008-02-29  4:55 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2008-02-28 17:29 [regression] nfs4: 30 second delay during umount of remote fs on system shutdown Frans Pop
2008-02-28 21:46 ` Trond Myklebust
2008-02-29  1:03   ` Frans Pop
2008-02-29  1:09     ` Jeff Layton
2008-02-29  4:55     ` Trond Myklebust

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).