nfs server patches for 2.6.25

nfs server patches for 2.6.25

[J. Bruce Fields]

Below is a summary of the nfs server patches which I expect to submit
(any day now) for 2.6.25; please let me know if you notice anything
wrong or missing.  I'll follow up to linux-nfs@vger.kernel.org with the
full list of patches.

The biggest changes come from Tom Tucker's rdma and server transport
switch patches, which Chuck Lever, Greg Banks, Neil Brown, and others
helped review (thanks!).  The rest is mostly bug fixes and cleanup.

As a reminder, you can see the patches I've got queued up any time from
the nfs-server-stable branch at:

	git://linux-nfs.org/~bfields/linux.git nfs-server-stable

--b.

Chuck Lever (9):
      SUNRPC: Prevent length underflow in read_flush()
      SUNRPC: Use unsigned string lengths in xdr_decode_string_inplace
      NLM: Fix sign of length of NLM variable length strings
      NFSD: Use unsigned length argument for decode_filename
      NFSD: File name length signage in nfsd request argument structures
      NFSD: Adjust filename length argument of nfsd_lookup
      NFSD: Use unsigned length argument for decode_pathname
      NFSD: Fix mixed sign comparison in nfs3svc_decode_symlinkargs
      NFSD: Path name length signage in nfsd request argument structures

Frank Filz (1):
      nfsd: Allow AIX client to read dir containing mountpoints

J. Bruce Fields (34):
      nfsd4: probe callback channel only once
      nfsd: move callback rpc_client creation into separate thread
      knfsd: fix broken length check in nfs4idmap.c
      knfsd: fix cache.c comment
      nfsd: Fix handling of negative lengths in read_buf()
      knfsd: cleanup nfsd4 properly on module init failure
      nfsd: cleanup nfsd module initialization cleanup
      nfsd: fail module init on reply cache init failure
      knfsd: cache unregistration needn't return error
      nfsd: select CONFIG_PROC_FS in nfsv4 and gss server cases
      nfsd: fail init on /proc/fs/nfs/exports creation failure
      nfsd: move cache proc (un)registration to separate function
      knfsd: allow cache_register to return error on failure
      nfsd: move nfsd/auth.h into fs/nfsd
      nfsd: minor fs/nfsd/auth.h cleanup
      nfsd4: kill some unneeded setclientid comments
      nfsd: eliminate final bogus case from setclientid logic
      nfsd: uniquify cl_confirm values
      nfsd4: kill unnecessary same_name() in setclientid_confirm
      nfsd4: remove unnecessary cl_verifier check from setclientid_confirm
      nfsd4: kill unneeded cl_confirm check
      nfsd: fix encode_entryplus_baggage() indentation
      nfsd4: make current_clientid local
      nfsd4: miscellaneous nfs4state.c style fixes
      nfsd4: recognize callback channel failure earlier
      nfsd4: fix bad seqid on lock request incompatible with open mode
      nfsd: allow root to set uid and gid on create
      nfsd: fix rsi_cache reference count leak
      sunrpc: gss: simplify rsi_parse logic
      nfsd4: clean up access_valid, deny_valid checks.
      svcrpc: ensure gss DESTROY tokens free contexts from cache
      knfsd: don't bother mapping putrootfh enoent to eperm
      lockd: minor log message fix
      nfsd: more careful input validation in nfsctl write methods

Jeff Layton (4):
      SUNRPC: spin svc_rqst initialization to its own function
      SUNRPC: export svc_sock_update_bufs
      NLM: Convert lockd to use kthreads
      NLM: have nlm_shutdown_hosts kill off all NLM RPC tasks

Neil Brown (1):
      knfsd: change mailing list for nfsd in MAINTAINERS

Oleg Drokin (3):
      lockd: fix reference count leaks in async locking case
      lockd: fix a leak in nlmsvc_testlock asynchronous request handling
      Leak in nlmsvc_testlock for async GETFL case

Prasad P (1):
      Fix incorrect assignment

Tom Tucker (47):
      svc: Add an svc transport class
      svc: Make svc_sock the tcp/udp transport
      svc: Change the svc_sock in the rqstp structure to a transport
      svc: Add a max payload value to the transport
      svc: Move sk_sendto and sk_recvfrom to svc_xprt_class
      svc: Add transport specific xpo_release function
      svc: Add per-transport delete functions
      svc: Add xpo_prep_reply_hdr
      svc: Add a transport function that checks for write space
      svc: Move close processing to a single place
      svc: Add xpo_accept transport function
      svc: Remove unnecessary call to svc_sock_enqueue
      svc: Move connection limit checking to its own function
      svc: Add a generic transport svc_create_xprt function
      svc: Change services to use new svc_create_xprt service
      svc: Change sk_inuse to a kref
      svc: Move sk_flags to the svc_xprt structure
      svc: Move sk_server and sk_pool to svc_xprt
      svc: Make close transport independent
      svc: Move sk_reserved to svc_xprt
      svc: Make the enqueue service transport neutral and export it.
      svc: Make svc_send transport neutral
      svc: Change svc_sock_received to svc_xprt_received and export it
      svc: Move accept call to svc_xprt_received to common code
      svc: Remove sk_lastrecv
      svc: Move the authinfo cache to svc_xprt.
      svc: Make deferral processing xprt independent
      svc: Move the sockaddr information to svc_xprt
      svc: Make svc_sock_release svc_xprt_release
      svc: Make svc_recv transport neutral
      svc: Make svc_age_temp_sockets svc_age_temp_transports
      svc: Move create logic to common code
      svc: Removing remaining references to rq_sock in rqstp
      svc: Make svc_check_conn_limits xprt independent
      svc: Move the xprt independent code to the svc_xprt.c file
      svc: Add transport hdr size for defer/revisit
      svc: Add /proc/sys/sunrpc/transport files
      svc: Add svc API that queries for a transport instance
      knfsd: Support adding transports by writing portlist file
      svc: Add svc_xprt_names service to replace svc_sock_names
      rdma: SVCRMDA Header File
      rdma: SVCRDMA Transport Module
      rdma: SVCRDMA Core Transport Services
      rdma: SVCRDMA recvfrom
      rdma: SVCRDMA sendto
      rdma: ONCRPC RDMA protocol marshalling
      rdma: makefile

nfs server patches not in 2.6.25

[J. Bruce Fields]

Just some idea what we might be working on for 2.6.26, besides continued
bug-fixing and cleanup:

Work that we already have patches for and that I expect to be included
in whole or in 2.6.26:

	- ipv6: Aurélien Charbon's patch to add ipv6 support to the
	  server's export interface is ready.  I'm not clear what else
	  remains for full ipv6 support.
	- failover and migration: Wendy Cheng's patches appear to be in
	  good shape, so I expect them or something with equivalent
	  functionality to be in 2.6.26.
	- gss callbacks: We have patches to add support for rpcsec_gss
	  on NFSv4's callback channel (allowing us to support
	  delegations on kerberos mounts), but they've been put on hold
	  pending improvements to the client's gssd upcall.  I hope to
	  get back to that work in the next few weeks.

Also in progress:

	- spkm3 and future gss mechanisms may generate context
	  initiation rpc's that are very large.  Olga Kornievskaia and I
	  have been working on fixing the server gssd interfaces to
	  permit this.

	- There are some mismatches between the semantics required for
	  nfsv4 delegations and what Linux's lease subsystem provides
	  us.  David Richter and I have done a little work on this.  We
	  need to start submitting it.

Three items I identified previously as issues I'd like fixed before we
removed the dependency of CONFIG_NFSD_V4 on CONFIG_EXPERIMENTAL:

	http://linux-nfs.org/pipermail/nfsv4/2006-December/005497.html

	- export paths consistent between v2/v3/v4:  We have some code
	  that fixes this entirely in userspace.  That approach doesn't
	  provide stable filehandles in the NFSv4 pseudofilesystem, and
	  there seems to be a general sentiment that it's overly
	  complicated.  It has the one advantage that we don't have to
	  commit to it, since it uses only existing kernel interfaces.
	  So I think we're probably going to apply that to nfs-utils as
	  a stopgap measure and start work on fixing this in the kernel
	  at the same time....

	- reboot recovery: there have been complaints about the
	  server-side nfsv4 reboot recovery code for a while, we've had
	  code that tries to fix it for a while, and it just hasn't
	  happened.  I'm hoping we can finally get this ready for
	  2.6.26.

	- export security: this was finished in 2.6.23; we now support
	  export options like sec=krb5:krb5i:krb5p, which have a few
	  advantages over the special gss/krb5 client names.  This could
	  be better documented, though.

I've probably left a lot out.  Let me know of ongoing projects and
todo's that I've forgotten....

--b.

Re: nfs server patches for 2.6.25

[Simon Holm Thøgersen]

fre, 25 01 2008 kl. 18:15 -0500, skrev J. Bruce Fields: 
> Below is a summary of the nfs server patches which I expect to submit
> (any day now) for 2.6.25; please let me know if you notice anything
> wrong or missing.  I'll follow up to linux-nfs@vger.kernel.org with the
> full list of patches.

There are at least two patches that don't compile and break
bisectability. The errors should be obvious, just checkout and make
6f310df and 23b4e2e (patch no. 29 and 65 from the posting of the patch
set on linux-nfs).

[...] 
> J. Bruce Fields (34):
>       nfsd4: kill unnecessary same_name() in setclientid_confirm
diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index 508b256..d026c24 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -858,8 +857,7 @@ nfsd4_setclientid_confirm(struct svc_rqst *rqstp,
                }
        } else if ((conf && !unconf) ||
            ((conf && unconf) && 
-            (!same_verf(&conf->cl_verifier, &unconf->cl_verifier) ||
-             !same_name(conf->cl_recdir, unconf->cl_recdir)))) {
+            !same_verf(&conf->cl_verifier, &unconf->cl_verifier)) {
missing ending parenthesis ---------------------------------------^


[...] 
> Tom Tucker (47):
>       svc: Make close transport independent
diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c
index f86538e..6dffdda 100644
--- a/net/sunrpc/svcsock.c
+++ b/net/sunrpc/svcsock.c
@@ -1912,66 +1911,63 @@ static void svc_sock_free(struct svc_xprt *xprt)
 }
 
 /*
- * Remove a dead socket
+ * Remove a dead transport
  */
-static void
-svc_delete_socket(struct svc_sock *svsk)
+static void svc_delete_xprt(struct svc_xprt *xprt)
 {
-       struct svc_serv *serv;
-       struct sock     *sk;
-
-       dprintk("svc: svc_delete_socket(%p)\n", svsk);
-
-       serv = svsk->sk_xprt.xpt_server;
-       sk = svsk->sk_sk;
+       struct svc_serv *serv = xprt->xpt_server;
 
-       svsk->sk_xprt.xpt_ops->xpo_detach(&svsk->sk_xprt);
+       dprintk("svc: svc_delete_xprt(%p)\n", xprt);
+       xprt->xpt_ops->xpo_detach(xprt);
 
        spin_lock_bh(&serv->sv_lock);
-
-       if (!test_and_set_bit(XPT_DETACHED, &svsk->sk_xprt.xpt_flags))
-               list_del_init(&svsk->sk_list);
+       if (!test_and_set_bit(XPT_DETACHED, &xprt->xpt_flags))
+               list_del_init(&xprt->xpt_list);
        /*
-        * We used to delete the svc_sock from whichever list
-        * it's sk_ready node was on, but we don't actually
+        * We used to delete the transport from whichever list
+        * it's sk_xprt.xpt_ready node was on, but we don't actually
         * need to.  This is because the only time we're called
         * while still attached to a queue, the queue itself
         * is about to be destroyed (in svc_destroy).
         */
-       if (!test_and_set_bit(XPT_DEAD, &svsk->sk_xprt.xpt_flags)) {
-               BUG_ON(atomic_read(&svsk->sk_xprt.xpt_ref.refcount) <
2);
-               if (test_bit(XPT_TEMP, &svsk->sk_xprt.xpt_flags))
+       if (!test_and_set_bit(XPT_DEAD, &xprt->xpt_flags)) {
+               BUG_ON(atomic_read(&xprt->xpt_ref.refcount) < 2);
+               if (test_bit(XPT_TEMP, &xprt->xpt_flags))
                        serv->sv_tmpcnt--;
                svc_xprt_put(&svsk->sk_xprt);
should have been
                svc_xprt_put(&xprt);

There might be more errors in the next 15 patches or so; I didn't test
them with this error corrected.



Simon Holm Thøgersen

Re: nfs server patches for 2.6.25

[J. Bruce Fields]

On Sun, Jan 27, 2008 at 09:42:23PM +0100, Simon Holm Thøgersen wrote:
> 
> fre, 25 01 2008 kl. 18:15 -0500, skrev J. Bruce Fields: 
> > Below is a summary of the nfs server patches which I expect to submit
> > (any day now) for 2.6.25; please let me know if you notice anything
> > wrong or missing.  I'll follow up to linux-nfs@vger.kernel.org with the
> > full list of patches.
> 
> There are at least two patches that don't compile and break
> bisectability. The errors should be obvious, just checkout and make
> 6f310df and 23b4e2e (patch no. 29 and 65 from the posting of the patch
> set on linux-nfs).

Whoops, thanks for finding both of those.

> 
> [...] 
> > J. Bruce Fields (34):
> >       nfsd4: kill unnecessary same_name() in setclientid_confirm
> diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
> index 508b256..d026c24 100644
> --- a/fs/nfsd/nfs4state.c
> +++ b/fs/nfsd/nfs4state.c
> @@ -858,8 +857,7 @@ nfsd4_setclientid_confirm(struct svc_rqst *rqstp,
>                 }
>         } else if ((conf && !unconf) ||
>             ((conf && unconf) && 
> -            (!same_verf(&conf->cl_verifier, &unconf->cl_verifier) ||
> -             !same_name(conf->cl_recdir, unconf->cl_recdir)))) {
> +            !same_verf(&conf->cl_verifier, &unconf->cl_verifier)) {
> missing ending parenthesis ---------------------------------------^

Yep; fixed.

> 
> 
> [...] 
> > Tom Tucker (47):
> >       svc: Make close transport independent
> diff --git a/net/sunrpc/svcsock.c b/net/sunrpc/svcsock.c
> index f86538e..6dffdda 100644
> --- a/net/sunrpc/svcsock.c
> +++ b/net/sunrpc/svcsock.c
> @@ -1912,66 +1911,63 @@ static void svc_sock_free(struct svc_xprt *xprt)
>  }
>  
>  /*
> - * Remove a dead socket
> + * Remove a dead transport
>   */
> -static void
> -svc_delete_socket(struct svc_sock *svsk)
> +static void svc_delete_xprt(struct svc_xprt *xprt)
>  {
> -       struct svc_serv *serv;
> -       struct sock     *sk;
> -
> -       dprintk("svc: svc_delete_socket(%p)\n", svsk);
> -
> -       serv = svsk->sk_xprt.xpt_server;
> -       sk = svsk->sk_sk;
> +       struct svc_serv *serv = xprt->xpt_server;
>  
> -       svsk->sk_xprt.xpt_ops->xpo_detach(&svsk->sk_xprt);
> +       dprintk("svc: svc_delete_xprt(%p)\n", xprt);
> +       xprt->xpt_ops->xpo_detach(xprt);
>  
>         spin_lock_bh(&serv->sv_lock);
> -
> -       if (!test_and_set_bit(XPT_DETACHED, &svsk->sk_xprt.xpt_flags))
> -               list_del_init(&svsk->sk_list);
> +       if (!test_and_set_bit(XPT_DETACHED, &xprt->xpt_flags))
> +               list_del_init(&xprt->xpt_list);
>         /*
> -        * We used to delete the svc_sock from whichever list
> -        * it's sk_ready node was on, but we don't actually
> +        * We used to delete the transport from whichever list
> +        * it's sk_xprt.xpt_ready node was on, but we don't actually
>          * need to.  This is because the only time we're called
>          * while still attached to a queue, the queue itself
>          * is about to be destroyed (in svc_destroy).
>          */
> -       if (!test_and_set_bit(XPT_DEAD, &svsk->sk_xprt.xpt_flags)) {
> -               BUG_ON(atomic_read(&svsk->sk_xprt.xpt_ref.refcount) <
> 2);
> -               if (test_bit(XPT_TEMP, &svsk->sk_xprt.xpt_flags))
> +       if (!test_and_set_bit(XPT_DEAD, &xprt->xpt_flags)) {
> +               BUG_ON(atomic_read(&xprt->xpt_ref.refcount) < 2);
> +               if (test_bit(XPT_TEMP, &xprt->xpt_flags))
>                         serv->sv_tmpcnt--;
>                 svc_xprt_put(&svsk->sk_xprt);
> should have been
>                 svc_xprt_put(&xprt);

Actually looks like it should be svc_xprt_put(xprt); fixed.

> There might be more errors in the next 15 patches or so; I didn't test
> them with this error corrected.

OK.  I've got a machine running

	for c in $(git rev-list origin..linux-nfs/nfs-server-stable^); do
		git checkout $c
		make -s -j4
	done

now just to make sure....  Thanks again!

--b.

Re: nfs server patches for 2.6.25

[J. Bruce Fields]

On Sun, Jan 27, 2008 at 05:10:17PM -0500, bfields wrote:
> OK.  I've got a machine running
> 
> 	for c in $(git rev-list origin..linux-nfs/nfs-server-stable^); do
> 		git checkout $c
> 		make -s -j4
> 	done
> 
> now just to make sure....  Thanks again!

Pfft, that should have been:

	for c in $(git rev-list --reverse origin..linux-nfs/nfs-server-stable^); do
		git checkout $c
		make -s -j4
	done

The results are a lot easier to understand if you do the compiles
in chronological order!

Anyway, that found one more compile failure, now fixed.  Hopefully
that's the last.  (Probably I did an incomplete job of reordering one of
the patches the other week....)

--b.