nfsd: Fix the NFSv4 pNFS MDS server for DS NFSERR_NOSPC

If a pNFS server's DS runs out of disk space, it replies
NFSERR_NOSPC to the client doing writing.  For the Linux
client, it then sends a LayoutError RPC to the server to
tell it about the error and keeps retrying, doing repeated
LayoutGet and Write RPCs to the DS.  The Linux client is
"stuck" until disk space on the DS is free'd up.
For a mirrored server configuration, the first mirror that
ran out of space was taken offline.  This does not make
much sense, since the other mirror(s) will run out of space
soon and the fix is a manual cleanup up disk space.

This patch changes the pNFS server to not disable a mirror
for the mirrored case when this occurs.

Further work is needed, since the Linux client expects the
MDS to reply NFSERR_NOSPC to LayoutGets once the DS is out
of space.  Without this further change, the above mentioned
looping occurs.

Found during a recent IEFT NFSv4 working group testing event.

MFC after:	2 weeks

2 files changed, 8 insertions, 6 deletions

diff --git a/sys/fs/nfsserver/nfs_nfsdserv.c b/sys/fs/nfsserver/nfs_nfsdserv.c
index fca1df3a716a..4d92cb61614d 100644
--- a/sys/fs/nfsserver/nfs_nfsdserv.c
+++ b/sys/fs/nfsserver/nfs_nfsdserv.c
@@ -5046,10 +5046,11 @@ nfsrvd_layouterror(struct nfsrv_descript *nd, __unused int isdgram,
 		opnum = fxdr_unsigned(int, *tl);
 		NFSD_DEBUG(4, "nfsrvd_layouterr op=%d stat=%d\n", opnum, stat);
 		/*
-		 * Except for NFSERR_ACCES and NFSERR_STALE errors,
-		 * disable the mirror.
+		 * Except for NFSERR_ACCES, NFSERR_STALE and NFSERR_NOSPC
+		 * errors, disable the mirror.
 		 */
-		if (stat != NFSERR_ACCES && stat != NFSERR_STALE)
+		if (stat != NFSERR_ACCES && stat != NFSERR_STALE &&
+		    stat != NFSERR_NOSPC)
 			nfsrv_delds(devid, curthread);
 	}
 nfsmout:
diff --git a/sys/fs/nfsserver/nfs_nfsdstate.c b/sys/fs/nfsserver/nfs_nfsdstate.c
index 750eda2027ec..797b9b0a466e 100644
--- a/sys/fs/nfsserver/nfs_nfsdstate.c
+++ b/sys/fs/nfsserver/nfs_nfsdstate.c
@@ -7009,10 +7009,11 @@ nfsrv_flexlayouterr(struct nfsrv_descript *nd, uint32_t *layp, int maxcnt,
 			NFSD_DEBUG(4, "flexlayouterr op=%d stat=%d\n", opnum,
 			    stat);
 			/*
-			 * Except for NFSERR_ACCES and NFSERR_STALE errors,
-			 * disable the mirror.
+			 * Except for NFSERR_ACCES, NFSERR_STALE and
+			 * NFSERR_NOSPC errors, disable the mirror.
 			 */
-			if (stat != NFSERR_ACCES && stat != NFSERR_STALE)
+			if (stat != NFSERR_ACCES && stat != NFSERR_STALE &&
+			    stat != NFSERR_NOSPC)
 				nfsrv_delds(devid, p);
 		}
 	}