nullfs: Fix handling of doomed vnodes in nullfs_unlink_lowervp()

nullfs_unlink_lowervp() is called with the lower vnode locked, so the
nullfs vnode is locked too.  The following can occur:
1. the vunref() call decrements the usecount 2->1,
2. a different thread calls vrele() on the vnode, decrements the
   usecount 0->1, then blocks on the vnode lock,
3. the first thread tests vp->v_usecount == 0 and observes that it is
   true,
4. the first thread incorrectly unlocks the lower vnode.

Fix this by testing VN_IS_DOOMED directly.  Since
nullfs_unlink_lowervp() holds the vnode lock, the value of the
VIRF_DOOMED flag is stable.

Thanks to leres@ for patiently helping to track this down.

PR:		288345
MFC after:	1 week
Reviewed by:	kib
Differential Revision:	https://reviews.freebsd.org/D55446

1 files changed, 4 insertions, 10 deletions

diff --git a/sys/fs/nullfs/null_vfsops.c b/sys/fs/nullfs/null_vfsops.c
index 170a3dd51cd8..c3eec727aeea 100644
--- a/sys/fs/nullfs/null_vfsops.c
+++ b/sys/fs/nullfs/null_vfsops.c
@@ -461,16 +461,12 @@ nullfs_unlink_lowervp(struct mount *mp, struct vnode *lowervp)
 	vhold(vp);
 	vunref(vp);
 
-	if (vp->v_usecount == 0) {
+	if (VN_IS_DOOMED(vp)) {
 		/*
-		 * If vunref() dropped the last use reference on the
-		 * nullfs vnode, it must be reclaimed, and its lock
-		 * was split from the lower vnode lock.  Need to do
-		 * extra unlock before allowing the final vdrop() to
-		 * free the vnode.
+		 * If the vnode is doomed, its lock was split from the lower
+		 * vnode lock.  Therefore we need to do an extra unlock before
+		 * allowing the final vdrop() to free the vnode.
 		 */
-		KASSERT(VN_IS_DOOMED(vp),
-		    ("not reclaimed nullfs vnode %p", vp));
 		VOP_UNLOCK(vp);
 	} else {
 		/*
@@ -480,8 +476,6 @@ nullfs_unlink_lowervp(struct mount *mp, struct vnode *lowervp)
 		 * relevant for future reclamations.
 		 */
 		ASSERT_VOP_ELOCKED(vp, "unlink_lowervp");
-		KASSERT(!VN_IS_DOOMED(vp),
-		    ("reclaimed nullfs vnode %p", vp));
 		xp->null_flags &= ~NULLV_NOUNLOCK;
 	}
 	vdrop(vp);