ifnet: Fix races in if_vmove_reclaim()

The thread running if_vmove_reclaim() may race with other threads those
running if_detach(), if_vmove_loan() or if_vmove_reclaim(). In case the
current thread loses race, two issues arise,

 1. It is unstable and unsafe to access ifp->if_vnet,
 2. The interface is removed from "active" list, hence if_unlink_ifnet()
    can fail.

For the first case, check against source prison's vnet instead, given
the interface is obtained from that vnet.

For the second one, return ENODEV to indicate the interface was on the
list but the current thread loses race, to distinguish from ENXIO, which
means the interface or child prison is not found. This is the same with
if_vmove_loan().

Reviewed by:	kp, pouria
Fixes:		a779388f8bb3 if: Protect V_ifnet in vnet_if_return()
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D55997

1 files changed, 9 insertions, 5 deletions

diff --git a/sys/net/if.c b/sys/net/if.c
index 41084ecf0516..bdb5671c1afb 100644
--- a/sys/net/if.c
+++ b/sys/net/if.c
@@ -1242,7 +1242,7 @@ if_vmove_reclaim(struct thread *td, char *ifname, int jid)
 	struct prison *pr;
 	struct vnet *vnet_dst;
 	struct ifnet *ifp;
-	int found __diagused;
+	int found;
 
 	/* Try to find the prison within our visibility. */
 	sx_slock(&allprison_lock);
@@ -1255,16 +1255,16 @@ if_vmove_reclaim(struct thread *td, char *ifname, int jid)
 
 	/* Make sure the named iface exists in the source prison/vnet. */
 	CURVNET_SET(pr->pr_vnet);
-	ifp = ifunit(ifname);		/* XXX Lock to avoid races. */
+	ifp = ifunit(ifname);
 	if (ifp == NULL) {
 		CURVNET_RESTORE();
 		prison_free(pr);
 		return (ENXIO);
 	}
 
-	/* Do not try to move the iface from and to the same prison. */
+	/* Do not try to move the iface from and to the same vnet. */
 	vnet_dst = TD_TO_VNET(td);
-	if (vnet_dst == ifp->if_vnet) {
+	if (vnet_dst == pr->pr_vnet) {
 		CURVNET_RESTORE();
 		prison_free(pr);
 		return (EEXIST);
@@ -1272,7 +1272,11 @@ if_vmove_reclaim(struct thread *td, char *ifname, int jid)
 
 	/* Get interface back from child jail/vnet. */
 	found = if_unlink_ifnet(ifp, true);
-	MPASS(found);
+	if (! found) {
+		CURVNET_RESTORE();
+		prison_free(pr);
+		return (ENODEV);
+	}
 	sx_xlock(&ifnet_detach_sxlock);
 	if_vmove(ifp, vnet_dst);
 	sx_xunlock(&ifnet_detach_sxlock);