Fix the design problem with delayed algorithm sync.

Currently, if the immutable algorithm like bsearch or radix_lockless
 receives rtable update notification, it schedules algorithm rebuild.
This rebuild is executed by the callout after ~50 milliseconds.

It is possible that a script adding an interface address and than route
 with the gateway bound to that address will fail. It can happen due
 to the fact that fib is not updated by the time the route addition
 request arrives.

Fix this by allowing synchronous algorithm rebuilds based on certain
 conditions. By default, these conditions assume:
1) less than net.route.algo.fib_sync_limit=100 routes
2) routes without gateway.

* Move algo instance build entirely under rib WLOCK.
 Rib lock is only used for control plane (except radix algo, but there
  are no rebuilds).
* Add rib_walk_ext_locked() function to allow RIB iteration with
 rib lock already held.
* Fix rare potential callout use-after-free for fds by binding fd
 callout to the relevant rib rmlock. In that case, callout_stop()
 under rib WLOCK guarantees no callout will be executed afterwards.

MFC after:	3 days

1 files changed, 12 insertions, 5 deletions

diff --git a/sys/net/route/route_helpers.c b/sys/net/route/route_helpers.c
index 88733fff419b..782e160ca51b 100644
--- a/sys/net/route/route_helpers.c
+++ b/sys/net/route/route_helpers.c
@@ -67,6 +67,17 @@ __FBSDID("$FreeBSD$");
  * RIB helper functions.
  */
 
+void
+rib_walk_ext_locked(struct rib_head *rnh, rib_walktree_f_t *wa_f,
+    rib_walk_hook_f_t *hook_f, void *arg)
+{
+	if (hook_f != NULL)
+		hook_f(rnh, RIB_WALK_HOOK_PRE, arg);
+	rnh->rnh_walktree(&rnh->head, (walktree_f_t *)wa_f, arg);
+	if (hook_f != NULL)
+		hook_f(rnh, RIB_WALK_HOOK_POST, arg);
+}
+
 /*
  * Calls @wa_f with @arg for each entry in the table specified by
  * @af and @fibnum.
@@ -86,11 +97,7 @@ rib_walk_ext_internal(struct rib_head *rnh, bool wlock, rib_walktree_f_t *wa_f,
 		RIB_WLOCK(rnh);
 	else
 		RIB_RLOCK(rnh);
-	if (hook_f != NULL)
-		hook_f(rnh, RIB_WALK_HOOK_PRE, arg);
-	rnh->rnh_walktree(&rnh->head, (walktree_f_t *)wa_f, arg);
-	if (hook_f != NULL)
-		hook_f(rnh, RIB_WALK_HOOK_POST, arg);
+	rib_walk_ext_locked(rnh, wa_f, hook_f, arg);
 	if (wlock)
 		RIB_WUNLOCK(rnh);
 	else