diff options
author | Alexander V. Chernikov <melifaro@FreeBSD.org> | 2020-12-25 10:39:52 +0000 |
---|---|---|
committer | Alexander V. Chernikov <melifaro@FreeBSD.org> | 2020-12-25 11:33:17 +0000 |
commit | f5baf8bb12f39d0e8d64508c47eb6c4386ef716d (patch) | |
tree | 0d15a6734d10c26415349a57b81d76a4332e2950 /sys/netinet/in_fib.c | |
parent | 760dbe84abfb0b3aa9b72f7474fb9a066bbdd0cf (diff) | |
download | src-f5baf8bb12f39d0e8d64508c47eb6c4386ef716d.tar.gz src-f5baf8bb12f39d0e8d64508c47eb6c4386ef716d.zip |
Add modular fib lookup framework.
This change introduces framework that allows to dynamically
attach or detach longest prefix match (lpm) lookup algorithms
to speed up datapath route tables lookups.
Framework takes care of handling initial synchronisation,
route subscription, nhop/nhop groups reference and indexing,
dataplane attachments and fib instance algorithm setup/teardown.
Framework features automatic algorithm selection, allowing for
picking the best matching algorithm on-the-fly based on the
amount of routes in the routing table.
Currently framework code is guarded under FIB_ALGO config option.
An idea is to enable it by default in the next couple of weeks.
The following algorithms are provided by default:
IPv4:
* bsearch4 (lockless binary search in a special IP array), tailored for
small-fib (<16 routes)
* radix4_lockless (lockless immutable radix, re-created on every rtable change),
tailored for small-fib (<1000 routes)
* radix4 (base system radix backend)
* dpdk_lpm4 (DPDK DIR24-8-based lookups), lockless datastrucure, optimized
for large-fib (D27412)
IPv6:
* radix6_lockless (lockless immutable radix, re-created on every rtable change),
tailed for small-fib (<1000 routes)
* radix6 (base system radix backend)
* dpdk_lpm6 (DPDK DIR24-8-based lookups), lockless datastrucure, optimized
for large-fib (D27412)
Performance changes:
Micro benchmarks (I7-7660U, single-core lookups, 2048k dst, code in D27604):
IPv4:
8 routes:
radix4: ~20mpps
radix4_lockless: ~24.8mpps
bsearch4: ~69mpps
dpdk_lpm4: ~67 mpps
700k routes:
radix4_lockless: 3.3mpps
dpdk_lpm4: 46mpps
IPv6:
8 routes:
radix6_lockless: ~20mpps
dpdk_lpm6: ~70mpps
100k routes:
radix6_lockless: 13.9mpps
dpdk_lpm6: 57mpps
Forwarding benchmarks:
+ 10-15% IPv4 forwarding performance (small-fib, bsearch4)
+ 25% IPv4 forwarding performance (full-view, dpdk_lpm4)
+ 20% IPv6 forwarding performance (full-view, dpdk_lpm6)
Control:
Framwork adds the following runtime sysctls:
List algos
* net.route.algo.inet.algo_list: bsearch4, radix4_lockless, radix4
* net.route.algo.inet6.algo_list: radix6_lockless, radix6, dpdk_lpm6
Debug level (7=LOG_DEBUG, per-route)
net.route.algo.debug_level: 5
Algo selection (currently only for fib 0):
net.route.algo.inet.algo: bsearch4
net.route.algo.inet6.algo: radix6_lockless
Support for manually changing algos in non-default fib will be added
soon. Some sysctl names will be changed in the near future.
Differential Revision: https://reviews.freebsd.org/D27401
Diffstat (limited to 'sys/netinet/in_fib.c')
-rw-r--r-- | sys/netinet/in_fib.c | 37 |
1 files changed, 37 insertions, 0 deletions
diff --git a/sys/netinet/in_fib.c b/sys/netinet/in_fib.c index ee54d524c3e2..8a6e64876b3d 100644 --- a/sys/netinet/in_fib.c +++ b/sys/netinet/in_fib.c @@ -49,6 +49,7 @@ __FBSDID("$FreeBSD$"); #include <net/route.h> #include <net/route/route_ctl.h> #include <net/route/route_var.h> +#include <net/route/fib_algo.h> #include <net/route/nhop.h> #include <net/toeplitz.h> #include <net/vnet.h> @@ -63,6 +64,10 @@ __FBSDID("$FreeBSD$"); /* Assert 'struct route_in' is compatible with 'struct route' */ CHK_STRUCT_ROUTE_COMPAT(struct route_in, ro_dst4); +#ifdef FIB_ALGO +VNET_DEFINE(struct fib_dp *, inet_dp); +#endif + #ifdef ROUTE_MPATH struct _hash_5tuple_ipv4 { struct in_addr src; @@ -103,6 +108,29 @@ fib4_calc_software_hash(struct in_addr src, struct in_addr dst, * one needs to pass NHR_REF as a flag. This will return referenced * nexthop. */ +#ifdef FIB_ALGO +struct nhop_object * +fib4_lookup(uint32_t fibnum, struct in_addr dst, uint32_t scopeid, + uint32_t flags, uint32_t flowid) +{ + struct nhop_object *nh; + struct fib_dp *dp = &V_inet_dp[fibnum]; + struct flm_lookup_key key = {.addr4 = dst }; + + nh = dp->f(dp->arg, key, scopeid); + if (nh != NULL) { + nh = nhop_select(nh, flowid); + /* Ensure route & ifp is UP */ + if (RT_LINK_IS_UP(nh->nh_ifp)) { + if (flags & NHR_REF) + nhop_ref_object(nh); + return (nh); + } + } + RTSTAT_INC(rts_unreach); + return (NULL); +} +#else struct nhop_object * fib4_lookup(uint32_t fibnum, struct in_addr dst, uint32_t scopeid, uint32_t flags, uint32_t flowid) @@ -142,6 +170,7 @@ fib4_lookup(uint32_t fibnum, struct in_addr dst, uint32_t scopeid, RTSTAT_INC(rts_unreach); return (NULL); } +#endif inline static int check_urpf_nhop(const struct nhop_object *nh, uint32_t flags, @@ -180,6 +209,7 @@ check_urpf(struct nhop_object *nh, uint32_t flags, return (check_urpf_nhop(nh, flags, src_if)); } +#ifndef FIB_ALGO static struct nhop_object * lookup_nhop(uint32_t fibnum, struct in_addr dst, uint32_t scopeid) { @@ -208,6 +238,7 @@ lookup_nhop(uint32_t fibnum, struct in_addr dst, uint32_t scopeid) return (nh); } +#endif /* * Performs reverse path forwarding lookup. @@ -223,8 +254,14 @@ fib4_check_urpf(uint32_t fibnum, struct in_addr dst, uint32_t scopeid, uint32_t flags, const struct ifnet *src_if) { struct nhop_object *nh; +#ifdef FIB_ALGO + struct fib_dp *dp = &V_inet_dp[fibnum]; + struct flm_lookup_key key = {.addr4 = dst }; + nh = dp->f(dp->arg, key, scopeid); +#else nh = lookup_nhop(fibnum, dst, scopeid); +#endif if (nh != NULL) return (check_urpf(nh, flags, src_if)); |