ena: Put taskqueues into correct domain if !RSS

When compiled without 'options RSS', the ena driver created taskqueues
using taskqueue_start_threads_cpuset passing a mask value of NULL,
both in the ena_setup_tx_resources path (for enqueues) and in the
ena_create_io_queues path (for the completion-processing).

In the default configuration, on most EC2 instances, this results in
taskqueues running in the right NUMA domain, but only by accident; in
non-default configurations (e.g. with with multiple EBS volumes
attached and associated NVMe taskqueues) the taskqueues may land in
the wrong NUMA domain even on instance types where the one-EBS-one-ENA
case produces the desired results.

Set (struct ena_que)->domain and use that to inform the choice of CPU
sets.  On a c8gn.48xlarge EC2 instance this doubles throughput on a
32-TCP-stream benchmark.

Reviewed by:	akiyano
MFC after:	7 days
Sponsored by:	Amazon
Differential Revision:	https://reviews.freebsd.org/D57918

1 files changed, 11 insertions, 1 deletions

diff --git a/sys/dev/ena/ena.c b/sys/dev/ena/ena.c
index 4f45d9103d76..79b9937991c1 100644
--- a/sys/dev/ena/ena.c
+++ b/sys/dev/ena/ena.c
@@ -726,6 +726,8 @@ ena_setup_tx_resources(struct ena_adapter *adapter, int qid)
 	snprintf(thread_name, sizeof(thread_name), "%s txeq %d",
 	    device_get_nameunit(adapter->pdev), que->cpu);
 #else
+	if (que->domain >= 0)
+		cpu_mask = &cpuset_domain[que->domain];
 	snprintf(thread_name, sizeof(thread_name), "%s txeq %d",
 	    device_get_nameunit(adapter->pdev), que->id);
 #endif
@@ -1671,6 +1673,9 @@ ena_create_io_queues(struct ena_adapter *adapter)
 
 #ifdef RSS
 		cpu_mask = &queue->cpu_mask;
+#else
+		if (queue->domain >= 0)
+			cpu_mask = &cpuset_domain[queue->domain];
 #endif
 		taskqueue_start_threads_cpuset(&queue->cleanup_tq, 1, PI_NET,
 		    cpu_mask, "%s queue %d cleanup",
@@ -1815,6 +1820,8 @@ ena_setup_io_intr(struct ena_adapter *adapter)
 	static int last_bind = 0;
 	int cur_bind;
 	int idx;
+#else
+	int domain;
 #endif
 	int irq_idx;
 
@@ -1827,6 +1834,9 @@ ena_setup_io_intr(struct ena_adapter *adapter)
 		last_bind = (last_bind + adapter->num_io_queues) % num_buckets;
 	}
 	cur_bind = adapter->first_bind;
+#else
+	if (bus_get_domain(adapter->pdev, &domain))
+		domain = -1;
 #endif
 
 	for (int i = 0; i < adapter->num_io_queues; i++) {
@@ -1860,7 +1870,7 @@ ena_setup_io_intr(struct ena_adapter *adapter)
 		}
 		adapter->que[i].domain = idx;
 #else
-		adapter->que[i].domain = -1;
+		adapter->que[i].domain = domain;
 #endif /* RSS */
 	}