x86: Add stub for Extended Destination ID support

Without an IOMMU, the APIC standard only allows 8 bits of Destination
ID for MSI messages, limiting us to 256 CPUs.  While IOMMUs can allow
for more than 256 CPUs to be supported, they are not necessarily
desirable in virtualized environments.

The Extended Destination ID standard authored by David Woodhouse uses
7 "Reserved" bits for the high bits of a 15-bit Extended Destination
ID in order to address this: http://david.woodhou.se/ExtDestId.pdf

Add a loader tunable machdep.apic_ext_dest_id to control the use of
this feature; the default value (-1) means "autodetect" while 0 and
1 mean disabled and enabled respectively.

Code to detect host support in Xen, Hyper-V, KVM, and Bhyve will come
in future commits, as will the code to use this setting in msi_map and
ioapic_program_intpin.

Tested on:	EC2 r8i.96xlarge
MFC after:	3 weeks
Sponsored by:	Amazon
Differential Revision:	https://reviews.freebsd.org/D55890

2 files changed, 16 insertions, 0 deletions

diff --git a/sys/x86/include/apicvar.h b/sys/x86/include/apicvar.h
index 551f5527ac00..c2f4414ec8db 100644
--- a/sys/x86/include/apicvar.h
+++ b/sys/x86/include/apicvar.h
@@ -265,6 +265,7 @@ device_t ioapic_get_dev(u_int apic_id);
 
 extern int x2apic_mode;
 extern int lapic_eoi_suppression;
+extern int apic_ext_dest_id;
 
 #ifdef _SYS_SYSCTL_H_
 SYSCTL_DECL(_hw_apic);
diff --git a/sys/x86/x86/local_apic.c b/sys/x86/x86/local_apic.c
index 888fd266d364..21934b41ea62 100644
--- a/sys/x86/x86/local_apic.c
+++ b/sys/x86/x86/local_apic.c
@@ -2086,6 +2086,17 @@ apic_setup_local(void *dummy __unused)
 }
 SYSINIT(apic_setup_local, SI_SUB_CPU, SI_ORDER_SECOND, apic_setup_local, NULL);
 
+/* Are we in a VM which supports the Extended Destination ID standard? */
+int apic_ext_dest_id = -1;
+SYSCTL_INT(_machdep, OID_AUTO, apic_ext_dest_id, CTLFLAG_RDTUN, &apic_ext_dest_id, 0,
+    "Use APIC Extended Destination IDs");
+
+/* Detect support for Extended Destination IDs. */
+static void
+detect_extended_dest_id(void)
+{
+}
+
 /*
  * Setup the I/O APICs.
  */
@@ -2097,6 +2108,10 @@ apic_setup_io(void *dummy __unused)
 	if (best_enum == NULL)
 		return;
 
+	/* Check hypervisor support for extended destination IDs. */
+	if (apic_ext_dest_id == -1)
+		detect_extended_dest_id();
+
 	/*
 	 * Local APIC must be registered before other PICs and pseudo PICs
 	 * for proper suspend/resume order.