arm64: Make local stores observable before sending IPIs

Add a synchronizing instruction to flush and wait until the local
CPU's writes are observable to other CPUs before sending IPIs.

This fixes an issue where recipient CPUs doing a rendezvous could
enter the rendezvous handling code before the initiator's writes
to the smp_rv_* variables were visible. This manifested as a
system hang, where a single CPU's increment of smp_rv_waiters[0]
actually happened "before" the initiator's zeroing of that field,
so all CPUs were stuck with the field appearing to be at
ncpus - 1.

Reviewed by:	andrew, markj
Approved by:	scottl (implicit)
MFC after:	1 week
Sponsored by:	Ampere Computing, Inc.
Differential Revision:	https://reviews.freebsd.org/D25798

1 files changed, 7 insertions, 0 deletions

diff --git a/sys/arm64/arm64/mp_machdep.c b/sys/arm64/arm64/mp_machdep.c
index d3dea3597106..8c8ceafe18e9 100644
--- a/sys/arm64/arm64/mp_machdep.c
+++ b/sys/arm64/arm64/mp_machdep.c
@@ -304,6 +304,13 @@ pic_ipi_send(void *arg, cpuset_t cpus, u_int ipi)
 {
 
 	KASSERT(intr_irq_root_dev != NULL, ("%s: no root attached", __func__));
+
+	/*
+	 * Ensure that this CPU's stores will be visible to IPI
+	 * recipients before starting to send the interrupts.
+	 */
+	dsb(ishst);
+
 	PIC_IPI_SEND(intr_irq_root_dev, arg, cpus, ipi);
 }