panic: add a switch and infrastructure for stopping other CPUs in SMP case

Historical behavior of letting other CPUs merily go on is a default for
time being.  The new behavior can be switched on via
kern.stop_scheduler_on_panic tunable and sysctl.

Stopping of the CPUs has (at least) the following benefits:
- more of the system state at panic time is preserved intact
- threads and interrupts do not interfere with dumping of the system
  state

Only one thread runs uninterrupted after panic if stop_scheduler_on_panic
is set.  That thread might call code that is also used in normal context
and that code might use locks to prevent concurrent execution of certain
parts.  Those locks might be held by the stopped threads and would never
be released.  To work around this issue, it was decided that instead of
explicit checks for panic context, we would rather put those checks
inside the locking primitives.

This change has substantial portions written and re-written by attilio
and kib at various times.  Other changes are heavily based on the ideas
and patches submitted by jhb and mdf.  bde has provided many insights
into the details and history of the current code.

The new behavior may cause problems for systems that use a USB keyboard
for interfacing with system console.  This is because of some unusual
locking patterns in the ukbd code which have to be used because on one
hand ukbd is below syscons, but on the other hand it has to interface
with other usb code that uses regular mutexes/Giant for its concurrency
protection.  Dumping to USB-connected disks may also be affected.

PR:			amd64/139614 (at least)
In cooperation with:	attilio, jhb, kib, mdf
Discussed with:		arch@, bde
Tested by:		Eugene Grosbein <eugen@grosbein.net>,
			gnn,
			Steven Hartland <killing@multiplay.co.uk>,
			glebius,
			Andrew Boyer <aboyer@averesystems.com>
			(various versions of the patch)
MFC after:		3 months (or never)

1 files changed, 13 insertions, 6 deletions

diff --git a/sys/sys/lockstat.h b/sys/sys/lockstat.h
index ed9cffa92b62..bdfb475cfc0e 100644
--- a/sys/sys/lockstat.h
+++ b/sys/sys/lockstat.h
@@ -185,17 +185,24 @@ extern uint64_t lockstat_nsecs(void);
 #define	LOCKSTAT_PROFILE_OBTAIN_LOCK_SUCCESS(probe, lp, c, wt, f, l)  do {   \
 	uint32_t id;							     \
 									     \
-    	lock_profile_obtain_lock_success(&(lp)->lock_object, c, wt, f, l);   \
-	if ((id = lockstat_probemap[(probe)])) 			     	     \
-		(*lockstat_probe_func)(id, (uintptr_t)(lp), 0, 0, 0, 0);     \
+	if (!SCHEDULER_STOPPED()) {					     \
+		lock_profile_obtain_lock_success(&(lp)->lock_object, c, wt,  \
+		    f, l);						     \
+		if ((id = lockstat_probemap[(probe)]))			     \
+			(*lockstat_probe_func)(id, (uintptr_t)(lp), 0, 0,    \
+			    0, 0);					     \
+	}								     \
 } while (0)
 
 #define	LOCKSTAT_PROFILE_RELEASE_LOCK(probe, lp)  do {			     \
 	uint32_t id;							     \
 									     \
-	lock_profile_release_lock(&(lp)->lock_object);			     \
-	if ((id = lockstat_probemap[(probe)])) 			     	     \
-		(*lockstat_probe_func)(id, (uintptr_t)(lp), 0, 0, 0, 0);     \
+	if (!SCHEDULER_STOPPED()) {					     \
+		lock_profile_release_lock(&(lp)->lock_object);		     \
+		if ((id = lockstat_probemap[(probe)])) 		     	     \
+			(*lockstat_probe_func)(id, (uintptr_t)(lp), 0, 0,    \
+			    0, 0);					     \
+	}								     \
 } while (0)
 
 #else	/* !KDTRACE_HOOKS */