Fix a long-standing deadlock issue with vnode backed md(4) devices:

On vnode backed md(4) devices over a certain, currently undetermined
size relative to the buffer cache our "lemming-syncer" can provoke
a buffer starvation which puts the md thread to sleep on wdrain.

This generally tends to grind the entire system to a stop because the
event that is supposed to wake up the thread will not happen until a fair
bit of the piled up I/O requests in the system finish, and since a lot
of those are on a md(4) vnode backed device which is currently waiting
on wdrain until a fair amount of the piled up ... you get the picture.

The cure is to issue all VOP_WRITES on the vnode backing the device
with IO_SYNC.

In addition to more closely emulating a real disk device with a
non-lying write-cache, this makes the writes exempt from rate-limited
(there to avoid starving the buffer cache) and consequently prevents
the deadlock.

Unfortunately performance takes a hit.

Add "async" option to give people who know what they are doing the
old behaviour.

2 files changed, 8 insertions, 1 deletions

diff --git a/sbin/mdconfig/mdconfig.8 b/sbin/mdconfig/mdconfig.8
index 0b97ddf1c120..4b2f8e246861 100644
--- a/sbin/mdconfig/mdconfig.8
+++ b/sbin/mdconfig/mdconfig.8
@@ -141,6 +141,9 @@ other devices.
 .It Fl o Oo Cm no Oc Ns Ar option
 Set or reset options.
 .Bl -tag -width indent
+.It Oo Cm no Oc Ns Cm async
+For vnode backed devices:  avoid IO_SYNC for increased performance but
+at the risk of deadlocking the entire kernel.
 .It Oo Cm no Oc Ns Cm reserve
 Allocate and reserve all needed storage from the start, rather than as needed.
 .It Oo Cm no Oc Ns Cm cluster
diff --git a/sbin/mdconfig/mdconfig.c b/sbin/mdconfig/mdconfig.c
index 6296da799268..b23e903831ab 100644
--- a/sbin/mdconfig/mdconfig.c
+++ b/sbin/mdconfig/mdconfig.c
@@ -117,7 +117,11 @@ main(int argc, char **argv)
 		case 'o':
 			if (cmdline != 2)
 				usage();
-			if (!strcmp(optarg, "cluster"))
+			if (!strcmp(optarg, "async"))
+				mdio.md_options |= MD_ASYNC;
+			else if (!strcmp(optarg, "noasync"))
+				mdio.md_options &= ~MD_ASYNC;
+			else if (!strcmp(optarg, "cluster"))
 				mdio.md_options |= MD_CLUSTER;
 			else if (!strcmp(optarg, "nocluster"))
 				mdio.md_options &= ~MD_CLUSTER;