dumpon(8): Provide seatbelt against weak RSA keys

The premise of dumpon -k foo.pem is that dump contents will be confidential
except to anyone holding the corresponding RSA private key.

This guarantee breaks down when weak RSA keys are used.  Small RSA keys
(e.g. 512 bits) can be broken on a single personal computer in tractible
time.  Marginal RSA keys (768 bits) can be broken by EC2 and a few dollars.
Even 1024 bit keys can probably be broken by sophisticated and wealthy
attackers.

NIST SP800-57 (2016) recommends a minimum of 2048 bit RSA keys, and
estimates this provides 112 bits of security.

It would also be good to protect users from weak values of 'e' (i.e., 3) and
perhaps sanity check that their public key .pem does not accidentally
contain their private key as well.  These considerations are left as future
work.

Reviewed by:	markj, darius AT dons.net.au (previous version)
Discussed with:	bjk
Differential Revision:	https://reviews.freebsd.org/D17678

2 files changed, 37 insertions, 1 deletions

diff --git a/sbin/dumpon/dumpon.8 b/sbin/dumpon/dumpon.8
index 79824f1b2bb7..937f85de8108 100644
--- a/sbin/dumpon/dumpon.8
+++ b/sbin/dumpon/dumpon.8
@@ -28,7 +28,7 @@
 .\"     From: @(#)swapon.8	8.1 (Berkeley) 6/5/93
 .\" $FreeBSD$
 .\"
-.Dd June 13, 2018
+.Dd October 26, 2018
 .Dt DUMPON 8
 .Os
 .Sh NAME
@@ -348,3 +348,15 @@ is taken, it is not possible to send crash dumps directly to a file.
 It is currently not possible to configure both compression and encryption.
 The encrypted dump format assumes that the kernel dump size is a multiple
 of the cipher block size, which may not be true when the dump is compressed.
+.Sh SECURITY CONSIDERATIONS
+RSA keys smaller than 1024 bits are practical to factor and therefore weak.
+Even 1024 bit keys may not be large enough to ensure privacy for many
+years, so NIST recommends a minimum of 2048 bit RSA keys.
+As a seatbelt,
+.Nm
+prevents users from configuring encrypted kernel dumps with weak RSA keys.
+If you do not care for cryptographic privacy guarantees, just use
+.Nm
+without specifying a
+.Fl k Ar pubkey
+option.
diff --git a/sbin/dumpon/dumpon.c b/sbin/dumpon/dumpon.c
index 998d0d48c256..48b32184a6dc 100644
--- a/sbin/dumpon/dumpon.c
+++ b/sbin/dumpon/dumpon.c
@@ -243,6 +243,30 @@ genkey(const char *pubkeyfile, struct diocskerneldump_arg *kdap)
 	if (pubkey == NULL)
 		errx(1, "Unable to read data from %s.", pubkeyfile);
 
+	/*
+	 * RSA keys under ~1024 bits are trivially factorable (2018).  OpenSSL
+	 * provides an API for RSA keys to estimate the symmetric-cipher
+	 * "equivalent" bits of security (defined in NIST SP800-57), which as
+	 * of this writing equates a 2048-bit RSA key to 112 symmetric cipher
+	 * bits.
+	 *
+	 * Use this API as a seatbelt to avoid suggesting to users that their
+	 * privacy is protected by encryption when the key size is insufficient
+	 * to prevent compromise via factoring.
+	 *
+	 * Future work: Sanity check for weak 'e', and sanity check for absence
+	 * of 'd' (i.e., the supplied key is a public key rather than a full
+	 * keypair).
+	 */
+#if OPENSSL_VERSION_NUMBER >= 0x10100000L
+	if (RSA_security_bits(pubkey) < 112)
+#else
+	if (RSA_size(pubkey) * 8 < 2048)
+#endif
+		errx(1, "Small RSA keys (you provided: %db) can be "
+		    "factored cheaply.  Please generate a larger key.",
+		    RSA_size(pubkey) * 8);
+
 	kdap->kda_encryptedkeysize = RSA_size(pubkey);
 	if (kdap->kda_encryptedkeysize > KERNELDUMP_ENCKEY_MAX_SIZE) {
 		errx(1, "Public key has to be at most %db long.",