aboutsummaryrefslogtreecommitdiff
path: root/sys/opencrypto
Commit message (Collapse)AuthorAgeFilesLines
...
* Add support for optional separate output buffers to in-kernel crypto.John Baldwin2020-05-256-339/+708
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Some crypto consumers such as GELI and KTLS for file-backed sendfile need to store their output in a separate buffer from the input. Currently these consumers copy the contents of the input buffer into the output buffer and queue an in-place crypto operation on the output buffer. Using a separate output buffer avoids this copy. - Create a new 'struct crypto_buffer' describing a crypto buffer containing a type and type-specific fields. crp_ilen is gone, instead buffers that use a flat kernel buffer have a cb_buf_len field for their length. The length of other buffer types is inferred from the backing store (e.g. uio_resid for a uio). Requests now have two such structures: crp_buf for the input buffer, and crp_obuf for the output buffer. - Consumers now use helper functions (crypto_use_*, e.g. crypto_use_mbuf()) to configure the input buffer. If an output buffer is not configured, the request still modifies the input buffer in-place. A consumer uses a second set of helper functions (crypto_use_output_*) to configure an output buffer. - Consumers must request support for separate output buffers when creating a crypto session via the CSP_F_SEPARATE_OUTPUT flag and are only permitted to queue a request with a separate output buffer on sessions with this flag set. Existing drivers already reject sessions with unknown flags, so this permits drivers to be modified to support this extension without requiring all drivers to change. - Several data-related functions now have matching versions that operate on an explicit buffer (e.g. crypto_apply_buf, crypto_contiguous_subsegment_buf, bus_dma_load_crp_buf). - Most of the existing data-related functions operate on the input buffer. However crypto_copyback always writes to the output buffer if a request uses a separate output buffer. - For the regions in input/output buffers, the following conventions are followed: - AAD and IV are always present in input only and their fields are offsets into the input buffer. - payload is always present in both buffers. If a request uses a separate output buffer, it must set a new crp_payload_start_output field to the offset of the payload in the output buffer. - digest is in the input buffer for verify operations, and in the output buffer for compute operations. crp_digest_start is relative to the appropriate buffer. - Add a crypto buffer cursor abstraction. This is a more general form of some bits in the cryptosoft driver that tried to always use uio's. However, compared to the original code, this avoids rewalking the uio iovec array for requests with multiple vectors. It also avoids allocate an iovec array for mbufs and populating it by instead walking the mbuf chain directly. - Update the cryptosoft(4) driver to support separate output buffers making use of the cursor abstraction. Sponsored by: Netflix Differential Revision: https://reviews.freebsd.org/D24545 Notes: svn path=/head/; revision=361481
* Correct the minimum key length for Camellia to 16 bytes (128 bits).John Baldwin2020-05-221-1/+1
| | | | | | | MFC after: 1 week Notes: svn path=/head/; revision=361393
* Improve support for stream ciphers in the software encryption interface.John Baldwin2020-05-223-38/+78
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add a 'native_blocksize' member to 'struct enc_xform' that ciphers can use if they support a partial final block. This is particular useful for stream ciphers, but can also apply to other ciphers. cryptosoft will only pass in native blocks to the encrypt and decrypt hooks. For the final partial block, 'struct enc_xform' now has new encrypt_last/decrypt_last hooks which accept the length of the final block. The multi_block methods are also retired. Mark AES-ICM (AES-CTR) as a stream cipher. This has some interesting effects on IPsec in that FreeBSD can now properly receive all packets sent by Linux when using AES-CTR, but FreeBSD can no longer interoperate with OpenBSD and older verisons of FreeBSD which assume AES-CTR packets have a payload padded to a 16-byte boundary. Kornel has offered to work on a patch to add a compatiblity sysctl to enforce additional padding for AES-CTR in esp_output to permit compatibility with OpenBSD and older versions of FreeBSD. AES-XTS continues to use a block size of a single AES block length. It is possible to adjust it to support partial final blocks by implementing cipher text stealing via encrypt_last/decrypt_last hooks, but I have not done so. Reviewed by: cem (earlier version) Tested by: Kornel Dulęba <mindal@semihalf.com> (AES-CTR with IPsec) Sponsored by: Netflix Differential Revision: https://reviews.freebsd.org/D24906 Notes: svn path=/head/; revision=361390
* Fix libstand build breakage after r361298.John Baldwin2020-05-201-0/+3
| | | | | | | | | | | | | | - Use enc_xform_aes_xts.setkey() directly instead of duplicating the code now that it no longer calls malloc(). - Rather than bringing back all of xform_userland.h, add a conditional #include of <stand.h> to xform_enc.h. - Update calls to encrypt/decrypt callbacks in enc_xform_aes_xts for separate input/output pointers. Pointy hat to: jhb Notes: svn path=/head/; revision=361306
* Various cleanups to the software encryption transform interface.John Baldwin2020-05-2011-310/+173
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Consistently use 'void *' for key schedules / key contexts instead of a mix of 'caddr_t', 'uint8_t *', and 'void *'. - Add a ctxsize member to enc_xform similar to what auth transforms use and require callers to malloc/zfree the context. The setkey callback now supplies the caller-allocated context pointer and the zerokey callback is removed. Callers now always use zfree() to ensure key contexts are zeroed. - Consistently use C99 initializers for all statically-initialized instances of 'struct enc_xform'. - Change the encrypt and decrypt functions to accept separate in and out buffer pointers. Almost all of the backend crypto functions already supported separate input and output buffers and this makes it simpler to support separate buffers in OCF. - Remove xform_userland.h shim to permit transforms to be compiled in userland. Transforms no longer call malloc/free directly. Reviewed by: cem (earlier version) Sponsored by: Netflix Differential Revision: https://reviews.freebsd.org/D24855 Notes: svn path=/head/; revision=361298
* Trim a few more things I missed from xform_enc.h.John Baldwin2020-05-131-2/+0
| | | | | | | | An extern declaration for the now-removed Blowfish encryption transform, and an include of the DES header. Notes: svn path=/head/; revision=361014
* Remove MD5 HMAC from OCF.John Baldwin2020-05-117-98/+2
| | | | | | | | | | | | There are no in-kernel consumers. Reviewed by: cem Relnotes: yes Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D24775 Notes: svn path=/head/; revision=360936
* opencrypto: Add missing ioctl exit SDTsKristof Provost2020-05-111-3/+17
| | | | | | | | | | | The opencrypto ioctl code has very useful probe points at the various exit points. These allow us to figure out exactly why a request failed. However, a few paths did not have these probe points. Add them here. Reviewed by: jhb Notes: svn path=/head/; revision=360935
* Remove support for DES and Triple DES from OCF.John Baldwin2020-05-116-250/+2
| | | | | | | | | | | | | It no longer has any in-kernel consumers via OCF. smbfs still uses single DES directly, so sys/crypto/des remains for that use case. Reviewed by: cem Relnotes: yes Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D24773 Notes: svn path=/head/; revision=360933
* Remove support for the Blowfish algorithm from OCF.John Baldwin2020-05-116-137/+0
| | | | | | | | | | | | It no longer has any in-kernel consumers. Reviewed by: cem Relnotes: yes Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D24772 Notes: svn path=/head/; revision=360931
* Remove support for the ARC4 algorithm from OCF.John Baldwin2020-05-114-26/+4
| | | | | | | | | | | | | There are no longer any in-kernel consumers. The software implementation was also a non-functional stub. Reviewed by: cem Relnotes: yes Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D24771 Notes: svn path=/head/; revision=360928
* Remove support for keyed MD5 and SHA1 authentication hashes.John Baldwin2020-05-116-95/+2
| | | | | | | | | | | | | They no longer have any in-tree consumers. Note that these are a different from MD5-HMAC and SHA1-HMAC and were only used with IPsec. Reviewed by: cem Relnotes: yes Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D24770 Notes: svn path=/head/; revision=360927
* Remove support for the skipjack encryption algorithm.John Baldwin2020-05-118-412/+0
| | | | | | | | | | | | | This was removed from IPsec in r286100 and no longer has any in-tree consumers. Reviewed by: cem Relnotes: yes Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D24769 Notes: svn path=/head/; revision=360924
* Remove support for the cast128 encryption algorithm.John Baldwin2020-05-119-930/+0
| | | | | | | | | | | | It no longer has any in-tree consumers. Reviewed by: cem Relnotes: yes Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D24768 Notes: svn path=/head/; revision=360923
* Remove incomplete support for plain MD5 from OCF.John Baldwin2020-05-113-13/+0
| | | | | | | | | | | | | | Although a few drivers supported this algorithm, there were never any in-kernel consumers. cryptosoft and cryptodev never supported it, and there was not a software xform auth_hash for it. Reviewed by: cem Relnotes: yes Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D24767 Notes: svn path=/head/; revision=360920
* Use a lookup table of algorithm types for alg_is_* helpers.John Baldwin2020-05-041-51/+62
| | | | | | | | | | | | This makes it easier to maintain these functions as algorithms are added or removed. Reviewed by: cem Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D24668 Notes: svn path=/head/; revision=360642
* Actually remove support for Triple DES, not just the warning.John Baldwin2020-05-021-3/+0
| | | | | | | Missed in r360562. Notes: svn path=/head/; revision=360565
* Remove support for the algorithms deprecated in r348876.John Baldwin2020-05-021-68/+0
| | | | | | | | | | | | | | | | | | | | | | This removes support for the following algorithms: - ARC4 - Blowfish - CAST128 - DES - 3DES - MD5-HMAC - Skipjack Since /dev/crypto no longer supports 3DES, stop testing the 3DES KAT vectors in cryptotest.py. Reviewed by: cem (previous version) Relnotes: yes Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D24346 Notes: svn path=/head/; revision=360562
* Retire the CRYPTO_F_IV_GENERATE flag.John Baldwin2020-04-203-36/+17
| | | | | | | | | | | | | | The sole in-tree user of this flag has been retired, so remove this complexity from all drivers. While here, add a helper routine drivers can use to read the current request's IV into a local buffer. Use this routine to replace duplicated code in nearly all drivers. Reviewed by: cem Sponsored by: Netflix Differential Revision: https://reviews.freebsd.org/D24450 Notes: svn path=/head/; revision=360136
* Avoid checking pointers that are never NULL.John Baldwin2020-04-021-8/+4
| | | | | | | | | | | | | Coverity noted that cod pointer is always non-NULL at the end of cryptodev_aead(). While here, fix cryptodev_op() to match by making one earlier failure case before cod and crp are allocated just return directly. CID: 1422185 Reported by: Coverity Notes: svn path=/head/; revision=359570
* Remove duplicate conditional.John Baldwin2020-04-021-2/+0
| | | | | | | | CID: 1422192 Reported by: Coverity Notes: svn path=/head/; revision=359569
* Refactor driver and consumer interfaces for OCF (in-kernel crypto).John Baldwin2020-03-279-1701/+2395
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - The linked list of cryptoini structures used in session initialization is replaced with a new flat structure: struct crypto_session_params. This session includes a new mode to define how the other fields should be interpreted. Available modes include: - COMPRESS (for compression/decompression) - CIPHER (for simply encryption/decryption) - DIGEST (computing and verifying digests) - AEAD (combined auth and encryption such as AES-GCM and AES-CCM) - ETA (combined auth and encryption using encrypt-then-authenticate) Additional modes could be added in the future (e.g. if we wanted to support TLS MtE for AES-CBC in the kernel we could add a new mode for that. TLS modes might also affect how AAD is interpreted, etc.) The flat structure also includes the key lengths and algorithms as before. However, code doesn't have to walk the linked list and switch on the algorithm to determine which key is the auth key vs encryption key. The 'csp_auth_*' fields are always used for auth keys and settings and 'csp_cipher_*' for cipher. (Compression algorithms are stored in csp_cipher_alg.) - Drivers no longer register a list of supported algorithms. This doesn't quite work when you factor in modes (e.g. a driver might support both AES-CBC and SHA2-256-HMAC separately but not combined for ETA). Instead, a new 'crypto_probesession' method has been added to the kobj interface for symmteric crypto drivers. This method returns a negative value on success (similar to how device_probe works) and the crypto framework uses this value to pick the "best" driver. There are three constants for hardware (e.g. ccr), accelerated software (e.g. aesni), and plain software (cryptosoft) that give preference in that order. One effect of this is that if you request only hardware when creating a new session, you will no longer get a session using accelerated software. Another effect is that the default setting to disallow software crypto via /dev/crypto now disables accelerated software. Once a driver is chosen, 'crypto_newsession' is invoked as before. - Crypto operations are now solely described by the flat 'cryptop' structure. The linked list of descriptors has been removed. A separate enum has been added to describe the type of data buffer in use instead of using CRYPTO_F_* flags to make it easier to add more types in the future if needed (e.g. wired userspace buffers for zero-copy). It will also make it easier to re-introduce separate input and output buffers (in-kernel TLS would benefit from this). Try to make the flags related to IV handling less insane: - CRYPTO_F_IV_SEPARATE means that the IV is stored in the 'crp_iv' member of the operation structure. If this flag is not set, the IV is stored in the data buffer at the 'crp_iv_start' offset. - CRYPTO_F_IV_GENERATE means that a random IV should be generated and stored into the data buffer. This cannot be used with CRYPTO_F_IV_SEPARATE. If a consumer wants to deal with explicit vs implicit IVs, etc. it can always generate the IV however it needs and store partial IVs in the buffer and the full IV/nonce in crp_iv and set CRYPTO_F_IV_SEPARATE. The layout of the buffer is now described via fields in cryptop. crp_aad_start and crp_aad_length define the boundaries of any AAD. Previously with GCM and CCM you defined an auth crd with this range, but for ETA your auth crd had to span both the AAD and plaintext (and they had to be adjacent). crp_payload_start and crp_payload_length define the boundaries of the plaintext/ciphertext. Modes that only do a single operation (COMPRESS, CIPHER, DIGEST) should only use this region and leave the AAD region empty. If a digest is present (or should be generated), it's starting location is marked by crp_digest_start. Instead of using the CRD_F_ENCRYPT flag to determine the direction of the operation, cryptop now includes an 'op' field defining the operation to perform. For digests I've added a new VERIFY digest mode which assumes a digest is present in the input and fails the request with EBADMSG if it doesn't match the internally-computed digest. GCM and CCM already assumed this, and the new AEAD mode requires this for decryption. The new ETA mode now also requires this for decryption, so IPsec and GELI no longer do their own authentication verification. Simple DIGEST operations can also do this, though there are no in-tree consumers. To eventually support some refcounting to close races, the session cookie is now passed to crypto_getop() and clients should no longer set crp_sesssion directly. - Assymteric crypto operation structures should be allocated via crypto_getkreq() and freed via crypto_freekreq(). This permits the crypto layer to track open asym requests and close races with a driver trying to unregister while asym requests are in flight. - crypto_copyback, crypto_copydata, crypto_apply, and crypto_contiguous_subsegment now accept the 'crp' object as the first parameter instead of individual members. This makes it easier to deal with different buffer types in the future as well as separate input and output buffers. It's also simpler for driver writers to use. - bus_dmamap_load_crp() loads a DMA mapping for a crypto buffer. This understands the various types of buffers so that drivers that use DMA do not have to be aware of different buffer types. - Helper routines now exist to build an auth context for HMAC IPAD and OPAD. This reduces some duplicated work among drivers. - Key buffers are now treated as const throughout the framework and in device drivers. However, session key buffers provided when a session is created are expected to remain alive for the duration of the session. - GCM and CCM sessions now only specify a cipher algorithm and a cipher key. The redundant auth information is not needed or used. - For cryptosoft, split up the code a bit such that the 'process' callback now invokes a function pointer in the session. This function pointer is set based on the mode (in effect) though it simplifies a few edge cases that would otherwise be in the switch in 'process'. It does split up GCM vs CCM which I think is more readable even if there is some duplication. - I changed /dev/crypto to support GMAC requests using CRYPTO_AES_NIST_GMAC as an auth algorithm and updated cryptocheck to work with it. - Combined cipher and auth sessions via /dev/crypto now always use ETA mode. The COP_F_CIPHER_FIRST flag is now a no-op that is ignored. This was actually documented as being true in crypto(4) before, but the code had not implemented this before I added the CIPHER_FIRST flag. - I have not yet updated /dev/crypto to be aware of explicit modes for sessions. I will probably do that at some point in the future as well as teach it about IV/nonce and tag lengths for AEAD so we can support all of the NIST KAT tests for GCM and CCM. - I've split up the exising crypto.9 manpage into several pages of which many are written from scratch. - I have converted all drivers and consumers in the tree and verified that they compile, but I have not tested all of them. I have tested the following drivers: - cryptosoft - aesni (AES only) - blake2 - ccr and the following consumers: - cryptodev - IPsec - ktls_ocf - GELI (lightly) I have not tested the following: - ccp - aesni with sha - hifn - kgssapi_krb5 - ubsec - padlock - safe - armv8_crypto (aarch64) - glxsb (i386) - sec (ppc) - cesa (armv7) - cryptocteon (mips64) - nlmsec (mips64) Discussed with: cem Relnotes: yes Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D23677 Notes: svn path=/head/; revision=359374
* Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many)Pawel Biernacki2020-02-261-1/+2
| | | | | | | | | | | | | | | | | | | r357614 added CTLFLAG_NEEDGIANT to make it easier to find nodes that are still not MPSAFE (or already are but aren’t properly marked). Use it in preparation for a general review of all nodes. This is non-functional change that adds annotations to SYSCTL_NODE and SYSCTL_PROC nodes using one of the soon-to-be-required flags. Mark all obvious cases as MPSAFE. All entries that haven't been marked as MPSAFE before are by default marked as NEEDGIANT Approved by: kib (mentor, blanket) Commented by: kib, gallatin, melifaro Differential Revision: https://reviews.freebsd.org/D23718 Notes: svn path=/head/; revision=358333
* Miscellaneous typo fixesEd Maste2020-02-071-1/+1
| | | | | | | | Submitted by: Gordon Bergling <gbergling_gmail.com> Differential Revision: https://reviews.freebsd.org/D23453 Notes: svn path=/head/; revision=357664
* Add stricter checking on mac key lengths.John Baldwin2020-01-091-2/+2
| | | | | | | | | | | | | | | Negative lengths are always invalid. The key length should also be zero for hash algorithms that do not accept a key. admbugs: 949 Reported by: Yuval Kanarenstein <yuvalk@ssd-disclosure.com> Reviewed by: cem MFC after: 1 week Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D23094 Notes: svn path=/head/; revision=356561
* Remove no-longer-used function prototype.John Baldwin2020-01-081-1/+0
| | | | | | | Reported by: amd64-gcc Notes: svn path=/head/; revision=356520
* Remove unneeded cdevsw methods and D_NEEDGIANT.John Baldwin2020-01-081-22/+0
| | | | | | | | | | Reviewed by: kib MFC after: 2 weeks Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D23079 Notes: svn path=/head/; revision=356509
* Use falloc_noinstall + finstall for crypto file descriptors.John Baldwin2020-01-081-7/+8
| | | | | | | | | | Reviewed by: cem, kib MFC after: 1 week Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D23078 Notes: svn path=/head/; revision=356508
* Add a reference count to cryptodev sessions.John Baldwin2020-01-081-24/+37
| | | | | | | | | | | | | | | | This prevents use-after-free races with crypto requests (which may sleep) and CIOCFSESSION as well as races from current CIOCFSESSION requests. admbugs: 949 Reported by: Yuval Kanarenstein <yuvalk@ssd-disclosure.com> Reviewed by: cem MFC after: 1 week Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D23077 Notes: svn path=/head/; revision=356507
* Add support for TLS 1.3 using AES-GCM to the OCF backend for KTLS.John Baldwin2019-12-181-18/+142
| | | | | | | | | Reviewed by: gallatin Sponsored by: Netflix Differential Revision: https://reviews.freebsd.org/D22802 Notes: svn path=/head/; revision=355873
* kTLS support for TLS 1.3Andrew Gallatin2019-09-271-1/+1
| | | | | | | | | | | | | | | | TLS 1.3 requires a few changes because 1.3 pretends to be 1.2 with a record type of application data. The "real" record type is then included at the end of the user-supplied plaintext data. This required adding a field to the mbuf_ext_pgs struct to save the record type, and passing the real record type to the sw_encrypt() ktls backend functions. Reviewed by: jhb, hselasky Sponsored by: Netflix Differential Revision: D21801 Notes: svn path=/head/; revision=352814
* Adjust the deprecated warnings for /dev/crypto to be less noisy.John Baldwin2019-08-271-17/+43
| | | | | | | | | | | | | | Warn when actual operations are performed instead of when sessions are created. The /dev/crypto engine in OpenSSL 1.0.x tries to create sessions for all possible algorithms each time it is initialized resulting in spurious warnings. Reported by: Mike Tancsa MFC after: 3 days Sponsored by: Chelsio Communications Notes: svn path=/head/; revision=351557
* Add kernel-side support for in-kernel TLS.John Baldwin2019-08-271-0/+308
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | KTLS adds support for in-kernel framing and encryption of Transport Layer Security (1.0-1.2) data on TCP sockets. KTLS only supports offload of TLS for transmitted data. Key negotation must still be performed in userland. Once completed, transmit session keys for a connection are provided to the kernel via a new TCP_TXTLS_ENABLE socket option. All subsequent data transmitted on the socket is placed into TLS frames and encrypted using the supplied keys. Any data written to a KTLS-enabled socket via write(2), aio_write(2), or sendfile(2) is assumed to be application data and is encoded in TLS frames with an application data type. Individual records can be sent with a custom type (e.g. handshake messages) via sendmsg(2) with a new control message (TLS_SET_RECORD_TYPE) specifying the record type. At present, rekeying is not supported though the in-kernel framework should support rekeying. KTLS makes use of the recently added unmapped mbufs to store TLS frames in the socket buffer. Each TLS frame is described by a single ext_pgs mbuf. The ext_pgs structure contains the header of the TLS record (and trailer for encrypted records) as well as references to the associated TLS session. KTLS supports two primary methods of encrypting TLS frames: software TLS and ifnet TLS. Software TLS marks mbufs holding socket data as not ready via M_NOTREADY similar to sendfile(2) when TLS framing information is added to an unmapped mbuf in ktls_frame(). ktls_enqueue() is then called to schedule TLS frames for encryption. In the case of sendfile_iodone() calls ktls_enqueue() instead of pru_ready() leaving the mbufs marked M_NOTREADY until encryption is completed. For other writes (vn_sendfile when pages are available, write(2), etc.), the PRUS_NOTREADY is set when invoking pru_send() along with invoking ktls_enqueue(). A pool of worker threads (the "KTLS" kernel process) encrypts TLS frames queued via ktls_enqueue(). Each TLS frame is temporarily mapped using the direct map and passed to a software encryption backend to perform the actual encryption. (Note: The use of PHYS_TO_DMAP could be replaced with sf_bufs if someone wished to make this work on architectures without a direct map.) KTLS supports pluggable software encryption backends. Internally, Netflix uses proprietary pure-software backends. This commit includes a simple backend in a new ktls_ocf.ko module that uses the kernel's OpenCrypto framework to provide AES-GCM encryption of TLS frames. As a result, software TLS is now a bit of a misnomer as it can make use of hardware crypto accelerators. Once software encryption has finished, the TLS frame mbufs are marked ready via pru_ready(). At this point, the encrypted data appears as regular payload to the TCP stack stored in unmapped mbufs. ifnet TLS permits a NIC to offload the TLS encryption and TCP segmentation. In this mode, a new send tag type (IF_SND_TAG_TYPE_TLS) is allocated on the interface a socket is routed over and associated with a TLS session. TLS records for a TLS session using ifnet TLS are not marked M_NOTREADY but are passed down the stack unencrypted. The ip_output_send() and ip6_output_send() helper functions that apply send tags to outbound IP packets verify that the send tag of the TLS record matches the outbound interface. If so, the packet is tagged with the TLS send tag and sent to the interface. The NIC device driver must recognize packets with the TLS send tag and schedule them for TLS encryption and TCP segmentation. If the the outbound interface does not match the interface in the TLS send tag, the packet is dropped. In addition, a task is scheduled to refresh the TLS send tag for the TLS session. If a new TLS send tag cannot be allocated, the connection is dropped. If a new TLS send tag is allocated, however, subsequent packets will be tagged with the correct TLS send tag. (This latter case has been tested by configuring both ports of a Chelsio T6 in a lagg and failing over from one port to another. As the connections migrated to the new port, new TLS send tags were allocated for the new port and connections resumed without being dropped.) ifnet TLS can be enabled and disabled on supported network interfaces via new '[-]txtls[46]' options to ifconfig(8). ifnet TLS is supported across both vlan devices and lagg interfaces using failover, lacp with flowid enabled, or lacp with flowid enabled. Applications may request the current KTLS mode of a connection via a new TCP_TXTLS_MODE socket option. They can also use this socket option to toggle between software and ifnet TLS modes. In addition, a testing tool is available in tools/tools/switch_tls. This is modeled on tcpdrop and uses similar syntax. However, instead of dropping connections, -s is used to force KTLS connections to switch to software TLS and -i is used to switch to ifnet TLS. Various sysctls and counters are available under the kern.ipc.tls sysctl node. The kern.ipc.tls.enable node must be set to true to enable KTLS (it is off by default). The use of unmapped mbufs must also be enabled via kern.ipc.mb_use_ext_pgs to enable KTLS. KTLS is enabled via the KERN_TLS kernel option. This patch is the culmination of years of work by several folks including Scott Long and Randall Stewart for the original design and implementation; Drew Gallatin for several optimizations including the use of ext_pgs mbufs, the M_NOTREADY mechanism for TLS records awaiting software encryption, and pluggable software crypto backends; and John Baldwin for modifications to support hardware TLS offload. Reviewed by: gallatin, hselasky, rrs Obtained from: Netflix Sponsored by: Netflix, Chelsio Communications Differential Revision: https://reviews.freebsd.org/D21277 Notes: svn path=/head/; revision=351522
* Stop using des_cblock * for arguments to DES functions.John Baldwin2019-08-262-12/+8
| | | | | | | | | | | | | | This amounts to a char ** since it is a char[8] *. Evil casts mostly resolved the fact that what was actually passed in were plain char *. Instead, change the DES functions to use 'unsigned char *' for keys and for input and output buffers. Reviewed by: cem, imp Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D21389 Notes: svn path=/head/; revision=351513
* Use 'const' for keys and IVs passed to software encryption algorithms.John Baldwin2019-08-2215-41/+41
| | | | | | | | | | | | Specifically, use 'const' for the key passed to the 'setkey' method and 'const' for the 'iv' passed to the 'reinit' method. Reviewed by: cem Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D21347 Notes: svn path=/head/; revision=351364
* Allow Kernel to link in both legacy libkern/zlib and new sys/contrib/zlib,Xin LI2019-08-012-28/+22
| | | | | | | | | | | | | | | | | | | | | | | | with an eventual goal to convert all legacl zlib callers to the new zlib version: * Move generic zlib shims that are not specific to zlib 1.0.4 to sys/dev/zlib. * Connect new zlib (1.2.11) to the zlib kernel module, currently built with Z_SOLO. * Prefix the legacy zlib (1.0.4) with 'zlib104_' namespace. * Convert sys/opencrypto/cryptodeflate.c to use new zlib. * Remove bundled zlib 1.2.3 from ZFS and adapt it to new zlib and make it depend on the zlib module. * Fix Z_SOLO build of new zlib. PR: 229763 Submitted by: Yoshihiro Ota <ota j email ne jp> Reviewed by: markm (sys/dev/zlib/zlib_kmod.c) Relnotes: yes Differential Revision: https://reviews.freebsd.org/D19706 Notes: svn path=/head/; revision=350496
* Move declaration of warninterval out from under COMPAT_FREEBSD32.John Baldwin2019-06-111-5/+5
| | | | | | | | | | This fixes builds of kernels without COMPAT_FREEBSD32. Reported by: tinderbox MFC after: 1 month Notes: svn path=/head/; revision=348974
* Make the warning intervals for deprecated crypto algorithms tunable.John Baldwin2019-06-111-1/+5
| | | | | | | | | | | | | | | | | New sysctl/tunables can now set the interval (in seconds) between rate-limited crypto warnings. The new sysctls are: - kern.cryptodev_warn_interval for /dev/crypto - net.inet.ipsec.crypto_warn_interval for IPsec - kern.kgssapi_warn_interval for KGSSAPI Reviewed by: cem MFC after: 1 month Relnotes: yes Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D20555 Notes: svn path=/head/; revision=348970
* Add warnings to /dev/crypto for deprecated algorithms.John Baldwin2019-06-101-0/+18
| | | | | | | | | | | | | | | | | | | | | These algorithms are deprecated algorithms that will have no in-kernel consumers in FreeBSD 13. Specifically, deprecate the following algorithms: - ARC4 - Blowfish - CAST128 - DES - 3DES - MD5-HMAC - Skipjack MFC after: 1 month Relnotes: yes Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D20554 Notes: svn path=/head/; revision=348876
* cryptodeflate: Drop z_stream zbuf.state->dummy from SDT probe.Xin LI2019-05-241-35/+18
| | | | | | | | | | | | | | For older versions of zlib, dummy was a workaround for compilers that do not handle opaque type definition well; on FreeBSD, it's representing a value that is not really useful for monitoring purposes, and the field would be gone in newer zlib versions. PR: 229763 Submitted by: Yoshihiro Ota <ota at j.email.ne.jp> Differential Revision: https://reviews.freebsd.org/D20222 Notes: svn path=/head/; revision=348222
* Don't panic for empty CCM requests.John Baldwin2019-04-241-3/+0
| | | | | | | | | | | | | | A request to encrypt an empty payload without any AAD is unusual, but it is defined behavior. Removing this assertion removes a panic and instead returns the correct tag for an empty buffer. Reviewed by: cem, sef MFC after: 2 weeks Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D20043 Notes: svn path=/head/; revision=346649
* Fix another bug introduced during the review process of r344140:Sean Eric Fagan2019-02-251-6/+22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | the tag wasn't being computed properly due to chaning a >= comparison to an == comparison. Specifically: CBC-MAC encodes the length of the authorization data into the the stream to be encrypted/hashed. For short data, this is two bytes (big-endian 16 bit value); for larger data, it's 6 bytes (a prefix of 0xff, 0xfe, followed by a 32-bit big-endian length). And there's a larger size, which is 10 bytes. These extra bytes weren't being accounted for with the post-review code. The other bit that then came into play was that OCF only calls the Update code with blksiz=16, which meant that I had to ignore the length variable. (It also means that it can't be called with a single buffer containing the AAD and payload; however, OCF doesn't do this for the software-only algorithsm.) I tested with this script: ALG=aes-ccm DEV=soft for aad in 0 1 2 3 4 14 16 24 30 32 34 36 1020 do for dln in 16 32 1024 2048 10240 do echo "Testing AAD length ${aad} data length ${dln}" /root/cryptocheck -A ${aad} -a ${ALG} -d ${DEV} ${dln} done done Reviewed by: cem Sponsored by: iXsystems Inc. Notes: svn path=/head/; revision=344547
* Fix another issue from r344141, having to do with size of a shift amount.Sean Eric Fagan2019-02-151-1/+1
| | | | | | | | | This did not show up in my testing. Differential Revision: https://reviews.freebsd.org/D18592 Notes: svn path=/head/; revision=344143
* Pasting in a source control line missed the last quote. Fixed.Sean Eric Fagan2019-02-151-1/+1
| | | | Notes: svn path=/head/; revision=344142
* Add AES-CCM encryption, and plumb into OCF.Sean Eric Fagan2019-02-156-6/+145
| | | | | | | | | | | | | | | | | | | | | | This commit essentially has three parts: * Add the AES-CCM encryption hooks. This is in and of itself fairly small, as there is only a small difference between CCM and the other ICM-based algorithms. * Hook the code into the OpenCrypto framework. This is the bulk of the changes, as the algorithm type has to be checked for, and the differences between it and GCM dealt with. * Update the cryptocheck tool to be aware of it. This is invaluable for confirming that the code works. This is a software-only implementation, meaning that the performance is very low. Sponsored by: iXsystems Inc. Differential Revision: https://reviews.freebsd.org/D19090 Notes: svn path=/head/; revision=344141
* Add CBC-MAC authentication.Sean Eric Fagan2019-02-154-1/+380
| | | | | | | | | | | | | | | | This adds the CBC-MAC code to the kernel, but does not hook it up to anything (that comes in the next commit). https://tools.ietf.org/html/rfc3610 describes the algorithm. Note that this is a software-only implementation, which means it is fairly slow. Sponsored by: iXsystems Inc Differential Revision: https://reviews.freebsd.org/D18592 Notes: svn path=/head/; revision=344140
* As struct cryptop is wrapped in #ifdef _KERNEL, userland doesn'tMarius Strobl2019-02-101-1/+1
| | | | | | | need to drag in <sys/_task.h> either. Notes: svn path=/head/; revision=343979
* Plug memory leak for AES_*_NIST_GMAC algorithms.Andrey V. Elsukov2018-12-131-0/+3
| | | | | | | | | | | swcr_newsession() allocates sw_ictx for these algorithms, thus we need to free() it in swcr_freesession(). PR: 233907 MFC after: 1 week Notes: svn path=/head/; revision=342030
* Generalize AES iov optimizationMatt Macy2018-12-132-0/+56
| | | | | | | | | | | | | | | | | | | | | | | | | Right now, aesni_cipher_alloc does a bit of special-casing for CRYPTO_F_IOV, to not do any allocation if the first uio is large enough for the requested size. While working on ZFS crypto port, I ran into horrible performance because the code uses scatter-gather, and many of the times the data to encrypt was in the second entry. This code looks through the list, and tries to see if there is a single uio that can contain the requested data, and, if so, uses that. This has a slight impact on the current consumers, in that the check is a little more complicated for the ones that use CRYPTO_F_IOV -- but none of them meet the criteria for testing more than one. Submitted by: sef at ixsystems.com Reviewed by: cem@ MFC after: 3 days Sponsored by: iX Systems Differential Revision: https://reviews.freebsd.org/D18522 Notes: svn path=/head/; revision=342024
* Add sha224 to the authctx union.John Baldwin2018-10-231-0/+2
| | | | | | | | MFC after: 2 months Sponsored by: Chelsio Communications Notes: svn path=/head/; revision=339665
generated by cgit v1.2.3 (git 2.25.1) at 2024-09-25 15:00:00 +0000