nvmf_che: NVMe-TCP offload support for Chelsio T7 adaptersHEADmain

This provides an alternative NVMe over TCP transport which uses PDU
offload for TOE connections on a T7.

Similar to iSCSI offload via cxgbei.ko, nvmf_che uses DDP when
possible to enable the NIC to DMA received data directly into I/O data
buffers (pages from a struct bio on the host side, pages from a CTL
I/O requests on the controller side) to avoid copying data on the host
CPU.  nvmf_che is also able to receive a stream of C2H or H2C PDUs for
a single data transfer when using DDP without processing the header of
each PDU.

Unlike cxgbei, nvmf_che aims to be mostly transparent to end users.
Notably, neither nvmecontrol or ctld have to be explicitly asked to
use an offload.  Instead, TCP queue pairs are claimed by this driver
whenever they are eligible (e.g., using TOE).

The main restriction of nvmf_che compared to the software TCP
transport is that Chelsio adapters have a restriction on the largest
PDU that can be sent and received.  When sending data, nvmf_che is
able to split large C2H or H2C data requests across multiple PDUs
without affecting nvmf(4) or nvmft(4).

To avoid overly large PDUs when using nvmf(4), nvmf_che reports a data
transfer limit that is honored by nvmf(4).  This ensures that the
remote controller's PDUs will never be too large (since the command
transfer size is limited to one PDU) and also ensures that nvmf(4)
will never to try to send a command PDU with ICD that is too large.

For nvmft(4), overly large command PDUs due to ICD are avoided by
clamping the size of the reported IOCCSZ in the controller data.
However, to ensure that H2C PDUs are sufficiently small, nvmf_che will
only claim queue pairs which advertised a suitable MAXH2CDATA
parameter during queue negotiation.  For ctld(8), this can be achieved
by setting the MAXH2CDATA option in a transport-group, e.g. for T7:

transport-group tg0 {
	discovery-auth-group no-authentication
	listen tcp 0.0.0.0
	listen tcp [::]
	listen discovery-tcp 0.0.0.0
	listen discovery-tcp [::]
	option MAXH2CDATA 32488
}

Sponsored by:	Chelsio Communications

0 files changed, 0 insertions, 0 deletions