re(4): re-arm the Tx doorbell when re_txeof() leaves a non-empty ring

On PCIe parts a TxPoll request can be lost when packets are queued
in quick succession, leaving owned descriptors with no transfer in
progress until the watchdog fires. re_txeof() runs from the interrupt
handlers, re_tick() and re_watchdog(), so re-writing TXSTART whenever
the ring is still non-empty turns a potential 5-second stall into at
most one tick.

One register write on a path that already took an interrupt;
fast path untouched.

* Sustained bidirectional load on RTL8168H; no Tx stalls, no throughput
  regression at 941 Mbps line rate.

Reviewed by:	adrian
Differential Revision:	https://reviews.freebsd.org/D58277
PR: kern/166724

1 files changed, 11 insertions, 0 deletions

diff --git a/sys/dev/re/if_re.c b/sys/dev/re/if_re.c
index cf7ba90c9a5a..4df75474bce3 100644
--- a/sys/dev/re/if_re.c
+++ b/sys/dev/re/if_re.c
@@ -2464,6 +2464,17 @@ re_txeof(struct rl_softc *sc)
 	/* No changes made to the TX ring, so no flush needed */
 
 	if (sc->rl_ldata.rl_tx_free != sc->rl_ldata.rl_tx_desc_cnt) {
+		/*
+		 * Some descriptors are still owned by the controller.  On PCIe
+		 * parts a TxPoll request can be lost when Tx packets are queued
+		 * too close together, leaving a non-empty ring with no transfer
+		 * in progress.  Re-arm the transmitter here -- this routine is
+		 * reached from the interrupt handlers, re_tick() and
+		 * re_watchdog() -- so a lost poll cannot stall the ring until
+		 * the watchdog fires.
+		 */
+		if ((sc->rl_flags & RL_FLAG_PCIE) != 0)
+			CSR_WRITE_1(sc, sc->rl_txstart, RL_TXSTART_START);
 #ifdef RE_TX_MODERATION
 		/*
 		 * If not all descriptors have been reaped yet, reload