x86/xen: fix accounted interrupt time

The current addition to the interrupt nesting level in
xen_arch_intr_handle_upcall() needs to be compensated in
xen_intr_handle_upcall(), otherwise interrupts dispatched by the upcall handler
end up seeing a td_intr_nesting_level of 2 or more, which makes them assume
there's been an interrupt nesting.

Such extra interrupt nesting count lead to statclock() reporting idle time as
interrupt, as the call from interrupt context will always be seen as a nested
one (td->td_intr_nesting_level >= 2) due to the nesting count increase done by
both xen_arch_intr_handle_upcall() and intr_execute_handlers().

Fix this by adjusting the nested interrupt count before dispatching interrupts
from xen_intr_handle_upcall().

PR: 277231
Reported by: Matthew Grooms <mgrooms@shrew.net>
Fixes: af610cabf1f4 ('xen/intr: adjust xen_intr_handle_upcall() to match driver filter')
Sponsored by: Cloud Software Group
Reviewed by: Elliott Mitchell <ehem+freebsd@m5p.com>

1 files changed, 20 insertions, 1 deletions

diff --git a/sys/dev/xen/bus/xen_intr.c b/sys/dev/xen/bus/xen_intr.c
index 609f31b5418a..bfe080b16f03 100644
--- a/sys/dev/xen/bus/xen_intr.c
+++ b/sys/dev/xen/bus/xen_intr.c
@@ -341,7 +341,7 @@ xen_intr_active_ports(const struct xen_intr_pcpu_data *const pcpu,
 /**
  * Interrupt handler for processing all Xen event channel events.
  * 
- * \param trap_frame  The trap frame context for the current interrupt.
+ * \param unused
  */
 int
 xen_intr_handle_upcall(void *unused __unused)
@@ -354,6 +354,15 @@ xen_intr_handle_upcall(void *unused __unused)
 	struct xen_intr_pcpu_data *pc;
 	u_long l1, l2;
 
+	/*
+	 * The upcall handler is an interrupt handler itself (that calls other
+	 * interrupt handlers), hence the caller has the responsibility to
+	 * increase td_intr_nesting_level ahead of dispatching the upcall
+	 * handler.
+	 */
+	KASSERT(curthread->td_intr_nesting_level > 0,
+	        ("Unexpected thread context"));
+
 	/* We must remain on the same vCPU during this function */
 	CRITICAL_ASSERT(curthread);
 
@@ -417,7 +426,17 @@ xen_intr_handle_upcall(void *unused __unused)
 				("Received unexpected event on vCPU#%u, event bound to vCPU#%u",
 				PCPU_GET(cpuid), isrc->xi_cpu));
 
+			/*
+			 * Reduce interrupt nesting level ahead of calling the
+			 * per-arch interrupt dispatch helper.  This is
+			 * required because the per-arch dispatcher will also
+			 * increase td_intr_nesting_level, and then handlers
+			 * would wrongly see td_intr_nesting_level = 2 when
+			 * there's no nesting at all.
+			 */
+			curthread->td_intr_nesting_level--;
 			xen_arch_intr_execute_handlers(isrc, trap_frame);
+			curthread->td_intr_nesting_level++;
 
 			/*
 			 * If this is the final port processed,