umtx: do not sleep on an unowned mutex after a spurious CAS failureHEADmain

On ll/sc architectures casueword32() may report a spurious
store-conditional failure (reservation lost to an interrupt, preemption,
or another CPU touching the same reservation granule), and this is
indistinguishable from a genuine comparison mismatch: both return 1.
That is intentional since D20772 and documented in casueword(9) ("The
store can fail on load-linked/store-conditional architectures."), so
callers must cope.

do_lock_normal() does not fully cope.  When the initial
UMUTEX_UNOWNED -> id acquire CAS fails spuriously, the observed owner is
still UMUTEX_UNOWNED, so neither the UMUTEX_CONTESTED branch nor the
real-owner case applies, and execution falls through past the "rv == 1
but not contested, likely store failure" comment into the sleep path.
There, the contested-bit CAS (expecting the observed owner, i.e.
UMUTEX_UNOWNED) succeeds because the mutex really is unowned, stamping
m_owner = UMUTEX_CONTESTED with no owner tid, and the thread sleeps on
"umtxn" forever: nobody owns the mutex, so no unlock and no wakeup ever
arrive.  In _UMUTEX_TRY mode the same situation returns a false EBUSY
for a free mutex.

Treat an observed owner of UMUTEX_UNOWNED like UMUTEX_CONTESTED: try to
acquire the mutex, setting the contested bit, instead of falling through
to the sleep path.  rv == 1 with the observed value equal to the
expected value can only mean a spurious store failure, so the mutex is
free.  If the acquire CAS fails again, the outer loop restarts and
re-evaluates ownership.  The contested bit set with no waiters present
only costs the matching unlock one trip through the kernel.

This was hit in practice on powerpc64le (POWER9): the Swift runtime's
Synchronization.Mutex issues _umtx_op(UMTX_OP_MUTEX_LOCK) directly with
no userspace fast path, so an uncontended lock of an unowned mutex runs
the kernel CAS exactly where a spurious failure deadlocks
(single-threaded process parked on "umtxn" with m_owner == 0x80000000,
observed as Foundation.Process.run() hanging).  libthr mostly masks the
bug because pthread_mutex_lock() enters the kernel only when there is a
real owner that will eventually issue a wakeup.

The mechanism was confirmed with an experimental powerpc kernel that
instead retried the ll/sc sequence inside casueword32()/casueword();
that also eliminated the hang, but is not proposed here since the
single-attempt semantics of casueword(9) are intentional.

Reviewed by:	kib
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D59338

0 files changed, 0 insertions, 0 deletions