Sometimes, especially with older firmware, mps(4) would have trouble initializing the card in one of these two steps. Add in a retry after a short delay. Sean Bruno and Stephen McConnell thought this was OK in the bug discussions, but never committed it. Steve indicated the delay might not be necessary, but the OP clearly needed to make it longer to make things work. I've kept the delay, and added the suggested comment. Ported the iocfacts part to mpr as well, since we see similar errors about once every month or two over a few thousand controllers at work. We've not seen it with IOC_INIT as far back as I can query the error log database, so I didn't port that forward. We'll see if this helps, but won't know for sure until next year (so I'm committing it now since it won't hurt and might help). We usually see this failure in connection with complicated recovery operations with a drive that's failing, though, at least in the last year's worth of failures. It's not clear this is the same as OP or not. PR: 212841 Sponsored by: Netflix Co-authored-by: imp (cherry picked from commit c0e0e530ced057502f51d7a6086857305e08fae0)
In preparation for adding a __result_use_check annotation to copyin() and related functions, start checking for errors from copyout() in the mpr(4) user command handler. This should make it easier to catch bugs. Reviewed by: imp, asomers MFC after: 1 month Differential Revision: https://reviews.freebsd.org/D43177 (cherry picked from commit 68cc77a3b73ffda1e8ac891b9852faca833e11b7)
xpt_path_string() is now a wrapper around xpt_path_sbuf(). Using it to than concatenate result to another sbuf makes no sense. Just call xpt_path_sbuf() directly. MFC after: 1 month (cherry picked from commit 8c4ee0b22c98fc1e208dd133f617bd329cd10728)
Most all of the memory used by the cards in the mpr(4) and mps(4) drivers is required, according to the specs and Broadcom developers, to be within a 4GB segment of memory. This includes: System Request Message Frames pool Reply Free Queues pool ReplyDescriptorPost Queues pool Chain Segments pool Sense Buffers pool SystemReply message pool We got a bug report from Dwight Engen, who ran into data corruption in the BAE port of FreeBSD: > We have a port of the FreeBSD mpr driver to our kernel and recently > I found an issue under heavy load where a DMA may go to the wrong > address. The test system is a Supermicro X10SRH-CLN4F with the > onboard SAS3008 controller setup with 2 enterprise Micron SSDs in > RAID 0 (striped). I have debugged the issue and narrowed down that > the errant DMA is one that has a segment that crosses a 4GB > physical boundary. There are more details I can provide if you'd > like, but with the attached patch in place I can no longer > re-create the issue. > I'm not sure if this is a known limit of the card (have not found a > datasheet/programming docs for the chip) or our system is just > doing something a bit different. Any helpful info or insight would > be welcome. > Anyway, just thought this might be helpful info if you want to > apply a similar fix to FreeBSD. You can ignore/discard the commit > message as it is my internal commit (blkio is our own tool we use > to write/read every block of a device with CRC verification which > is how I found the problem). The commit message was: > [PATCH 8/9] mpr: fix memory corrupting DMA when sg segment crosses > 4GB boundary > Test case was two SSD's in RAID 0 (stripe). The logical disk was > then partitioned into two partitions. One partition had lots of > filesystem I/O and the other was initially filled using blkio with > CRCable data and then read back with blkio CRC verify in a loop. > Eventually blkio would report a bad CRC block because the physical > page being read-ahead into didn't contain the right data. If the > physical address in the arq/segs was for example 0x500003000 the > data would actually be DMAed to 0x400003000. The original patch was against mpr(4) before busdma templates were introduced, and only affected the buffer pool (sc->buffer_dmat) in the mpr(4) driver. After some discussion with Dwight and the LSI/Broadcom developers and looking through the driver, it looks like most of the queues in the driver are ok, because they limit the memory used to memory below 4GB. The buffer queue and the chain frames seem to be the exceptions. This is pretty much the same between the mpr(4) and mps(4) drivers. So, apply a 4GB boundary limitation for the buffer and chain frame pools in the mpr(4) and mps(4) drivers. Reported by: Dwight Engen <dwight.engen@gmail.com> Reviewed by: imp Obtained from: Dwight Engen <dwight.engen@gmail.com> Differential Revision: <https://reviews.freebsd.org/D43008> (cherry picked from commit 264610a86e14f8e123d94c3c3bd9632d75c078a3)
Depending on the card's firmware version, it may return different length responses for MPI2_FUNCTION_IOC_FACTS. But the first part of the response contains the length of the rest, so query it first to get the length and then use that to size the buffer for the full response. Also, correctly zero-initialize MPI2_IOC_FACTS_REQUEST. It only worked by luck before. PR: 264848 Reported by: Julien Cigar <julien@perdition.city> Sponsored by: Axcient Reviewed by: scottl, imp Differential Revision: https://reviews.freebsd.org/D38739 (cherry picked from commit 7d154c4dc64e61af7ca536c4e9927fa07c675a83)
Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/
Remove /^\s*\*\n \*\s+\$FreeBSD\$$\n/
moving -> removing (we're removing the device) CAM_REQ_CMO_ERROR -> CAM_REQ_ERR (the former isn't a thing) Sponsored by: Netflix
Pointed out by: imp Sponsored by: Klara Inc.
Before the commit 6cc44223cb6717795afdac4348bbe7e2a968a07d the field event_mask was fully copied to the EventMasks field. After this commit the event_mask (uint8_t) is 4 times casted to EventMask (uint32_t). Because of that 24 bits of each event_mask array is lost. This commits brings back simple copying of field, and after words converting 32 bits field to the requested endian. I don't think we need more sophisticated method, as the array is of size 4 (for 32 bits version). Reviewed by: imp MFC after: 1 week Sponsored by: Klara Inc. Differential Revision: https://reviews.freebsd.org/D39562