diff options
author | Justin T. Gibbs <gibbs@FreeBSD.org> | 2004-02-04 16:38:38 +0000 |
---|---|---|
committer | Justin T. Gibbs <gibbs@FreeBSD.org> | 2004-02-04 16:38:38 +0000 |
commit | 4164174aff63c9454809f7f9b7b3ce9021514c81 (patch) | |
tree | 9266eaaec6e0c5fda1f7f285063c76d2dd2f56db | |
parent | df3ac0421182689aed21d0d811aac0a53f9b33c3 (diff) | |
download | src-4164174aff63c9454809f7f9b7b3ce9021514c81.tar.gz src-4164174aff63c9454809f7f9b7b3ce9021514c81.zip |
aic79xx.c:
aic79xx.seq:
Convert the COMPLETE_DMA_SCB list to an "stailq". This allows us to
safely keep the SCB that is currently being DMA'ed back the host on
the head of the list while processing completions off of the bus. The
newly completed SCBs are appended to the tail of the queue. In the
past, we just dequeued the SCB that was in flight from the list, but
this could result in a lost completion should the host perform certain
types of error recovery that must cancel all in-flight SCB DMA operations.
Switch from using a 16bit completion entry, holding just the tag and the
completion valid bit, to a 64bit completion entry that also contains a
"status packet valid" indicator. This solves two problems:
o The SCB DMA engine on at least Rev B. silicon does not properly deal
with a PCI disconnect that occurs at a non-64bit aligned offset in the
chips "source buffer". When the transfer is resumed, the DMA engine
continues at the correct offset, but may wrap to the head of the buffer
causing duplicate completions to be reported to the host. By using a
completion buffer in host memory that is 64bit aligned and using 64bit
completion entries, such disconnects should only occur at aligned addresses.
This assumes that the host bridge will only disconnect on cache-line
boundaries and that cache-lines are multpiles of 64bits.
o By embedding the status information in the completion entry we can avoid
an extra memory reference to the HSCB for commands that complete without
error.
Use the comparison of a "host freeze count" and a "sequencer freeze count"
to allow the host to process most SCBs that complete with non-zero status
without having to clear critical sections. Instead the host can just pause the
sequencer, performs any necessary cleanup in the waiting for selection list,
increments its freeze count on the controller, and unpauses. This is only
possible because the sequencer defers completions of SCBs with bad status
until after all pending selections have completed. The sequencer then avoids
referencing any data structures the host may touch during completion of the
SCB until the freeze counts match.
aic79xx.c:
Change the strategy for allocating our sentinal HSCB for the QINFIFO. In
the past, this allocation was tacked onto the QOUTFIFO allocation. Now that
the qoutfifo has grown to accomodate larger completion entries, the old
approach will result in a 64byte allocation that costs an extra page of
coherent memory. We now do this extra allocation via ahd_alloc_scbs()
where the "unused space" can be used to allocate "normal" HSCBs.
In our packetized busfree handler, use the ENSELO bit to differentiate
between packetized and non-packetized unexpected busfree events that
occur just after selection, but before the sequencer has had the oportunity
to service the selection.
When cleaning out the waiting for selection list, use the SCSI mode
instead of the command channel mode. The SCB pointer in the command
channel mode may be referenced by the SCB dma engine even while the
sequencer is paused, whereas the SCSI mode SCB pointer is only accessed
by the sequencer.
Print the "complete on qfreeze" sequencer SCB completion list in
ahd_dump_card_state(). This list holds all SCB completions that are deferred
until a pending select-out qfreeze event has taken effect.
aic79xx.h:
Add definitions and structures to handle the new SCB completion scheme.
Add a controller flag that indicates if the controller is in HostRAID
mode.
aic79xx.reg:
Remove macros used for toggling from one data fifo mode to the other.
They have not been in use for some time.
Add scratch ram fields for our new qfreeze count scheme, converting
the complete dma list into an "stailq", and providing for the "complete
on qfreeze" SCB completion list. Some other fields were moved to retain
proper field alignment (alignment >= field size in bytes).
aic79xx.seq:
Add code to our idle loop to:
o Process deferred completions once a qfreeze event has taken full
effect.
o Thaw the queue once the sequencer and host qfreeze counts match.
Generate 64bit completion entries passing the SCB_SGPTR field as the
"good status" indicator. The first bit in this field is only set if
we have a valid status packet to send to the host.
Convert the COMPLETE_DMA_SCB list to an "stailq".
When using "setjmp" to register an idle loop handler, do not combine
the "ret" with the block move to pop the stack address in the same
instruction. At least on the A, this results in a return to the setjmp
caller, not to the new address at the top of the stack. Since we want
the latter (we want the newly registered handler to only be invoked from
the idle loop), we must use a separate ret instruction.
Add a few missing critical sections.
Close a race condition that can occur on Rev A. silicon. If both FIFOs
happen to be allocated before the sequencer has a chance to service the
FIFO that was allocated first, we must take special care to service the
FIFO that is not active on the SCSI bus first. This guarantees that a
FIFO will be freed to handle any snapshot requests for the FIFO that is
still on the bus. Chosing the incorrect FIFO will result in deadlock.
Update comments.
aic79xx_inline.h
Correct the offset calculation for the syncing of our qoutfifo.
Update ahd_check_cmdcmpltqueues() for the larger completion entries.
aic79xx_pci.c:
Attach to HostRAID controllers by default. In the future I may add a
sysctl to modify the behavior, but since FreeBSD does not have any
HostRAID drivers, failing to attach just results in more email and
bug reports for the author.
MFC After: 1week
Notes
Notes:
svn path=/head/; revision=125448
-rw-r--r-- | sys/dev/aic7xxx/aic79xx.c | 182 | ||||
-rw-r--r-- | sys/dev/aic7xxx/aic79xx.h | 35 | ||||
-rw-r--r-- | sys/dev/aic7xxx/aic79xx.reg | 57 | ||||
-rw-r--r-- | sys/dev/aic7xxx/aic79xx.seq | 147 | ||||
-rw-r--r-- | sys/dev/aic7xxx/aic79xx_inline.h | 13 | ||||
-rw-r--r-- | sys/dev/aic7xxx/aic79xx_pci.c | 28 |
6 files changed, 327 insertions, 135 deletions
diff --git a/sys/dev/aic7xxx/aic79xx.c b/sys/dev/aic7xxx/aic79xx.c index e224edb37361..c615863fc235 100644 --- a/sys/dev/aic7xxx/aic79xx.c +++ b/sys/dev/aic7xxx/aic79xx.c @@ -37,7 +37,7 @@ * IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE * POSSIBILITY OF SUCH DAMAGES. * - * $Id: //depot/aic7xxx/aic7xxx/aic79xx.c#224 $ + * $Id: //depot/aic7xxx/aic7xxx/aic79xx.c#238 $ */ #ifdef __linux__ @@ -54,6 +54,7 @@ __FBSDID("$FreeBSD$"); /******************************** Globals *************************************/ struct ahd_softc_tailq ahd_tailq = TAILQ_HEAD_INITIALIZER(ahd_tailq); +uint32_t ahd_attach_to_HostRAID_controllers = 1; /***************************** Lookup Tables **********************************/ char *ahd_chip_names[] = @@ -445,10 +446,21 @@ rescan_fifos: ahd_outb(ahd, SCB_SGPTR, ahd_inb_scbram(ahd, SCB_SGPTR) | SG_STATUS_VALID); - ahd_outw(ahd, SCB_TAG, SCB_GET_TAG(scb)); + ahd_outw(ahd, SCB_TAG, scbid); + ahd_outw(ahd, SCB_NEXT_COMPLETE, SCB_LIST_NULL); comp_head = ahd_inw(ahd, COMPLETE_DMA_SCB_HEAD); - ahd_outw(ahd, SCB_NEXT_COMPLETE, comp_head); - ahd_outw(ahd, COMPLETE_DMA_SCB_HEAD, SCB_GET_TAG(scb)); + if (SCBID_IS_NULL(comp_head)) { + ahd_outw(ahd, COMPLETE_DMA_SCB_HEAD, scbid); + ahd_outw(ahd, COMPLETE_DMA_SCB_TAIL, scbid); + } else { + u_int tail; + + tail = ahd_inw(ahd, COMPLETE_DMA_SCB_TAIL); + ahd_set_scbptr(ahd, tail); + ahd_outw(ahd, SCB_NEXT_COMPLETE, scbid); + ahd_outw(ahd, COMPLETE_DMA_SCB_TAIL, scbid); + ahd_set_scbptr(ahd, scbid); + } } else ahd_complete_scb(ahd, scb); } @@ -514,6 +526,24 @@ rescan_fifos: scbid = next_scbid; } ahd_outw(ahd, COMPLETE_DMA_SCB_HEAD, SCB_LIST_NULL); + ahd_outw(ahd, COMPLETE_DMA_SCB_TAIL, SCB_LIST_NULL); + + scbid = ahd_inw(ahd, COMPLETE_ON_QFREEZE_HEAD); + while (!SCBID_IS_NULL(scbid)) { + + ahd_set_scbptr(ahd, scbid); + next_scbid = ahd_inw_scbram(ahd, SCB_NEXT_COMPLETE); + scb = ahd_lookup_scb(ahd, scbid); + if (scb == NULL) { + printf("%s: Warning - Complete Qfrz SCB %d invalid\n", + ahd_name(ahd), scbid); + continue; + } + + ahd_complete_scb(ahd, scb); + scbid = next_scbid; + } + ahd_outw(ahd, COMPLETE_ON_QFREEZE_HEAD, SCB_LIST_NULL); scbid = ahd_inw(ahd, COMPLETE_SCB_HEAD); while (!SCBID_IS_NULL(scbid)) { @@ -797,9 +827,20 @@ clrchn: } } +/* + * Look for entries in the QoutFIFO that have completed. + * The valid_tag completion field indicates the validity + * of the entry - the valid value toggles each time through + * the queue. We use the sg_status field in the completion + * entry to avoid referencing the hscb if the completion + * occurred with no errors and no residual. sg_status is + * a copy of the first byte (little endian) of the sgptr + * hscb field. + */ void ahd_run_qoutfifo(struct ahd_softc *ahd) { + struct ahd_completion *completion; struct scb *scb; u_int scb_index; @@ -807,11 +848,13 @@ ahd_run_qoutfifo(struct ahd_softc *ahd) panic("ahd_run_qoutfifo recursion"); ahd->flags |= AHD_RUNNING_QOUTFIFO; ahd_sync_qoutfifo(ahd, BUS_DMASYNC_POSTREAD); - while ((ahd->qoutfifo[ahd->qoutfifonext] - & QOUTFIFO_ENTRY_VALID_LE) == ahd->qoutfifonext_valid_tag) { + for (;;) { + completion = &ahd->qoutfifo[ahd->qoutfifonext]; + + if (completion->valid_tag != ahd->qoutfifonext_valid_tag) + break; - scb_index = aic_le16toh(ahd->qoutfifo[ahd->qoutfifonext] - & ~QOUTFIFO_ENTRY_VALID_LE); + scb_index = aic_le16toh(completion->tag); scb = ahd_lookup_scb(ahd, scb_index); if (scb == NULL) { printf("%s: WARNING no command for scb %d " @@ -819,12 +862,15 @@ ahd_run_qoutfifo(struct ahd_softc *ahd) ahd_name(ahd), scb_index, ahd->qoutfifonext); ahd_dump_card_state(ahd); - } else - ahd_complete_scb(ahd, scb); + } else if ((completion->sg_status & SG_STATUS_VALID) != 0) { + ahd_handle_scb_status(ahd, scb); + } else { + ahd_done(ahd, scb); + } ahd->qoutfifonext = (ahd->qoutfifonext+1) & (AHD_QOUT_SIZE-1); if (ahd->qoutfifonext == 0) - ahd->qoutfifonext_valid_tag ^= QOUTFIFO_ENTRY_VALID_LE; + ahd->qoutfifonext_valid_tag ^= QOUTFIFO_ENTRY_VALID; } ahd->flags &= ~AHD_RUNNING_QOUTFIFO; } @@ -1653,7 +1699,15 @@ ahd_handle_scsiint(struct ahd_softc *ahd, u_int intstat) clear_fifo = 0; packetized = (lqostat1 & LQOBUSFREE) != 0; if (!packetized - && ahd_inb(ahd, LASTPHASE) == P_BUSFREE) + && ahd_inb(ahd, LASTPHASE) == P_BUSFREE + && ((ahd_inb(ahd, SSTAT0) & SELDO) == 0 + || (ahd_inb(ahd, SCSISEQ0) & ENSELO) == 0)) + /* + * Assume packetized if we are not + * on the bus in a non-packetized + * capacity and any pending selection + * was a packetized selection. + */ packetized = 1; break; } @@ -5959,6 +6013,22 @@ ahd_alloc_scbs(struct ahd_softc *ahd) hscb = (struct hardware_scb *)hscb_map->vaddr; hscb_busaddr = hscb_map->busaddr; scb_data->scbs_left = PAGE_SIZE / sizeof(*hscb); + if (ahd->next_queued_hscb == NULL) { + /* + * We need one HSCB to serve as the "next HSCB". Since + * the tag identifier in this HSCB will never be used, + * there is no point in using a valid SCB from the + * free pool for it. So, we allocate this "sentinel" + * specially. + */ + ahd->next_queued_hscb = hscb; + ahd->next_queued_hscb_map = hscb_map; + memset(hscb, 0, sizeof(*hscb)); + hscb->hscb_busaddr = aic_htole32(hscb_busaddr); + hscb++; + hscb_busaddr += sizeof(*hscb); + scb_data->scbs_left--; + } } if (scb_data->sgs_left != 0) { @@ -6042,12 +6112,12 @@ ahd_alloc_scbs(struct ahd_softc *ahd) scb_data->scbs_left -= newcount; scb_data->sgs_left -= newcount; for (i = 0; i < newcount; i++) { - u_int col_tag; - struct scb_platform_data *pdata; + u_int col_tag; #ifndef __linux__ int error; #endif + next_scb = (struct scb *)malloc(sizeof(*next_scb), M_DEVBUF, M_NOWAIT); if (next_scb == NULL) @@ -6218,8 +6288,7 @@ ahd_init(struct ahd_softc *ahd) * for the target mode role, we must additionally provide space for * the incoming target command fifo. */ - driver_data_size = AHD_SCB_MAX * sizeof(uint16_t) - + sizeof(struct hardware_scb); + driver_data_size = AHD_SCB_MAX * sizeof(*ahd->qoutfifo); if ((ahd->features & AHD_TARGETMODE) != 0) driver_data_size += AHD_TMODE_CMDS * sizeof(struct target_cmd); if ((ahd->bugs & AHD_PKT_BITBUCKET_BUG) != 0) @@ -6253,10 +6322,10 @@ ahd_init(struct ahd_softc *ahd) ahd->shared_data_map.vaddr, driver_data_size, ahd_dmamap_cb, &ahd->shared_data_map.busaddr, /*flags*/0); - ahd->qoutfifo = (uint16_t *)ahd->shared_data_map.vaddr; + ahd->qoutfifo = (struct ahd_completion *)ahd->shared_data_map.vaddr; next_vaddr = (uint8_t *)&ahd->qoutfifo[AHD_QOUT_SIZE]; next_baddr = ahd->shared_data_map.busaddr - + AHD_QOUT_SIZE*sizeof(uint16_t); + + AHD_QOUT_SIZE*sizeof(struct ahd_completion); if ((ahd->features & AHD_TARGETMODE) != 0) { ahd->targetcmds = (struct target_cmd *)next_vaddr; next_vaddr += AHD_TMODE_CMDS * sizeof(struct target_cmd); @@ -6269,17 +6338,6 @@ ahd_init(struct ahd_softc *ahd) next_baddr += PKT_OVERRUN_BUFSIZE; } - /* - * We need one SCB to serve as the "next SCB". Since the - * tag identifier in this SCB will never be used, there is - * no point in using a valid HSCB tag from an SCB pulled from - * the standard free pool. So, we allocate this "sentinel" - * specially from the DMA safe memory chunk used for the QOUTFIFO. - */ - ahd->next_queued_hscb = (struct hardware_scb *)next_vaddr; - ahd->next_queued_hscb_map = &ahd->shared_data_map; - ahd->next_queued_hscb->hscb_busaddr = aic_htole32(next_baddr); - ahd->init_level++; /* Allocate SCB data now that buffer_dmat is initialized */ @@ -6583,10 +6641,10 @@ ahd_chip_init(struct ahd_softc *ahd) /* All of our queues are empty */ ahd->qoutfifonext = 0; - ahd->qoutfifonext_valid_tag = QOUTFIFO_ENTRY_VALID_LE; - ahd_outb(ahd, QOUTFIFO_ENTRY_VALID_TAG, QOUTFIFO_ENTRY_VALID >> 8); + ahd->qoutfifonext_valid_tag = QOUTFIFO_ENTRY_VALID; + ahd_outb(ahd, QOUTFIFO_ENTRY_VALID_TAG, QOUTFIFO_ENTRY_VALID); for (i = 0; i < AHD_QOUT_SIZE; i++) - ahd->qoutfifo[i] = 0; + ahd->qoutfifo[i].valid_tag = 0; ahd_sync_qoutfifo(ahd, BUS_DMASYNC_PREREAD); ahd->qinfifonext = 0; @@ -6619,11 +6677,15 @@ ahd_chip_init(struct ahd_softc *ahd) ahd_outw(ahd, COMPLETE_SCB_HEAD, SCB_LIST_NULL); ahd_outw(ahd, COMPLETE_SCB_DMAINPROG_HEAD, SCB_LIST_NULL); ahd_outw(ahd, COMPLETE_DMA_SCB_HEAD, SCB_LIST_NULL); + ahd_outw(ahd, COMPLETE_DMA_SCB_TAIL, SCB_LIST_NULL); + ahd_outw(ahd, COMPLETE_ON_QFREEZE_HEAD, SCB_LIST_NULL); /* * The Freeze Count is 0. */ + ahd->qfreeze_cnt = 0; ahd_outw(ahd, QFREEZE_COUNT, 0); + ahd_outw(ahd, KERNEL_QFREEZE_COUNT, 0); /* * Tell the sequencer where it can find our arrays in memory. @@ -6983,18 +7045,17 @@ ahd_pause_and_flushwork(struct ahd_softc *ahd) { u_int intstat; u_int maxloops; - u_int qfreeze_cnt; maxloops = 1000; ahd->flags |= AHD_ALL_INTERRUPTS; ahd_pause(ahd); /* - * Increment the QFreeze Count so that the sequencer - * will not start new selections. We do this only + * Freeze the outgoing selections. We do this only * until we are safely paused without further selections * pending. */ - ahd_outw(ahd, QFREEZE_COUNT, ahd_inw(ahd, QFREEZE_COUNT) + 1); + ahd->qfreeze_cnt--; + ahd_outw(ahd, KERNEL_QFREEZE_COUNT, ahd->qfreeze_cnt); ahd_outb(ahd, SEQ_FLAGS2, ahd_inb(ahd, SEQ_FLAGS2) | SELECTOUT_QFROZEN); do { struct scb *waiting_scb; @@ -7036,17 +7097,8 @@ ahd_pause_and_flushwork(struct ahd_softc *ahd) printf("Infinite interrupt loop, INTSTAT = %x", ahd_inb(ahd, INTSTAT)); } - qfreeze_cnt = ahd_inw(ahd, QFREEZE_COUNT); - if (qfreeze_cnt == 0) { - printf("%s: ahd_pause_and_flushwork with 0 qfreeze count!\n", - ahd_name(ahd)); - } else { - qfreeze_cnt--; - } - ahd_outw(ahd, QFREEZE_COUNT, qfreeze_cnt); - if (qfreeze_cnt == 0) - ahd_outb(ahd, SEQ_FLAGS2, - ahd_inb(ahd, SEQ_FLAGS2) & ~SELECTOUT_QFROZEN); + ahd->qfreeze_cnt++; + ahd_outw(ahd, KERNEL_QFREEZE_COUNT, ahd->qfreeze_cnt); ahd_flush_qoutfifo(ahd); @@ -7387,6 +7439,7 @@ ahd_search_qinfifo(struct ahd_softc *ahd, int target, char channel, * appropriate, traverse the SCBs of each "their id" * looking for matches. */ + ahd_set_modes(ahd, AHD_MODE_SCSI, AHD_MODE_SCSI); savedscbptr = ahd_get_scbptr(ahd); tid_next = ahd_inw(ahd, WAITING_TID_HEAD); tid_prev = SCB_LIST_NULL; @@ -7456,7 +7509,7 @@ ahd_search_scb_list(struct ahd_softc *ahd, int target, char channel, u_int prev; int found; - AHD_ASSERT_MODES(ahd, AHD_MODE_CCHAN_MSK, AHD_MODE_CCHAN_MSK); + AHD_ASSERT_MODES(ahd, AHD_MODE_SCSI_MSK, AHD_MODE_SCSI_MSK); found = 0; prev = SCB_LIST_NULL; next = *list_head; @@ -7523,7 +7576,7 @@ static void ahd_stitch_tid_list(struct ahd_softc *ahd, u_int tid_prev, u_int tid_cur, u_int tid_next) { - AHD_ASSERT_MODES(ahd, AHD_MODE_CCHAN_MSK, AHD_MODE_CCHAN_MSK); + AHD_ASSERT_MODES(ahd, AHD_MODE_SCSI_MSK, AHD_MODE_SCSI_MSK); if (SCBID_IS_NULL(tid_cur)) { @@ -7563,7 +7616,7 @@ ahd_rem_wscb(struct ahd_softc *ahd, u_int scbid, { u_int tail_offset; - AHD_ASSERT_MODES(ahd, AHD_MODE_CCHAN_MSK, AHD_MODE_CCHAN_MSK); + AHD_ASSERT_MODES(ahd, AHD_MODE_SCSI_MSK, AHD_MODE_SCSI_MSK); if (!SCBID_IS_NULL(prev)) { ahd_set_scbptr(ahd, prev); ahd_outw(ahd, SCB_NEXT, next); @@ -7987,14 +8040,16 @@ void ahd_handle_scsi_status(struct ahd_softc *ahd, struct scb *scb) { struct hardware_scb *hscb; - u_int qfreeze_cnt; int paused; /* * The sequencer freezes its select-out queue * anytime a SCSI status error occurs. We must - * handle the error and decrement the QFREEZE count - * to allow the sequencer to continue. + * handle the error and increment our qfreeze count + * to allow the sequencer to continue. We don't + * bother clearing critical sections here since all + * operations are on data structures that the sequencer + * is not touching once the queue is frozen. */ hscb = scb->hscb; @@ -8008,16 +8063,8 @@ ahd_handle_scsi_status(struct ahd_softc *ahd, struct scb *scb) /* Freeze the queue until the client sees the error. */ ahd_freeze_devq(ahd, scb); aic_freeze_scb(scb); - qfreeze_cnt = ahd_inw(ahd, QFREEZE_COUNT); - if (qfreeze_cnt == 0) { - printf("%s: Bad status with 0 qfreeze count!\n", ahd_name(ahd)); - } else { - qfreeze_cnt--; - ahd_outw(ahd, QFREEZE_COUNT, qfreeze_cnt); - } - if (qfreeze_cnt == 0) - ahd_outb(ahd, SEQ_FLAGS2, - ahd_inb(ahd, SEQ_FLAGS2) & ~SELECTOUT_QFROZEN); + ahd->qfreeze_cnt++; + ahd_outw(ahd, KERNEL_QFREEZE_COUNT, ahd->qfreeze_cnt); if (paused == 0) ahd_unpause(ahd); @@ -8938,6 +8985,15 @@ ahd_dump_card_state(struct ahd_softc *ahd) scb_index = ahd_inw_scbram(ahd, SCB_NEXT_COMPLETE); } printf("\n"); + printf("Sequencer On QFreeze and Complete list: "); + scb_index = ahd_inw(ahd, COMPLETE_ON_QFREEZE_HEAD); + i = 0; + while (!SCBID_IS_NULL(scb_index) && i++ < AHD_SCB_MAX) { + ahd_set_scbptr(ahd, scb_index); + printf("%d ", scb_index); + scb_index = ahd_inw_scbram(ahd, SCB_NEXT_COMPLETE); + } + printf("\n"); ahd_set_scbptr(ahd, saved_scb_index); dffstat = ahd_inb(ahd, DFFSTAT); for (i = 0; i < 2; i++) { diff --git a/sys/dev/aic7xxx/aic79xx.h b/sys/dev/aic7xxx/aic79xx.h index a9b7ae181c3d..ab07ad8f3e68 100644 --- a/sys/dev/aic7xxx/aic79xx.h +++ b/sys/dev/aic7xxx/aic79xx.h @@ -37,7 +37,7 @@ * IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE * POSSIBILITY OF SUCH DAMAGES. * - * $Id: //depot/aic7xxx/aic7xxx/aic79xx.h#101 $ + * $Id: //depot/aic7xxx/aic7xxx/aic79xx.h#106 $ * * $FreeBSD$ */ @@ -75,8 +75,7 @@ struct scb_platform_data; #define INITIATOR_WILDCARD (~0) #define SCB_LIST_NULL 0xFF00 #define SCB_LIST_NULL_LE (aic_htole16(SCB_LIST_NULL)) -#define QOUTFIFO_ENTRY_VALID 0x8000 -#define QOUTFIFO_ENTRY_VALID_LE (aic_htole16(0x8000)) +#define QOUTFIFO_ENTRY_VALID 0x80 #define SCBID_IS_NULL(scbid) (((scbid) & 0xFF00 ) == SCB_LIST_NULL) #define SCSIID_TARGET(ahd, scsiid) \ @@ -202,6 +201,8 @@ do { \ #define AHD_BUSRESET_DELAY 25 /******************* Chip Characteristics/Operating Settings *****************/ +extern uint32_t ahd_attach_to_HostRAID_controllers; + /* * Chip Type * The chip order is from least sophisticated to most sophisticated. @@ -376,7 +377,8 @@ typedef enum { AHD_UPDATE_PEND_CMDS = 0x400000, AHD_RUNNING_QOUTFIFO = 0x800000, AHD_HAD_FIRST_SEL = 0x1000000, - AHD_SHUTDOWN_RECOVERY = 0x2000000 /* Terminate recovery thread. */ + AHD_SHUTDOWN_RECOVERY = 0x2000000, /* Terminate recovery thread. */ + AHD_HOSTRAID_BOARD = 0x4000000 } ahd_flag; /************************* Hardware SCB Definition ***************************/ @@ -1059,6 +1061,14 @@ typedef uint8_t ahd_mode_state; typedef void ahd_callback_t (void *); +struct ahd_completion +{ + uint16_t tag; + uint8_t sg_status; + uint8_t pad[4]; + uint8_t valid_tag; +}; + struct ahd_softc { bus_space_tag_t tags[2]; bus_space_handle_t bshs[2]; @@ -1152,16 +1162,23 @@ struct ahd_softc { ahd_flag flags; struct seeprom_config *seep_config; - /* Values to store in the SEQCTL register for pause and unpause */ - uint8_t unpause; - uint8_t pause; - /* Command Queues */ + struct ahd_completion *qoutfifo; uint16_t qoutfifonext; uint16_t qoutfifonext_valid_tag; uint16_t qinfifonext; uint16_t qinfifo[AHD_SCB_MAX]; - uint16_t *qoutfifo; + + /* + * Our qfreeze count. The sequencer compares + * this value with its own counter to determine + * whether to allow selections to occur. + */ + uint16_t qfreeze_cnt; + + /* Values to store in the SEQCTL register for pause and unpause */ + uint8_t unpause; + uint8_t pause; /* Critical Section Data */ struct cs *critical_sections; diff --git a/sys/dev/aic7xxx/aic79xx.reg b/sys/dev/aic7xxx/aic79xx.reg index 449fe95ae624..cff63d6fd6cc 100644 --- a/sys/dev/aic7xxx/aic79xx.reg +++ b/sys/dev/aic7xxx/aic79xx.reg @@ -39,7 +39,7 @@ * * $FreeBSD$ */ -VERSION = "$Id: //depot/aic7xxx/aic7xxx/aic79xx.reg#70 $" +VERSION = "$Id: //depot/aic7xxx/aic7xxx/aic79xx.reg#75 $" /* * This file is processed by the aic7xxx_asm utility for use in assembling @@ -65,13 +65,6 @@ VERSION = "$Id: //depot/aic7xxx/aic7xxx/aic79xx.reg#70 $" mvi MODE_PTR, MK_MODE(src, dst); \ } -#define TOGGLE_DFF_MODE \ - if ((ahd->bugs & AHD_SET_MODE_BUG) != 0) { \ - call toggle_dff_mode_work_around; \ - } else { \ - xor MODE_PTR, MK_MODE(M_DFF1, M_DFF1); \ - } - #define RESTORE_MODE(mode) \ if ((ahd->bugs & AHD_SET_MODE_BUG) != 0) { \ mov mode call set_mode_work_around; \ @@ -3542,10 +3535,34 @@ scratch_ram { COMPLETE_DMA_SCB_HEAD { size 2 } - /* Counting semaphore to prevent new select-outs */ + /* + * tail of list of SCBs that have + * completed but need to be uploaded + * to the host prior to being completed. + */ + COMPLETE_DMA_SCB_TAIL { + size 2 + } + /* + * head of list of SCBs that have + * been uploaded to the host, but cannot + * be completed until the QFREEZE is in + * full effect (i.e. no selections pending). + */ + COMPLETE_ON_QFREEZE_HEAD { + size 2 + } + /* + * Counting semaphore to prevent new select-outs + * The queue is frozen so long as the sequencer + * and kernel freeze counts differ. + */ QFREEZE_COUNT { size 2 } + KERNEL_QFREEZE_COUNT { + size 2 + } /* * Mode to restore on legacy idle loop exit. */ @@ -3625,6 +3642,17 @@ scratch_ram { size 1 } /* + * Kernel and sequencer offsets into the queue of + * incoming target mode command descriptors. The + * queue is full when the KERNEL_TQINPOS == TQINPOS. + */ + KERNEL_TQINPOS { + size 1 + } + TQINPOS { + size 1 + } + /* * Base address of our shared data with the kernel driver in host * memory. This includes the qoutfifo and target mode * incoming command queue. @@ -3639,17 +3667,6 @@ scratch_ram { QOUTFIFO_NEXT_ADDR { size 4 } - /* - * Kernel and sequencer offsets into the queue of - * incoming target mode command descriptors. The - * queue is full when the KERNEL_TQINPOS == TQINPOS. - */ - KERNEL_TQINPOS { - size 1 - } - TQINPOS { - size 1 - } ARG_1 { size 1 mask SEND_MSG 0x80 diff --git a/sys/dev/aic7xxx/aic79xx.seq b/sys/dev/aic7xxx/aic79xx.seq index 68c539beedac..7bd3b417b1c4 100644 --- a/sys/dev/aic7xxx/aic79xx.seq +++ b/sys/dev/aic7xxx/aic79xx.seq @@ -40,7 +40,7 @@ * $FreeBSD$ */ -VERSION = "$Id: //depot/aic7xxx/aic7xxx/aic79xx.seq#107 $" +VERSION = "$Id: //depot/aic7xxx/aic7xxx/aic79xx.seq#118 $" PATCH_ARG_LIST = "struct ahd_softc *ahd" PREFIX = "ahd_" @@ -68,13 +68,47 @@ no_error_set: } SET_MODE(M_SCSI, M_SCSI) test SCSISEQ0, ENSELO|ENARBO jnz idle_loop_checkbus; - test SEQ_FLAGS2, SELECTOUT_QFROZEN jnz idle_loop_checkbus; + test SEQ_FLAGS2, SELECTOUT_QFROZEN jz check_waiting_list; + /* + * If the kernel has caught up with us, thaw the queue. + */ + mov A, KERNEL_QFREEZE_COUNT; + cmp QFREEZE_COUNT, A jne check_frozen_completions; + mov A, KERNEL_QFREEZE_COUNT[1]; + cmp QFREEZE_COUNT[1], A jne check_frozen_completions; + and SEQ_FLAGS2, ~SELECTOUT_QFROZEN; + jmp check_waiting_list; +check_frozen_completions: + test SSTAT0, SELDO|SELINGO jnz idle_loop_checkbus; +BEGIN_CRITICAL; + /* + * If we have completions stalled waiting for the qfreeze + * to take effect, move them over to the complete_scb list + * now that no selections are pending. + */ + cmp COMPLETE_ON_QFREEZE_HEAD[1],SCB_LIST_NULL je idle_loop_checkbus; + /* + * Find the end of the qfreeze list. The first element has + * to be treated specially. + */ + bmov SCBPTR, COMPLETE_ON_QFREEZE_HEAD, 2; + cmp SCB_NEXT_COMPLETE[1], SCB_LIST_NULL je join_lists; + /* + * Now the normal loop. + */ + bmov SCBPTR, SCB_NEXT_COMPLETE, 2; + cmp SCB_NEXT_COMPLETE[1], SCB_LIST_NULL jne . - 1; +join_lists: + bmov SCB_NEXT_COMPLETE, COMPLETE_SCB_HEAD, 2; + bmov COMPLETE_SCB_HEAD, COMPLETE_ON_QFREEZE_HEAD, 2; + mvi COMPLETE_ON_QFREEZE_HEAD[1], SCB_LIST_NULL; + jmp idle_loop_checkbus; +check_waiting_list: cmp WAITING_TID_HEAD[1], SCB_LIST_NULL je idle_loop_checkbus; /* * ENSELO is cleared by a SELDO, so we must test for SELDO * one last time. */ -BEGIN_CRITICAL; test SSTAT0, SELDO jnz select_out; END_CRITICAL; call start_selection; @@ -192,14 +226,14 @@ scbdma_tohost_done: * wait until any select-out activity has halted, and * then queue the completion. */ - test SCB_SCSI_STATUS, 0xff jz scbdma_queue_completion; - SET_MODE(M_SCSI, M_SCSI) - test SCSISEQ0, ENSELO jnz return; - test SSTAT0, (SELDO|SELINGO) jnz return; - SET_MODE(M_CCHAN, M_CCHAN) -scbdma_queue_completion: and CCSCBCTL, ~(CCARREN|CCSCBEN); bmov COMPLETE_DMA_SCB_HEAD, SCB_NEXT_COMPLETE, 2; + cmp SCB_NEXT_COMPLETE[1], SCB_LIST_NULL jne . + 2; + mvi COMPLETE_DMA_SCB_TAIL[1], SCB_LIST_NULL; + test SCB_SCSI_STATUS, 0xff jz scbdma_queue_completion; + bmov SCB_NEXT_COMPLETE, COMPLETE_ON_QFREEZE_HEAD, 2; + bmov COMPLETE_ON_QFREEZE_HEAD, SCBPTR, 2 ret; +scbdma_queue_completion: bmov SCB_NEXT_COMPLETE, COMPLETE_SCB_HEAD, 2; bmov COMPLETE_SCB_HEAD, SCBPTR, 2 ret; fill_qoutfifo_dmadone: @@ -327,14 +361,15 @@ fill_qoutfifo: * Keep track of the SCBs we are dmaing just * in case the DMA fails or is aborted. */ - mov A, QOUTFIFO_ENTRY_VALID_TAG; bmov COMPLETE_SCB_DMAINPROG_HEAD, COMPLETE_SCB_HEAD, 2; mvi CCSCBCTL, CCSCBRESET; bmov SCBHADDR, QOUTFIFO_NEXT_ADDR, 4; bmov SCBPTR, COMPLETE_SCB_HEAD, 2; fill_qoutfifo_loop: - mov CCSCBRAM, SCBPTR; - or CCSCBRAM, A, SCBPTR[1]; + bmov CCSCBRAM, SCBPTR, 2; + mov CCSCBRAM, SCB_SGPTR[0]; + bmov CCSCBRAM, ALLZEROS, 4; + mov CCSCBRAM, QOUTFIFO_ENTRY_VALID_TAG; mov NONE, SDSCB_QOFF; inc INT_COALESCING_CMDCOUNT; add CMDS_PENDING, -1; @@ -357,7 +392,6 @@ dma_complete_scb: bmov SCBPTR, COMPLETE_DMA_SCB_HEAD, 2; bmov SCBHADDR, SCB_BUSADDR, 4; mvi CCARREN|CCSCBEN|CCSCBRESET jmp dma_scb; -END_CRITICAL; /* * Either post or fetch an SCB from host memory. The caller @@ -374,9 +408,19 @@ dma_scb: mvi SCBHCNT, SCB_TRANSFER_SIZE; mov CCSCBCTL, SINDEX ret; -BEGIN_CRITICAL; setjmp: - bmov LONGJMP_ADDR, STACK, 2 ret; + /* + * At least on the A, a return in the same + * instruction as the bmov results in a return + * to the caller, not to the new address at the + * top of the stack. Since we want the latter + * (we use setjmp to register a handler from an + * interrupt context but not invoke that handler + * until we return to our idle loop), use a + * separate ret instruction. + */ + bmov LONGJMP_ADDR, STACK, 2; + ret; setjmp_inline: bmov LONGJMP_ADDR, STACK, 2; longjmp: @@ -395,11 +439,6 @@ set_mode_work_around: mvi SEQINTCTL, INTVEC1DSL; mov MODE_PTR, SINDEX; clr SEQINTCTL ret; - -toggle_dff_mode_work_around: - mvi SEQINTCTL, INTVEC1DSL; - xor MODE_PTR, MK_MODE(M_DFF1, M_DFF1); - clr SEQINTCTL ret; } @@ -1033,15 +1072,9 @@ not_found_ITloop: /* * We received a "command complete" message. Put the SCB on the complete * queue and trigger a completion interrupt via the idle loop. Before doing - * so, check to see if there - * is a residual or the status byte is something other than STATUS_GOOD (0). - * In either of these conditions, we upload the SCB back to the host so it can - * process this information. In the case of a non zero status byte, we - * additionally interrupt the kernel driver synchronously, allowing it to - * decide if sense should be retrieved. If the kernel driver wishes to request - * sense, it will fill the kernel SCB with a request sense command, requeue - * it to the QINFIFO and tell us not to post to the QOUTFIFO by setting - * RETURN_1 to SEND_SENSE. + * so, check to see if there is a residual or the status byte is something + * other than STATUS_GOOD (0). In either of these conditions, we upload the + * SCB back to the host so it can process this information. */ mesgin_complete: @@ -1086,6 +1119,7 @@ complete_nomsg: call queue_scb_completion; jmp await_busfree; +BEGIN_CRITICAL; freeze_queue: /* Cancel any pending select-out. */ test SSTAT0, SELDO|SELINGO jnz . + 2; @@ -1096,6 +1130,7 @@ freeze_queue: adc QFREEZE_COUNT[1], A; or SEQ_FLAGS2, SELECTOUT_QFROZEN; mov A, ACCUM_SAVE ret; +END_CRITICAL; /* * Complete the current FIFO's SCB if data for this same @@ -1133,9 +1168,16 @@ upload_scb: */ bmov SCB_TAG, SCBPTR, 2; BEGIN_CRITICAL; - bmov SCB_NEXT_COMPLETE, COMPLETE_DMA_SCB_HEAD, 2; + or SCB_SGPTR, SG_STATUS_VALID; + mvi SCB_NEXT_COMPLETE[1], SCB_LIST_NULL; + cmp COMPLETE_DMA_SCB_HEAD[1], SCB_LIST_NULL jne add_dma_scb_tail; bmov COMPLETE_DMA_SCB_HEAD, SCBPTR, 2; - or SCB_SGPTR, SG_STATUS_VALID ret; + bmov COMPLETE_DMA_SCB_TAIL, SCBPTR, 2 ret; +add_dma_scb_tail: + bmov REG0, SCBPTR, 2; + bmov SCBPTR, COMPLETE_DMA_SCB_TAIL, 2; + bmov SCB_NEXT_COMPLETE, REG0, 2; + bmov COMPLETE_DMA_SCB_TAIL, REG0, 2 ret; END_CRITICAL; /* @@ -1340,6 +1382,47 @@ service_fifo: test CCSGCTL, CCSGENACK jnz return; /* + * Should the other FIFO get the S/G cache first? If + * both FIFOs have been allocated since we last checked + * any FIFO, it is important that we service a FIFO + * that is not actively on the bus first. This guarantees + * that a FIFO will be freed to handle snapshot requests for + * any FIFO that is still on the bus. Chips with RTI do not + * perform snapshots, so don't bother with this test there. + */ + if ((ahd->features & AHD_RTI) == 0) { + /* + * If we're not still receiving SCSI data, + * it is safe to allocate the S/G cache to + * this FIFO. + */ + test DFCNTRL, SCSIEN jz idle_sgfetch_start; + + /* + * Switch to the other FIFO. Non-RTI chips + * also have the "set mode" bug, so we must + * disable interrupts during the switch. + */ + mvi SEQINTCTL, INTVEC1DSL; + xor MODE_PTR, MK_MODE(M_DFF1, M_DFF1); + + /* + * If the other FIFO needs loading, then it + * must not have claimed the S/G cache yet + * (SG_CACHE_AVAIL would have been cleared in + * the orginal FIFO mode and we test this above). + * Return to the idle loop so we can process the + * FIFO not currently on the bus first. + */ + test SG_STATE, LOADING_NEEDED jz idle_sgfetch_okay; + clr SEQINTCTL ret; +idle_sgfetch_okay: + xor MODE_PTR, MK_MODE(M_DFF1, M_DFF1); + clr SEQINTCTL; + } + +idle_sgfetch_start: + /* * We fetch a "cacheline aligned" and sized amount of data * so we don't end up referencing a non-existant page. * Cacheline aligned is in quotes because the kernel will @@ -1350,7 +1433,7 @@ service_fifo: mvi SGHCNT, SG_PREFETCH_CNT; if ((ahd->bugs & AHD_REG_SLOW_SETTLE_BUG) != 0) { /* - * Need two instruction between "touches" of SGHADDR. + * Need two instructions between "touches" of SGHADDR. */ nop; } diff --git a/sys/dev/aic7xxx/aic79xx_inline.h b/sys/dev/aic7xxx/aic79xx_inline.h index fabda5ba4fda..80989edc23e1 100644 --- a/sys/dev/aic7xxx/aic79xx_inline.h +++ b/sys/dev/aic7xxx/aic79xx_inline.h @@ -37,7 +37,7 @@ * IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE * POSSIBILITY OF SUCH DAMAGES. * - * $Id: //depot/aic7xxx/aic7xxx/aic79xx_inline.h#55 $ + * $Id: //depot/aic7xxx/aic7xxx/aic79xx_inline.h#56 $ * * $FreeBSD$ */ @@ -838,7 +838,8 @@ static __inline void ahd_sync_qoutfifo(struct ahd_softc *ahd, int op) { aic_dmamap_sync(ahd, ahd->shared_data_dmat, ahd->shared_data_map.dmamap, - /*offset*/0, /*len*/AHD_SCB_MAX * sizeof(uint16_t), op); + /*offset*/0, + /*len*/AHD_SCB_MAX * sizeof(struct ahd_completion), op); } static __inline void @@ -868,10 +869,10 @@ ahd_check_cmdcmpltqueues(struct ahd_softc *ahd) retval = 0; aic_dmamap_sync(ahd, ahd->shared_data_dmat, ahd->shared_data_map.dmamap, - /*offset*/ahd->qoutfifonext, /*len*/2, - BUS_DMASYNC_POSTREAD); - if ((ahd->qoutfifo[ahd->qoutfifonext] - & QOUTFIFO_ENTRY_VALID_LE) == ahd->qoutfifonext_valid_tag) + /*offset*/ahd->qoutfifonext * sizeof(*ahd->qoutfifo), + /*len*/sizeof(*ahd->qoutfifo), BUS_DMASYNC_POSTREAD); + if (ahd->qoutfifo[ahd->qoutfifonext].valid_tag + == ahd->qoutfifonext_valid_tag) retval |= AHD_RUN_QOUTFIFO; #ifdef AHD_TARGET_MODE if ((ahd->flags & AHD_TARGETROLE) != 0 diff --git a/sys/dev/aic7xxx/aic79xx_pci.c b/sys/dev/aic7xxx/aic79xx_pci.c index 004e163e97a3..ec5bd4ca123f 100644 --- a/sys/dev/aic7xxx/aic79xx_pci.c +++ b/sys/dev/aic7xxx/aic79xx_pci.c @@ -38,7 +38,7 @@ * IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE * POSSIBILITY OF SUCH DAMAGES. * - * $Id: //depot/aic7xxx/aic7xxx/aic79xx_pci.c#84 $ + * $Id: //depot/aic7xxx/aic7xxx/aic79xx_pci.c#86 $ */ #ifdef __linux__ @@ -65,10 +65,10 @@ ahd_compose_id(u_int device, u_int vendor, u_int subdevice, u_int subvendor) } #define ID_ALL_MASK 0xFFFFFFFFFFFFFFFFull -#define ID_ALL_IROC_MASK 0xFFFFFF7FFFFFFFFFull +#define ID_ALL_IROC_MASK 0xFF7FFFFFFFFFFFFFull #define ID_DEV_VENDOR_MASK 0xFFFFFFFF00000000ull #define ID_9005_GENERIC_MASK 0xFFF0FFFF00000000ull -#define ID_9005_GENERIC_IROC_MASK 0xFFF0FF7F00000000ull +#define ID_9005_GENERIC_IROC_MASK 0xFF70FFFF00000000ull #define ID_AIC7901 0x800F9005FFFF9005ull #define ID_AHA_29320A 0x8000900500609005ull @@ -92,10 +92,11 @@ ahd_compose_id(u_int device, u_int vendor, u_int subdevice, u_int subvendor) #define ID_AIC7902_PCI_REV_B0 0x10 #define SUBID_HP 0x0E11 +#define DEVID_9005_HOSTRAID(id) ((id) & 0x80) + #define DEVID_9005_TYPE(id) ((id) & 0xF) #define DEVID_9005_TYPE_HBA 0x0 /* Standard Card */ #define DEVID_9005_TYPE_HBA_2EXT 0x1 /* 2 External Ports */ -#define DEVID_9005_TYPE_IROC 0x8 /* Raid(0,1,10) Card */ #define DEVID_9005_TYPE_MB 0xF /* On Motherboard */ #define DEVID_9005_MFUNC(id) ((id) & 0x10) @@ -198,7 +199,7 @@ struct ahd_pci_identity ahd_pci_ident_table [] = }, /* Generic chip probes for devices we don't know 'exactly' */ { - ID_AIC7901 & ID_DEV_VENDOR_MASK, + ID_AIC7901 & ID_9005_GENERIC_MASK, ID_DEV_VENDOR_MASK, "Adaptec AIC7901 Ultra320 SCSI adapter", ahd_aic7901_setup @@ -282,6 +283,14 @@ ahd_find_pci_device(aic_dev_softc_t pci) subdevice, subvendor); + /* + * If we are configured to attach to HostRAID + * controllers, mask out the IROC/HostRAID bit + * in the + */ + if (ahd_attach_to_HostRAID_controllers) + full_id &= ID_ALL_IROC_MASK; + for (i = 0; i < ahd_num_pci_devs; i++) { entry = &ahd_pci_ident_table[i]; if (entry->full_id == (full_id & entry->id_mask)) { @@ -301,12 +310,21 @@ ahd_pci_config(struct ahd_softc *ahd, struct ahd_pci_identity *entry) u_long l; u_int command; uint32_t devconfig; + uint16_t device; uint16_t subvendor; int error; shared_scb_data = NULL; ahd->description = entry->name; /* + * Record if this is a HostRAID board. + */ + device = aic_pci_read_config(ahd->dev_softc, + PCIR_DEVICE, /*bytes*/2); + if (DEVID_9005_HOSTRAID(device)) + ahd->flags |= AHD_HOSTRAID_BOARD; + + /* * Record if this is an HP board. */ subvendor = aic_pci_read_config(ahd->dev_softc, |