- Fix and enable support for flow control.
- Partially revert r172334; as it turns out the DELAYs in gem_reset_{r,t}x()
  are actually necessary although bus space barriers and gem_bitwait() are
  used, otherwise the controller may trigger an IOMMU errors on at least
  sparc64. This is in line with what Linux and OpenSolaris do.

Convert the PHY drivers to honor the mii_flags passed down and convert
the NIC drivers as well as the PHY drivers to take advantage of the
mii_attach() introduced in r213878 (MFC'ed to stable/8 in r214684) to
get rid of certain hacks. For the most part these were:
- Artificially limiting miibus_{read,write}reg methods to certain PHY
  addresses; we now let mii_attach() only probe the PHY at the desired
  address(es) instead.
- PHY drivers setting MIIF_* flags based on the NIC driver they hang
  off from, partly even based on grabbing and using the softc of the
  parent; we now pass these flags down from the NIC to the PHY drivers
  via mii_attach(). This got us rid of all such hacks except those of
  brgphy() in combination with bce(4) and bge(4), which is way beyond
  what can be expressed with simple flags.

While at it, I took the opportunity to change the NIC drivers to pass
up the error returned by mii_attach() (previously by mii_phy_probe())
and unify the error message used in this case where and as appropriate
as mii_attach() actually can fail for a number of reasons, not just
because of no PHY(s) being present at the expected address(es).

Reviewed by:	jhb, yongari

Merge r207585 (MFC'ed to stable/8 in r208086) from cas(4):
- Don't probe for PHYs if we already know to use a SERDES. Unlike as with
  cas(4) this only serves to speed up the the device attach though and can
  only be determined via the OFW device tree but not from the VPD.
- Don't touch the MIF when using a SERDES.
- Add some missing bus space barriers, mainly in the PCS code path.

According to the Linux sungem driver, in case of Apple (K2) GMACs
GEM_MIF_CONFIG_MDI0 cannot be trusted when the firmware has powered
down the chip so the internal transceiver has to be hardcoded. This
is also in line with the AppleGMACEthernet driver, which just doesn't
distinguish between internal/external transceiver and MDIO/MDI1
respectively in the first place. Tested by: Andreas Tobler

IF_ADDR_UNLOCK() across network device drivers when accessing the
per-interface multicast address list, if_multiaddrs.  This will
allow us to change the locking strategy without affecting our driver
programming interface or binary interface.

For two wireless drivers, remove unnecessary locking, since they
don't actually access the multicast address list.

Approved by:	re (kib)
MFC after:	6 weeks

call ether_ifdetach(9) before stopping the controller and the
callouts. The consensus is that the latter is now safe to do and
should also solve the problem of active BPF listeners clearing
promiscuous mode can result in the tick callout being restarted
which in turn will trigger a panic once it's actually gone.

  the PHYs as some PHY drivers use it (but probably shouldn't). How
  gem(4) has worked with brgphy(4) on powerpc without this so far is
  unclear to me.
- Introduce a dying flag which is set during detach and checked in
  gem_ioctl() in order to prevent active BPF listeners to clear
  promiscuous mode which may lead to the tick callout being restarted
  which will trigger a panic once it's actually gone.
- In gem_stop() reset rather than just disable the transmitter and
  receiver in order to ensure we're not unloading DMA maps still in
  use by the hardware. [1]
- The blanking time is specified in PCI clocks so we should use twice
  the value when operating at 66MHz.
- Spell some 2 as ETHER_ALIGN and a 19 as GEM_STATUS_TX_COMPLETION_SHFT
  to make the actual intentions clear.
- As we don't unload the peak attempts counter ignore its overflow
  interrupts.
- Remove a stale setting of a variable to GEM_TD_INTERRUPT_ME which
  isn't used afterwards.
- For optimum performance increment the TX kick register in multiples
  of 4 if possible as suggested by the documentation.
- Partially revert r164931; drivers should only clear the watchdog
  timer if all outstanding TX descriptors are done.
- Fix some debugging strings.
- Add a missing BUS_DMASYNC_POSTWRITE in gem_rint().
- As the error paths in the interrupt handler are generally unlikely
  predict them as false.
- Add support for the SBus version of the GEM controller. [2]
- Add some lock assertions.
- Improve some comments.
- Fix some more or less cosmetic issues in the code of the PCI front-end.
- Change some softc members to be unsigned where more appropriate and
  remove unused ones.

Approved by:	re (kib)
Obtained from:	NetBSD (partially) [2], OpenBSD [1]
MFC after:	2 weeks

  PHY only and not also in the case of an external PHY currently
  doing full duplex, which accidentally got broken in r172334.
  It's still not clear to me why we need to enable the buffer for
  an internal PHY though.
- Count excess and late collisions as output errors. [1]
- Count receive errors as input errors. [1]

Obtained from:	NetBSD [1]
MFC after:	3 days

  some time now so collapse calls accordingly.
o Given that gem_load_txmbuf() is allowed to fail resulting in a packet
  drop also for quite some time now implement the functionality of
  gem_txcksum() by means of m_pullup(9), which de-obfuscates the code
  and allows to always retrieve the correct length of the IP header.
o Add missing BUS_DMASYNC_PREREAD when syncing the control DMA maps in
  gem_rint() and gem_start_locked().
o Correct some bus_barrier(9) calls to do a read/write barrier as we
  do a read after a write. Add some missing ones in gem_mii_readreg()
  and gem_mii_writereg().
o According to the Apple GMAC driver, the GEM ASIC specification and
  the OpenSolaris eri(7D) the TX FIFO threshold has to be set to 0x4ff
  for the Gigabit variants and 0x100 for the ERI in order do avoid TX
  underruns.
o In gem_init_locked():
  - be conservative and enable the RX and TX MACs,
  - don't clear GEM_LINK otherwise we don't ever mark the link as up
    again if gem_init_locked() is called from gem_watchdog(),
  - remove superfluous setting of sc_ifflags.
o Don't bother to check whether the interface is running or whether its
  queue is empty before calling gem_start_locked() in gem_tint(), the
  former will check these anyway.
o Call gem_start_locked() in gem_watchdog() in order to try to get
  some more packets going.
o In gem_mii_writereg() after reseting the PCS restore its configuration.

GMAC testing:	grehan, marcel
MFC after:	2 weeks

Obtained from:	NetBSD (loosely)
MFC after:	2 weeks