aboutsummaryrefslogtreecommitdiff
path: root/sys/dev/dpaa
Commit message (Collapse)AuthorAgeFilesLines
* dpaa: Split FMan port driver into distinct TX/RXJustin Hibbits2 days4-630/+848
| | | | | | There are several port functions that are only for one type or the other, so they don't make sense to be together. Splitting these up also simplifies adding support for the Offline/Host Command ports.
* dpaa: Add LRO and receive callback batchingJustin Hibbits3 days8-15/+142
| | | | | | | | | | | | | | Reduce the code executed in the DQRR dequeue loop, and move the heavy-weight operations to post-dequeue loop. * Batch if_input() after DQRR dispatch loop completes. Only do the DQRR_CI_CINH write at the end of the loop, so only up to 16 entries will be processed. * Add software LRO per FQ. Each per-CPU RX FQ gets its own LRO tracking structure. Since LRO is configured at FQ initialization time, allocate the ifnet earlier in attach to prevent a panic.
* dpaa/eth: distribute RX across per-CPU FQs via FMan KeyGenJustin Hibbits3 days6-48/+266
| | | | | | | | | | | | | | | | | | | | | | | | Grow sc_nrxfqs from 1 to the CPU total, and hash the RX 5-tuple across the range with the KG driver from the prior commit. Each FQ lands on its own per-CPU QMan channel, so a given core drains only its own share of RX work and gets frame annotation + data-head stashed into its cache. * Add alignment parameter to qman_alloc_fqid_range() to meet KeyGen requirements. * Initialize 1 frame queue (FQ) per CPU in dpaa_eth_fm_port_rx_init(), using a 5-tuple to spread the load across CPUs. * Channel ownership for TX confirms moved from rx_init/free to tx_init/free -- sc_rx_channel is now a TX-confirm-only per-port pool channel. Fallbacks/degradation: * If any per-CPU channel is -1 (no portal attached) the port fails to attach with a clear message. * If the FQID range can't be allocated aligned, the port fails attach. * If KG scheme allocation fails at port setup, the port keeps its N FQs but only FQ #0 sees traffic.
* dpaa/fman: KeyGen (Parse-Classify-Distribute) driverJustin Hibbits3 days5-20/+434
| | | | | | | | | | | | | | | | | | | | | Add sys/dev/dpaa/fman_keygen.[ch]. Public API is four functions: * fman_kg_init(sc) -- Initialize KeyGen subsystem, clear out any stale config. * fman_kg_fini(sc) -- Teardown KeyGen * fman_kg_alloc_hash_scheme(sc, port, base_fqid, nfqs) -- Allocate a scheme, program it for RSS-over-IP-5-tuple hashing to nfqs FQs starting at base_fqid, bind it to port. * fman_kg_free_hash_scheme(sc, port) -- Remove a scheme added by fman_kg_alloc_hash_scheme(). KeyGen state (bitmap + port->scheme table) is added to the fman softc. Future work may allow configuring the KG hash inputs, but what we have now (5-tuple of src/src-port/dst/dst-port/IPSec SPI field) is sufficient.
* dpaa_eth: refactor RX FQ state into an array (N=1)Justin Hibbits3 days2-13/+45
| | | | | | | Preparation for FMan KeyGen-driven multi-queue RX. Replace the single sc_rx_fq / sc_rx_fqid pair with a sc_rx_fqs[] array (currently one entry) and sc_rx_fqid_base. Each entry carries a back-pointer to the softc for use by the RX callback.
* dpaa/qman: per-CPU pool channel service + FQID range allocatorJustin Hibbits3 days3-17/+98
| | | | | | | | | | | | | | | | | | | | | | | | Add plumbing for future FMan KeyGen-driven multi-queue RX. Pure infrastructure; no behavioural change for existing single-FQ consumers. qman: * New qman_percpu_channel(cpu) to get the per-CPU channel, needed for receive-side scaling. * New qman_alloc_fqid_range(count, *basep) / qman_free_fqid_range() reserve a contiguous FQID range so a later KeyGen-distribution caller can compute FQID = base + (hash & mask) and create each FQ individually with force_fqid=true (each landing on its own per-CPU channel). qman_fq_create: * Honor the force_fqid / fqid_or_align parameters: when force_fqid is set, use the caller-supplied FQID and skip the internal vmem_alloc. The fqids_num != 1 restriction is lifted; qman_fq_list[] now records the handle at every FQID slot in the range so DQRR dispatch works for the whole range. * Add fqid_count and force_fqid to struct qman_fq so qman_fq_free() can retire every FQ in the range.
* dpaa/qman: Fix various pool allocation bugsJustin Hibbits5 days2-7/+6
| | | | | | | | | | * Set qman_channel_base after determining if QMan is v3, otherwise this global stays at 0x21, which messes up the shift in qman_portal_static_dequeue_channel(). * Fix the base shift in qman_portal_static_dequeue_channel(), there are only 15 channels available, not 16, so starting at a shift of 15 yields shifting into the portal-specific channel. * Correct vmem pool names for QMan resource pools.
* dpaa/eth: fast-path single-page TX frame buildJustin Hibbits5 days1-15/+42
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The TX SG-build loop in dpaa_eth_if_start_locked() walked page boundaries with PAGE_MASK arithmetic even for buffers that lived entirely within one page -- the common case, since MCLBYTES is smaller than PAGE_SIZE. Add a fast path that emits a single SGT entry for wholly-in-one-page segments and skips the inner while entirely. Fix the following bugs while we're here: 1. "if (m->m_len == 0) continue;" in the outer loop never advanced m -- any zero-length mbuf hung the TX path in an infinite loop. Fix this by switching to a for loop, with the advancement in the post-clause. 2. In the inner (page-splitting) loop, the cap "if (m->m_len < ssize) ssize = m->m_len;" compared against the mbuf's original length, not the remaining bytes. A single mbuf whose data started mid-page and ran into a second page would produce a second SGT entry with ssize > rem, over-reading past the buffer end into whatever followed in kernel memory. Fixed by tracking a local rem and capping ssize against it. 3. If the whole mbuf chain consisted of zero-length segments, the final-flag store "fi_sgt[i - 1].final = 1" wrote to index -1. Reject empty frames up front now instead. As part of this, rename the inner counter from dsize to rem for clarity instead of playing double-duty in both inner and outer loops.
* dpaa/dtsec: advertise RX/TX csumJustin Hibbits6 days3-3/+23
| | | | | | | | Apply 6464974 to dTSEC, since it supports the same offload capabilities as mEMAC. The DPAA_CSUM_TX_OFFLOAD macro moves from if_memac.c to the shared dpaa_eth.h since both drivers now reference it.
* dpaa/qman: Make `show fqid <N>` actually workJustin Hibbits6 days3-38/+62
| | | | | Set the verb correctly so the query works. Also print out the programmable FQID fields as well.
* DPAA: Minor performance improvementsJustin Hibbits6 days6-44/+225
| | | | | | | | | | | | | | | | | * Add interrupt coalescing for DQRR and MR, with thresholds and period as tunable sysctls under the `hw.qman` tree. * Do lazy/sloppy buffer management to avoid constantly checking thresholds via QMan portal round-trips. * Add cache stashing to prewarm caches, reducing latency. * Fix the definition of Context_A in the init_fq MC command/result structures, they're 64-bit fields, not 32-bit. * Reorder the dpaa_eth_frame_info as a bit of cleanup. * Take advantage of the fact that UMA small allocations are returned in the DMAP, and avoid pmap_kextract(). These changes together improve throughput by ~1.5% (925Mbps->935-940Mbps) consistently, and reduce CPU usage by a bit, increasing idle CPU from 30%->35% minimum.
* fman: Fix duplicate port identifiersJustin Hibbits6 days1-2/+2
| | | | | fman_qman_channel_id returns the QMan FMan channel for a given port. If a port isn't found, the wrong channel number will be returned.
* dpaa/fman_port: Fix register config typoJustin Hibbits6 days1-1/+2
|
* dpaa: Restore Semihalf license headerJustin Hibbits2026-05-201-0/+25
| | | | | | | | This is a "new" file, but is mostly copied from if_dtsec_fdt.c, so need to retain the original license header in addition to the new one. Reviewed by: ziaee Differential Revision: https://reviews.freebsd.org/D57123
* dpaa: Fix copyright headersJustin Hibbits2026-05-0421-107/+47
| | | | | Reported by: ziaee@ Fixes: fd8d34ce27 ("dpaa: Migrate from NCSW base to a home-grown driver")
* dpaa: Support jumbo frames as multi-buffer framesJustin Hibbits2026-05-032-4/+24
|
* dpaa: Enable checksum offloading for IPv4 and IPv6Justin Hibbits2026-05-036-29/+216
| | | | | | | | | | | | | Enable the FMan hardware parser to take advantage of some offloading. This enables receive and transmit checksum offloading for both IPv4 and IPv6. Additional offloading capabilities the DPAA supports include: * vlan parsing * Transmit rate limiting * IEEE-1588 timestamps * Soft parsing for custom protocol checking * Congestion handling
* dpaa: Migrate from NCSW base to a home-grown driverJustin Hibbits2026-05-0336-3074/+5946
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The NCSW reference base requires tuning for each target, and currently is configured for FMANv2. This doesn't readily work on FMANv3 devices, such as the T-series powerpc, or the LS1043 ARM. Since Freescale/NXP abandoned the NCSW driver a decade ago, it makes sense to abandon it ourselves as well. This new driver uses a combination of the NCSW driver and the Linux driver (BSD/GPL dual licensed) as a reference, but contains no actual code from them. The DPAA (Data Path Acceleration Architecture) subsystem consists of the following components: * BMan -- Buffer Manager. Manages buffer pools of different sizes (one size per pool, up to 64 pools) * QMan -- Queue Manager. Manages the interfaces between DPAA-based components and the CPU(s). * FMan -- Frame Manager. Responsible for all ethernet-related processing. Consists itself of the following components: * Ports -- interfaces to the QMan. An ethernet interface consists of 2 ports. Ports use "Next-invoked action" (NIA) descriptors to form a pipeline for processing on receive and transmit. * Parser -- performs protocol header parsing and validation. Both hardware and software parsers are available. * KeyGen -- Key generator, used to start the classification process (for the Policer), generating FQIDs and other keys based on the frame input. * Policer -- performs traffic shaping and classification * MAC -- SoC specific ethernet MAC (dTSEC, TGEC, mEMAC). Currently supports dTSEC and mEMAC, along with their MDIO blocks. Additional components not yet handled: SEC -- Security engine (crypto) RE -- RAID engine RapidIO DCE -- Decompression/Compression engine, supports ZLIB, DEFLATE, and GZIP, as well as base64 encoding and decoding. BMan and QMan are accessed via cache-coherent portals, using ring buffers as I/O. The intent is for portals to be per-CPU (core/thread) to reduce locking contention and improve performance. This driver pins interrupt handlers to the CPU "owning" a given portal, and uses critical sections to prevent switching while accessing the portal.
* chore: replace {0, 0} with {DEV,KOBJ}METHOD_ENDEnji Cooper2026-02-255-7/+7
| | | | | | | | | | | | | | Both of the aforementioned macros have been present in FreeBSD for well over a decade: 2009 for `KOBJMETHOD_END`; 2011 for `DEVMETHOD_END`. Adapt all hardcoded references of `{0, 0}` with `DEVMETHOD_END` and `KOBJMETHOD_END` as appropriate. This helps ensure that future adaptations to drivers following patterns documented in driver(9) can be made more easily/without issue. MFC after: 1 week Differential Revision: https://reviews.freebsd.org/D55414
* dpaa: Simplify CPU binding for bman and qmanJustin Hibbits2026-01-212-61/+18
| | | | | | | If cpu-handle property doesn't exist simply iterate and assign the CPUs in sequence rather than following the convoluted search which may not bear fruit in some cases. If cpu-handle doesn't exist for one portal it probably doesn't exist for any of them.
* bus_alloc_resource: Pass rid by value to BUS_ALLOC_RESOURCE DEVMETHODJohn Baldwin2025-12-092-5/+5
| | | | | | | | | The wrapper functions such as bus_alloc_resource_any() still support passing the rid by value or pointer, but the underlying implementation now passes by value. Reviewed by: imp Differential Revision: https://reviews.freebsd.org/D53402
* Replace calls to bus_generic_attach with bus_attach_childrenJohn Baldwin2024-12-063-3/+6
| | | | | Reviewed by: imp Differential Revision: https://reviews.freebsd.org/D47675
* net: Remove unneeded NULL check for the allocated ifnetZhenlei Huang2024-06-281-6/+0
| | | | | | | | | | | Change 4787572d0580 made if_alloc_domain() never fail, then also do the wrappers if_alloc(), if_alloc_dev(), and if_gethandle(). No functional change intended. Reviewed by: kp, imp, glebius, stevek MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D45740
* dpaa: uma_zcreate() does not failMark Johnston2024-04-241-4/+0
| | | | | | No functional change intended. MFC after: 1 week
* new-bus: Remove the 'rid' and 'type' arguments from BUS_RELEASE_RESOURCEJohn Baldwin2024-03-132-8/+7
| | | | | | | | | | The public bus_release_resource() API still accepts both forms, but the internal kobj method no longer passes the arguments. Implementations which need the rid or type now use rman_get_rid() or rman_get_type() to fetch the value from the allocated resource. Reviewed by: imp Differential Revision: https://reviews.freebsd.org/D44131
* new-bus: Remove the 'rid' and 'type' arguments from BUS_*ACTIVATE_RESOURCEJohn Baldwin2024-03-132-5/+4
| | | | | | | | | | The public bus_activate/deactivate_resource() API still accepts both forms, but the internal kobj methods no longer pass the arguments. Implementations which need the rid or type now use rman_get_rid() or rman_get_type() to fetch the value from the allocated resource. Reviewed by: imp Differential Revision: https://reviews.freebsd.org/D44130
* sys: Set the type of allocated bus resourcesJohn Baldwin2024-03-131-0/+1
| | | | | | | | Use rman_set_type to set the type of allocated resources everywhere rman_set_rid is currently called. Reviewed by: imp Differential Revision: https://reviews.freebsd.org/D44123
* sys: Automated cleanup of cdefs and other formattingWarner Losh2023-11-2714-14/+5
| | | | | | | | | | | | | | | | Apply the following automated changes to try to eliminate no-longer-needed sys/cdefs.h includes as well as now-empty blank lines in a row. Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/ Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/ Remove /\n+#if.*\n#endif.*\n+/ Remove /^#if.*\n#endif.*\n/ Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/ Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/ Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/ Sponsored by: Netflix
* dtsec(4): Restore IFCAP_JUMBO_MTU lost in IfAPI conversionJustin Hibbits2023-09-021-0/+1
| | | | | | | Also add IFCAP_VLAN_MTU, since it's supported. Fixes: 0083fc5c7 ("Mechanically convert dtsec(4) to IfAPI") MFC after: 1 week
* dtsec: Support multicast receive.Justin Hibbits2023-08-261-1/+32
| | | | | | | | Implemented based on the tsec(4) multicast support. This is the minimum required to support VLANs. The hardware does support vlan tagging, among other acceleration features, which will be added at a later time. MFC after: 2 weeks
* sys: Remove $FreeBSD$: one-line .c patternWarner Losh2023-08-1614-28/+0
| | | | Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/
* sys: Remove $FreeBSD$: two-line .h patternWarner Losh2023-08-167-14/+0
| | | | Remove /^\s*\*\n \*\s+\$FreeBSD\$$\n/
* dtsec: Complete mechanical conversion to IfAPIJustin Hibbits2023-03-113-12/+12
| | | | | | Some changes were missed in 0083fc5c76. Sponsored by: Juniper Networks, Inc.
* mi_switch(): clean up switch types and their usageMitchell Horne2023-02-092-4/+2
| | | | | | | | | | | | | | | | | | | Overall, this is a non-functional change, except for kernels built with SCHED_STATS. However, the switch types are useful for communicating the intent of the caller. 1. Ensure that every caller provides a type. In most cases, we upgrade the basic yield to sched_relinquish() aka SWT_RELINQUISH. 2. The case of sched_bind() is distinct, so add a new switch type SWT_BIND. 3. Remove the two unused types, SWT_PREEMPT and SWT_SLEEPQTIMO. 4. Remove SWT_NONE altogether and assert that callers always provide a type flag. 5. Reference the mi_switch(9) man page in the comments, as these flags will be documented there. Reviewed by: kib, markj Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D38184
* Mechanically convert dtsec(4) to IfAPIJustin Hibbits2023-01-242-25/+24
| | | | | Sponsored by: Juniper Networks, Inc. Differential Revision: https://reviews.freebsd.org/D37815
* dpaa: Don't assume the MDIO is on the same fman as the MACJustin Hibbits2022-11-282-31/+17
| | | | | The P5040 has the MDIO for FMAN2 on FMAN1 for some reason. Instead of trying to manually find the MDIO, use a real xref.
* dpaa: Don't probe disabled devicesJustin Hibbits2022-11-272-0/+6
| | | | Probing disabled devices just adds noise.
* dpaa: Use BUS_PASS_SUPPORTDEV for pqmdioJustin Hibbits2022-11-261-1/+1
| | | | | | | | BUS_PASS_SUPPORTDEV is semantically better than BUS_PASS_RESOURCE, since it's a support device for dtsec, so only needs probed before dtsec, not before interrupts. Suggested by: manu
* dpaa: Probe and attach pqmdio at BUS_PASS_RESOURCEJustin Hibbits2022-11-261-1/+2
| | | | | Avoid the fragile nature of depending on ordering of the devices in the FDT by forcing the MDIO driver to precede the DTSEC driver.
* dpaa: Account for MDIO device not being foundJustin Hibbits2022-11-261-0/+3
| | | | | | | | | It's possible the MDIO device hasn't been and attached, or is incorrect in the device tree so can't probe and attach. In this case, ofw_bus_find_child_device_by_phandle() will fail, and return NULL. Return an error from find_mdio() here to prevent the MAC from attaching, rather than worry about a NULL pointer dereference later on when accessing the PHY.
* dpaa: Go to cleanup exit point on fman attach errorJustin Hibbits2022-11-061-1/+1
| | | | | | | | | If fman_init() fails it can leave things in a state where it cannot attach at all in the future, because it would simply exit without tearing down everything that was already set up. Go to the exit point to clean up on error instead, so that it can try again later. MFC after: 1 week
* dpaa: Remove unused devclass arguments to DRIVER_MODULE.John Baldwin2022-05-105-18/+9
|
* dpaa bman: Remove unused bman_devclass.John Baldwin2022-05-101-2/+0
|
* Remove unused miibus_devclass and miibus_fdt_devclass.John Baldwin2022-05-062-2/+2
|
* dpaa: Remove unused variables.John Baldwin2022-04-132-4/+0
|
* [PowerPC] [Book-E] Remove obsolete interrupt binding workaroundBrandon Bergren2020-02-251-7/+0
| | | | | | | | | | | | | | | | | | Remove an old workaround that is no longer necessary since rS343824. There used to be a problem with FMan interrupts firing on multiple CPUS at the same time. This ended up being due to multicast interrupts being unsupported in the Freescale PIC (so instead of using a selection algorithm, it would do some unspecified action, such as interrupting multiple cpus at random.) Reviewed by: jhibbits Sponsored by: Tag1 Consulting, Inc. Differential Revision: https://reviews.freebsd.org/D23829 Notes: svn path=/head/; revision=358322
* [PowerPC] [Book-E] Fix dpaa interrupt binding.Brandon Bergren2020-02-251-2/+1
| | | | | | | | | | | | | | | | | | | | | | After the network epoch was added, we lost the ability to migrate the ithread in the middle of dispatch, as being in the network epoch will pin the current thread (for safety reasons.) Luckily, we don't actually have to do this workaround in the first place, as we can just bind it to the correct cpu when we preallocate it. Pass dev through to XX_PreallocAndBindIntr() and actually bind it to the cpu like it was supposed to in the first place, instad of leaving it floating and moving it to the correct cpu the first time it fires. This fixes panics while bringing up dtsec on my X5000. Reviewed by: jhibbits Sponsored by: Tag1 Consulting, Inc. Differential Revision: https://reviews.freebsd.org/D23826 Notes: svn path=/head/; revision=358307
* Although most of the NIC drivers are epoch ready, due to peer pressureGleb Smirnoff2020-02-241-1/+1
| | | | | | | | | | | | | | | | | | | switch over to opt-in instead of opt-out for epoch. Instead of IFF_NEEDSEPOCH, provide IFF_KNOWSEPOCH. If driver marks itself with IFF_KNOWSEPOCH, then ether_input() would not enter epoch when processing its packets. Now this will create recursive entrance in epoch in >90% network drivers, but will guarantee safeness of the transition. Mark several tested drivers as IFF_KNOWSEPOCH. Reviewed by: hselasky, jeff, bz, gallatin Differential Revision: https://reviews.freebsd.org/D23674 Notes: svn path=/head/; revision=358301
* Introduce flag IFF_NEEDSEPOCH that marks Ethernet interfaces thatGleb Smirnoff2020-01-231-1/+1
| | | | | | | | | | | supposedly may call into ether_input() without network epoch. They all need to be reviewed before 13.0-RELEASE. Some may need be fixed. The flag is not planned to be used in the kernel for a long time. Notes: svn path=/head/; revision=357010
* schedlock 4/4Jeff Roberson2019-12-152-4/+2
| | | | | | | | | | | | | | | | | | | | | Don't hold the scheduler lock while doing context switches. Instead we unlock after selecting the new thread and switch within a spinlock section leaving interrupts and preemption disabled to prevent local concurrency. This means that mi_switch() is entered with the thread locked but returns without. This dramatically simplifies scheduler locking because we will not hold the schedlock while spinning on blocked lock in switch. This change has not been made to 4BSD but in principle it would be more straightforward. Discussed with: markj Reviewed by: kib Tested by: pho Differential Revision: https://reviews.freebsd.org/D22778 Notes: svn path=/head/; revision=355784
generated by cgit v1.3 (git 2.53.0) at 2026-08-27 11:03:11 +0000