aboutsummaryrefslogtreecommitdiff
path: root/sys/dev/netmap
Commit message (Collapse)AuthorAgeFilesLines
...
* netmap: several typo fixesVincenzo Maffione2021-04-0211-37/+37
| | | | No functional changes intended.
* netmap: fix typo bug in netmap_compute_buf_lenVincenzo Maffione2021-04-022-2/+2
|
* netmap: monitor: add a flag to distinguish packet directionVincenzo Maffione2021-03-291-2/+6
| | | | | | | | The netmap monitor intercepts any TX/RX packets on the monitored port. However, before this change there was no way to tell whether an intercepted packet was being transmitted or received on the monitored port. A TXMON flag in the netmap slot has been added for this purpose.
* netmap: add kernel support for the "offsets" featureVincenzo Maffione2021-03-2911-325/+853
| | | | | | | | | | | | | This feature enables applications to ask netmap to transmit or receive packets starting at a user-specified offset from the beginning of the netmap buffer. This is meant to ease those packet manipulation operations such as pushing or popping packet headers, that may be useful to implement software switches, routers and other packet processors. To use the feature, drivers (e.g., iflib, vtnet, etc.) must have explicit support. This change does not add support for any driver, but introduces the necessary kernel changes. However, offsets support is already included for VALE ports and pipes.
* netmap: fix issues in nm_os_extmem_create()Vincenzo Maffione2021-03-201-6/+10
| | | | | | | | | | - Call vm_object_reference() before vm_map_lookup_done(). - Use vm_mmap_to_errno() to convert vm_map_* return values to errno. - Fix memory leak of e->obj. Reported by: markj Reviewed by: markj MFC after: 1 week
* netmap: fix memory leak in NETMAP_REQ_PORT_INFO_GETVincenzo Maffione2021-03-151-0/+4
| | | | | | | | | | | | The netmap_ioctl() function has a reference counting bug in case of NETMAP_REQ_PORT_INFO_GET command. When `hdr->nr_name[0] == '\0'`, the function does not decrease the refcount of "nmd", which is increased by netmap_mem_find(), causing a refcount leak. Reported by: Xiyu Yang <sherllyyang00@gmail.com> Submitted by: Carl Smith <carl.smith@alliedtelesis.co.nz> MFC after: 3 days PR: 254311
* netmap: Stop printing a line to the dmesg in netmap_init()Mark Johnston2021-03-051-0/+2
| | | | | | | | | | | netmap is compiled into the kernel by default so initialization was always reported, and netmap uses a formatting convention not used in the rest of the kernel. Reviewed by: vmaffione MFC after: 1 week Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D29099
* netmap: simplify parameter passingVincenzo Maffione2021-01-244-44/+45
| | | | Changes imported from the netmap github.
* netmap: vtnet: fix RX initialization after netmap_reset()Vincenzo Maffione2021-01-111-7/+12
| | | | | | | At device reset, we must not publish those netmap receive buffers that are owned by userspace (nm_kr_rxspace). MFC after: 1 week
* netmap: restore hwofs and support it in iflibVincenzo Maffione2021-01-101-13/+34
| | | | | | | | | | | | | | | | Restore the hwofs functionality temporarily disabled by 7ba6ecf216fb15e8b147db2 to prevent issues with iflib. This patch brings the necessary changes to iflib to enable howfs to allow interface restarts without disrupting netmap applications actively using its rings. After this change, it becomes possible for multiple non-cooperating netmap applications to use non-overlapping subsets of the available netmap rings without clashing with each other. PR: 252453 MFC after: 1 week
* netmap: vtnet: enable/disable krings on any interface reinitVincenzo Maffione2021-01-102-6/+4
| | | | | | | See 3d65fd97e85ab807f3b for a detailed explanation. PR: 252453 MFC after: 1 week
* netmap: vtnet: stop krings during interface resetVincenzo Maffione2021-01-091-1/+7
| | | | | | | | Similarly to what done for iflib in 1d238b07d5d4d9660ae0e, this patch prevents access to the krings during the interface reset triggered by netmap_register(). MFC after: 1 week
* netmap: refactor netmap_resetVincenzo Maffione2021-01-091-45/+20
| | | | | | | | | | | | | | | | The netmap_reset() function is meant to be called by the driver when they initialize (or re-initialize) a hardware ring. However, since the introduction of support for opening (in netmap mode) a subset of the available rings, netmap_reset() may be called multiple times on actively used rings, causing both kring and netmap ring to transition to an inconsistent state. This changes improves the situation by resetting all the indices fields of the kring to 0, as expected after the reinitialization of a hardware ring. PR: 252518 MFC after: 1 week
* netmap: iflib: stop krings during interface resetVincenzo Maffione2021-01-091-1/+1
| | | | | | | | | | | | | | | | | When different processes open separate subsets of the available rings of a same netmap interface, a device reset may be performed while one of the processes is actively using some rings (e.g., caused by another process executing a nmport_open()). With this patch, such situation will cause the active process to get a POLLERR, so that it can have a chance to detect the situation. We also guarantee that no process is running a txsync or rxsync (ioctl or poll) while an iflib device reset is in progress. PR: 252453 MFC after: 1 week
* netmap: fix mutex double unlock bugVincenzo Maffione2020-10-221-1/+0
| | | | | | | | | | https://github.com/luigirizzo/netmap/pull/733 Submitted by: brian90013 MFC after: 3 days Notes: svn path=/head/; revision=366952
* netmap: use FreeBSD guards for epoch callsVincenzo Maffione2020-08-241-1/+5
| | | | | | | | | | EPOCH calls are FreeBSD specific. Use guards to protect these, so that the code can compile under Linux. MFC after: 1 week Notes: svn path=/head/; revision=364731
* netmap: fix parsing of legacy nmr->nr_ringidVincenzo Maffione2020-08-181-2/+2
| | | | | | | | | | | | | | | | Code was checking for NETMAP_{SW,HW}_RING in req->nr_ringid which had already been masked by NETMAP_RING_MASK. Therefore, the comparisons always failed and set NR_REG_ALL_NIC. Check against the original nmr structure. Submitted by: bpoole@packetforensics.com Reported by: bpoole@packetforensics.com Reviewed by: giuseppe.lettieri@unipi.it Approved by: vmaffione MFC after: 1 week Notes: svn path=/head/; revision=364341
* netmap: vtnet: fix races in vtnet_netmap_reg()Vincenzo Maffione2020-06-141-47/+11
| | | | | | | | | | | | | | | | | | | | | The nm_register callback needs to call nm_set_native_flags() or nm_clear_native_flags() once the device has been stopped. However, in the current implementation this is not true, as the device is stopped by vtnet_init_locked(). This causes race conditions where the driver crashes as soon as it dequeues netmap buffers assuming they are mbufs (or the other way around). To fix the issue, we extend vtnet_init_locked() with a second argument that, if not zero, will set/clear the netmap flags. This results in a huge simplification of the nm_register callback itself. Also, use netmap_reset() to check if a ring is going to be re-initialized in netmap mode. MFC after: 1 week Notes: svn path=/head/; revision=362183
* netmap: introduce netmap_kring_on()Vincenzo Maffione2020-06-112-19/+18
| | | | | | | | | | | | This function returns NULL if the ring identified by queue id and direction is in netmap mode. Otherwise return the corresponding kring. Use this function to replace vtnet_netmap_queue_on(). MFC after: 1 week Notes: svn path=/head/; revision=362076
* netmap: vtnet: clean up rxsync disabled logsVincenzo Maffione2020-06-031-4/+2
| | | | | | | MFC after: 1 week Notes: svn path=/head/; revision=361760
* netmap: vtnet: fix race condition in rxsyncVincenzo Maffione2020-06-031-3/+18
| | | | | | | | | | | | This change prevents a race that happens when rxsync dequeues N-1 rx packets (with N being the size of the netmap rx ring). In this situation, the loop exits without re-enabling the rx interrupts, thus causing the VQ to stall. MFC after: 1 week Notes: svn path=/head/; revision=361759
* netmap: vtnet: add vtnrx_nm_refill index to receive queuesVincenzo Maffione2020-06-031-21/+31
| | | | | | | | | | | | | | | The new index tracks the next netmap slot that is going to be enqueued into the virtqueue. The index is necessary to prevent the receive VQ and the netmap rx ring from going out of sync, considering that we never enqueue N slots, but at most N-1. This change fixes a bug that causes the VQ and the netmap ring to go out of sync after N-1 packets have been received. MFC after: 1 week Notes: svn path=/head/; revision=361758
* netmap: vale: fix disabled logsVincenzo Maffione2020-06-031-2/+2
| | | | | | | MFC after: 1 week Notes: svn path=/head/; revision=361747
* netmap: vtnet: remove leftover memory barriersVincenzo Maffione2020-06-031-2/+0
| | | | | | | MFC after: 1 week Notes: svn path=/head/; revision=361746
* netmap: if_vtnet: avoid netmap ring wraparoundVincenzo Maffione2020-06-011-2/+2
| | | | | | | | | | | netmap assumes the one "slot" is left unused to distinguish the empty ring and full ring conditions. This assumption was violated by vtnet_netmap_rxq_populate(). MFC after: 1 week Notes: svn path=/head/; revision=361698
* netmap: if_vtnet: replace vtnet_free_used()Vincenzo Maffione2020-06-011-38/+2
| | | | | | | | | | | The functionality contained in this function is duplicated, as it is already available in vtnet_txq_free_mbufs() and vtnet_rxq_free_mbufs(). MFC after: 1 week Notes: svn path=/head/; revision=361697
* netmap: vtnet: fix RX virtqueue initialization bugVincenzo Maffione2020-06-011-7/+19
| | | | | | | | | | | | | | | | The vtnet_netmap_rxq_populate() function erroneously assumed that kring->nr_hwcur = 0, i.e. the kring was in the initial state. However, this is not always the case: for example, when a vtnet reinit is triggered by some changes in the interface flags or capenable. This patch changes the behaviour of vtnet_netmap_kring_refill() so that it always starts publishing the netmap buffers starting from the current value of kring->nr_hwcur. MFC after: 1 week Notes: svn path=/head/; revision=361696
* Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many)Pawel Biernacki2020-02-261-1/+2
| | | | | | | | | | | | | | | | | | | r357614 added CTLFLAG_NEEDGIANT to make it easier to find nodes that are still not MPSAFE (or already are but aren’t properly marked). Use it in preparation for a general review of all nodes. This is non-functional change that adds annotations to SYSCTL_NODE and SYSCTL_PROC nodes using one of the soon-to-be-required flags. Mark all obvious cases as MPSAFE. All entries that haven't been marked as MPSAFE before are by default marked as NEEDGIANT Approved by: kib (mentor, blanket) Commented by: kib, gallatin, melifaro Differential Revision: https://reviews.freebsd.org/D23718 Notes: svn path=/head/; revision=358333
* Use NET_TASK_INIT() and NET_GROUPTASK_INIT() for drivers that processGleb Smirnoff2020-02-111-3/+4
| | | | | | | | | | incoming packets in taskqueue context. Reviewed by: hselasky Differential Revision: https://reviews.freebsd.org/D23518 Notes: svn path=/head/; revision=357772
* netmap: improve netmap(4) and vale(4) man pagesVincenzo Maffione2020-02-071-2/+0
| | | | | | | | | | | | Clean up obsolete sysctl descriptions and add missing ones. PR: 243838 Reviewed by: bcr MFC after: 1 week Differential Revision: https://reviews.freebsd.org/D23546 Notes: svn path=/head/; revision=357663
* netmap_mem_unmap: fix NULL pointer dereferenceVincenzo Maffione2020-01-261-1/+2
| | | | | | | MFC after: 3 days Notes: svn path=/head/; revision=357159
* In netmap() call ether_input() within the network epoch.Gleb Smirnoff2020-01-231-0/+5
| | | | Notes: svn path=/head/; revision=357008
* netmap: disable passthrough with no hypervisor supportVincenzo Maffione2020-01-132-1/+3
| | | | | | | | | | | | | | | The netmap passthrough subsystem requires proper support in the hypervisor. In particular, two PCI device ids (from the Red Hat PCI vendor id 0x1b36) need to be assigned to the two netmap virtual devices. We then disable these devices until the ids have not been assigned, in order to avoid conflicts with other virtual devices emulated by upstream QEMU. PR: 241774 MFC after: 3 days Notes: svn path=/head/; revision=356704
* Make page busy state deterministic on free. Pages must be xbusy whenJeff Roberson2019-12-221-4/+1
| | | | | | | | | | | | | | | | | | | removed from objects including calls to free. Pages must not be xbusy when freed and not on an object. Strengthen assertions to match these expectations. In practice very little code had to change busy handling to meet these rules but we can now make stronger guarantees to busy holders and avoid conditionally dropping busy in free. Refine vm_page_remove() and vm_page_replace() semantics now that we have stronger guarantees about busy state. This removes redundant and potentially problematic code that has proliferated. Discussed with: markj Reviewed by: kib Differential Revision: https://reviews.freebsd.org/D22822 Notes: svn path=/head/; revision=356002
* add valectl to the system commandsVincenzo Maffione2019-10-311-1/+1
| | | | | | | | | | | | | | The valectl(4) program is used to manage vale(4) switches. Add it to the system commands so that it can be used right away. This program was previously called vale-ctl, and stored in tools/tools/netmap Reviewed by: hrs, bcr, lwhsu, kevans MFC after: 1 week Differential Revision: https://reviews.freebsd.org/D22146 Notes: svn path=/head/; revision=354229
* netmap: enter NET_EPOCH on generic txsyncVincenzo Maffione2019-10-281-0/+9
| | | | | | | | | | | | | | | | After r353292, netmap generic adapter on if_vlan interfaces panics on asserting the NET_EPOCH. In more detail, this happens when nm_os_generic_xmit_frame() is called, that is in the generic txsync routine. Fix the issue by entering the NET_EPOCH during the generic txsync. We amortize the cost of entering/exiting over a whole batch of transmissions. PR: 241489 Reported by: Aleksandr Fedorov <aleksandr.fedorov@itglobal.com> Notes: svn path=/head/; revision=354137
* netmap: minor misc improvementsVincenzo Maffione2019-10-203-4/+5
| | | | | | | | | | | - use ring->head rather than ring->cur in lb(8) - use strlcat() rather than strncat() - fix bandwidth computation in pkt-gen(8) MFC after: 1 week Notes: svn path=/head/; revision=353775
* tap: add support for virtio-net offloadsVincenzo Maffione2019-10-181-304/+6
| | | | | | | | | | | | | | | | | | | | | | | | This patch is part of an effort to make bhyve networking (in particular TCP) faster. The key strategy to enhance TCP throughput is to let the whole packet datapath work with TSO/LRO packets (up to 64KB each), so that the per-packet overhead is amortized over a large number of bytes. This capability is supported in the guest by means of the vtnet(4) driver, which is able to handle TSO/LRO packets leveraging the virtio-net header (see struct virtio_net_hdr and struct virtio_net_hdr_mrg_rxbuf). A bhyve VM exchanges packets with the host through a network backend, which can be vale(4) or if_tap(4). While vale(4) supports TSO/LRO packets, if_tap(4) does not. This patch extends if_tap(4) with the ability to understand the virtio-net header, so that a tapX interface can process TSO/LRO packets. A couple of ioctl commands have been added to configure and probe the virtio-net header. Once the virtio-net header is set, the tapX interface acquires all the IFCAP capabilities necessary for TSO/LRO. Reviewed by: kevans Differential Revision: https://reviews.freebsd.org/D21263 Notes: svn path=/head/; revision=353741
* (4/6) Protect page valid with the busy lock.Jeff Roberson2019-10-151-1/+1
| | | | | | | | | | | | | | Atomics are used for page busy and valid state when the shared busy is held. The details of the locking protocol and valid and dirty synchronization are in the updated vm_page.h comments. Reviewed by: kib, markj Tested by: pho Sponsored by: Netflix, Intel Differential Revision: https://reviews.freebsd.org/D21594 Notes: svn path=/head/; revision=353539
* Change synchonization rules for vm_page reference counting.Mark Johnston2019-09-091-2/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There are several mechanisms by which a vm_page reference is held, preventing the page from being freed back to the page allocator. In particular, holding the page's object lock is sufficient to prevent the page from being freed; holding the busy lock or a wiring is sufficent as well. These references are protected by the page lock, which must therefore be acquired for many per-page operations. This results in false sharing since the page locks are external to the vm_page structures themselves and each lock protects multiple structures. Transition to using an atomically updated per-page reference counter. The object's reference is counted using a flag bit in the counter. A second flag bit is used to atomically block new references via pmap_extract_and_hold() while removing managed mappings of a page. Thus, the reference count of a page is guaranteed not to increase if the page is unbusied, unmapped, and the object's write lock is held. As a consequence of this, the page lock no longer protects a page's identity; operations which move pages between objects are now synchronized solely by the objects' locks. The vm_page_wire() and vm_page_unwire() KPIs are changed. The former requires that either the object lock or the busy lock is held. The latter no longer has a return value and may free the page if it releases the last reference to that page. vm_page_unwire_noq() behaves the same as before; the caller is responsible for checking its return value and freeing or enqueuing the page as appropriate. vm_page_wire_mapped() is introduced for use in pmap_extract_and_hold(). It fails if the page is concurrently being unmapped, typically triggering a fallback to the fault handler. vm_page_wire() no longer requires the page lock and vm_page_unwire() now internally acquires the page lock when releasing the last wiring of a page (since the page lock still protects a page's queue state). In particular, synchronization details are no longer leaked into the caller. The change excises the page lock from several frequently executed code paths. In particular, vm_object_terminate() no longer bounces between page locks as it releases an object's pages, and direct I/O and sendfile(SF_NOCACHE) completions no longer require the page lock. In these latter cases we now get linear scalability in the common scenario where different threads are operating on different files. __FreeBSD_version is bumped. The DRM ports have been updated to accomodate the KPI changes. Reviewed by: jeff (earlier version) Tested by: gallatin (earlier version), pho Sponsored by: Netflix Differential Revision: https://reviews.freebsd.org/D20486 Notes: svn path=/head/; revision=352110
* netmap: import changes from upstream (SHA 137f537eae513)Vincenzo Maffione2019-09-014-73/+147
| | | | | | | | | | | - Rework option processing. - Use larger integers for memory size values in the memory management code. MFC after: 2 weeks Notes: svn path=/head/; revision=351657
* netmap: remove obsolete fileVincenzo Maffione2019-08-251-1498/+0
| | | | | | | | | | | The netmap_pt.c module has become obsolete after the refactoring that added netmap_kloop.c. Remove it and unlink it from the build system. MFC after: 1 week Notes: svn path=/head/; revision=351488
* netmap: fix bug introduced by r349752Vincenzo Maffione2019-07-131-1/+2
| | | | | | | | | | | r349752 introduced a NULL pointer reference bug in the emulated netmap code. Reported by: lwhsu MFC after: 3 days Notes: svn path=/head/; revision=349966
* netmap: Remove pointer leakage in netmap_mem2.cVincenzo Maffione2019-07-041-2/+2
| | | | | | | | | | PR: 238641 Submitted by: Fuqian Huang <huangfq.daxian@gmail.com> Reviewed by: vmaffione MFC after: 1 week Notes: svn path=/head/; revision=349753
* netmap: fix kernel pointer printing in netmap_generic.cVincenzo Maffione2019-07-041-2/+2
| | | | | | | | | | | | | Print the adapter name rather than the address of the adapter to avoid kernel address leakage. PR: Bug 238642 Submitted by: Fuqian Huang <huangfq.daxian@gmail.com> Reviewed by: vmaffione MFC after: 1 week Notes: svn path=/head/; revision=349752
* netmap: fix two panics with emulated adapterVincenzo Maffione2019-07-011-0/+3
| | | | | | | | | | | | | | | This patch fixes 2 panics. The first one is due to the current VNET not being set in the emulated adapter transmission path. The second one is caused by the M_PKTHDR flag not being set when preallocated mbufs are recycled in the transmit path. Submitted by: aleksandr.fedorov@itglobal.com Reviewed by: vmaffione MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D20824 Notes: svn path=/head/; revision=349581
* Add two missing eventhandler.h headersConrad Meyer2019-05-211-0/+1
| | | | | | | | | | | | | These are obviously missing from the .c files, but don't show up in any tinderbox configuration (due to latent header pollution of some kind). It seems some configurations don't have this pollution, and the includes are obviously missing, so go ahead and add them. Reported by: Peter Jeremy <peter AT rulingia.com> X-MFC-With: r347984 Notes: svn path=/head/; revision=348022
* netmap: align if_ptnet to the changes introduced by r347233Vincenzo Maffione2019-05-171-12/+2
| | | | | | | | | | This removes non-functional SCTP checksum offload support. More information in the log message of r347233. MFC after: 2 weeks Notes: svn path=/head/; revision=347944
* netmap: add support for multiple host ringsVincenzo Maffione2019-03-183-2/+52
| | | | | | | | | | | | | | | | Some applications forward from/to host rings most or all the traffic received or sent on a physical interface. In this cases it is desirable to have more than a pair of RX/TX host rings, and use multiple threads to speed up forwarding. This change adds support for multiple host rings. On registering a netmap port, the user can specify the number of desired receive and transmit host rings in the nr_host_tx_rings and nr_host_rx_rings fields of the nmreq_register structure. MFC after: 2 weeks Notes: svn path=/head/; revision=345269
* netmap: remove redundant call to nm_set_native_flags()Vincenzo Maffione2019-02-251-1/+0
| | | | | | | | | | This redundant call was introduced by mistake in r343772. MFC after: 3 days Sponsored by: Sunny Valley Networks Notes: svn path=/head/; revision=344510
generated by cgit v1.2.3 (git 2.25.1) at 2026-02-28 21:41:52 +0000