aboutsummaryrefslogtreecommitdiff
path: root/sys/cam/ctl/ctl_tpc.c
Commit message (Collapse)AuthorAgeFilesLines
* Remove pointless lun->be_lun checks.Alexander Motin2021-02-261-3/+3
| | | | | | There is no such thing as LUN without backend, at least for years. MFC after: 1 week
* Save context switch per I/O for iSCSI and IOCTL frontends.Alexander Motin2021-02-191-1/+1
| | | | | | | | | | | | | | | Introduce new CTL core KPI ctl_run(), preprocessing I/Os in the caller context instead of scheduling another thread just for that. This call may sleep, that is not acceptable for some frontends like the original CAM/FC one, but iSCSI already has separate sleepable per-connection RX threads, and another thread scheduling is mostly just a waste of time. IOCTL frontend actually waits for the I/O completion in the caller thread, so the use of another thread for this has even less sense. With this change I can measure ~5% IOPS improvement on 4KB iSCSI I/Os to ZFS. MFC after: 1 month
* cam: clean up empty lines in .c and .h filesMateusz Guzik2020-09-011-1/+0
| | | | Notes: svn path=/head/; revision=365225
* Rework CTL frontend & backend options to use nv(3), allow creating multipleMarcelo Araujo2018-05-101-4/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ioctl frontend ports. This revision introduces two changes to CTL: - Changes the way options are passed to CTL_LUN_REQ and CTL_PORT_REQ ioctls. Removes ctl_be_arg structure and associated logic and replaces it with nv(3)-based logic for passing in and out arguments. - Allows creating multiple ioctl frontend ports using either ctladm(8) or ctld(8). New frontend ports are represented by /dev/cam/ctl<pp>.<vp> nodes, eg /dev/cam/ctl5.3. Those device nodes respond only to CTL_IO ioctl. New command-line options for ctladm: # creates new ioctl frontend port with using free pp and vp=0 ctladm port -c # creates new ioctl frontend port with pp=10 and vp=0 ctladm port -c -O pp=10 # creates new ioctl frontend port with pp=11 and vp=12 ctladm port -c -O pp=11 -O vp=12 # removes port with number 4 (it's a "targ_port" number, not pp number) ctladm port -r -p 4 New syntax for ctl.conf: target ... { port ioctl/<pp> ... } target ... { port ioctl/<pp>/<vp> ... Note: Most of this work was made by jceel@, thank you. Submitted by: jceel Reworked by: myself Reviewed by: mav (earlier versions and recently during the rework) Obtained from: FreeNAS and TrueOS Relnotes: Yes Sponsored by: iXsystems Inc. Differential Revision: https://reviews.freebsd.org/D9299 Notes: svn path=/head/; revision=333446
* SPDX: finish tagging sys/cam.Pedro F. Giffuni2018-01-161-0/+2
| | | | Notes: svn path=/head/; revision=328070
* Polish handling of different reset flavours.Alexander Motin2017-02-271-0/+15
| | | | | | | | | | The biggest change is that ctl_remove_initiator() now generates I_T NEXUS LOSS event, cleaning part of LUs state related to the initiator. MFC after: 2 weeks Notes: svn path=/head/; revision=314338
* Change XCOPY memory allocations.Alexander Motin2017-02-181-13/+12
| | | | | | | | | | | | | | | | | Before this change XCOPY code could allocate memory in chunks up to 16-32MB (VMware does XCOPY in 4MB chunks by default), that could be difficult for VM subsystem to do due to KVA fragmentation, that sometimes created huge allocation delays, blocking any I/O for respective LU for that time. This change limits allocations down to TPC_MAX_IO_SIZE, which is 1MB now. 1MB is also not a cookie, but ZFS also can do that for large blocks, so it should be less dramatic. As drawback this increases CPU overhead, but it still look acceptable comparing to time consumed by ZFS read/write. MFC after: 1 week Notes: svn path=/head/; revision=313910
* Remove writing 'residual' field of struct ctl_scsiio.Alexander Motin2017-01-171-77/+14
| | | | | | | | | | | | | | | This field has no practical use and never readed. Initiators already receive respective residual size from frontends. Removed field had different semantics, which looks useless, and was never passed through by any frontend. While there, fix kern_data_resid field support in case of HA, missed in r312291. MFC after: 13 days Notes: svn path=/head/; revision=312348
* Make CTL frontends report kern_data_resid for under-/overruns.Alexander Motin2017-01-161-11/+0
| | | | | | | | | | | | | | | | | It seems like kern_data_resid was never really implemented. This change finally does it. Now frontends update this field while transferring data, while CTL/backends getting it can more flexibly handle the result. At this point behavior should not change significantly, still reporting errors on write overrun, but that may be changed later, if we decide so. CAM target frontend still does not properly handle overruns due to CAM API limitations. We may need to add some fields to struct ccb_accept_tio to pass information about initiator requested transfer size(s). MFC after: 2 weeks Notes: svn path=/head/; revision=312291
* Improve use of I/O's private area.Alexander Motin2016-12-291-37/+14
| | | | | | | | | | | - Since I/Os are allocates from per-port pools, make allocations store pointer to CTL softc there, and use it where needed instead of global. - Created bunch of helper macros to access LUN, port and CTL softc. MFC after: 2 weeks Notes: svn path=/head/; revision=310778
* Remove CTL_MAX_LUNS from places where it is not required.Alexander Motin2016-12-251-3/+3
| | | | | | | MFC after: 2 weeks Notes: svn path=/head/; revision=310539
* Improve third-party copy error reporting.Alexander Motin2016-12-251-39/+155
| | | | | | | | | | | | | | | For EXTENDED COPY: - improve parameters checking to report some errors before copy start; - forward sense data from copy target as descriptor in case of error; - report which CSCD reported error in sense key specific information. For WRITE USING TOKEN: - pass through real sense data from copy target instead of reporting our copy error, since for initiator its a "simple" write, not a copy. MFC after: 2 weeks Notes: svn path=/head/; revision=310534
* When reporting "Logical block address out of range" error, report the LBAAlexander Motin2016-12-191-6/+11
| | | | | | | | | in sense data INFORMATION field. MFC after: 2 weeks Notes: svn path=/head/; revision=310285
* Fix previous commit to report proper error code.Alexander Motin2016-05-101-6/+28
| | | | | | | MFC after: 2 weeks Notes: svn path=/head/; revision=299348
* Validate XCOPY range offsets and lengths.Alexander Motin2016-05-101-3/+11
| | | | | | | MFC after: 2 weeks Notes: svn path=/head/; revision=299347
* More XCOPY parameters validation.Alexander Motin2016-05-101-5/+55
| | | | | | | MFC after: 2 weeks Notes: svn path=/head/; revision=299346
* Improve validation of some POPULATE TOKEN parameters.Alexander Motin2016-05-101-8/+22
| | | | | | | MFC after: 2 weeks Notes: svn path=/head/; revision=299329
* More aggressively fill WUT read pipeline.Alexander Motin2015-10-011-5/+4
| | | | | | | On some tests I've measured 5% copy speedup from this. Notes: svn path=/head/; revision=288458
* Make zero WUT use WRITE SAME with recently allowed NDOB flag.Alexander Motin2015-10-011-5/+3
| | | | Notes: svn path=/head/; revision=288450
* Fix arguments order.Alexander Motin2015-09-291-8/+8
| | | | Notes: svn path=/head/; revision=288367
* Report number of failed XCOPY segment.Alexander Motin2015-09-171-8/+32
| | | | Notes: svn path=/head/; revision=287913
* Improve XCOPY error reporting.Alexander Motin2015-09-121-16/+28
| | | | Notes: svn path=/head/; revision=287715
* Report that we have no limit on POPULATE TOKEN segment size.Alexander Motin2015-09-121-2/+1
| | | | Notes: svn path=/head/; revision=287714
* Reimplement CTL High Availability.Alexander Motin2015-09-101-10/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | CTL HA functionality was originally implemented by Copan many years ago, but large part of the sources was never published. This change includes clean room implementation of the missing code and fixes for many bugs. This code supports dual-node HA with ALUA in four modes: - Active/Unavailable without interlink between nodes; - Active/Standby with second node handling only basic LUN discovery and reservation, synchronizing with the first node through the interlink; - Active/Active with both nodes processing commands and accessing the backing storage, synchronizing with the first node through the interlink; - Active/Active with second node working as proxy, transfering all commands to the first node for execution through the interlink. Unlike original Copan's implementation, depending on specific hardware, this code uses simple custom TCP-based protocol for interlink. It has no authentication, so it should never be enabled on public interfaces. The code may still need some polishing, but generally it is functional. Relnotes: yes Sponsored by: iXsystems, Inc. Notes: svn path=/head/; revision=287621
* Drop "internal" CTL frontend.Alexander Motin2015-08-151-1/+0
| | | | | | | | | Its idea was to be a simple initiator and execute several commands from kernel level, but FreeBSD never had consumer for that functionality, while its implementation polluted many unrelated places.. Notes: svn path=/head/; revision=286806
* Issue all reads of single XCOPY segment simultaneously.Alexander Motin2015-08-051-5/+4
| | | | | | | | | | | | | | | | | | | | During vMotion and Clone VMware by default runs multiple sequential 4MB XCOPY requests same time. If CTL issues reads sequentially in 1MB chunks for each XCOPY command, reads from different commands are not detected as sequential by serseq option code and allowed to execute simultaneously. Such read pattern confused ZFS prefetcher, causing suboptimal disk access. Issuing all reads same time make serseq code work properly, serializing reads both within each XCOPY command and between them. My tests with ZFS pool of 14 disks in RAID10 shows prefetcher efficiency improved from 37% to 99.7%, copying speed improved by 10-60%, average read latency reduced twice on HDD layer and by five times on zvol layer. MFC after: 2 weeks Sponsored by: iXsystems, Inc. Notes: svn path=/head/; revision=286320
* Introduce separate lock for tokens to reduce ctl_lock scope.Alexander Motin2015-06-201-13/+18
| | | | Notes: svn path=/head/; revision=284639
* Make XCOPY and WUT commands respect physical block size/offset.Alexander Motin2015-02-121-19/+63
| | | | | | | | | | This change by 2-3 times improves performance of misaligned XCOPY and WUT commands by avoiding unneeded read-modify-write cycles inside ZFS. MFC after: 1 week Notes: svn path=/head/; revision=278625
* Fix wrong LUN reference in XCOPY block-to-block operation.Alexander Motin2015-01-241-1/+1
| | | | | | | | | | This could cause data corruption due to accessing wrong LUN in case of retries on write errors. Failed writes were retried to read LUN. MFC after: 3 days Notes: svn path=/head/; revision=277647
* Reduce number of places where global control_softc is used.Alexander Motin2014-12-191-21/+28
| | | | | | | | | | At some point we may want to have several CTL instances, and that is not really impossible. MFC after: 2 weeks Notes: svn path=/head/; revision=275942
* Plug memory leaks on UNMAP and XCOPY with invalid parameters.Alexander Motin2014-12-031-2/+14
| | | | | | | MFC after: 1 week Notes: svn path=/head/; revision=275446
* Coalesce last data move and command status for read commands.Alexander Motin2014-11-251-9/+7
| | | | | | | | | | | | | | | | | | Make CTL core and block backend set success status before initiating last data move for read commands. Make CAM target and iSCSI frontends detect such condition and send command status together with data. New I/O flag allows to skip duplicate status sending on later fe_done() call. For Fibre Channel this change saves one of three interrupts per read command, increasing performance from 126K to 160K IOPS. For iSCSI this change saves one of three PDUs per read command, increasing performance from 1M to 1.2M IOPS. MFC after: 1 month Sponsored by: iXsystems, Inc. Notes: svn path=/head/; revision=275058
* Replace home-grown CTL IO allocator with UMA.Alexander Motin2014-11-241-32/+0
| | | | | | | | | | | | | | | | | | | | | Old allocator created significant lock congestion protecting its lists of preallocated I/Os, while UMA provides much better SMP scalability. The downside of UMA is lack of reliable preallocation, that could guarantee successful allocation in non-sleepable environments. But careful code review shown, that only CAM target frontend really has that requirement. Fix that making that frontend preallocate and statically bind CTL I/O for every ATIO/INOT it preallocates any way. That allows to avoid allocations in hot I/O path. Other frontends either may sleep in allocation context or can properly handle allocation errors. On 40-core server with 6 ZVOL-backed LUNs and 7 iSCSI client connections this change increases peak performance from ~700K to >1M IOPS! Yay! :) MFC after: 1 month Sponsored by: iXsystems, Inc. Notes: svn path=/head/; revision=274962
* Fix couple issues with ROD tokens content.Alexander Motin2014-10-011-3/+14
| | | | | | | MFC after: 3 days Notes: svn path=/head/; revision=272355
* Fix tpc_create_token() introduced in r269497 to encode CREATOR LOGICAL UNITAlexander Motin2014-09-171-2/+7
| | | | | | | | | | DESCRIPTOR field as Identification Descriptor CSCD descriptor, not just as Identification Descriptor. MFC after: 3 days Notes: svn path=/head/; revision=271702
* Fix lock recursion on LUN shutdown, introduced on r269497.Alexander Motin2014-08-191-2/+1
| | | | | | | MFC after: 3 days Notes: svn path=/head/; revision=270176
* Reimplement WRITE USING TOKEN with Block Zero token using WRITE SAME.Alexander Motin2014-08-051-27/+93
| | | | | | | | | On my ZVOL of SSDs that increases speed of zero writing in that way from 1 to 2.5GB/s by reducing CPU overhead. MFC after: 2 weeks Notes: svn path=/head/; revision=269587
* Add support for Windows dialect of EXTENDED COPY command, aka Microsoft ODX.Alexander Motin2014-08-041-32/+863
| | | | | | | | | | | | This allows to avoid extra network traffic when copying files on NTFS iSCSI disks within one storage host by drag'n'dropping them in Windows Explorer of Windows 8/2012. It should also accelerate Hyper-V VM operations, etc. MFC after: 2 weeks Sponsored by: iXsystems, Inc. Notes: svn path=/head/; revision=269497
* Rework r269444 to work also for lists without IDs.Alexander Motin2014-08-021-2/+2
| | | | | | | MFC after: 3 days Notes: svn path=/head/; revision=269450
* Plug EXTENDED COPY request data memory leak.Alexander Motin2014-08-021-0/+2
| | | | | | | MFC after: 3 days Notes: svn path=/head/; revision=269444
* Fix some bugs in RECEIVE COPY STATUS data.Alexander Motin2014-08-021-9/+7
| | | | | | | MFC after: 3 days Notes: svn path=/head/; revision=269442
* Add missing comparisons to make list IDs in EXTENDED COPY per-initiator,Alexander Motin2014-08-021-33/+26
| | | | | | | | | as they should be. Wrap it into a function to not duplicate the code. MFC after: 3 days Notes: svn path=/head/; revision=269441
* Increase maximal number of SCSI ports in CTL from 32 to 128.Alexander Motin2014-07-171-1/+1
| | | | | | | | | | | After I gave each iSCSI target its own port, the old limit appeared to be not so big. This change almost proportionally increases per-LUN memory use, but it is still three times better then it was before r268807. MFC after: 2 weeks Notes: svn path=/head/; revision=268808
* Add support for VMWare dialect of EXTENDED COPY command, aka VAAI Clone.Alexander Motin2014-07-161-0/+1370
This allows to clone VMs and move them between LUNs inside one storage host without generating extra network traffic to the initiator and back, and without being limited by network bandwidth. LUNs participating in copy operation should have UNIQUE NAA or EUI IDs set. For LUNs without these IDs VMWare will use traditional copy operations. Beware: the above LUN IDs explicitly set to values non-unique from the VM cluster point of view may cause data corruption if wrong LUN is addressed! MFC after: 2 weeks Sponsored by: iXsystems, Inc. Notes: svn path=/head/; revision=268767