aboutsummaryrefslogtreecommitdiff
path: root/cddl
Commit message (Collapse)AuthorAgeFilesLines
* Remove local variables missed in r321842.Mark Johnston2017-08-011-4/+0
| | | | | | | X-MFC with: r321842 Notes: svn path=/head/; revision=321843
* Let lockstat use ksyms(4)'s mmap interface.Mark Johnston2017-08-011-24/+0
| | | | | | | | | The workaround described in the deleted comment is no longer needed. MFC after: 1 week Notes: svn path=/head/; revision=321842
* Add an auxiliary subroutine to generate some events for testingLi-Wen Hsu2017-07-261-0/+14
| | | | | | | | | | | This test is also timeout on a quiet system because there is nobody triggering read probefunc while test execution. Reviewed by: gnn, markj, ngie Differential Revision: https://reviews.freebsd.org/D11731 Notes: svn path=/head/; revision=321518
* The test case common.funcs.t_dtrace_contrib.tst_basename_d generates aLi-Wen Hsu2017-07-251-0/+8
| | | | | | | | | | | | verifying script which needs being run to complete the test. While here, add missing shebang. Reviewed by: gnn, markj, ngie Differential Revision: https://reviews.freebsd.org/D11716 Notes: svn path=/head/; revision=321472
* Modify glob patterns and expected output to match FreeBSD's implementation.Li-Wen Hsu2017-07-252-7/+15
| | | | | | | | Reviewed by: gnn, markj, ngie Differential Revision: https://reviews.freebsd.org/D11713 Notes: svn path=/head/; revision=321470
* Make this test case accepts basename() in D script returns "" or "."Li-Wen Hsu2017-07-252-4/+22
| | | | | | | | | | | | | | | In Solaris, basename(1) and basename(3) both return "." while being given an empty string (""), while in BSD (and Linux) basename(1) returns "" and basename(3) returns "." While here, also change #!/usr/bin/ksh to #!/usr/bin/env ksh to find ksh in $PATH Reviewed by: gnn, markj (earlier version), ngie (earlier version) Differential Revision: https://reviews.freebsd.org/D11707 Notes: svn path=/head/; revision=321469
* Explicitly set dynamic variable buffer size.Li-Wen Hsu2017-07-251-0/+1
| | | | | | | | | | | | | We added too many variable assignments in BEGIN block, which will run out of default auto-configured variable buffer space. The test VM has 4G RAM which should be enough for most cases so it's reasonable to increase limitation to these case. Reviewed by: gnn Differential Revision: https://reviews.freebsd.org/D11676 Notes: svn path=/head/; revision=321468
* Explicitly set dynamic variable buffer size.Li-Wen Hsu2017-07-251-0/+1
| | | | | | | | | | | | | We added too many variable assignments in BEGIN block, which will run out of default auto-configured variable buffer space. The test VM has 4G RAM which should be enough for most cases so it's reasonable to increase limitation to these case. Reviewed by: gnn, markj, ngie Differential Revision: https://reviews.freebsd.org/D11674 Notes: svn path=/head/; revision=321467
* Add an auxiliary subroutine to generate read(2) event while testing.Li-Wen Hsu2017-07-251-0/+13
| | | | | | | | Reviewed by: gnn, ngie Differential Revision: https://reviews.freebsd.org/D11673 Notes: svn path=/head/; revision=321466
* Add a simple script which calls open(2) and others to generate events forLi-Wen Hsu2017-07-253-0/+32
| | | | | | | | | | | | | testing. This test times-out on a quiet system because there is nobody triggers syscall::open:entry or syscall::: probe while test execution. Reviewed by: gnn, markj (earlier version) Differential Revision: https://reviews.freebsd.org/D11671 Notes: svn path=/head/; revision=321465
* Add a simple program which calls sigtimedwait(2) to generate events for testingLi-Wen Hsu2017-07-253-0/+49
| | | | | | | | | | | This test timeout on a quiet system because there is nobody triggers 'syscall::*wait*:entry' probe while test execution. Reviewed by: gnn, markj, ngie Differential Revision: https://reviews.freebsd.org/D11668 Notes: svn path=/head/; revision=321464
* Fix whitespace on a line in fix(..) accidentally missed in r321424Enji Cooper2017-07-241-1/+1
| | | | | | | | MFC after: 1 month MFC with: r321424 Notes: svn path=/head/; revision=321425
* Style cleanup: delete spurious trailing whitespaceEnji Cooper2017-07-241-4/+4
| | | | | | | MFC after: 1 month Notes: svn path=/head/; revision=321424
* Don't use incorrect hardcoded path to ksh -- use /usr/bin/envEnji Cooper2017-07-231-1/+1
| | | | | | | | | to find ksh instead MFC after: 1 month Notes: svn path=/head/; revision=321387
* zfsd(8): Remove pidfile on shutdownAlan Somers2017-06-201-1/+1
| | | | | | | | MFC after: 3 weeks Sponsored by: Spectra Logic Corp Notes: svn path=/head/; revision=320167
* MFV r318946: 8021 ARC buf data scatter-izationAndriy Gapon2017-06-204-49/+79
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | illumos/illumos-gate@770499e185d15678ccb0be57ebc626ad18d93383 https://github.com/illumos/illumos-gate/commit/770499e185d15678ccb0be57ebc626ad18d93383 https://www.illumos.org/issues/8021 The ARC buf data project (known simply as "ABD" since its genesis in the ZoL community) changes the way the ARC allocates `b_pdata` memory from using linear `void *` buffers to using scatter/gather lists of fixed-size 1KB chunks. This improves ZFS's performance by helping to defragment the address space occupied by the ARC, in particular for cases where compressed ARC is enabled. It could also ease future work to allocate pages directly from `segkpm` for minimal- overhead memory allocations, bypassing the `kmem` subsystem. This is essentially the same change as the one which recently landed in ZFS on Linux, although they made some platform-specific changes while adapting this work to their codebase: 1. Implemented the equivalent of the `segkpm` suggestion for future work mentioned above to bypass issues that they've had with the Linux kernel memory allocator. 2. Changed the internal representation of the ABD's scatter/gather list so it could be used to pass I/O directly into Linux block device drivers. (This feature is not available in the illumos block device interface yet.) FreeBSD notes: - the actual (default) chunk size is 4KB (despite the text above saying 1KB) - we can try to reimplement ABDs, so that they are not permanently mapped into the KVA unless explicitly requested, especially on platforms with scarce KVA - we can try to use unmapped I/O and avoid intermediate allocation of a linear, virtual memory mapped buffer - we can try to avoid extra data copying by referring to chunks / pages in the original ABD Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: George Wilson <george.wilson@delphix.com> Reviewed by: Paul Dagnelie <pcd@delphix.com> Reviewed by: John Kennedy <john.kennedy@delphix.com> Reviewed by: Prakash Surya <prakash.surya@delphix.com> Reviewed by: Prashanth Sreenivasa <pks@delphix.com> Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com> Reviewed by: Chris Williamson <chris.williamson@delphix.com> Approved by: Richard Lowe <richlowe@richlowe.net> Author: Dan Kimmel <dan.kimmel@delphix.com> MFC after: 3 weeks Notes: svn path=/head/; revision=320156
* Utilize SYSROOT from r320119 in places where DESTDIR may be wanting WORLDTMP.Bryan Drewery2017-06-191-10/+4
| | | | | | | | | | | | | | Since buildenv exports SYSROOT all of these uses will now look in WORLDTMP by default. sys/boot/efi/loader/Makefile A LIBSTAND hack is no longer required for buildenv. MFC after: 2 weeks Sponsored by: Dell EMC Isilon Notes: svn path=/head/; revision=320122
* MFV r319945,r319946: 8264 want support for promoting datasets in libzfs_coreAndriy Gapon2017-06-143-14/+33
| | | | | | | | | | | | | | | | | | illumos/illumos-gate@a4b8c9aa65a0a735aba318024a424a90d7b06c37 https://github.com/illumos/illumos-gate/commit/a4b8c9aa65a0a735aba318024a424a90d7b06c37 https://www.illumos.org/issues/8264 Oddly there is a lzc_clone function, but no lzc_promote function. Reviewed by: Andriy Gapon <avg@FreeBSD.org> Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Dan McDonald <danmcd@kebe.com> Approved by: Dan McDonald <danmcd@kebe.com> Author: Andrew Stormont <astormont@racktopsystems.com> MFC after: 1 week Notes: svn path=/head/; revision=319947
* Override the locale so that file lists get a consistent sort order.Mark Johnston2017-06-101-0/+2
| | | | | | | | Reported by: avg MFC after: 1 week Notes: svn path=/head/; revision=319792
* follow up to r319746: add the new test files to the make fileAndriy Gapon2017-06-101-0/+2
| | | | | | | | | Reported by: markj MFC after: 2 days X-MFC with: r319746 Notes: svn path=/head/; revision=319769
* MFV r319740: 8168 NULL pointer dereference in zfs_create()Andriy Gapon2017-06-091-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | illumos/illumos-gate@690031d326342fa4ea28b5e80f1ad6a16281519d https://github.com/illumos/illumos-gate/commit/690031d326342fa4ea28b5e80f1ad6a16281519d https://www.illumos.org/issues/8168 If we manage to export the pool on which we are creating a dataset (filesystem or zvol) between entering libzfs`zfs_create() and libzfs`zpool_open() call (for which we never check the return value) we end up dereferencing a NULL pointer in libzfs`zpool_close(). This was discovered on ZFS on Linux. The same issue can be reproduced on Illumos running in parallel: while :; do zpool import -d /tmp testpool ; zpool export testpool ; done while :; do zfs create testpool/fs; zfs destroy testpool/fs ; done Eventually this will result in several core dumps like this one: [root@52-54-00-d3-7a-01 /cores]# mdb core.zfs.4244 Loading modules: [ libumem.so.1 libc.so.1 libtopo.so.1 libavl.so.1 libnvpair.so.1 ld.so.1 ] > ::stack libzfs.so.1`zpool_close+0x17(0, 0, 0, 8047450) libzfs.so.1`zfs_create+0x1bb(8090548, 8047e6f, 1, 808cba8) zfs_do_create+0x545(2, 8047d74, 80778a0, 801, 0, 3) main+0x22c(8047d2c, fef5c6e8, 8047d64, 8055a17, 3, 8047d70) _start+0x83(3, 8047e64, 8047e68, 8047e6f, 0, 8047e7b) > Fix and reproducer (systemtap): https://github.com/zfsonlinux/zfs/pull/6096 Reviewed by: Matt Ahrens <mahrens@delphix.com> Approved by: Robert Mustacchi <rm@joyent.com> Author: loli10K <ezomori.nozomu@gmail.com> MFC after: 2 weeks Notes: svn path=/head/; revision=319751
* MFV r319744,r319745: 8269 dtrace stddev aggregation is normalized incorrectlyAndriy Gapon2017-06-092-0/+49
| | | | | | | | | | | | | | | | | | | | | | | | illumos/illumos-gate@79809f9cf402f130667349b2d4007ecd65d63c6f https://github.com/illumos/illumos-gate/commit/79809f9cf402f130667349b2d4007ecd65d63c6f https://www.illumos.org/issues/8269 It seems that currently normalization of stddev aggregation is done incorrectly. We divide both the sum of values and the sum of their squares by the normalization factor. But we should divide the sum of squares by the normalization factor squared to scale the original values properly. FreeBSD note: the actual change was committed in r316853, this commit adds the test files and record merge information. Reviewed by: Bryan Cantrill <bryan@joyent.com> Approved by: Robert Mustacchi <rm@joyent.com> Author: Andriy Gapon <avg@FreeBSD.org> MFC after: 1 week Sponsored by: Panzura Notes: svn path=/head/; revision=319746
* New sentences start on new lines, fix two violationsAllan Jude2017-06-081-4/+8
| | | | | | | | Reviewed by: bcr Sponsored by: BSDCan Dev Summit Notes: svn path=/head/; revision=319672
* SHA-512 and Skein have been supported by the boot loader for some time.Allan Jude2017-06-081-11/+1
| | | | | | | | | Submitted by: lifanov Reviewed by: bcr Sponsored by: BSDCan Dev Summit Notes: svn path=/head/; revision=319671
* MFV r316922: 5380 receive of a send -p stream doesn't need to try renaming ↵Andriy Gapon2017-05-241-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | snapshots illumos/illumos-gate@471a88e499c660844f4590487ce7c4d5a7090294 https://github.com/illumos/illumos-gate/commit/471a88e499c660844f4590487ce7c4d5a7090294 https://www.illumos.org/issues/5380 A stream created with zfs send -p -I contains properties of all snapshots of a given dataset as opposed to only properties of snapshots in a given range. Not only this is suboptimal but the receive code also does not filter properties by the range. So, properties of earlier snapshots would be updated even though the snapshots themselves are not in the stream (just their properties). Given that modifying the snapshot properties requires a TXG sync and that the snapshots are updated one by one the described behavior may lead to a sever performance penalty. Reviewed by: Paul Dagnelie <pcd@delphix.com> Reviewed by: Matt Ahrens <mahrens@delphix.com> Approved by: Dan McDonald <danmcd@omniti.com> Author: Andriy Gapon <avg@FreeBSD.org> MFC after: 3 weeks Notes: svn path=/head/; revision=318831
* MFC r316908: 7541 zpool import/tryimport ioctl returns ENOMEM because ↵Andriy Gapon2017-05-242-4/+7
| | | | | | | | | | | | | | | | | | | | | | | | | provided buffer is too small for config illumos/illumos-gate@8b65a70b763232c90a91f31eb2010314c02ed338 https://github.com/illumos/illumos-gate/commit/8b65a70b763232c90a91f31eb2010314c02ed338 https://www.illumos.org/issues/7541 When calling zpool import, zpool does a few ioctls to ZFS. zpool allocates a buffer in userland and passes it to the kernel so that ZFS can copy info into it. ZFS will use it to put the nvlist that describes the pool configuration. If the allocated buffer is too small, ZFS will return ENOMEM and the call will have to be redone. This wastes CPU time and slows down the import process. This happens very often for the ZFS_IOC_POOL_TRYIMPORT call. Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Dan Kimmel <dan.kimmel@delphix.com> Approved by: Dan McDonald <danmcd@omniti.com> Author: Pavel Zakharov <pavel.zakharov@delphix.com> MFC after: 2 weeks Notes: svn path=/head/; revision=318819
* MFC r316904: 7729 libzfs_core`lzc_rollback() leaks result nvlAndriy Gapon2017-05-241-0/+2
| | | | | | | | | | | | | | | | | | illumos/illumos-gate@ac428481f96be89add7a1edf43ae47dd71038553 https://github.com/illumos/illumos-gate/commit/ac428481f96be89add7a1edf43ae47dd71038553 https://www.illumos.org/issues/7729 libzfs_core`lzc_rollback() doesn't free the result nvl after lzc_ioctl() call. Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Prakash Surya <prakash.surya@delphix.com> Approved by: Dan McDonald <danmcd@omniti.com> Author: Yuri Pankov <yuri.pankov@nexenta.com> MFC after: 2 weeks Notes: svn path=/head/; revision=318814
* MFV r316860: 7545 zdb should disable reference trackingAndriy Gapon2017-05-241-0/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | illumos/illumos-gate@4dd77f9e38ef05b39db128ff7608d926fd3218c6 https://github.com/illumos/illumos-gate/commit/4dd77f9e38ef05b39db128ff7608d926fd3218c6 https://www.illumos.org/issues/7545 When evicting from the ARC, we manipulate some refcount_t's, e.g. arcs_size. When using zdb to examine a large amount of data (e.g. zdb -bb on a large pool with small blocks), the ARC may have a large number of entries. If reference tracking is enabled, there will be ~1 reference for each block in the ARC. When evicting, we decrement the refcount and have to search all the references to find the one that we are removing, which is very slow. Since zdb is typically used to find problems with the on-disk format, and not with the code it is running, we should disable reference tracking in zdb. Reviewed by: Dan Kimmel <dan.kimmel@delphix.com> Reviewed by: Steve Gonczi <steve.gonczi@delphix.com> Reviewed by: George Wilson <george.wilson@delphix.com> Approved by: Robert Mustacchi <rm@joyent.com> Author: Matthew Ahrens <mahrens@delphix.com> MFC after: 2 weeks Notes: svn path=/head/; revision=318812
* Commit the 64-bit inode project.Konstantin Belousov2017-05-231-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Extend the ino_t, dev_t, nlink_t types to 64-bit ints. Modify struct dirent layout to add d_off, increase the size of d_fileno to 64-bits, increase the size of d_namlen to 16-bits, and change the required alignment. Increase struct statfs f_mntfromname[] and f_mntonname[] array length MNAMELEN to 1024. ABI breakage is mitigated by providing compatibility using versioned symbols, ingenious use of the existing padding in structures, and by employing other tricks. Unfortunately, not everything can be fixed, especially outside the base system. For instance, third-party APIs which pass struct stat around are broken in backward and forward incompatible ways. Kinfo sysctl MIBs ABI is changed in backward-compatible way, but there is no general mechanism to handle other sysctl MIBS which return structures where the layout has changed. It was considered that the breakage is either in the management interfaces, where we usually allow ABI slip, or is not important. Struct xvnode changed layout, no compat shims are provided. For struct xtty, dev_t tty device member was reduced to uint32_t. It was decided that keeping ABI compat in this case is more useful than reporting 64-bit dev_t, for the sake of pstat. Update note: strictly follow the instructions in UPDATING. Build and install the new kernel with COMPAT_FREEBSD11 option enabled, then reboot, and only then install new world. Credits: The 64-bit inode project, also known as ino64, started life many years ago as a project by Gleb Kurtsou (gleb). Kirk McKusick (mckusick) then picked up and updated the patch, and acted as a flag-waver. Feedback, suggestions, and discussions were carried by Ed Maste (emaste), John Baldwin (jhb), Jilles Tjoelker (jilles), and Rick Macklem (rmacklem). Kris Moore (kris) performed an initial ports investigation followed by an exp-run by Antoine Brodin (antoine). Essential and all-embracing testing was done by Peter Holm (pho). The heavy lifting of coordinating all these efforts and bringing the project to completion were done by Konstantin Belousov (kib). Sponsored by: The FreeBSD Foundation (emaste, kib) Differential revision: https://reviews.freebsd.org/D10439 Notes: svn path=/head/; revision=318736
* Add a little helper program for tst.exitcore.ksh.Mark Johnston2017-05-223-9/+40
| | | | | | | | | | | sleep(1) is capsicumized, which means that we cannot rely on it to dump core as required by the test. MFC after: 1 week Sponsored by: Dell EMC Isilon Notes: svn path=/head/; revision=318680
* Fix time handling in cv_timedwait_hires().Alexander Motin2017-05-191-3/+8
| | | | | | | | | | pthread_cond_timedwait() receives absolute time, not relative. Passing wrong time there caused two threads of zdb to spin in a tight loop. MFC after: 1 week Notes: svn path=/head/; revision=318516
* Remove the EXFAIL annotation for tests which pass as of r309596.Mark Johnston2017-05-191-5/+0
| | | | | | | | Reported by: bdrewery Sponsored by: Dell EMC Isilon Notes: svn path=/head/; revision=318508
* MFV 316894Josh Paetzel2017-04-257-44/+140
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 7252 7628 compressed zfs send / receive illumos/illumos-gate@5602294fda888d923d57a78bafdaf48ae6223dea https://github.com/illumos/illumos-gate/commit/5602294fda888d923d57a78bafdaf48ae6223dea https://www.illumos.org/issues/7252 This feature includes code to allow a system with compressed ARC enabled to send data in its compressed form straight out of the ARC, and receive data in its compressed form directly into the ARC. https://www.illumos.org/issues/7628 We should have longer, more readable versions of the ZFS send / recv options. 7628 create long versions of ZFS send / receive options Reviewed by: George Wilson <george.wilson@delphix.com> Reviewed by: John Kennedy <john.kennedy@delphix.com> Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Paul Dagnelie <pcd@delphix.com> Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com> Reviewed by: Sebastien Roy <sebastien.roy@delphix.com> Reviewed by: David Quigley <dpquigl@davequigley.com> Reviewed by: Thomas Caputi <tcaputi@datto.com> Approved by: Dan McDonald <danmcd@omniti.com> Author: Dan Kimmel <dan.kimmel@delphix.com> Notes: svn path=/head/; revision=317414
* MFV 316891Josh Paetzel2017-04-215-22/+115
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 7386 zfs get does not work properly with bookmarks illumos/illumos-gate@edb901aab9c738b5eb15aa55933e82b0f2f9d9a2 https://github.com/illumos/illumos-gate/commit/edb901aab9c738b5eb15aa55933e82b0f2f9d9a2 https://www.illumos.org/issues/7386 The zfs get command does not work with the bookmark parameter while it works properly with both filesystem and snapshot: # zfs get -t all -r creation rpool/test NAME PROPERTY VALUE SOURCE rpool/test creation Fri Sep 16 15:00 2016 - rpool/test@snap creation Fri Sep 16 15:00 2016 - rpool/test#bkmark creation Fri Sep 16 15:00 2016 - # zfs get -t all -r creation rpool/test@snap NAME PROPERTY VALUE SOURCE rpool/test@snap creation Fri Sep 16 15:00 2016 - # zfs get -t all -r creation rpool/test#bkmark cannot open 'rpool/test#bkmark': invalid dataset name # The zfs get command should be modified to work properly with bookmarks too. Reviewed by: Simon Klinkert <simon.klinkert@gmail.com> Reviewed by: Paul Dagnelie <pcd@delphix.com> Approved by: Matthew Ahrens <mahrens@delphix.com> Author: Marcel Telka <marcel@telka.sk> Notes: svn path=/head/; revision=317267
* MFV 316855Alan Somers2017-04-141-12/+8
| | | | | | | | | | | | | | | | | | 7900 zdb shouldn't print the path of a znode at verbosity < 5 Reviewed by: Paul Dagnelie <pcd@delphix.com> Reviewed by: Matt Ahrens <mahrens@delphix.com> Approved by: Dan McDonald <danmcd@omniti.com> Author: Alan Somers <asomers@freebsd.org> illumos/illumos-gate@e548d2fa41d1baa06662ed9abbb8bcec86e27dd9 https://www.illumos.org/issues/7900 MFC after: 3 weeks Sponsored by: Spectra Logic Corp Notes: svn path=/head/; revision=316856
* dtrace: fix normalization of stddev aggregationAndriy Gapon2017-04-141-0/+2
| | | | | | | | | | | To be upstreamed. Discussed with: Bryan Cantrill <bryancantrill@gmail.com> MFC after: 2 weeks Sponsored by: Panzura Notes: svn path=/head/; revision=316853
* MFV r316693:Pedro F. Giffuni2017-04-101-1/+1
| | | | | | | | | | | | | | | | | | 8046 Let calloc() do the multiplication in libzfs_fru_refresh https://github.com/illumos/illumos-gate/commit/5697e03e6e3e2697f56ae341c7c8ce79680d6a2e https://www.illumos.org/issues/8046 Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Paul Dagnelie <pcd@delphix.com> Approved by: Robert Mustacchi <rm@joyent.com> Author: Pedro Giffuni <pfg@freebsd.org> MFC after: 3 days Notes: svn path=/head/; revision=316695
* MFV r315290, r315291: 7303 dynamic metaslab selectionAlexander Motin2017-03-243-5/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | illumos/illumos-gate@8363e80ae72609660f6090766ca8c2c18aa53f0c https://github.com/illumos/illumos-gate/commit/8363e80ae72609660f6090766ca8c2c18 https://www.illumos.org/issues/7303 This change introduces a new weighting algorithm to improve metaslab selection. The new weighting algorithm relies on the SPACEMAP_HISTOGRAM feature. As a result, the metaslab weight now encodes the type of weighting algorithm used (size-based vs segment-based). This also introduce a new allocation tracing facility and two new dcmds to help debug allocation problems. Each zio now contains a zio_alloc_list_t structure that is populated as the zio goes through the allocations stage. Here's an example of how to use the tracing facility: > c5ec000::print zio_t io_alloc_list | ::walk list | ::metaslab_trace MSID DVA ASIZE WEIGHT RESULT VDEV - 0 400 0 NOT_ALLOCATABLE ztest.0a - 0 400 0 NOT_ALLOCATABLE ztest.0a - 0 400 0 ENOSPC ztest.0a - 0 200 0 NOT_ALLOCATABLE ztest.0a - 0 200 0 NOT_ALLOCATABLE ztest.0a - 0 200 0 ENOSPC ztest.0a 1 0 400 1 x 8M 17b1a00 ztest.0a > 1ff2400::print zio_t io_alloc_list | ::walk list | ::metaslab_trace MSID DVA ASIZE WEIGHT RESULT VDEV - 0 200 0 NOT_ALLOCATABLE mirror-2 - 0 200 0 NOT_ALLOCATABLE mirror-0 1 0 200 1 x 4M 112ae00 mirror-1 - 1 200 0 NOT_ALLOCATABLE mirror-2 - 1 200 0 NOT_ALLOCATABLE mirror-0 1 1 200 1 x 4M 112b000 mirror-1 - 2 200 0 NOT_ALLOCATABLE mirror-2 If the metaslab is using segment-based weighting then the WEIGHT column will display the number of segments available in the bucket where the allocation attempt was made. Author: George Wilson <george.wilson@delphix.com> Reviewed by: Alex Reece <alex@delphix.com> Reviewed by: Chris Siden <christopher.siden@delphix.com> Reviewed by: Dan Kimmel <dan.kimmel@delphix.com> Reviewed by: Matthew Ahrens <mahrens@delphix.com> Reviewed by: Paul Dagnelie <paul.dagnelie@delphix.com> Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com> Reviewed by: Prakash Surya <prakash.surya@delphix.com> Reviewed by: Don Brady <don.brady@intel.com> Approved by: Richard Lowe <richlowe@richlowe.net> Notes: svn path=/head/; revision=315896
* cddl: normalize paths using SRCTOP-relative paths or :H when possibleEnji Cooper2017-03-0424-256/+235
| | | | | | | | | | | | | This simplifies make logic/output While here, remove bogus CFLAGS which look for headers in cddl/lib/libumem. There aren't any source files there (just Makefiles) MFC after: 1 month Sponsored by: Dell EMC Isilon Notes: svn path=/head/; revision=314654
* Fix memory leaks in error cases in libdtrace.Mark Johnston2017-02-232-3/+9
| | | | | | | | | Submitted by: Tom Rix <trix@juniper.net> MFC after: 1 week Differential Revision: https://reviews.freebsd.org/D9705 Notes: svn path=/head/; revision=314154
* Fix a memory leak in an error case in libctf.Mark Johnston2017-02-231-0/+1
| | | | | | | | Submitted by: Tom Rix <trix@juniper.net> MFC after: 1 week Notes: svn path=/head/; revision=314153
* When patching USDT probes, use non-unique names for aliases of weak symbols.Mark Johnston2017-02-101-32/+35
| | | | | | | | | | | | | | | Aliases are normally given names that include a key that's unique for each input object file. This, for example, ensures that aliases for identically named local symbols in different object files don't conflict. However, in some cases the static linker will leave an undefined alias after merging identical weak symbols, resulting in a link error. A non-unique name allows the aliases to be merged as well. PR: 216871 X-MFC With: r313262 Notes: svn path=/head/; revision=313504
* Search for _DTRACE_VERSION in sys/sdt.h rather than unistd.h.Mark Johnston2017-02-051-3/+3
| | | | | | | MFC after: 1 week Notes: svn path=/head/; revision=313265
* Avoid using Sun compiler-specific flags.Mark Johnston2017-02-051-1/+1
| | | | | | | MFC after: 1 week Notes: svn path=/head/; revision=313264
* Fix a double free of libelf data buffers in the USDT link code.Mark Johnston2017-02-051-3/+5
| | | | | | | | | | | | | | | libdtrace needs to append to the input object files' string and symbol tables. Currently it does so by allocating a larger buffer, copying the existing sections into them, and swapping pointers in the libelf data descriptors. However, it also frees those buffers when its processing is complete, which leads to a double free since the elftoolchain libelf owns them and also frees them in elf_end(3). Instead, free the buffers originally allocated by libelf. MFC after: 2 weeks Notes: svn path=/head/; revision=313263
* Use PC-relative relocations for USDT probe sites on i386 and amd64.Mark Johnston2017-02-053-107/+55
| | | | | | | | | | | | | | | | | | | | | | | When recording probe site addresses in the output DOF file, dtrace -G needs to emit relocations for the .SUNW_dof section in order to obtain the addresses of functions containing probe sites. DTrace expects the addresses to be relative to the base address of the final ELF file, and the amd64 USDT implementation was relying on some unspecified and incorrect behaviour in the base system GNU ld to achieve this. This change reimplements the probe site relocation handling to allow USDT to be used with lld and newer GNU binutils. Specifically, it makes use of R_X86_64_PC64/R_386_PC32 relocations to obtain the probe site address relative to the DOF file address, and adds and uses a new DOF relocation type which computes the final probe site address using these relative offsets. Reported by and discussed with: Rafael Espíndola MFC after: 1 month Differential Revision: https://reviews.freebsd.org/D9374 Notes: svn path=/head/; revision=313262
* Add an mbuf to ipinfo_t translator to finish cleanup of mbuf passing to TCP ↵George V. Neville-Neil2017-02-011-0/+18
| | | | | | | | | | | | probes. Reviewed by: markj MFC after: 2 weeks Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D9401 Notes: svn path=/head/; revision=313045
* Fix an unchecked return value in zfsdAlan Somers2017-01-181-2/+5
| | | | | | | | | | | | It's pretty unlikely to actually hit this, but good to check it anyway Reported by: Coverity CID: 1362018 MFC after: 4 weeks Sponsored by: Spectra Logic Corp Notes: svn path=/head/; revision=312396
* Convert ipv4_flags and ipv4_offset fields into host byte order.Andrey V. Elsukov2016-12-291-2/+2
| | | | | | | | | | | Also save only high bits in the ipv4_flags, because it is defined as uint8_t. So now it will show DF and MF flags as 0x40 and 0x20. Reviewed by: markj@ MFC after: 1 week Notes: svn path=/head/; revision=310785
* Avoid modifying the object string table when patching USDT probes.Mark Johnston2016-12-201-5/+8
| | | | | | | | | | | | | | | | | dtrace converts pairs of consecutive underscores in a probe name to dashes. When dtrace -G processes relocations corresponding to USDT probe sites, it performs this conversion on the corresponding symbol names prior to looking up the resulting probe names in the USDT provider definition. However, in so doing it would actually modify the input object's string table, which breaks the string suffix merging done by recent binutils. Because we don't care about the symbol name once the probe site is recorded, just perform the probe lookup using a temporary copy. Reported by: hrs, swills MFC after: 3 weeks Notes: svn path=/head/; revision=310332