Introduce "feature flags" for ZFS pools (bump SPA version to 5000).

Add first feature "com.delphix:async_destroy" (asynchronous destroy
of ZFS datasets).
Implement features support in ZFS boot code.

Illumos revisions merged:
13700:2889e2596bd6
13701:1949b688d5fb
2619 asynchronous destruction of ZFS file systems
2747 SPA versioning with zfs feature flags

References:
https://www.illumos.org/issues/2619
https://www.illumos.org/issues/2747

Obtained from:	illumos (issue #2619, #2747)
MFC after:	1 month

1 files changed, 174 insertions, 0 deletions

diff --git a/cddl/contrib/opensolaris/cmd/zpool/zpool-features.5 b/cddl/contrib/opensolaris/cmd/zpool/zpool-features.5
new file mode 100644
index 000000000000..d590fdc9fe4c
--- /dev/null
+++ b/cddl/contrib/opensolaris/cmd/zpool/zpool-features.5
@@ -0,0 +1,174 @@
+'\" te
+.\" Copyright (c) 2012, Martin Matuska <mm@FreeBSD.org>.
+.\" All Rights Reserved.
+.\"
+.\" The contents of this file are subject to the terms of the
+.\" Common Development and Distribution License (the "License").
+.\" You may not use this file except in compliance with the License.
+.\"
+.\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE
+.\" or http://www.opensolaris.org/os/licensing.
+.\" See the License for the specific language governing permissions
+.\" and limitations under the License.
+.\"
+.\" When distributing Covered Code, include this CDDL HEADER in each
+.\" file and include the License file at usr/src/OPENSOLARIS.LICENSE.
+.\" If applicable, add the following below this CDDL HEADER, with the
+.\" fields enclosed by brackets "[]" replaced with your own identifying
+.\" information: Portions Copyright [yyyy] [name of copyright owner]
+.\"
+.\" Copyright (c) 2012 by Delphix. All rights reserved.
+.\"
+.\" $FreeBSD$
+.\"
+.Dd May 28, 2012
+.Dt ZPOOL-FEATURES 8
+.Os
+.Sh NAME
+.Nm zpool-features
+.Nd ZFS pool feature descriptions
+.Sh DESCRIPTION
+ZFS pool on\-disk format versions are specified via "features" which replace
+the old on\-disk format numbers (the last supported on\-disk format number is
+28).
+To enable a feature on a pool use the
+.Xr zpool 8
+command to set the
+.Sy feature@feature_name
+property to
+.Ar enabled .
+.Pp
+The pool format does not affect file system version compatibility or the ability
+to send file systems between pools.
+.Pp
+Since most features can be enabled independently of each other the on\-disk
+format of the pool is specified by the set of all features marked as
+.Sy active
+on the pool. If the pool was created by another software version this set may
+include unsupported features.
+.Ss Identifying features
+Every feature has a guid of the form
+.Sy com.example:feature_name .
+The reverse DNS name ensures that the feature's guid is unique across all ZFS
+implementations. When unsupported features are encountered on a pool they will
+be identified by their guids.
+Refer to the documentation for the ZFS implementation that created the pool
+for information about those features.
+.Pp
+Each supported feature also has a short name.
+By convention a feature's short name is the portion of its guid which follows
+the ':' (e.g.
+.Sy com.example:feature_name
+would have the short name
+.Sy feature_name ),
+however a feature's short name may differ across ZFS implementations if
+following the convention would result in name conflicts.
+.Ss Feature states
+Features can be in one of three states:
+.Bl -tag
+.It Sy active
+This feature's on\-disk format changes are in effect on the pool.
+Support for this feature is required to import the pool in read\-write mode.
+If this feature is not read-only compatible, support is also required to
+import the pool in read\-only mode (see "Read\-only compatibility").
+.It Sy enabled
+An administrator has marked this feature as enabled on the pool, but the
+feature's on\-disk format changes have not been made yet.
+The pool can still be imported by software that does not support this feature,
+but changes may be made to the on\-disk format at any time which will move
+the feature to the
+.Sy active
+state.
+Some features may support returning to the
+.Sy enabled
+state after becoming
+.Sy active .
+See feature\-specific documentation for details.
+.It Sy disabled
+This feature's on\-disk format changes have not been made and will not be made
+unless an administrator moves the feature to the
+.Sy enabled
+state.
+Features cannot be disabled once they have been enabled.
+.El
+The state of supported features is exposed through pool properties of the form
+.Sy feature@short_name .
+.Ss Read\-only compatibility
+Some features may make on\-disk format changes that do not interfere with other
+software's ability to read from the pool.
+These features are referred to as "read\-only compatible".
+If all unsupported features on a pool are read\-only compatible, the pool can
+be imported in read\-only mode by setting the
+.Sy readonly
+property during import (see
+.Xr zpool 8
+for details on importing pools).
+.Ss Unsupported features
+For each unsupported feature enabled on an imported pool a pool property
+named
+.Sy unsupported@feature_guid
+will indicate why the import was allowed despite the unsupported feature.
+Possible values for this property are:
+.Bl -tag
+.It Sy inactive
+The feature is in the
+.Sy enabled
+state and therefore the pool's on\-disk format is still compatible with
+software that does not support this feature.
+.It Sy readonly
+The feature is read\-only compatible and the pool has been imported in
+read\-only mode.
+.El
+.Ss Feature dependencies
+Some features depend on other features being enabled in order to function
+properly.
+Enabling a feature will automatically enable any features it depends on.
+.Sh FEATURES
+The following features are supported on this system:
+.Bl -tag
+.It Sy async_destroy
+.Bl -column "READ\-ONLY COMPATIBLE" "com.delphix:async_destroy"
+.It GUID Ta com.delphix:async_destroy
+.It READ\-ONLY COMPATIBLE Ta yes
+.It DEPENDENCIES Ta none
+.El
+.Pp
+Destroying a file system requires traversing all of its data in order to
+return its used space to the pool.
+Without
+.Sy async_destroy
+the file system is not fully removed until all space has been reclaimed.
+If the destroy operation is interrupted by a reboot or power outage the next
+attempt to open the pool will need to complete the destroy operation
+synchronously.
+.Pp
+When
+.Sy async_destroy
+is enabled the file system's data will be reclaimed by a background process,
+allowing the destroy operation to complete without traversing the entire file
+system.
+The background process is able to resume interrupted destroys after the pool
+has been opened, eliminating the need to finish interrupted destroys as part
+of the open operation.
+The amount of space remaining to be reclaimed by the background process is
+available through the
+.Sy freeing
+property.
+.Sh SEE ALSO
+.Xr zpool 8
+.Sh AUTHORS
+This manual page is a
+.Xr mdoc 7
+reimplementation of the
+.Tn illumos
+manual page
+.Em zpool-features(5) ,
+modified and customized for
+.Fx
+and licensed under the Common Development and Distribution License
+.Pq Tn CDDL .
+.Pp
+The
+.Xr mdoc 7
+implementation of this manual page was initially written by
+.An Martin Matuska Aq mm@FreeBSD.org .