3 files changed, 540 insertions, 0 deletions
diff --git a/contrib/atf/doc/atf-test-case.4 b/contrib/atf/doc/atf-test-case.4
new file mode 100644
index 000000000000..34f5e1bce1e9
--- /dev/null
+++ b/contrib/atf/doc/atf-test-case.4
@@ -0,0 +1,321 @@
+.\" Copyright (c) 2007 The NetBSD Foundation, Inc.
+.\" All rights reserved.
+.\"
+.\" Redistribution and use in source and binary forms, with or without
+.\" modification, are permitted provided that the following conditions
+.\" are met:
+.\" 1. Redistributions of source code must retain the above copyright
+.\"    notice, this list of conditions and the following disclaimer.
+.\" 2. Redistributions in binary form must reproduce the above copyright
+.\"    notice, this list of conditions and the following disclaimer in the
+.\"    documentation and/or other materials provided with the distribution.
+.\"
+.\" THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND
+.\" CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES,
+.\" INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
+.\" MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
+.\" IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS BE LIABLE FOR ANY
+.\" DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
+.\" GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+.\" INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+.\" IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
+.\" OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN
+.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+.Dd March 6, 2017
+.Dt ATF-TEST-CASE 4
+.Os
+.Sh NAME
+.Nm atf-test-case
+.Nd generic description of test cases
+.Sh DESCRIPTION
+A
+.Em test case
+is a piece of code that stress-tests a specific feature of the software.
+This feature is typically self-contained enough, either in the amount of
+code that implements it or in the general idea that describes it, to
+warrant its independent testing.
+Given this, test cases are very fine-grained, but they attempt to group
+similar smaller tests which are semantically related.
+.Pp
+A test case is defined by three components regardless of the language it is
+implemented in: a header, a body and a cleanup routine.
+The
+.Em header
+is, basically, a declarative piece of code that defines several
+properties to describe what the test case does and how it behaves.
+In other words: it defines the test case's
+.Em meta-data ,
+further described in the
+.Sx Meta-data
+section.
+The
+.Em body
+is the test case itself.
+It executes all actions needed to reproduce the test, and checks for
+failures.
+This body is only executed if the abstract conditions specified by the
+header are met.
+The
+.Em cleanup
+routine is a piece of code always executed after the body, regardless of
+the exit status of the test case.
+It can be used to undo side-effects of the test case.
+Note that almost all side-effects of a test case are automatically cleaned
+up by the library; this is explained in more detail in the rest of this
+document.
+.Pp
+It is extremely important to keep the separation between a test case's
+header and body well-defined, because the header is
+.Em always
+parsed, whereas the body is only executed when the conditions defined in
+the header are met and when the user specifies that test case.
+.Pp
+At last, test cases are always contained into test programs.
+The test programs act as a front-end to them, providing a consistent
+interface to the user and several APIs to ease their implementation.
+.Ss Results
+Upon termination, a test case reports a status and, optionally, a textual
+reason describing why the test reported such status.
+The caller must ensure that the test case really performed the task that its
+status describes, as the test program may be bogus and therefore providing a
+misleading result, e.g., providing a result that indicates success but the
+error code of the program says otherwise.
+.Pp
+The possible exit status of a test case are one of the following:
+.Bl -tag -width expectedXfailureXX
+.It expected_death
+The test case expects to terminate abruptly.
+.It expected_exit
+The test case expects to exit cleanly.
+.It expected_failure
+The test case expects to exit with a controller fatal/non-fatal failure.
+If this happens, the test program exits with a success error code.
+.It expected_signal
+The test case expects to receive a signal that makes it terminate.
+.It expected_timeout
+The test case expects to execute for longer than its timeout.
+.It passed
+The test case was executed successfully.
+The test program exits with a success error code.
+.It skipped
+The test case could not be executed because some preconditions were not
+met.
+This is not a failure because it can typically be resolved by adjusting
+the system to meet the necessary conditions.
+This is always accompanied by a
+.Em reason ,
+a message describing why the test was skipped.
+The test program exits with a success error code.
+.It failed
+An error appeared during the execution of the test case.
+This is always accompanied by a
+.Em reason ,
+a message describing why the test failed.
+The test program exits with a failure error code.
+.El
+.Pp
+The usefulness of the
+.Sq expected_*
+results comes when writing test cases that verify known failures caused,
+in general, due to programming errors (aka bugs).
+Whenever the faulty condition that the
+.Sq expected_*
+result is trying to cover is fixed, then the test case will be reported as
+.Sq failed
+and the developer will have to adjust it to match its new condition.
+.Pp
+It is important to note that all
+.Sq expected_*
+results are only provided as a
+.Em hint
+to the caller; the caller must verify that the test case did actually terminate
+as the expected condition says.
+.Ss Input/output
+Test cases are free to print whatever they want to their
+.Xr stdout 4
+and
+.Xr stderr 4
+file descriptors.
+They are, in fact, encouraged to print status information as they execute
+to keep the user informed of their actions.
+This is specially important for long test cases.
+.Pp
+Test cases will log their results to an auxiliary file, which is then
+collected by the test program they are contained in.
+The developer need not care about this as long as he uses the correct
+APIs to implement the test cases.
+.Pp
+The standard input of the test cases is unconditionally connected to
+.Sq /dev/zero .
+.Ss Meta-data
+The following list describes all meta-data properties interpreted
+internally by ATF.
+You are free to define new properties in your test cases and use them as
+you wish, but non-standard properties must be prefixed by
+.Sq X- .
+.Bl -tag -width requireXmachineXX
+.It descr
+Type: textual.
+Required.
+.Pp
+A brief textual description of the test case's purpose.
+Will be shown to the user in reports.
+Also good for documentation purposes.
+.It has.cleanup
+Type: boolean.
+Optional.
+.Pp
+If set to true, specifies that the test case has a cleanup routine that has
+to be executed by the runtime engine during the cleanup phase of the execution.
+This property is automatically set by the framework when defining a test case
+with a cleanup routine, so it should never be set by hand.
+.It ident
+Type: textual.
+Required.
+.Pp
+The test case's identifier.
+Must be unique inside the test program and should be short but descriptive.
+.It require.arch
+Type: textual.
+Optional.
+.Pp
+A whitespace separated list of architectures that the test case can be run
+under without causing errors due to an architecture mismatch.
+.It require.config
+Type: textual.
+Optional.
+.Pp
+A whitespace separated list of configuration variables that must be defined
+to execute the test case.
+If any of the required variables is not defined, the test case is
+.Em skipped .
+.It require.diskspace
+Type: integer.
+Optional.
+Specifies the minimum amount of available disk space needed by the test.
+The value can have a size suffix such as
+.Sq K ,
+.Sq M ,
+.Sq G
+or
+.Sq T
+to make the amount of bytes easier to type and read.
+.It require.files
+Type: textual.
+Optional.
+.Pp
+A whitespace separated list of files that must be present to execute the
+test case.
+The names of these files must be absolute paths.
+If any of the required files is not found, the test case is
+.Em skipped .
+.It require.machine
+Type: textual.
+Optional.
+.Pp
+A whitespace separated list of machine types that the test case can be run
+under without causing errors due to a machine type mismatch.
+.It require.memory
+Type: integer.
+Optional.
+Specifies the minimum amount of physical memory needed by the test.
+The value can have a size suffix such as
+.Sq K ,
+.Sq M ,
+.Sq G
+or
+.Sq T
+to make the amount of bytes easier to type and read.
+.It require.progs
+Type: textual.
+Optional.
+.Pp
+A whitespace separated list of programs that must be present to execute
+the test case.
+These can be given as plain names, in which case they are looked in the
+user's
+.Ev PATH ,
+or as absolute paths.
+If any of the required programs is not found, the test case is
+.Em skipped .
+.It require.user
+Type: textual.
+Optional.
+.Pp
+The required privileges to execute the test case.
+Can be one of
+.Sq root
+or
+.Sq unprivileged .
+.Pp
+If the test case is running as a regular user and this property is
+.Sq root ,
+the test case is
+.Em skipped .
+.Pp
+If the test case is running as root and this property is
+.Sq unprivileged ,
+the runtime engine will automatically drop the privileges if the
+.Sq unprivileged-user
+configuration property is set; otherwise the test case is
+.Em skipped .
+.It timeout
+Type: integral.
+Optional; defaults to
+.Sq 300 .
+.Pp
+Specifies the maximum amount of time the test case can run.
+This is particularly useful because some tests can stall either because they
+are incorrectly coded or because they trigger an anomalous behavior of the
+program.
+It is not acceptable for these tests to stall the whole execution of the
+test program.
+.Pp
+Can optionally be set to zero, in which case the test case has no run-time
+limit.
+This is discouraged.
+.El
+.Ss Environment
+Every time a test case is executed, several environment variables are
+cleared or reseted to sane values to ensure they do not make the test fail
+due to unexpected conditions.
+These variables are:
+.Bl -tag -width LCXMESSAGESXX
+.It Ev HOME
+Set to the work directory's path.
+.It Ev LANG
+Undefined.
+.It Ev LC_ALL
+Undefined.
+.It Ev LC_COLLATE
+Undefined.
+.It Ev LC_CTYPE
+Undefined.
+.It Ev LC_MESSAGES
+Undefined.
+.It Ev LC_MONETARY
+Undefined.
+.It Ev LC_NUMERIC
+Undefined.
+.It Ev LC_TIME
+Undefined.
+.It Ev TZ
+Hardcoded to
+.Sq UTC .
+.El
+.Ss Work directories
+The test program always creates a temporary directory
+and switches to it before running the test case's body.
+This way the test case is free to modify its current directory as it
+wishes, and the runtime engine will be able to clean it up later on in a
+safe way, removing any traces of its execution from the system.
+To do so, the runtime engine will perform a recursive removal of the work
+directory without crossing mount points; if a mount point is found, the
+file system will be unmounted (if possible).
+.Ss File creation mode mask (umask)
+Test cases are always executed with a file creation mode mask (umask) of
+.Sq 0022 .
+The test case's code is free to change this during execution.
+.Sh SEE ALSO
+.Xr atf-test-program 1
diff --git a/contrib/atf/doc/atf-test-program.1 b/contrib/atf/doc/atf-test-program.1
new file mode 100644
index 000000000000..0ca19635549f
--- /dev/null
+++ b/contrib/atf/doc/atf-test-program.1
@@ -0,0 +1,99 @@
+.\" Copyright (c) 2007 The NetBSD Foundation, Inc.
+.\" All rights reserved.
+.\"
+.\" Redistribution and use in source and binary forms, with or without
+.\" modification, are permitted provided that the following conditions
+.\" are met:
+.\" 1. Redistributions of source code must retain the above copyright
+.\"    notice, this list of conditions and the following disclaimer.
+.\" 2. Redistributions in binary form must reproduce the above copyright
+.\"    notice, this list of conditions and the following disclaimer in the
+.\"    documentation and/or other materials provided with the distribution.
+.\"
+.\" THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND
+.\" CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES,
+.\" INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
+.\" MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
+.\" IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS BE LIABLE FOR ANY
+.\" DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
+.\" GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+.\" INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+.\" IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
+.\" OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN
+.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+.Dd March 2, 2014
+.Dt ATF-TEST-PROGRAM 1
+.Os
+.Sh NAME
+.Nm atf-test-program
+.Nd common interface to ATF test programs
+.Sh SYNOPSIS
+.Nm
+.Op Fl r Ar resfile
+.Op Fl s Ar srcdir
+.Op Fl v Ar var1=value1 Op .. Fl v Ar varN=valueN
+.Ar test_case
+.Nm
+.Fl l
+.Sh DESCRIPTION
+Test programs written using the ATF libraries all share a common user
+interface, which is what this manual page describes.
+.Em NOTE: There is no binary known as
+.Nm ;
+.Em what is described in this manual page is the command-line interface
+.Em exposed by the atf-c, atf-c++ and atf-sh bindings .
+.Pp
+In the first synopsis form, the test program will execute the provided
+test case and print its results to the standard output, unless otherwise
+stated by the
+.Fl r
+flag.
+Optionally, the test case name can be suffixed by
+.Sq :cleanup ,
+in which case the cleanup routine of the test case will be executed
+instead of the test case body; see
+.Xr atf-test-case 4 .
+Note that the test case is
+.Em executed without isolation ,
+so it can and probably will create and modify files in the current directory.
+To execute test cases in a controller manner, you need a runtime engine
+that understands the ATF interface.
+The recommended runtime engine is
+.Xr kyua 1 .
+You should only execute test cases by hand for debugging purposes.
+.Pp
+In the second synopsis form, the test program will list all available
+test cases alongside their meta-data properties in a format that is
+machine parseable.
+This list is processed by
+.Xr kyua 1
+to know how to execute the test cases of a given test program.
+.Pp
+The following options are available:
+.Bl -tag -width XvXvarXvalueXX
+.It Fl l
+Lists available test cases alongside a brief description for each of them.
+.It Fl r Ar resfile
+Specifies the file that will receive the test case result.
+If not specified, the test case prints its results to stdout.
+If the result of a test case needs to be parsed by another program, you must
+use this option to redirect the result to a file and then read the resulting
+file from the other program.
+Note:
+.Em do not try to process the stdout of the test case
+because your program may break in the future.
+.It Fl s Ar srcdir
+The path to the directory where the test program is located.
+This is needed in all cases, except when the test program is being executed
+from the current directory.
+The test program will use this path to locate any helper data files or
+utilities.
+.It Fl v Ar var=value
+Sets the configuration variable
+.Ar var
+to the value
+.Ar value .
+.El
+.Sh SEE ALSO
+.Xr kyua 1
diff --git a/contrib/atf/doc/atf.7.in b/contrib/atf/doc/atf.7.in
new file mode 100644
index 000000000000..ded00c112031
--- /dev/null
+++ b/contrib/atf/doc/atf.7.in
@@ -0,0 +1,120 @@
+.\" Copyright (c) 2007 The NetBSD Foundation, Inc.
+.\" All rights reserved.
+.\"
+.\" Redistribution and use in source and binary forms, with or without
+.\" modification, are permitted provided that the following conditions
+.\" are met:
+.\" 1. Redistributions of source code must retain the above copyright
+.\"    notice, this list of conditions and the following disclaimer.
+.\" 2. Redistributions in binary form must reproduce the above copyright
+.\"    notice, this list of conditions and the following disclaimer in the
+.\"    documentation and/or other materials provided with the distribution.
+.\"
+.\" THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND
+.\" CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES,
+.\" INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
+.\" MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
+.\" IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS BE LIABLE FOR ANY
+.\" DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
+.\" GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+.\" INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
+.\" IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
+.\" OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN
+.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+.Dd September 14, 2014
+.Dt ATF 7
+.Os
+.Sh NAME
+.Nm ATF
+.Nd introduction to the Automated Testing Framework
+.Sh DESCRIPTION
+The Automated Testing Framework
+.Pf ( Nm )
+is a
+.Em collection of libraries
+to implement test programs in a variety of languages.
+These libraries all offer similar functionality and any test program
+written with them exposes a consistent user interface.
+.Pp
+Test programs using the
+.Nm
+libraries rely on a separate runtime engine to execute them in a
+deterministic fashion.
+The runtime engine isolates the test programs from the rest of the system
+and ensures some common side-effects are cleaned up.
+The runtime engine is also responsible for gathering the results of all
+tests and composing reports.
+The current runtime of choice is Kyua, described in
+.Xr kyua 1 .
+.Pp
+If your operating systems distributes
+.Nm ,
+it should also provide an introductory
+.Xr tests 7
+manual page.
+You are encouraged to read it now.
+.Pp
+The rest of this manual page serves as a cross-reference to all the other
+documentation shipped with
+.Nm .
+.Ss Language bindings
+.Bl -tag -width atfXtestXprogramXXXXX
+.It Xr atf-c 3
+C programming interface.
+.It Xr atf-c++ 3
+C++ programming interface.
+.It Xr atf-sh 3
+.Xr sh 1
+programming interface.
+.El
+.Ss Miscellaneous pages
+.Bl -tag -width atfXtestXprogramXXXXX
+.It Xr atf-test-case 4
+Generic description of test cases, independent of the language they are
+implemented in.
+.It Xr atf-test-program 1
+Common interface provided by the test programs written using the
+.Nm
+libraries.
+.El
+.Sh SEE ALSO
+.Xr kyua 1 ,
+.Xr tests 7
+.Sh HISTORY
+.Nm
+started as a Google Summer of Code 2007 project mentored by The NetBSD
+Foundation.
+Its original goal was to provide a testing framework for the
+.Nx
+operating system, but it grew as an independent project because the
+framework itself did not need to be tied to a specific operating system.
+.Pp
+Originally,
+.Nm
+shipped the collection of libraries described in this manual page as well
+as a runtime engine.
+The runtime engine has since been replaced by Kyua and the old tools were
+removed in
+.Nm 0.20 ,
+which shipped in early 2014.
+.Pp
+As of late 2014, both
+.Fx
+and
+.Nx
+ship
+.Nm
+in their base systems and provide extensive test suites based on it.
+.Pp
+For more details on historical changes, refer to:
+.Bd -literal -offset indent
+.Pa __DOCDIR__/NEWS
+.Ed
+.Sh AUTHORS
+For more details on the people that made
+.Nm
+possible, refer to:
+.Bd -literal -offset indent
+.Pa __DOCDIR__/AUTHORS
+.Ed