diff options
Diffstat (limited to 'contrib/atf/doc/atf-formats.5')
| -rw-r--r-- | contrib/atf/doc/atf-formats.5 | 231 |
1 files changed, 231 insertions, 0 deletions
diff --git a/contrib/atf/doc/atf-formats.5 b/contrib/atf/doc/atf-formats.5 new file mode 100644 index 000000000000..bb919f483a9c --- /dev/null +++ b/contrib/atf/doc/atf-formats.5 @@ -0,0 +1,231 @@ +.\" +.\" Automated Testing Framework (atf) +.\" +.\" Copyright (c) 2007 The NetBSD Foundation, Inc. +.\" All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND +.\" CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, +.\" INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +.\" MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. +.\" IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS BE LIABLE FOR ANY +.\" DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE +.\" GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +.\" INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER +.\" IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR +.\" OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN +.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +.\" +.Dd December 20, 2011 +.Dt ATF-FORMATS 5 +.Os +.Sh NAME +.Nm atf-formats +.Nd machine-parseable data formats used by ATF +.Sh DESCRIPTION +This manual page describes the multiple data formats used in ATF. +These formats affect configuration files, control files and any data that +is externalized or internalized by the tools. +.Pp +Data files are always organized as follows: +.Bd -literal -offset indent +Header1: Value1 \\ + ... | head +HeaderN: ValueN / + mandatory blank line +Free-form text contents \\ + ... | body + ... / +.Ed +.Pp +A file must always contain a +.Sq Content-Type +header and must always separate that header from the body with a blank +line, even if the body is empty. +.Pp +The +.Sq Content-Type +is always of the form: +.Bd -literal -offset indent +Content-Type: application/X-atf-<subtype>; version="<version>" +.Ed +.Pp +where +.Sq subtype +indicates the specific file format and +.Sq version +its format version. +This header must be the first one of the file. +.Pp +The main purpose of the +.Sq Content-Type +header, aside from determining the format used in the file, is to allow +future changes to a given format. +Whenever an incompatible change is made, the version is bumped by one. +By keeping the header in the first line, future versions may even remove +the need for such a header -- e.g. if some format was replaced by XML +files, which have their own mandatory header. +.Pp +The rest of this document details the different format types. +.Ss Format: application/X-atf-atffile, version: 1 +Atffiles are logically divided into three sections: +.Bl -bullet +.It +Test programs: the list of test programs that define the test suite +described by the Atffile. +.It +Meta-data properties: these define some constant values applicable to +all the test programs defined in the file. +In some sense they define the properties that describe the test suite. +.It +Configuration variables: defaults for configuration variables that +can be overridden through configuration files or the command line. +.El +.Pp +The grammar for Atffiles is the following: +.Bd -literal -offset indent +DATA ::= ( ( CONF | PROP | TP )? COMMENT? NEWLINE )* EOF +CONF ::= 'conf:' WORD EQUAL STRING +PROP ::= 'prop:' WORD EQUAL STRING +TP ::= TPFILE | TPGLOB +TPFILE ::= 'tp: ' STRING +TPGLOB ::= 'tp-glob: ' STRING +STRING ::= WORD | '"' WORD* '"' +.Ed +.Pp +The meaning of the constructions above is: +.Bl -tag -width TPGLOBXX +.It CONF +Definition of a configuration variable. +.It PROP +Definition of a meta-data property. +.It TPFILE +Addition of a test program into the test suite. +The string is taken literally as the program's name, and this program +must exist. +.It TPGLOB +Addition of multiple test programs into the test suite. +The string is taken as a glob pattern, which may have or not have any +matches in the current directory. +.El +.Pp +An example: +.Bd -literal -offset indent +prop: test-suite = utilities + +conf: unprivileged-user = nobody + +tp: t_cp +tp: t_mv +tp: t_df +tp-glob: t_dir_* +.Ed +.Ss Format: application/X-atf-config, version: 1 +Configuration files are very simple: they only contain a list of variable +name/variable value pairs. +Their grammar is: +.Bd -literal -offset indent +DATA ::= ( VAR? COMMENT? NEWLINE )* EOF +VAR ::= WORD EQUAL STRING +COMMENT ::= HASH WORD* +STRING ::= WORD | '"' WORD* '"' +.Ed +.Pp +An example: +.Bd -literal -offset indent +# This is the system-wide configuration file for ATF. +# The above and this line are comments placed on their own line. + +var1 = this is a variable value +var2 = this is another one # Optional comment at the end. +.Ed +.Ss Format: application/X-atf-tps, version: 3 +The +.Sq application/X-atf-tps +format multiplexes the standard output, standard error and results output +streams from multiple test programs into a single data file. +This format is used by +.Xr atf-run 1 +to report the execution of several test programs and is later parsed by +.Xr atf-report 1 +to inform the user of this process. +It has the following grammar: +.Bd -literal -offset indent +DATA ::= INFO* TPS-COUNT TP-STANZA* INFO* EOF +INFO ::= 'info' COLON STRING COMMA STRING NEWLINE +TPS-COUNT ::= 'tps-count' COLON POSITIVE-NUMBER NEWLINE +TP-STANZA ::= TP-START TC-STANZA* TC-END +TP-START ::= 'tp-start' COLON TIMESTAMP COMMA STRING COMMA + POSITIVE-NUMBER NEWLINE +TP-END ::= 'tc-end' COLON TIMESTAMP COMMA STRING (COMMA STRING)? +TC-STANZA ::= TC-START (TC-SO | TC-SE)* TC-END +TC-START ::= 'tc-start' COLON TIMESTAMP COMMA STRING NEWLINE +TC-SO ::= 'tc-so' COLON STRING NEWLINE +TC-SE ::= 'tc-se' COLON STRING NEWLINE +TC-END ::= 'tc-end' COLON TIMESTAMP COMMA STRING COMMA TCR NEWLINE +TCR ::= 'passed' | ('failed' | 'skipped') COMMA STRING +TIMESTAMP ::= [0-9]+.[0-9]+ +.Ed +.Pp +The meaning of the constructions above is: +.Bl -tag -width TPSXCOUNTXX +.It TPS-COUNT +Indicates the number of test programs that will be executed. +There will be this exact amount of +.Sq TP-STANZA +constructions following it. +.It TP-START +Indicates the beginning of a test program. +This includes the program's name and the amount of test cases that +will follow. +.It TP-END +Indicates the completion of a test program. +This is followed by the program's name and, if the program ended +prematurely, an error message indicating the reason of its failure. +A successful execution of a test program (regardless of the status of its +test cases) must not be accompanied by any reason. +.It TC-START +Indicates the beginning of a test case. +This is accompanied by the test case's name. +.It TC-SO +Contains a text line sent to the standard output stream during the +execution of the test case. +Leading and trailing space is preserved. +.It TC-SE +Contains a text line sent to the standard error stream during the +execution of the test case. +Leading and trailing space is preserved. +.It TC-END +Indicates the completion of a test case. +This is accompanied by the test case's name, its result and the reason +associated with this result (if applicable). +.El +.Pp +An example: +.Bd -literal -offset indent +tps-count: 2 +tp-start: calculator, 1324318951.838923, 2 +tc-start: add, 1324318951.839101 +tc-end: add, 1324318951.839500, passed +tc-start: subtract, 1324318951.840001 +tc-so: 3-2 expected to return 1 but got 0 +tc-end: subtract, 1324318952.000123, failed, Calculated an unexpected value +tp-end: calculator, 1324318952.002301 +tp-start: files, 1, 1324318952.502348 +tc-start: copy, 1324318952.508291 +tc-se: could not find the cp(1) utility +tc-end: copy, 1324318953.203145, skipped +tp-end: files, 1324318953.203800 +.Ed +.Sh SEE ALSO +.Xr atf 7 |