diff options
Diffstat (limited to 'share/doc/psd')
142 files changed, 44625 insertions, 0 deletions
diff --git a/share/doc/psd/01.cacm/Makefile b/share/doc/psd/01.cacm/Makefile new file mode 100644 index 000000000000..14a2f703e5b9 --- /dev/null +++ b/share/doc/psd/01.cacm/Makefile @@ -0,0 +1,16 @@ +# @(#)Makefile 8.1 (Berkeley) 6/8/93 +# $FreeBSD$ + +VOLUME= psd/01.cacm +SRCS= stubs p.mac p1 p2 p3 p4 p5 p6 +EXTRA= ref.bib +MACROS= -ms +USE_REFER= +USE_TBL= +CLEANFILES= stubs + +stubs: + @(echo .R1; echo database ${.CURDIR}/ref.bib; \ + echo accumulate; echo .R2) > ${.TARGET} + +.include <bsd.doc.mk> diff --git a/share/doc/psd/01.cacm/p.mac b/share/doc/psd/01.cacm/p.mac new file mode 100644 index 000000000000..5450b5dc96f7 --- /dev/null +++ b/share/doc/psd/01.cacm/p.mac @@ -0,0 +1,31 @@ +.\" This module is believed to contain source code proprietary to AT&T. +.\" Use and redistribution is subject to the Berkeley Software License +.\" Agreement and your Software Agreement with AT&T (Western Electric). +.\" +.\" @(#)p.mac 8.1 (Berkeley) 6/8/93 +.\" +.\" $FreeBSD$ +.de P1 +.DS +.. +.de P2 +.DE +.. +.de UL +.lg 0 +.if n .ul +\%\&\\$3\f3\\$1\fR\&\\$2 +.lg +.. +.de UC +\&\\$3\s-1\\$1\\s0\&\\$2 +.. +.de IT +.lg 0 +.if n .ul +\%\&\\$3\f2\\$1\fR\&\\$2 +.lg +.. +.de SP +.sp \\$1 +.. diff --git a/share/doc/psd/01.cacm/p1 b/share/doc/psd/01.cacm/p1 new file mode 100644 index 000000000000..88987e02bb06 --- /dev/null +++ b/share/doc/psd/01.cacm/p1 @@ -0,0 +1,567 @@ +.\" This module is believed to contain source code proprietary to AT&T. +.\" Use and redistribution is subject to the Berkeley Software License +.\" Agreement and your Software Agreement with AT&T (Western Electric). +.\" +.\" @(#)p1 8.1 (Berkeley) 6/8/93 +.\" +.\" $FreeBSD$ +.OH 'The UNIX Time-Sharing System''PSD:1-%' +.EH 'PSD:1-%''The UNIX Time-Sharing System' +.ds n \s+2 +.hw above-mentioned +.ds s \s-2 +.ds m \v'-.3'.\v'.3' +.TL +The UNIX +Time-Sharing System\f1\s10\v'-.2n'*\v'.2n'\s0\fP +.AU +D. M. Ritchie and K. Thompson +.AB +.FS +* Copyright 1974, +Association for Computing Machinery, Inc., +reprinted by permission. +This is a revised version of an article +that appeared in Communications of the \*sACM\*n, +.IT 17 , +No. 7 (July 1974), pp. 365-375. +That article was a +revised version of a paper presented +at the Fourth \*sACM\*n Symposium on Operating +Systems Principles, +\*sIBM\*n Thomas J. Watson Research Center, +Yorktown Heights, +New York, +October 15-17, 1973. +.FE +.UX +is a general-purpose, multi-user, interactive +operating system for the larger Digital Equipment Corporation +\*sPDP\*n-11 and +the Interdata 8/32 computers. +It offers a number of features +seldom found even in larger operating +systems, including +.IP i +A hierarchical file system incorporating +demountable volumes, +.IP ii +Compatible file, device, and inter-process I/O, +.IP iii +The ability to initiate asynchronous processes, +.IP iv +System command language selectable on a per-user basis, +.IP v +Over 100 subsystems including a dozen languages, +.IP vi +High degree of portability. +.LP +This paper discusses the nature +and implementation of the file system +and of the user command interface. +.AE +.NH +INTRODUCTION +.PP +There have been four versions of +the +.UX +time-sharing system. +.hy 12 +The earliest (circa 1969-70) ran on +the Digital Equipment Corporation \*sPDP\*n-7 and -9 computers. +The second version ran on the unprotected +\*sPDP\*n-11/20 computer. +The third incorporated multiprogramming and ran +on the \*sPDP\*n-11/34, /40, /45, /60, and /70 computers; +it is the one described in the previously published version +of this paper, and is also the most widely used today. +.hy 14 +This paper describes only the +fourth, current +system that runs on the \*sPDP\*n-11/70 and the +Interdata 8/32 computers. +In fact, the differences among the various systems is +rather small; +most of the revisions made to the originally published version of this +paper, +aside from those concerned with style, +had to do with details of the implementation of the file system. +.PP +Since +\*sPDP\*n-11 +.UX +became operational +in February, 1971, +over 600 installations have been put into service. +Most of them are engaged in applications such as +computer science education, +the preparation and formatting of documents +and other textual material, +the collection and processing of trouble data +from various switching machines within the Bell System, +and recording and checking telephone service +orders. +Our own installation is used mainly for research +in operating systems, languages, +computer networks, +and other topics in computer science, and also for +document preparation. +.PP +Perhaps the most important achievement of +.UX +is to demonstrate +that +a powerful operating system for interactive use +need not be expensive either in equipment or in human +effort: +it +can run on hardware costing as little as $40,000, and +less than two man-years were spent on the main system +software. +We hope, however, that users find +that the +most important characteristics of the system +are its simplicity, elegance, and ease of use. +.PP +Besides the operating system proper, some major programs +available under +.UX +are +.DS +.nf +C compiler +Text editor based on \*sQED\*n +.[ +qed lampson +.] +Assembler, linking loader, symbolic debugger +Phototypesetting and equation setting programs +.[ +cherry kernighan typesetting mathematics cacm +.] +.[ +kernighan lesk ossanna document preparation bstj +%Q This issue +.] +.fi +.in +3n +.ll -5n +.ti -3n +Dozens of languages including +Fortran 77, Basic, Snobol, \*sAPL\*n, Algol 68, M6, \*sTMG\*n, Pascal +.in +.ll +.DE +There is a host of maintenance, utility, recreation and novelty programs, +all written locally. +The +.UX +user community, which numbers in the thousands, +has contributed many more programs and languages. +It is worth noting that the system is totally self-supporting. +All +.UX +software is maintained on +the +system; +likewise, this paper and all other +documents +in this issue +were generated and formatted by the +.UX +editor and text formatting +programs. +.SH +II. HARDWARE AND SOFTWARE ENVIRONMENT +.PP +The \*sPDP\*n-11/70 on which the Research +.UX +system is installed is a 16-bit +word (8-bit byte) computer with 768K bytes of core memory; +the system kernel +occupies 90K bytes +about equally divided between code +and data tables. +This system, however, includes a very large number of +device drivers +and enjoys a generous allotment +of space for I/O buffers and system tables; +a minimal system capable of running the software +mentioned above can +require as little as 96K bytes +of core altogether. +There are even larger installations; +see the description of the +\*sPWB/UNIX\*n systems, +.[ +dolotta mashey workbench software engineering +.] +.[ +dolotta haight mashey workbench bstj +%Q This issue +.] +for example. +There are also much smaller, though somewhat restricted, +versions of the system. +.[ +lycklama microprocessor bstj +%Q This issue +.] +.PP +Our own \*sPDP\*n-11 has two +200-Mb moving-head disks +for file system storage and swapping. +There are 20 variable-speed +communications interfaces +attached to 300- and 1200-baud data sets, +and an additional 12 communication lines +hard-wired to 9600-baud terminals and +satellite computers. +There are also several 2400- and 4800-baud +synchronous communication interfaces +used for machine-to-machine file transfer. +Finally, there is a variety +of miscellaneous +devices including +nine-track magnetic tape, +a line printer, +a voice synthesizer, +a phototypesetter, +a digital switching network, +and a chess machine. +.PP +The preponderance of +.UX +software is written in the +abovementioned C language. +.[ +c programming language kernighan ritchie prentice-hall +.] +Early versions of the operating system were written in assembly language, +but during the summer of 1973, it was rewritten in C. +The size of the new system was about one-third greater +than that of the old. +Since the new system not only became much easier to +understand and to modify but also +included +many functional improvements, +including multiprogramming and the ability to +share reentrant code among several user programs, +we consider this increase in size quite acceptable. +.SH +III. THE FILE SYSTEM +.PP +The most important role of +the system +is to provide +a file system. +From the point of view of the user, there +are three kinds of files: ordinary disk files, +directories, and special files. +.SH +3.1 Ordinary files +.PP +A file +contains whatever information the user places on it, +for example, symbolic or binary +(object) programs. +No particular structuring is expected by the system. +A file of text consists simply of a string +of characters, with lines demarcated by the newline character. +Binary programs are sequences of words as +they will appear in core memory when the program +starts executing. +A few user programs manipulate files with more +structure; +for example, the assembler generates, and the loader +expects, an object file in a particular format. +However, +the structure of files is controlled by +the programs that use them, not by the system. +.SH +3.2 Directories +.PP +Directories provide +the mapping between the names of files +and the files themselves, and thus +induce a structure on the file system as a whole. +Each user has a directory of his own files; +he may also create subdirectories to contain +groups of files conveniently treated together. +A directory behaves exactly like an ordinary file except that it +cannot be written on by unprivileged programs, so that the system +controls the contents of directories. +However, anyone with +appropriate permission may read a directory just like any other file. +.PP +The system maintains several directories +for its own use. +One of these is the +.UL root +directory. +All files in the system can be found by tracing +a path through a chain of directories +until the desired file is reached. +The starting point for such searches is often the +.UL root . +Other system directories contain all the programs provided +for general use; that is, all the +.IT commands . +As will be seen, however, it is by no means necessary +that a program reside in one of these directories for it +to be executed. +.PP +Files are named by sequences of 14 or +fewer characters. +When the name of a file is specified to the +system, it may be in the form of a +.IT path +.IT name , +which +is a sequence of directory names separated by slashes, ``/\^'', +and ending in a file name. +If the sequence begins with a slash, the search begins in the +root directory. +The name +.UL /alpha/beta/gamma +causes the system to search +the root for directory +.UL alpha , +then to search +.UL alpha +for +.UL beta , +finally to find +.UL gamma +in +.UL beta . +.UL \&gamma +may be an ordinary file, a directory, or a special +file. +As a limiting case, the name ``/\^'' refers to the root itself. +.PP +A path name not starting with ``/\^'' causes the system to begin the +search in the user's current directory. +Thus, the name +.UL alpha/beta +specifies the file named +.UL beta +in +subdirectory +.UL alpha +of the current +directory. +The simplest kind of name, for example, +.UL alpha , +refers to a file that itself is found in the current +directory. +As another limiting case, the null file name refers +to the current directory. +.PP +The same non-directory file may appear in several directories under +possibly different names. +This feature is called +.IT linking ; +a directory entry for a file is sometimes called a link. +The +.UX +system +differs from other systems in which linking is permitted +in that all links to a file have equal status. +That is, a file does not exist within a particular directory; +the directory entry for a file consists merely +of its name and a pointer to the information actually +describing the file. +Thus a file exists independently of any +directory entry, although in practice a file is made to +disappear along with the last link to it. +.PP +Each directory always has at least two entries. +The name +``\|\fB.\|\fP'' in each directory refers to the directory itself. +Thus a program +may read the current directory under the name ``\fB\|.\|\fP'' without knowing +its complete path name. +The name ``\fB\|.\|.\|\fP'' by convention refers to the parent of the +directory in which it appears, that is, to the directory in which +it was created. +.PP +The directory structure is constrained to have the form +of a rooted tree. +Except for the special entries ``\|\fB\|.\|\fP'' and ``\fB\|.\|.\|\fP'', each directory +must appear as an entry in exactly one other directory, which is its +parent. +The reason for this is to simplify the writing of programs +that visit subtrees of the directory structure, and more +important, to avoid the separation of portions of the hierarchy. +If arbitrary links to directories were permitted, it would +be quite difficult to detect when the last connection from +the root to a directory was severed. +.SH +3.3 Special files +.PP +Special files constitute the most unusual feature of the +.UX +file system. +Each supported I/O device +is associated with at least one such file. +Special files are read and written just like ordinary +disk files, but requests to read or write result in activation of the associated +device. +An entry for each special file resides in directory +.UL /dev , +although a link may be made to one of these files +just as it may to an ordinary file. +Thus, for example, +to write on a magnetic tape +one may write on the file +.UL /dev/mt . +Special files exist for each communication line, each disk, +each tape drive, +and for physical main memory. +Of course, +the active disks +and the memory special file are protected from +indiscriminate access. +.PP +There is a threefold advantage in treating +I/O devices this way: +file and device I/O +are as similar as possible; +file and device names have the same +syntax and meaning, so that +a program expecting a file name +as a parameter can be passed a device +name; finally, +special files are subject to the same +protection mechanism as regular files. +.SH +3.4 Removable file systems +.PP +Although the root of the file system is always stored on the same +device, +it is not necessary that the entire file system hierarchy +reside on this device. +There is a +.UL mount +system request with two arguments: +the name of an existing ordinary file, and the name of a special +file whose associated +storage volume (e.g., a disk pack) should have the structure +of an independent file system +containing its own directory hierarchy. +The effect of +.UL mount +is to cause +references to the heretofore ordinary file +to refer instead to the root directory +of the file system on the removable volume. +In effect, +.UL mount +replaces a leaf of the hierarchy tree (the ordinary file) +by a whole new subtree (the hierarchy stored on the +removable volume). +After the +.UL mount , +there is virtually no distinction +between files on the removable volume and those in the +permanent file system. +In our installation, for example, +the root directory resides +on a small partition of one of +our disk drives, +while the other drive, +which contains the user's files, +is mounted by the system initialization +sequence. +A mountable file system is generated by +writing on its corresponding special file. +A utility program is available to create +an empty file system, +or one may simply copy an existing file system. +.PP +There is only one exception to the rule of identical +treatment of files on different devices: +no link may exist between one file system hierarchy and +another. +This restriction is enforced so as to avoid +the elaborate bookkeeping +that would otherwise be required to assure removal of the links +whenever the removable volume is dismounted. +.SH +3.5 Protection +.PP +Although the access control scheme +is quite simple, it has some unusual features. +Each user of the system is assigned a unique +user identification number. +When a file is created, it is marked with +the user \*sID\*n of its owner. +Also given for new files +is a set of ten protection bits. +Nine of these specify +independently read, write, and execute permission +for the +owner of the file, +for other members of his group, +and for all remaining users. +.PP +If the tenth bit is on, the system +will temporarily change the user identification +(hereafter, user \*sID\*n) +of the current user to that of the creator of the file whenever +the file is executed as a program. +This change in user \*sID\*n is effective only +during the execution of the program that calls for it. +The set-user-\*sID\*n feature provides +for privileged programs that may use files +inaccessible to other users. +For example, a program may keep an accounting file +that should neither be read nor changed +except by the program itself. +If the set-user-\*sID\*n bit is on for the +program, it may access the file although +this access might be forbidden to other programs +invoked by the given program's user. +Since the actual user \*sID\*n +of the invoker of any program +is always available, +set-user-\*sID\*n programs +may take any measures desired to satisfy themselves +as to their invoker's credentials. +This mechanism is used to allow users to execute +the carefully written +commands +that call privileged system entries. +For example, there is a system entry +invokable only by the ``super-user'' (below) +that creates +an empty directory. +As indicated above, directories are expected to +have entries for ``\fB\|.\|\fP'' and ``\fB\|.\|.\|\fP''. +The command which creates a directory +is owned by the super-user +and has the set-user-\*sID\*n bit set. +After it checks its invoker's authorization to +create the specified directory, +it creates it and makes the entries +for ``\fB\|.\|\fP'' and ``\fB\|.\|.\|\fP''. +.PP +Because anyone may set the set-user-\*sID\*n +bit on one of his own files, +this mechanism is generally +available without administrative intervention. +For example, +this protection scheme easily solves the \*sMOO\*n +accounting problem posed by ``Aleph-null.'' +.[ +aleph null software practice +.] +.PP +The system recognizes one particular user \*sID\*n (that of the ``super-user'') as +exempt from the usual constraints on file access; thus (for example), +programs may be written to dump and reload the file +system without +unwanted interference from the protection +system. diff --git a/share/doc/psd/01.cacm/p2 b/share/doc/psd/01.cacm/p2 new file mode 100644 index 000000000000..8373c0ed9d58 --- /dev/null +++ b/share/doc/psd/01.cacm/p2 @@ -0,0 +1,448 @@ +.\" This module is believed to contain source code proprietary to AT&T. +.\" Use and redistribution is subject to the Berkeley Software License +.\" Agreement and your Software Agreement with AT&T (Western Electric). +.\" +.\" @(#)p2 8.1 (Berkeley) 6/8/93 +.\" +.\" $FreeBSD$ +.SH +3.6 I/O calls +.PP +The system calls to do I/O are designed to eliminate +the differences between the various devices and styles of +access. +There is no distinction between ``random'' +and ``sequential'' I/O, nor is any logical record size imposed +by the system. +The size of an ordinary file is determined +by the number of bytes written on it; +no predetermination of the size of a file is necessary +or possible. +.PP +To illustrate the essentials of I/O, +some of the basic calls are +summarized below +in an anonymous language that will +indicate the required parameters without getting into the +underlying +complexities. +Each call to the system may potentially result in an error +return, which for simplicity is not represented +in the calling sequence. +.PP +To read or write a file assumed to exist already, it must +be opened by the following call: +.P1 +filep = open\|(\|name, flag\|) +.P2 +where +.UL name +indicates the name of the file. +An arbitrary path name may be given. +The +.UL flag +argument indicates whether the file is to be read, written, +or ``updated,'' that is, read and written simultaneously. +.PP +The returned value +.UL filep +is called a +.IT "file descriptor" . +It is a small integer used to identify the file +in subsequent calls to read, write, +or otherwise manipulate the file. +.PP +To create a new file or completely rewrite an old one, +there is a +.UL create +system call that +creates the given file if it does not exist, +or truncates it to zero length +if it does exist; +.UL create +also opens the new file for writing +and, like +.UL open , +returns a file descriptor. +.PP +The file system maintains no locks visible to the user, nor is there any +restriction on the number of users who may have a file +open for reading or writing. +Although it is possible for the contents of a file +to become scrambled when two users write on it simultaneously, +in practice difficulties do not arise. +We take the view that locks are neither +necessary nor sufficient, in our environment, +to prevent interference between users of the same file. +They are unnecessary because we are not +faced with large, single-file data bases +maintained by independent processes. +They are insufficient because +locks in the ordinary sense, whereby +one user is prevented from writing on a file that another +user is reading, +cannot prevent confusion +when, for example, both users are editing +a file with an editor that makes +a copy of the file being edited. +.PP +There are, however, +sufficient internal interlocks to maintain +the logical consistency of the file system +when two users engage simultaneously in +activities such as writing on +the same file, +creating files in the same directory, +or deleting each other's open files. +.PP +Except as indicated below, reading and writing +are sequential. +This means that if a particular +byte in the file was the last byte written (or read), +the next I/O call implicitly refers to the +immediately following byte. +For each open file there is a pointer, maintained +inside the system, +that indicates the next byte to be read +or written. +If +.IT n +bytes are read or written, the pointer advances +by +.IT n +bytes. +.PP +Once a file is open, the following calls +may be used: +.P1 +n = read\|(\|filep, buffer, count\|) +n = write\|(\|filep, buffer, count\|) +.P2 +Up to +.UL count +bytes are transmitted between the file specified +by +.UL filep +and the byte array +specified by +.UL buffer . +The returned value +.UL n +is the number of bytes actually transmitted. +In the +.UL write +case, +.UL n +is the same as +.UL count +except under exceptional conditions, such as I/O errors or +end of physical medium on special files; +in a +.UL read , +however, +.UL n +may without error be less than +.UL count . +If the read pointer is so near the end of the +file that reading +.UL count +characters +would cause reading beyond the end, only sufficient +bytes are transmitted to reach the end of the +file; +also, typewriter-like terminals +never return more than one line of input. +When a +.UL read +call returns with +.UL n +equal +to zero, the end of the file has been reached. +For disk files this occurs when the read pointer +becomes equal to the current +size of the file. +It is possible to generate an end-of-file +from a terminal by use of an escape +sequence that depends on the device used. +.PP +Bytes written affect only those parts of a file implied by +the position of the write pointer and the +count; no other part of the file +is changed. +If the last byte lies beyond the end of the file, the +file is made to grow as needed. +.PP +To do random (direct-access) I/O +it is only necessary to move the read or write pointer +to the appropriate location in the file. +.P1 +location = lseek\|(\|filep, offset, base\|) +.P2 +The pointer +associated with +.UL filep +is moved to a position +.UL offset +bytes from the beginning of the file, from the current position +of the pointer, or from the end of the file, +depending on +.UL base. +.UL \&offset +may be negative. +For some devices (e.g., paper +tape and +terminals) seek calls are +ignored. +The actual offset from the beginning of the file +to which the pointer was moved is returned +in +.UL location . +.PP +There are several additional system entries +having to do with I/O and with the file +system that will not be discussed. +For example: +close a file, +get the status of a file, +change the protection mode or the owner +of a file, +create a directory, +make a link to an existing file, +delete a file. +.SH +IV. IMPLEMENTATION OF THE FILE SYSTEM +.PP +As mentioned in Section 3.2 above, a directory entry contains +only a name for the associated file and a pointer to the +file itself. +This pointer is an integer called the +.IT i-number +(for index number) +of the file. +When the file is accessed, +its i-number is used as an index into +a system table (the +.IT i-list \|) +stored in a known +part of the device on which +the directory resides. +The entry found thereby (the file's +.IT i-node \|) +contains +the description of the file: +.IP i +the user and group-\*sID\*n of its owner +.IP ii +its protection bits +.IP iii +the physical disk or tape addresses for the file contents +.IP iv +its size +.IP v +time of creation, last use, and last modification +.IP vi +the number of links to the file, that is, the number of times it appears in a directory +.IP vii +a code indicating whether the file is a directory, an ordinary file, or a special file. +.LP +The purpose of an +.UL open +or +.UL create +system call is to turn the path name given by the user +into an i-number +by searching the explicitly or implicitly named directories. +Once a file is open, +its device, i-number, and read/write pointer are stored in a system table +indexed by the file descriptor returned by the +.UL open +or +.UL create . +Thus, during a subsequent +call to read or write the +file, +the descriptor +may be easily related to the information necessary to access the file. +.PP +When a new file is created, +an i-node is allocated for it and a directory entry is made +that contains the name of the file and the i-node +number. +Making a link to an existing file involves +creating a directory entry with the new name, +copying the i-number from the original file entry, +and incrementing the link-count field of the i-node. +Removing (deleting) a file is done by +decrementing the +link-count of the i-node specified by its directory entry +and erasing the directory entry. +If the link-count drops to 0, +any disk blocks in the file +are freed and the i-node is de-allocated. +.PP +The space on all disks that +contain a file system is divided into a number of +512-byte +blocks logically addressed from 0 up to a limit that +depends on the device. +There is space in the i-node of each file for 13 device addresses. +For nonspecial files, +the first 10 device addresses point at the first +10 blocks of the file. +If the file is larger than 10 blocks, +the 11 device address points to an +indirect block containing up to 128 addresses +of additional blocks in the file. +Still larger files use the twelfth device address +of the i-node to point to +a double-indirect block naming +128 indirect blocks, +each +pointing to 128 blocks of the file. +If required, +the thirteenth device address is +a triple-indirect block. +Thus files may conceptually grow to +[\|(10+128+128\u\s62\s0\d+128\u\s63\s0\d)\*m512\|] bytes. +Once opened, +bytes numbered below 5120 can be read with a single +disk access; +bytes in the range 5120 to 70,656 +require two accesses; +bytes in the range 70,656 +to 8,459,264 +require three accesses; +bytes from there to the +largest file +(1,082,201,088) +require four accesses. +In practice, +a device cache mechanism +(see below) +proves effective in eliminating +most of the indirect fetches. +.PP +The foregoing discussion applies to ordinary files. +When an I/O request is made to a file whose i-node indicates that it +is special, +the last 12 device address words are immaterial, +and the first specifies +an internal +.IT "device name" , +which is interpreted as a pair of numbers +representing, +respectively, a device type +and subdevice number. +The device type indicates which +system routine will deal with I/O on that device; +the subdevice number selects, for example, a disk drive +attached to a particular controller or one of several +similar terminal interfaces. +.PP +In this environment, the implementation of the +.UL mount +system call (Section 3.4) is quite straightforward. +.UL \&mount +maintains a system table whose +argument is the i-number and device name of the +ordinary file specified +during the +.UL mount , +and whose corresponding value is the +device name of the indicated special file. +This table is searched for each i-number/device pair +that turns up while a path name is being scanned +during an +.UL open +or +.UL create ; +if a match is found, +the i-number is replaced by the i-number of the root +directory +and the device name is replaced by the table value. +.PP +To the user, both reading and writing of files appear to +be synchronous and unbuffered. +That is, immediately after +return from a +.UL read +call the data are available; conversely, +after a +.UL write +the user's workspace may be reused. +In fact, the system maintains a rather complicated +buffering mechanism that reduces greatly the number +of I/O operations required to access a file. +Suppose a +.UL write +call is made specifying transmission +of a single byte. +The system +will search its buffers to see +whether the affected disk block currently resides in main memory; +if not, it will be read in from the device. +Then the affected byte is replaced in the buffer and an +entry is made in a list of blocks to be written. +The return from the +.UL write +call may then take place, +although the actual I/O may not be completed until a later time. +Conversely, if a single byte is read, the system determines +whether the secondary storage block in which the byte is located is already +in one of the system's buffers; if so, the byte can be returned immediately. +If not, the block is read into a buffer and the byte picked out. +.PP +The system recognizes when +a program has +made accesses to +sequential blocks of a file, +and asynchronously +pre-reads the next block. +This significantly reduces +the running time of most programs +while adding little to +system overhead. +.PP +A program that reads or writes files in units of 512 bytes +has an advantage over a program that reads or writes +a single byte at a time, but the gain is not immense; +it comes mainly from the avoidance of system overhead. +If a program is used rarely or does +no great volume of I/O, it may quite reasonably +read and write in units as small as it wishes. +.PP +The notion of the i-list is an unusual feature +of +.UX . +In practice, this method of organizing the file system +has proved quite reliable and easy to deal with. +To the system itself, one of its strengths is +the fact that each file has a short, unambiguous name +related in a simple way to the protection, addressing, +and other information needed to access the file. +It also permits a quite simple and rapid +algorithm for checking the consistency of a file system, +for example, verification +that the portions of each device containing useful information +and those free to be allocated are disjoint and together +exhaust the space on the device. +This algorithm is independent +of the directory hierarchy, because it need only scan +the linearly organized i-list. +At the same time the notion of the i-list induces certain +peculiarities not found in other file system organizations. +For example, there is the question of who is to be charged +for the space a file occupies, +because all directory entries for a file have equal status. +Charging the owner of a file is unfair in general, +for one user may create a file, another may link to +it, and the first user may delete the file. +The first user is still the owner of the +file, but it should be charged +to the second user. +The simplest reasonably fair algorithm +seems to be to spread the charges +equally among users who have links to a file. +Many installations +avoid the +issue by not charging any fees at all. diff --git a/share/doc/psd/01.cacm/p3 b/share/doc/psd/01.cacm/p3 new file mode 100644 index 000000000000..2dc86d28e755 --- /dev/null +++ b/share/doc/psd/01.cacm/p3 @@ -0,0 +1,190 @@ +.\" This module is believed to contain source code proprietary to AT&T. +.\" Use and redistribution is subject to the Berkeley Software License +.\" Agreement and your Software Agreement with AT&T (Western Electric). +.\" +.\" @(#)p3 8.1 (Berkeley) 6/8/93 +.\" +.\" $FreeBSD$ +.SH +V. PROCESSES AND IMAGES +.PP +An +.IT image +is a computer execution environment. +It includes a memory image, +general register values, +status of open files, +current directory and the like. +An image is the current state of a pseudo-computer. +.PP +A +.IT process +is the execution of an image. +While the processor is executing on behalf of a process, +the image must reside in main memory; +during the execution of other processes it remains in main memory +unless the appearance of an active, higher-priority +process +forces it to be swapped out to the disk. +.PP +The user-memory part of an image is divided into three logical segments. +The program text segment begins at location 0 in the virtual address space. +During execution, this segment is write-protected +and a single copy of it is shared among +all processes executing the same program. +At the first hardware protection byte boundary above the program text segment in the +virtual address space begins a non-shared, writable data segment, +the size of which may be extended by a system call. +Starting at the highest +address in the virtual address space is a stack segment, +which automatically grows downward +as the stack pointer fluctuates. +.SH +5.1 Processes +.PP +Except while +the system +is bootstrapping itself into operation, a new +process can come into existence only +by use of the +.UL fork +system call: +.P1 +processid = fork\|(\|\|)\| +.P2 +When +.UL fork +is executed, the process +splits into two independently executing processes. +The two processes have independent +copies of the original memory image, +and share all open files. +The new processes differ only in that one is considered +the parent process: +in the parent, +the returned +.UL processid +actually identifies the child process +and is never 0, +while in the child, +the returned value is always 0. +.PP +Because the values returned by +.UL fork +in the parent and child process are distinguishable, +each process may determine whether +it is the parent or child. +.SH +5.2 Pipes +.PP +Processes may communicate +with related processes using the same system +.UL read +and +.UL write +calls that are used for file-system I/O. +The call: +.P1 +filep = pipe\|(\|\|)\| +.P2 +returns a file descriptor +.UL filep +and +creates an inter-process channel called a +.IT pipe . +This channel, like other open files, is passed from parent to child process in +the image by the +.UL fork +call. +A +.UL read +using a pipe file descriptor +waits until another process writes using the +file descriptor for the same pipe. +At this point, data are passed between the images of the +two processes. +Neither process need know that a pipe, +rather than an ordinary file, +is involved. +.PP +Although +inter-process communication +via pipes is a quite valuable tool +(see Section 6.2), +it is not a completely general +mechanism, +because the pipe must be set up by a common ancestor +of the processes involved. +.SH +5.3 Execution of programs +.PP +Another major system primitive +is invoked by +.P1 +execute\|(\|file, arg\*s\d1\u\*n, arg\*s\d2\u\*n, .\|.\|. , arg\*s\dn\u\*n\|)\| +.P2 +which requests the system to read in and execute the program +named by +.UL file , +passing it string arguments +.UL arg\v'.3'\*s1\*n\v'-.3'\| , +.UL arg\v'.3'\*s2\*n\v'-.3'\| , +.UL .\|.\|.\|\| , +.UL arg\v'.3'\*sn\*n\v'-.3' . +All the code and data in the process invoking +.UL execute +is replaced from the +.UL file , +but +open files, current directory, and +inter-process relationships are unaltered. +Only if the call fails, for example +because +.UL file +could not be found or because +its execute-permission bit was not set, does a return +take place from the +.UL execute +primitive; +it resembles a ``jump'' machine instruction +rather than a subroutine call. +.SH +5.4 Process synchronization +.PP +Another process control system call: +.P1 +processid = wait\|(\|status\|)\| +.P2 +causes its caller to suspend +execution until one of its children has completed execution. +Then +.UL wait +returns the +.UL processid +of the terminated process. +An error return is taken if the calling process has no +descendants. +Certain status from the child process +is also available. +.SH +5.5 Termination +.PP +Lastly: +.P1 +exit\|(\|status\|)\| +.P2 +terminates a process, +destroys its image, +closes its open files, +and generally obliterates it. +The parent is notified through +the +.UL wait +primitive, +and +.UL status +is made available +to it. +Processes may also terminate as a result of +various illegal actions or user-generated signals +(Section VII below). diff --git a/share/doc/psd/01.cacm/p4 b/share/doc/psd/01.cacm/p4 new file mode 100644 index 000000000000..09adb2bb6e8b --- /dev/null +++ b/share/doc/psd/01.cacm/p4 @@ -0,0 +1,524 @@ +.\" This module is believed to contain source code proprietary to AT&T. +.\" Use and redistribution is subject to the Berkeley Software License +.\" Agreement and your Software Agreement with AT&T (Western Electric). +.\" +.\" @(#)p4 8.1 (Berkeley) 6/8/93 +.\" +.\" $FreeBSD$ +.SH +VI. THE SHELL +.PP +For most users, +communication with +the system +is carried on with the +aid of a program called the \&shell. +The \&shell is a +command-line interpreter: it reads lines typed by the user and +interprets them as requests to execute +other programs. +(The \&shell is described fully elsewhere, +.[ +bourne shell bstj +%Q This issue +.] +so this section will discuss only the theory of its operation.) +In simplest form, a command line consists of the command +name followed by arguments to the command, all separated +by spaces: +.P1 +command arg\*s\d1\u\*n arg\*s\d2\u\*n .\|.\|. arg\*s\dn\u\*n +.P2 +The \&shell splits up the command name and the arguments into +separate strings. +Then a file with name +.UL command +is sought; +.UL command +may be a path name including the ``/'' character to +specify any file in the system. +If +.UL command +is found, it is brought into +memory and executed. +The arguments +collected by the \&shell are accessible +to the command. +When the command is finished, the \&shell +resumes its own execution, and indicates its readiness +to accept another command by typing a prompt character. +.PP +If file +.UL command +cannot be found, +the \&shell generally prefixes a string +such as +.UL /\|bin\|/ +to +.UL command +and +attempts again to find the file. +Directory +.UL /\|bin +contains commands +intended to be generally used. +(The sequence of directories to be searched +may be changed by user request.) +.SH +6.1 Standard I/O +.PP +The discussion of I/O in Section III above seems to imply that +every file used by a program must be opened or created by the program in +order to get a file descriptor for the file. +Programs executed by the \&shell, however, start off with +three open files with file descriptors +0, 1, and 2. +As such a program begins execution, file 1 is open for writing, +and is best understood as the standard output file. +Except under circumstances indicated below, this file +is the user's terminal. +Thus programs that wish to write informative +information ordinarily use file descriptor 1. +Conversely, file 0 starts off open for reading, and programs that +wish to read messages typed by the user +read this file. +.PP +The \&shell is able to change the standard assignments of +these file descriptors from the +user's terminal printer and keyboard. +If one of the +arguments to a command is prefixed by ``>'', file descriptor +1 will, for the duration of the command, refer to the +file named after the ``>''. +For example: +.P1 +ls +.P2 +ordinarily lists, on the typewriter, the names of the files in the current +directory. +The command: +.P1 +ls >there +.P2 +creates a file called +.UL there +and places the listing there. +Thus the argument +.UL >there +means +``place output on +.UL there .'' +On the other hand: +.P1 +ed +.P2 +ordinarily enters the editor, which takes requests from the +user via his keyboard. +The command +.P1 +ed <script +.P2 +interprets +.UL script +as a file of editor commands; +thus +.UL <script +means ``take input from +.UL script .'' +.PP +Although the file name following ``<'' or ``>'' appears +to be an argument to the command, in fact it is interpreted +completely by the \&shell and is not passed to the +command at all. +Thus no special coding to handle I/O redirection is needed within each +command; the command need merely use the standard file +descriptors 0 and 1 where appropriate. +.PP +File descriptor 2 is, like file 1, +ordinarily associated with the terminal output stream. +When an output-diversion request with ``>'' is specified, +file 2 remains attached to the terminal, so that commands +may produce diagnostic messages that +do not silently end up in the output file. +.SH +6.2 Filters +.PP +An extension of the standard I/O notion is used +to direct output from one command to +the input of another. +A sequence of commands separated by +vertical bars causes the \&shell to +execute all the commands simultaneously and to arrange +that the standard output of each command +be delivered to the standard input of +the next command in the sequence. +Thus in the command line: +.P1 +ls | pr \(mi2 | opr +.P2 +.UL ls +lists the names of the files in the current directory; +its output is passed to +.UL pr , +which +paginates its input with dated headings. +(The argument ``\(mi2'' requests +double-column output.) +Likewise, the output from +.UL pr +is input to +.UL opr ; +this command spools its input onto a file for off-line +printing. +.PP +This procedure could have been carried out +more clumsily by: +.P1 +ls >temp1 +pr \(mi2 <temp1 >temp2 +opr <temp2 +.P2 +followed by removal of the temporary files. +In the absence of the ability +to redirect output and input, +a still clumsier method would have been to +require the +.UL ls +command +to accept user requests to paginate its output, +to print in multi-column format, and to arrange +that its output be delivered off-line. +Actually it would be surprising, and in fact +unwise for efficiency reasons, +to expect authors of +commands such as +.UL ls +to provide such a wide variety of output options. +.PP +A program +such as +.UL pr +which copies its standard input to its standard output +(with processing) +is called a +.IT filter . +Some filters that we have found useful +perform +character transliteration, +selection of lines according to a pattern, +sorting of the input, +and encryption and decryption. +.SH +6.3 Command separators; multitasking +.PP +Another feature provided by the \&shell is relatively straightforward. +Commands need not be on different lines; instead they may be separated +by semicolons: +.P1 +ls; ed +.P2 +will first list the contents of the current directory, then enter +the editor. +.PP +A related feature is more interesting. +If a command is followed +by ``\f3&\f1,'' the \&shell will not wait for the command to finish before +prompting again; instead, it is ready immediately +to accept a new command. +For example: +.bd 3 +.P1 +as source >output & +.P2 +causes +.UL source +to be assembled, with diagnostic +output going to +.UL output ; +no matter how long the +assembly takes, the \&shell returns immediately. +When the \&shell does not wait for +the completion of a command, +the identification number of the +process running that command is printed. +This identification may be used to +wait for the completion of the command or to +terminate it. +The ``\f3&\f1'' may be used +several times in a line: +.P1 +as source >output & ls >files & +.P2 +does both the assembly and the listing in the background. +In these examples, an output file +other than the terminal was provided; if this had not been +done, the outputs of the various commands would have been +intermingled. +.PP +The \&shell also allows parentheses in the above operations. +For example: +.P1 +(\|date; ls\|) >x & +.P2 +writes the current date and time followed by +a list of the current directory onto the file +.UL x . +The \&shell also returns immediately for another request. +.SH 1 +6.4 The \&shell as a command; command files +.PP +The \&shell is itself a command, and may be called recursively. +Suppose file +.UL tryout +contains the lines: +.P1 +as source +mv a.out testprog +testprog +.P2 +The +.UL mv +command causes the file +.UL a.out +to be renamed +.UL testprog. +.UL \&a.out +is the (binary) output of the assembler, ready to be executed. +Thus if the three lines above were typed on the keyboard, +.UL source +would be assembled, the resulting program renamed +.UL testprog , +and +.UL testprog +executed. +When the lines are in +.UL tryout , +the command: +.P1 +sh <tryout +.P2 +would cause the \&shell +.UL sh +to execute the commands +sequentially. +.PP +The \&shell has further capabilities, including the +ability to substitute parameters +and +to construct argument lists from a specified +subset of the file names in a directory. +It also provides general conditional and looping constructions. +.SH 1 +6.5 Implementation of the \&shell +.PP +The outline of the operation of the \&shell can now be understood. +Most of the time, the \&shell +is waiting for the user to type a command. +When the +newline character ending the line +is typed, the \&shell's +.UL read +call returns. +The \&shell analyzes the command line, putting the +arguments in a form appropriate for +.UL execute . +Then +.UL fork +is called. +The child process, whose code +of course is still that of the \&shell, attempts +to perform an +.UL execute +with the appropriate arguments. +If successful, this will bring in and start execution of the program whose name +was given. +Meanwhile, the other process resulting from the +.UL fork , +which is the +parent process, +.UL wait s +for the child process to die. +When this happens, the \&shell knows the command is finished, so +it types its prompt and reads the keyboard to obtain another +command. +.PP +Given this framework, the implementation of background processes +is trivial; whenever a command line contains ``\f3&\f1,'' +the \&shell merely refrains from waiting for the process +that it created +to execute the command. +.PP +Happily, all of this mechanism meshes very nicely with +the notion of standard input and output files. +When a process is created by the +.UL fork +primitive, it +inherits not only the memory image of its parent +but also all the files currently open in its parent, +including those with file descriptors 0, 1, and 2. +The \&shell, of course, uses these files to read command +lines and to write its prompts and diagnostics, and in the ordinary case +its children\(emthe command programs\(eminherit them automatically. +When an argument with ``<'' or ``>'' is given, however, the +offspring process, just before it performs +.UL execute, +makes the standard I/O +file descriptor (0 or 1, respectively) refer to the named file. +This is easy +because, by agreement, +the smallest unused file descriptor is assigned +when a new file is +.UL open ed +(or +.UL create d); +it is only necessary to close file 0 (or 1) +and open the named file. +Because the process in which the command program runs simply terminates +when it is through, the association between a file +specified after ``<'' or ``>'' and file descriptor 0 or 1 is ended +automatically when the process dies. +Therefore +the \&shell need not know the actual names of the files +that are its own standard input and output, because it need +never reopen them. +.PP +Filters are straightforward extensions +of standard I/O redirection with pipes used +instead of files. +.PP +In ordinary circumstances, the main loop of the \&shell never +terminates. +(The main loop includes the +branch of the return from +.UL fork +belonging to the +parent process; that is, the branch that does a +.UL wait , +then +reads another command line.) +The one thing that causes the \&shell to terminate is +discovering an end-of-file condition on its input file. +Thus, when the \&shell is executed as a command with +a given input file, as in: +.P1 +sh <comfile +.P2 +the commands in +.UL comfile +will be executed until +the end of +.UL comfile +is reached; then the instance of the \&shell +invoked by +.UL sh +will terminate. +Because this \&shell process +is the child of another instance of the \&shell, the +.UL wait +executed in the latter will return, and another +command may then be processed. +.SH +6.6 Initialization +.PP +The instances of the \&shell to which users type +commands are themselves children of another process. +The last step in the initialization of +the system +is the creation of +a single process and the invocation (via +.UL execute ) +of a program called +.UL init . +The role of +.UL init +is to create one process +for each terminal channel. +The various subinstances of +.UL init +open the appropriate terminals +for input and output +on files 0, 1, and 2, +waiting, if necessary, for carrier to be established on dial-up lines. +Then a message is typed out requesting that the user log in. +When the user types a name or other identification, +the appropriate instance of +.UL init +wakes up, receives the log-in +line, and reads a password file. +If the user's name is found, and if +he is able to supply the correct password, +.UL init +changes to the user's default current directory, sets +the process's user \*sID\*n to that of the person logging in, and performs +an +.UL execute +of the \&shell. +At this point, the \&shell is ready to receive commands +and the logging-in protocol is complete. +.PP +Meanwhile, the mainstream path of +.UL init +(the parent of all +the subinstances of itself that will later become \&shells) +does a +.UL wait . +If one of the child processes terminates, either +because a \&shell found an end of file or because a user +typed an incorrect name or password, this path of +.UL init +simply recreates the defunct process, which in turn reopens the appropriate +input and output files and types another log-in message. +Thus a user may log out simply by typing the end-of-file +sequence to the \&shell. +.SH +6.7 Other programs as \&shell +.PP +The \&shell as described above is designed to allow users +full access to the facilities of the system, because it will +invoke the execution of any program +with appropriate protection mode. +Sometimes, however, a different interface to the system +is desirable, and this feature is easily arranged for. +.PP +Recall that after a user has successfully logged in by supplying +a name and password, +.UL init +ordinarily invokes the \&shell +to interpret command lines. +The user's entry +in the password file may contain the name +of a program to be invoked after log-in instead of the \&shell. +This program is free to interpret the user's messages +in any way it wishes. +.PP +For example, the password file entries +for users of a secretarial editing system +might +specify that the +editor +.UL ed +is to be used instead of the \&shell. +Thus when users of the editing system log in, they are inside the editor and +can begin work immediately; also, they can be prevented from +invoking +programs not intended for their use. +In practice, it has proved desirable to allow a temporary +escape from the editor +to execute the formatting program and other utilities. +.PP +Several of the games (e.g., chess, blackjack, 3D tic-tac-toe) +available on +the system +illustrate +a much more severely restricted environment. +For each of these, an entry exists +in the password file specifying that the appropriate game-playing +program is to be invoked instead of the \&shell. +People who log in as a player +of one of these games find themselves limited to the +game and unable to investigate the (presumably more interesting) +offerings of +the +.UX +system +as a whole. diff --git a/share/doc/psd/01.cacm/p5 b/share/doc/psd/01.cacm/p5 new file mode 100644 index 000000000000..cf40f2d6b016 --- /dev/null +++ b/share/doc/psd/01.cacm/p5 @@ -0,0 +1,235 @@ +.\" This module is believed to contain source code proprietary to AT&T. +.\" Use and redistribution is subject to the Berkeley Software License +.\" Agreement and your Software Agreement with AT&T (Western Electric). +.\" +.\" @(#)p5 8.1 (Berkeley) 6/8/93 +.\" +.\" $FreeBSD$ +.SH +VII. TRAPS +.PP +The \*sPDP\*n-11 hardware detects a number of program faults, +such as references to non-existent memory, unimplemented instructions, +and odd addresses used where an even address is required. +Such faults cause the processor to trap to a system routine. +Unless other arrangements have been made, +an illegal action causes the system +to terminate the process and to write its +image +on file +.UL core +in the current directory. +A debugger can be used to determine +the state of the program at the time of the fault. +.PP +Programs that are looping, that produce unwanted output, or about which +the user has second thoughts may be halted by the use of the +.UL interrupt +signal, which is generated by typing the ``delete'' +character. +Unless special action has been taken, this +signal simply causes the program to cease execution +without producing a +.UL core +file. +There is also a +.UL quit +signal +used to force an image file to be produced. +Thus programs that loop unexpectedly may be +halted and the remains inspected without prearrangement. +.PP +The hardware-generated faults +and the interrupt and quit signals +can, by request, be either ignored or caught by a process. +For example, +the \&shell ignores quits to prevent +a quit from logging the user out. +The editor catches interrupts and returns +to its command level. +This is useful for stopping long printouts +without losing work in progress (the editor +manipulates a copy of the file it is editing). +In systems without floating-point hardware, +unimplemented instructions are caught +and floating-point instructions are +interpreted. +.SH +VIII. PERSPECTIVE +.PP +Perhaps paradoxically, +the success of +the +.UX +system +is largely due to the fact that it was not +designed to meet any +predefined objectives. +The first version was written when one of us +(Thompson), +dissatisfied with the available computer facilities, +discovered a little-used \*sPDP\*n-7 +and set out to create a more +hospitable environment. +This (essentially personal) effort was +sufficiently successful +to gain the interest of the other author +and several colleagues, +and later to justify the acquisition +of the \*sPDP\*n-11/20, specifically to support +a text editing and formatting system. +When in turn the 11/20 was outgrown, +the system +had proved useful enough to persuade management to +invest in the \*sPDP\*n-11/45, +and later in the +\*sPDP\*n-11/70 and Interdata 8/32 machines, +upon which it developed to its present form. +Our goals throughout the effort, +when articulated at all, have always been to build +a comfortable relationship with the machine +and to explore ideas and inventions in operating systems +and other software. +We have not been faced with the need to satisfy someone +else's requirements, +and for this freedom we are grateful. +.PP +Three considerations that influenced the design of +.UX +are visible in retrospect. +.PP +First: +because we are programmers, +we naturally designed the system to make it easy to +write, test, and run programs. +The most important expression of our desire for +programming convenience +was that the system +was arranged for interactive use, +even though the original version only +supported one user. +We believe that a properly designed +interactive system is much more +productive +and satisfying to use than a ``batch'' system. +Moreover, such a system is rather easily +adaptable to noninteractive use, while the converse is not true. +.PP +Second: +there have always been fairly severe size constraints +on the system and its software. +Given the partially antagonistic desires for reasonable efficiency and +expressive power, +the size constraint has encouraged +not only economy, but also a certain elegance of design. +This may be a thinly disguised version of the ``salvation +through suffering'' philosophy, +but in our case it worked. +.PP +Third: nearly from the start, the system was able to, and did, maintain itself. +This fact is more important than it might seem. +If designers of a system are forced to use that system, +they quickly become aware of its functional and superficial deficiencies +and are strongly motivated to correct them before it is too late. +Because all source programs were always available +and easily modified on-line, +we were willing to revise and rewrite the system and its software +when new ideas were invented, discovered, +or suggested by others. +.PP +The aspects of +.UX +discussed in this paper exhibit clearly +at least the first two of these +design considerations. +The interface to the file +system, for example, is extremely convenient from +a programming standpoint. +The lowest possible interface level is designed +to eliminate distinctions +between +the various devices and files and between +direct and sequential access. +No large ``access method'' routines +are required +to insulate the programmer from the +system calls; +in fact, all user programs either call the system +directly or +use a small library program, less than a page long, +that buffers a number of characters +and reads or writes them all at once. +.PP +Another important aspect of programming +convenience is that there are no ``control blocks'' +with a complicated structure partially maintained by +and depended on by the file system or other system calls. +Generally speaking, the contents of a program's address space +are the property of the program, and we have tried to +avoid placing restrictions +on the data structures within that address space. +.PP +Given the requirement +that all programs should be usable with any file or +device as input or output, +it is also desirable +to push device-dependent considerations +into the operating system itself. +The only alternatives seem to be to load, +with all programs, +routines for dealing with each device, +which is expensive in space, +or to depend on some means of dynamically linking to +the routine appropriate to each device when it is actually +needed, +which is expensive either in overhead or in hardware. +.PP +Likewise, +the process-control scheme and the command interface +have proved both convenient and efficient. +Because the \&shell operates as an ordinary, swappable +user program, +it consumes no ``wired-down'' space in the system proper, +and it may be made as powerful as desired +at little cost. +In particular, +given the framework in which the \&shell executes +as a process that spawns other processes to +perform commands, +the notions of I/O redirection, background processes, +command files, and user-selectable system interfaces +all become essentially trivial to implement. +.SH +Influences +.PP +The success of +.UX +lies +not so much in new inventions +but rather in the full exploitation of a carefully selected +set of fertile ideas, +and especially in showing that +they can be keys to the implementation of a small +yet powerful operating system. +.PP +The +.UL fork +operation, essentially as we implemented it, was +present in the \*sGENIE\*n time-sharing system. +.[ +lampson deutsch 930 manual 1965 system preliminary +.] +On a number of points we were influenced by Multics, +which suggested the particular form of the I/O system calls +.[ +multics input output feiertag organick +.] +and both the name of the \&shell and its general functions. +The notion that the \&shell should create a process +for each command was also suggested to us by +the early design of Multics, although in that +system it was later dropped for efficiency reasons. +A similar scheme is used by \*sTENEX\*n. +.[ +bobrow burchfiel tenex +.] diff --git a/share/doc/psd/01.cacm/p6 b/share/doc/psd/01.cacm/p6 new file mode 100644 index 000000000000..77af7e762292 --- /dev/null +++ b/share/doc/psd/01.cacm/p6 @@ -0,0 +1,72 @@ +.\" This module is believed to contain source code proprietary to AT&T. +.\" Use and redistribution is subject to the Berkeley Software License +.\" Agreement and your Software Agreement with AT&T (Western Electric). +.\" +.\" @(#)p6 8.1 (Berkeley) 6/8/93 +.\" +.\" $FreeBSD$ +.SH +IX. STATISTICS +.PP +The following numbers +are presented to suggest the scale of the Research +.UX +operation. +Those of our users +not involved in document preparation +tend to use the system for +program development, especially language work. +There are few important +``applications'' programs. +.PP +Overall, we have today: +.PP +.SP .5 +.TS +center; +r5 l. +125 user population +33 maximum simultaneous users +1,630 directories +28,300 files +301,700 512-byte secondary storage blocks used +.TE +.SP .5 +There is a ``background'' process that +runs at the lowest possible priority; it is used +to soak up any idle \*sCPU\*n time. +It has been used to produce a million-digit +approximation to the constant \fIe\fR, +and other semi-infinite problems. +Not counting this background work, we average daily: +.SP .5 +.TS +center; +r 5 l. +13,500 commands +9.6 \*sCPU\*n hours +230 connect hours +62 different users +240 log-ins +.TE +.SP .5 +.SH +X. ACKNOWLEDGMENTS +.PP +The contributors to +.UX +are, in the traditional but here especially apposite +phrase, too numerous to mention. +Certainly, collective salutes are due to our colleagues in the +Computing Science Research Center. +R. H. Canaday contributed much to the basic design of the +file system. +We are particularly appreciative +of the inventiveness, +thoughtful criticism, +and constant support of +R. Morris, M. D. McIlroy, +and J. F. Ossanna. +.[ +$LIST$ +.] diff --git a/share/doc/psd/01.cacm/ref.bib b/share/doc/psd/01.cacm/ref.bib new file mode 100644 index 000000000000..c4283b5806c3 --- /dev/null +++ b/share/doc/psd/01.cacm/ref.bib @@ -0,0 +1,113 @@ +# $FreeBSD$ + +%A L. P. Deutsch +%A B. W. Lampson +%T An online editor +%J Comm. Assoc. Comp. Mach. +%V 10 +%N 12 +%D December 1967 +%P 793-799, 803 +%K qed + +%K cstr +%R Comp. Sci. Tech. Rep. No. 17 +%I Bell Laboratories +%C Murray Hill, New Jersey +%A B. W. Kernighan +%A L. L. Cherry +%T A System for Typesetting Mathematics +%d May 1974, revised April 1977 +%J Comm. Assoc. Comp. Mach. +%K acm cacm +%V 18 +%P 151-157 +%D March 1975 + +%T U\s-2NIX\s0 Time-Sharing System: Document Preparation +%K unix bstj +%A B. W. Kernighan +%A M. E. Lesk +%A J. F. Ossanna +%J Bell Sys. Tech. J. +%V 57 +%N 6 +%P 2115-2135 +%D 1978 + +%A T. A. Dolotta +%A J. R. Mashey +%T An Introduction to the Programmer's Workbench +%J Proc. 2nd Int. Conf. on Software Engineering +%D October 13-15, 1976 +%P 164-168 + +%T U\s-2NIX\s0 Time-Sharing System: The Programmer's Workbench +%A T. A. Dolotta +%A R. C. Haight +%A J. R. Mashey +%J Bell Sys. Tech. J. +%V 57 +%N 6 +%P 2177-2200 +%D 1978 +%K unix bstj + +%T U\s-2NIX\s0 Time-Sharing System: U\s-2NIX\s0 on a Microprocessor +%K unix bstj +%A H. Lycklama +%J Bell Sys. Tech. J. +%V 57 +%N 6 +%P 2087-2101 +%D 1978 + +%T The C Programming Language +%A B. W. Kernighan +%A D. M. Ritchie +%I Prentice-Hall +%C Englewood Cliffs, New Jersey +%D 1978 + +%T Computer Recreations +%A Aleph-null +%J Software Practice and Experience +%V 1 +%N 2 +%D April-June 1971 +%P 201-204 + +%T U\s-2NIX\s0 Time-Sharing System: The U\s-2NIX\s0 Shell +%A S. R. Bourne +%K unix bstj +%J Bell Sys. Tech. J. +%V 57 +%N 6 +%P 1971-1990 +%D 1978 + +%A L. P. Deutsch +%A B. W. Lampson +%T \*sSDS\*n 930 time-sharing system preliminary reference manual +%R Doc. 30.10.10, Project \*sGENIE\*n +%C Univ. Cal. at Berkeley +%D April 1965 + +%A R. J. Feiertag +%A E. I. Organick +%T The Multics input-output system +%J Proc. Third Symposium on Operating Systems Principles +%D October 18-20, 1971 +%P 35-41 + +%A D. G. Bobrow +%A J. D. Burchfiel +%A D. L. Murphy +%A R. S. Tomlinson +%T \*sTENEX\*n, a Paged Time Sharing System for the \*sPDP\*n-10 +%J Comm. Assoc. Comp. Mach. +%V 15 +%N 3 +%D March 1972 +%K tenex +%P 135-143 diff --git a/share/doc/psd/02.implement/Makefile b/share/doc/psd/02.implement/Makefile new file mode 100644 index 000000000000..89d0dc3c3b95 --- /dev/null +++ b/share/doc/psd/02.implement/Makefile @@ -0,0 +1,17 @@ +# @(#)Makefile 8.1 (Berkeley) 6/8/93 +# $FreeBSD$ + +VOLUME= psd/02.implement +SRCS= stubs implement +EXTRA= ref.bib +MACROS= -ms +USE_PIC= +USE_REFER= +USE_SOELIM= +CLEANFILES= stubs + +stubs: + @(echo .R1; echo database ${.CURDIR}/ref.bib; \ + echo accumulate; echo .R2) > ${.TARGET} + +.include <bsd.doc.mk> diff --git a/share/doc/psd/02.implement/fig1.pic b/share/doc/psd/02.implement/fig1.pic new file mode 100644 index 000000000000..2d4b3d3c2231 --- /dev/null +++ b/share/doc/psd/02.implement/fig1.pic @@ -0,0 +1,100 @@ +.\" Copyright (c) 1986, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Copyright (C) Caldera International Inc. 2001-2002. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions are +.\" met: +.\" +.\" Redistributions of source code and documentation must retain the above +.\" copyright notice, this list of conditions and the following +.\" disclaimer. +.\" +.\" Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" +.\" All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" +.\" This product includes software developed or owned by Caldera +.\" International, Inc. Neither the name of Caldera International, Inc. +.\" nor the names of other contributors may be used to endorse or promote +.\" products derived from this software without specific prior written +.\" permission. +.\" +.\" USE OF THE SOFTWARE PROVIDED FOR UNDER THIS LICENSE BY CALDERA +.\" INTERNATIONAL, INC. AND CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR +.\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +.\" WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +.\" DISCLAIMED. IN NO EVENT SHALL CALDERA INTERNATIONAL, INC. BE LIABLE +.\" FOR ANY DIRECT, INDIRECT INCIDENTAL, SPECIAL, EXEMPLARY, OR +.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR +.\" BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, +.\" WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE +.\" OR OTHERWISE) RISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN +.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +.\" +.\" @(#)fig1.pic 8.1 (Berkeley) 6/8/93 +.\" +.\" $FreeBSD$ +.PS +.ps 9 +[ + PT: [ + T: box invis ht .2i "Process Table"; move down .125i + A: box ht .25i; down + PTE: box "Process" "Table" "Entry"; down + C: box ht .25i + ] + move right 1.5i + TT: [ + T: box invis ht .2i "Text Table"; move down .125i + A: box ht .25i; down + TTE: box "Text" "Table" "Entry"; down + C: box ht .25i + ] + move down 1i from TT.C.s + move right 0.5i + UTS: [ + box ht 0.75i wid 0.75i "User" "Text" "Segment" + ] + move left 1.5i from UTS.w + DS: [ + SDS: box "System" "Data" "Segment" ; move down .5i from SDS.n ; + UDS: box ht 0.75i "User" "Data" "Segment" + ] + move left 1i from DS.UDS.w + move down 0.25i + UAS: [ + box invis "User" "Address" "Space" + ] + line from UAS.ne to UAS.se + line from UAS.nw to UAS.sw + line right 0.15i from UAS.nw + line right 0.15i from UAS.sw + line left 0.15i from UAS.ne + line left 0.15i from UAS.se + arrow from 1/4 of the way between PT.PTE.ne and PT.PTE.se right 1.875i + arrow from TT.TTE.e right .5i then down to UTS.n + arrow from PT.PTE.e right .875i then down to DS.SDS.n + arrow from 3/4 of the way between PT.PTE.ne and PT.PTE.se right .25i then down 1.5i then right .25i + arrow from 1/4 of the way between UAS.ne and UAS.se right .375i then up .25i then right .25i + arrow from 3/4 of the way between UAS.ne and UAS.se right 2.375i then up .875i then right .5i + move up 1.3175i from UAS.nw + move left .75i + line right 5.625i + move left 5.25i + move up .3125i + RS: [ + box invis ht 0.2i "Resident" + ] + move down .8i + SW: [ + box invis ht 0.2i "Swapped" + ] + arrow <-> from RS.s to SW.n +] +.PE diff --git a/share/doc/psd/02.implement/fig2.pic b/share/doc/psd/02.implement/fig2.pic new file mode 100644 index 000000000000..2dc291556d02 --- /dev/null +++ b/share/doc/psd/02.implement/fig2.pic @@ -0,0 +1,110 @@ +.\" Copyright (c) 1986, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Copyright (C) Caldera International Inc. 2001-2002. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions are +.\" met: +.\" +.\" Redistributions of source code and documentation must retain the above +.\" copyright notice, this list of conditions and the following +.\" disclaimer. +.\" +.\" Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" +.\" All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" +.\" This product includes software developed or owned by Caldera +.\" International, Inc. Neither the name of Caldera International, Inc. +.\" nor the names of other contributors may be used to endorse or promote +.\" products derived from this software without specific prior written +.\" permission. +.\" +.\" USE OF THE SOFTWARE PROVIDED FOR UNDER THIS LICENSE BY CALDERA +.\" INTERNATIONAL, INC. AND CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR +.\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +.\" WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +.\" DISCLAIMED. IN NO EVENT SHALL CALDERA INTERNATIONAL, INC. BE LIABLE +.\" FOR ANY DIRECT, INDIRECT INCIDENTAL, SPECIAL, EXEMPLARY, OR +.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR +.\" BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, +.\" WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE +.\" OR OTHERWISE) RISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN +.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +.\" +.\" @(#)fig2.pic 8.1 (Berkeley) 6/8/93 +.\" +.\" $FreeBSD$ +.PS +.ps 9 +[ + PUOFT: [ + A: box invis ht .4i wid 1i "Per-User Open" "File Table" + B: box ht .25i with .n at A.s + C: box with .n at B.s + D: box ht .25i with .n at C.s + ] + move down 1.0625i left 1.25i from PUOFT.D.s + OFT: [ + A: box invis ht .4i wid 1i "Open File" "Table" + B: box ht .25i with .n at A.s + C: box with .n at B.s + D: box ht .25i with .n at C.s + ] + move down 1.0625i right 1.25i from PUOFT.D.s + AIT: [ + A: box invis ht .4i wid 1i "Active I-node" "Table" + B: box ht .25i with .n at A.s + C: box with .n at B.s + D: box ht .25i with .n at C.s + ] + move down 2.5i from PUOFT.D.s + IF: [ + A: box ht .25i + B: box ht .25i "I-node" with .n at A.s + C: box ht .25i with .n at B.s + D: box ht .25i "File" with .n at C.s + E: box ht .25i with .n at D.s + ] + move right 1.5i from IF.D.w + FMA: [ + box invis "File" "Mapping" "Algorithms" + ] + line from FMA.ne to FMA.se + line from FMA.nw to FMA.sw + line left .15i from FMA.se + line left .15i from FMA.ne + line right .15i from FMA.nw + line right .15i from FMA.sw + + arrow from FMA.w to IF.D.e + arrow from AIT.C.e right .25i then down 2.125i then left .5i + arrow from OFT.C.e to AIT.C.w + arrow from PUOFT.C.w left .5i then down 1.625i then left .5i + arrow <-> from IF.B.e right .5i then up 1.5i then right .5i + + move up .1875i from OFT.A.nw + line right 5i + move left 5i down 1.9375i + line right 5i + + move up 1.63475i right 2.75i from PUOFT.D.s + line right .1i down .1i then down .6i then right .1i down .1i then left .1i down .1i then down .6i then left .1i down .1i + move down .34375i right 2.75i from PUOFT.D.s + line right .1i down .1i then down .6i then right .1i down .1i then left .1i down .1i then down .6i then left .1i down .1i + move down 2.34375i right 2.75i from PUOFT.D.s + line right .1i down .1i then down .6i then right .1i down .1i then left .1i down .1i then down .6i then left .1i down .1i + + move up 0.817375i right 2.9i from PUOFT.D.s + box invis "Swapped" "Per User" + move down 1.15625i right 2.9i from PUOFT.D.s + box invis wid 1i "Resident" "Per System" + move down 3.15675i right 2.9i from PUOFT.D.s + box invis ht 1i wid 1i "Secondary" "Storage" "Per" "File System" +] +.PE diff --git a/share/doc/psd/02.implement/implement b/share/doc/psd/02.implement/implement new file mode 100644 index 000000000000..f6ad7d759191 --- /dev/null +++ b/share/doc/psd/02.implement/implement @@ -0,0 +1,1282 @@ +.\" Copyright (c) 1986, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Copyright (C) Caldera International Inc. 2001-2002. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions are +.\" met: +.\" +.\" Redistributions of source code and documentation must retain the above +.\" copyright notice, this list of conditions and the following +.\" disclaimer. +.\" +.\" Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" +.\" All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" +.\" This product includes software developed or owned by Caldera +.\" International, Inc. Neither the name of Caldera International, Inc. +.\" nor the names of other contributors may be used to endorse or promote +.\" products derived from this software without specific prior written +.\" permission. +.\" +.\" USE OF THE SOFTWARE PROVIDED FOR UNDER THIS LICENSE BY CALDERA +.\" INTERNATIONAL, INC. AND CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR +.\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +.\" WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +.\" DISCLAIMED. IN NO EVENT SHALL CALDERA INTERNATIONAL, INC. BE LIABLE +.\" FOR ANY DIRECT, INDIRECT INCIDENTAL, SPECIAL, EXEMPLARY, OR +.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR +.\" BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, +.\" WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE +.\" OR OTHERWISE) RISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN +.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +.\" +.\" @(#)implement 8.1 (Berkeley) 6/8/93 +.\" +.\" $FreeBSD$ +.EH 'PSD:2-%''UNIX Implementation' +.OH 'UNIX Implementation''PSD:2-%' +.de P1 +.DS +.. +.de P2 +.DE +.. +.de UL +.lg 0 +.if n .ul +\%\&\\$3\f3\\$1\fR\&\\$2 +.lg +.. +.de UC +\&\\$3\s-1\\$1\\s0\&\\$2 +.. +.de IT +.lg 0 +.if n .ul +\%\&\\$3\f2\\$1\fR\&\\$2 +.lg +.. +.de SP +.sp \\$1 +.. +.hw device +.TL +UNIX Implementation +.AU "MH 2C-523" 2394 +K. Thompson +.AI +AT&T Bell Laboratories +Murray Hill, NJ +.AB +This paper describes in high-level terms the +implementation of the resident +.UX +kernel. +This discussion is broken into three parts. +The first part describes +how the +.UX +system views processes, users, and programs. +The second part describes the I/O system. +The last part describes the +.UX +file system. +.AE +.NH +INTRODUCTION +.PP +The +.UX +kernel consists of about 10,000 +lines of C code and about 1,000 lines of assembly code. +The assembly code can be further broken down into +200 lines included for +the sake of efficiency +(they could have been written in C) +and 800 lines to perform hardware +functions not possible in C. +.PP +This code represents 5 to 10 percent of what has +been lumped into the broad expression +``the +.UX +operating system.'' +The kernel is the only +.UX +code that +cannot be substituted by a user to his +own liking. +For this reason, +the kernel should make as few real +decisions as possible. +This does not mean to allow the user +a million options to do the same thing. +Rather, it means to allow only one way to +do one thing, +but have that way be the least-common divisor +of all the options that might have been provided. +.PP +What is or is not implemented in the kernel +represents both a great responsibility and a great power. +It is a soap-box platform on +``the way things should be done.'' +Even so, if +``the way'' is too radical, +no one will follow it. +Every important decision was weighed +carefully. +Throughout, +simplicity has been substituted for efficiency. +Complex algorithms are used only if +their complexity can be localized. +.NH +PROCESS CONTROL +.PP +In the +.UX +system, +a user executes programs in an +environment called a user process. +When a system function is required, +the user process calls the system +as a subroutine. +At some point in this call, +there is a distinct switch of environments. +After this, +the process is said to be a system process. +In the normal definition of processes, +the user and system processes are different +phases of the same process +(they never execute simultaneously). +For protection, +each system process has its own stack. +.PP +The user process may execute +from a read-only text segment, +which is shared by all processes +executing the same code. +There is no +.IT functional +benefit +from shared-text segments. +An +.IT efficiency +benefit comes from the fact +that there is no need to swap read-only +segments out because the original +copy on secondary memory is still current. +This is a great benefit to interactive +programs that tend to be swapped while +waiting for terminal input. +Furthermore, +if two processes are +executing +simultaneously +from the same copy of a read-only segment, +only one copy needs to reside in +primary memory. +This is a secondary effect, +because +simultaneous execution of a program +is not common. +It is ironic that this effect, +which reduces the use of primary memory, +only comes into play when there is +an overabundance of primary memory, +that is, +when there is enough memory +to keep waiting processes loaded. +.PP +All current read-only text segments in the +system are maintained from the +.IT "text table" . +A text table entry holds the location of the +text segment on secondary memory. +If the segment is loaded, +that table also holds the primary memory location +and the count of the number of processes +sharing this entry. +When this count is reduced to zero, +the entry is freed along with any +primary and secondary memory holding the segment. +When a process first executes a shared-text segment, +a text table entry is allocated and the +segment is loaded onto secondary memory. +If a second process executes a text segment +that is already allocated, +the entry reference count is simply incremented. +.PP +A user process has some strictly private +read-write data +contained in its +data segment. +As far as possible, +the system does not use the user's +data segment to hold system data. +In particular, +there are no I/O buffers in the +user address space. +.PP +The user data segment has two growing boundaries. +One, increased automatically by the system +as a result of memory faults, +is used for a stack. +The second boundary is only grown (or shrunk) by +explicit requests. +The contents of newly allocated primary memory +is initialized to zero. +.PP +Also associated and swapped with +a process is a small fixed-size +system data segment. +This segment contains all +the data about the process +that the system needs only when the +process is active. +Examples of the kind of data contained +in the system data segment are: +saved central processor registers, +open file descriptors, +accounting information, +scratch data area, +and the stack for the system phase +of the process. +The system data segment is not +addressable from the user process +and is therefore protected. +.PP +Last, +there is a process table with +one entry per process. +This entry contains all the data +needed by the system when the process +is +.IT not +active. +Examples are +the process's name, +the location of the other segments, +and scheduling information. +The process table entry is allocated +when the process is created, and freed +when the process terminates. +This process entry is always directly +addressable by the kernel. +.PP +Figure 1 shows the relationships +between the various process control +data. +In a sense, +the process table is the +definition of all processes, +because +all the data associated with a process +may be accessed +starting from the process table entry. +.KF +.if t .in .375i +.so fig1.pic +.if t .in -.375i +.sp 2v +.ce +Fig. 1\(emProcess control data structure. +.KE +.NH 2 +Process creation and program execution +.PP +Processes are created by the system primitive +.UL fork . +The newly created process (child) is a copy of the original process (parent). +There is no detectable sharing of primary memory between the two processes. +(Of course, +if the parent process was executing from a read-only +text segment, +the child will share the text segment.) +Copies of all writable data segments +are made for the child process. +Files that were open before the +.UL fork +are +truly shared after the +.UL fork . +The processes are informed as to their part in the +relationship to +allow them to select their own +(usually non-identical) +destiny. +The parent may +.UL wait +for the termination of +any of its children. +.PP +A process may +.UL exec +a file. +This consists of exchanging the current text and data +segments of the process for new text and data +segments specified in the file. +The old segments are lost. +Doing an +.UL exec +does +.IT not +change processes; +the process that did the +.UL exec +persists, +but +after the +.UL exec +it is executing a different program. +Files that were open +before the +.UL exec +remain open after the +.UL exec . +.PP +If a program, +say the first pass of a compiler, +wishes to overlay itself with another program, +say the second pass, +then it simply +.UL exec s +the second program. +This is analogous +to a ``goto.'' +If a program wishes to regain control +after +.UL exec ing +a second program, +it should +.UL fork +a child process, +have the child +.UL exec +the second program, and +have the parent +.UL wait +for the child. +This is analogous to a ``call.'' +Breaking up the call into a binding followed by +a transfer is similar to the subroutine linkage in +SL-5. +.[ +griswold hanson sl5 overview +.] +.NH 2 +Swapping +.PP +The major data associated with a process +(the user data segment, +the system data segment, and +the text segment) +are swapped to and from secondary +memory, as needed. +The user data segment and the system data segment +are kept in contiguous primary memory to reduce +swapping latency. +(When low-latency devices, such as bubbles, +.UC CCD s, +or scatter/gather devices, +are used, +this decision will have to be reconsidered.) +Allocation of both primary +and secondary memory is performed +by the same simple first-fit algorithm. +When a process grows, +a new piece of primary memory is allocated. +The contents of the old memory is copied to the new memory. +The old memory is freed +and the tables are updated. +If there is not enough primary memory, +secondary memory is allocated instead. +The process is swapped out onto the +secondary memory, +ready to be swapped in with +its new size. +.PP +One separate process in the kernel, +the swapping process, +simply swaps the other +processes in and out of primary memory. +It examines the +process table looking for a process +that is swapped out and is +ready to run. +It allocates primary memory for that +process and +reads its segments into +primary memory, where that process competes for the +central processor with other loaded processes. +If no primary memory is available, +the swapping process makes memory available +by examining the process table for processes +that can be swapped out. +It selects a process to swap out, +writes it to secondary memory, +frees the primary memory, +and then goes back to look for a process +to swap in. +.PP +Thus there are two specific algorithms +to the swapping process. +Which of the possibly many processes that +are swapped out is to be swapped in? +This is decided by secondary storage residence +time. +The one with the longest time out is swapped in first. +There is a slight penalty for larger processes. +Which of the possibly many processes that +are loaded is to be swapped out? +Processes that are waiting for slow events +(i.e., not currently running or waiting for +disk I/O) +are picked first, +by age in primary memory, +again with size penalties. +The other processes are examined +by the same age algorithm, +but are not taken out unless they are +at least of some age. +This adds +hysteresis to the swapping and +prevents total thrashing. +.PP +These swapping algorithms are the +most suspect in the system. +With limited primary memory, +these algorithms cause total swapping. +This is not bad in itself, because +the swapping does not impact the +execution of the resident processes. +However, if the swapping device must +also be used for file storage, +the swapping traffic severely +impacts the file system traffic. +It is exactly these small systems +that tend to double usage of limited disk +resources. +.NH 2 +Synchronization and scheduling +.PP +Process synchronization is accomplished by having processes +wait for events. +Events are represented by arbitrary integers. +By convention, +events are chosen to be addresses of +tables associated with those events. +For example, a process that is waiting for +any of its children to terminate will wait +for an event that is the address of +its own process table entry. +When a process terminates, +it signals the event represented by +its parent's process table entry. +Signaling an event on which no process +is waiting has no effect. +Similarly, +signaling an event on which many processes +are waiting will wake all of them up. +This differs considerably from +Dijkstra's P and V +synchronization operations, +.[ +dijkstra sequential processes 1968 +.] +in that +no memory is associated with events. +Thus there need be no allocation of events +prior to their use. +Events exist simply by being used. +.PP +On the negative side, +because there is no memory associated with events, +no notion of ``how much'' +can be signaled via the event mechanism. +For example, +processes that want memory might +wait on an event associated with +memory allocation. +When any amount of memory becomes available, +the event would be signaled. +All the competing processes would then wake +up to fight over the new memory. +(In reality, +the swapping process is the only process +that waits for primary memory to become available.) +.PP +If an event occurs +between the time a process decides +to wait for that event and the +time that process enters the wait state, +then +the process will wait on an event that has +already happened (and may never happen again). +This race condition happens because there is no memory associated with +the event to indicate that the event has occurred; +the only action of an event is to change a set of processes +from wait state to run state. +This problem is relieved largely +by the fact that process switching can +only occur in the kernel by explicit calls +to the event-wait mechanism. +If the event in question is signaled by another +process, +then there is no problem. +But if the event is signaled by a hardware +interrupt, +then special care must be taken. +These synchronization races pose the biggest +problem when +.UX +is adapted to multiple-processor configurations. +.[ +hawley meyer multiprocessing unix +.] +.PP +The event-wait code in the kernel +is like a co-routine linkage. +At any time, +all but one of the processes has called event-wait. +The remaining process is the one currently executing. +When it calls event-wait, +a process whose event has been signaled +is selected and that process +returns from its call to event-wait. +.PP +Which of the runable processes is to run next? +Associated with each process is a priority. +The priority of a system process is assigned by the code +issuing the wait on an event. +This is roughly equivalent to the response +that one would expect on such an event. +Disk events have high priority, +teletype events are low, +and time-of-day events are very low. +(From observation, +the difference in system process priorities +has little or no performance impact.) +All user-process priorities are lower than the +lowest system priority. +User-process priorities are assigned +by an algorithm based on the +recent ratio of the amount of compute time to real time consumed +by the process. +A process that has used a lot of +compute time in the last real-time +unit is assigned a low user priority. +Because interactive processes are characterized +by low ratios of compute to real time, +interactive response is maintained without any +special arrangements. +.PP +The scheduling algorithm simply picks +the process with the highest priority, +thus +picking all system processes first and +user processes second. +The compute-to-real-time ratio is updated +every second. +Thus, +all other things being equal, +looping user processes will be +scheduled round-robin with a +1-second quantum. +A high-priority process waking up will +preempt a running, low-priority process. +The scheduling algorithm has a very desirable +negative feedback character. +If a process uses its high priority +to hog the computer, +its priority will drop. +At the same time, if a low-priority +process is ignored for a long time, +its priority will rise. +.NH +I/O SYSTEM +.PP +The I/O system +is broken into two completely separate systems: +the block I/O system and the character I/O system. +In retrospect, +the names should have been ``structured I/O'' +and ``unstructured I/O,'' respectively; +while the term ``block I/O'' has some meaning, +``character I/O'' is a complete misnomer. +.PP +Devices are characterized by a major device number, +a minor device number, and +a class (block or character). +For each class, +there is an array of entry points into the device drivers. +The major device number is used to index the array +when calling the code for a particular device driver. +The minor device number is passed to the +device driver as an argument. +The minor number has no significance other +than that attributed to it by the driver. +Usually, +the driver uses the minor number to access +one of several identical physical devices. +.PP +The use of the array of entry points +(configuration table) +as the only connection between the +system code and the device drivers is +very important. +Early versions of the system had a much +less formal connection with the drivers, +so that it was extremely hard to handcraft +differently configured systems. +Now it is possible to create new +device drivers in an average of a few hours. +The configuration table in most cases +is created automatically by a program +that reads the system's parts list. +.NH 2 +Block I/O system +.PP +The model block I/O device consists +of randomly addressed, secondary +memory blocks of 512 bytes each. +The blocks are uniformly addressed +0, 1, .\|.\|. up to the size of the device. +The block device driver has the job of +emulating this model on a +physical device. +.PP +The block I/O devices are accessed +through a layer of buffering software. +The system maintains a list of buffers +(typically between 10 and 70) +each assigned a device name and +a device address. +This buffer pool constitutes a data cache +for the block devices. +On a read request, +the cache is searched for the desired block. +If the block is found, +the data are made available to the +requester without any physical I/O. +If the block is not in the cache, +the least recently used block in the cache is renamed, +the correct device driver is called to +fill up the renamed buffer, and then the +data are made available. +Write requests are handled in an analogous manner. +The correct buffer is found +and relabeled if necessary. +The write is performed simply by marking +the buffer as ``dirty.'' +The physical I/O is then deferred until +the buffer is renamed. +.PP +The benefits in reduction of physical I/O +of this scheme are substantial, +especially considering the file system implementation. +There are, +however, +some drawbacks. +The asynchronous nature of the +algorithm makes error reporting +and meaningful user error handling +almost impossible. +The cavalier approach to I/O error +handling in the +.UX +system is partly due to the asynchronous +nature of the block I/O system. +A second problem is in the delayed writes. +If the system stops unexpectedly, +it is almost certain that there is a +lot of logically complete, +but physically incomplete, +I/O in the buffers. +There is a system primitive to +flush all outstanding I/O activity +from the buffers. +Periodic use of this primitive helps, +but does not solve, the problem. +Finally, +the associativity in the buffers +can alter the physical I/O sequence +from that of the logical I/O sequence. +This means that there are times +when data structures on disk are inconsistent, +even though the software is careful +to perform I/O in the correct order. +On non-random devices, +notably magnetic tape, +the inversions of writes can be disastrous. +The problem with magnetic tapes is ``cured'' by +allowing only one outstanding write request +per drive. +.NH 2 +Character I/O system +.PP +The character I/O system consists of all +devices that do not fall into the block I/O model. +This includes the ``classical'' character devices +such as communications lines, paper tape, and +line printers. +It also includes magnetic tape and disks when +they are not used in a stereotyped way, +for example, 80-byte physical records on tape +and track-at-a-time disk copies. +In short, +the character I/O interface +means ``everything other than block.'' +I/O requests from the user are sent to the +device driver essentially unaltered. +The implementation of these requests is, of course, +up to the device driver. +There are guidelines and conventions +to help the implementation of +certain types of device drivers. +.NH 3 +Disk drivers +.PP +Disk drivers are implemented +with a queue of transaction records. +Each record holds a read/write flag, +a primary memory address, +a secondary memory address, and +a transfer byte count. +Swapping is accomplished by passing +such a record to the swapping device driver. +The block I/O interface is implemented by +passing such records with requests to +fill and empty system buffers. +The character I/O interface to the disk +drivers create a transaction record that +points directly into the user area. +The routine that creates this record also insures +that the user is not swapped during this +I/O transaction. +Thus by implementing the general disk driver, +it is possible to use the disk +as a block device, +a character device, and a swap device. +The only really disk-specific code in normal +disk drivers is the pre-sort of transactions to +minimize latency for a particular device, and +the actual issuing of the I/O request. +.NH 3 +Character lists +.PP +Real character-oriented devices may +be implemented using the common +code to handle character lists. +A character list is a queue of characters. +One routine puts a character on a queue. +Another gets a character from a queue. +It is also possible to ask how many +characters are currently on a queue. +Storage for all queues in the system comes +from a single common pool. +Putting a character on a queue will allocate +space from the common pool and link the +character onto the data structure defining the queue. +Getting a character from a queue returns +the corresponding space to the pool. +.PP +A typical character-output device +(paper tape punch, for example) +is implemented by passing characters +from the user onto a character queue until +some maximum number of characters is on the queue. +The I/O is prodded to start as +soon as there is anything on the queue +and, once started, +it is sustained by hardware completion interrupts. +Each time there is a completion interrupt, +the driver gets the next character from the queue +and sends it to the hardware. +The number of characters on the queue is checked and, +as the count falls through some intermediate level, +an event (the queue address) is signaled. +The process that is passing characters from +the user to the queue can be waiting on the event, and +refill the queue to its maximum +when the event occurs. +.PP +A typical character input device +(for example, a paper tape reader) +is handled in a very similar manner. +.PP +Another class of character devices is the terminals. +A terminal is represented by three +character queues. +There are two input queues (raw and canonical) +and an output queue. +Characters going to the output of a terminal +are handled by common code exactly as described +above. +The main difference is that there is also code +to interpret the output stream as +.UC ASCII +characters and to perform some translations, +e.g., escapes for deficient terminals. +Another common aspect of terminals is code +to insert real-time delay after certain control characters. +.PP +Input on terminals is a little different. +Characters are collected from the terminal and +placed on a raw input queue. +Some device-dependent code conversion and +escape interpretation is handled here. +When a line is complete in the raw queue, +an event is signaled. +The code catching this signal then copies a +line from the raw queue to a canonical queue +performing the character erase and line kill editing. +User read requests on terminals can be +directed at either the raw or canonical queues. +.NH 3 +Other character devices +.PP +Finally, +there are devices that fit no general category. +These devices are set up as character I/O drivers. +An example is a driver that reads and writes +unmapped primary memory as an I/O device. +Some devices are too +fast to be treated a character at time, +but do not fit the disk I/O mold. +Examples are fast communications lines and +fast line printers. +These devices either have their own buffers +or ``borrow'' block I/O buffers for a while and +then give them back. +.NH +THE FILE SYSTEM +.PP +In the +.UX +system, +a file is a (one-dimensional) array of bytes. +No other structure of files is implied by the +system. +Files are attached anywhere +(and possibly multiply) +onto a hierarchy of directories. +Directories are simply files that +users cannot write. +For a further discussion +of the external view of files and directories, +see Ref.\0 +.[ +ritchie thompson unix bstj 1978 +%Q This issue +.]. +.PP +The +.UX +file system is a disk data structure +accessed completely through +the block I/O system. +As stated before, +the canonical view of a ``disk'' is +a randomly addressable array of +512-byte blocks. +A file system breaks the disk into +four self-identifying regions. +The first block (address 0) +is unused by the file system. +It is left aside for booting procedures. +The second block (address 1) +contains the so-called ``super-block.'' +This block, +among other things, +contains the size of the disk and +the boundaries of the other regions. +Next comes the i-list, +a list of file definitions. +Each file definition is +a 64-byte structure, called an i-node. +The offset of a particular i-node +within the i-list is called its i-number. +The combination of device name +(major and minor numbers) and i-number +serves to uniquely name a particular file. +After the i-list, +and to the end of the disk, +come free storage blocks that +are available for the contents of files. +.PP +The free space on a disk is maintained +by a linked list of available disk blocks. +Every block in this chain contains a disk address +of the next block in the chain. +The remaining space contains the address of up to +50 disk blocks that are also free. +Thus with one I/O operation, +the system obtains 50 free blocks and a +pointer where to find more. +The disk allocation algorithms are +very straightforward. +Since all allocation is in fixed-size +blocks and there is strict accounting of +space, +there is no need to compact or garbage collect. +However, +as disk space becomes dispersed, +latency gradually increases. +Some installations choose to occasionally compact +disk space to reduce latency. +.PP +An i-node contains 13 disk addresses. +The first 10 of these addresses point directly at +the first 10 blocks of a file. +If a file is larger than 10 blocks (5,120 bytes), +then the eleventh address points at a block +that contains the addresses of the next 128 blocks of the file. +If the file is still larger than this +(70,656 bytes), +then the twelfth block points at up to 128 blocks, +each pointing to 128 blocks of the file. +Files yet larger +(8,459,264 bytes) +use the thirteenth address for a ``triple indirect'' address. +The algorithm ends here with the maximum file size +of 1,082,201,087 bytes. +.PP +A logical directory hierarchy is added +to this flat physical structure simply +by adding a new type of file, the directory. +A directory is accessed exactly as an ordinary file. +It contains 16-byte entries consisting of +a 14-byte name and an i-number. +The root of the hierarchy is at a known i-number +(\fIviz.,\fR 2). +The file system structure allows an arbitrary, directed graph +of directories with regular files linked in +at arbitrary places in this graph. +In fact, +very early +.UX +systems used such a structure. +Administration of such a structure became so +chaotic that later systems were restricted +to a directory tree. +Even now, +with regular files linked multiply +into arbitrary places in the tree, +accounting for space has become a problem. +It may become necessary to restrict the entire +structure to a tree, +and allow a new form of linking that +is subservient to the tree structure. +.PP +The file system allows +easy creation, +easy removal, +easy random accessing, +and very easy space allocation. +With most physical addresses confined +to a small contiguous section of disk, +it is also easy to dump, restore, and +check the consistency of the file system. +Large files suffer from indirect addressing, +but the cache prevents most of the implied physical I/O +without adding much execution. +The space overhead properties of this scheme are quite good. +For example, +on one particular file system, +there are 25,000 files containing 130M bytes of data-file content. +The overhead (i-node, indirect blocks, and last block breakage) +is about 11.5M bytes. +The directory structure to support these files +has about 1,500 directories containing 0.6M bytes of directory content +and about 0.5M bytes of overhead in accessing the directories. +Added up any way, +this comes out to less than a 10 percent overhead for actual +stored data. +Most systems have this much overhead in +padded trailing blanks alone. +.NH 2 +File system implementation +.PP +Because the i-node defines a file, +the implementation of the file system centers +around access to the i-node. +The system maintains a table of all active +i-nodes. +As a new file is accessed, +the system locates the corresponding i-node, +allocates an i-node table entry, and reads +the i-node into primary memory. +As in the buffer cache, +the table entry is considered to be the current +version of the i-node. +Modifications to the i-node are made to +the table entry. +When the last access to the i-node goes +away, +the table entry is copied back to the +secondary store i-list and the table entry is freed. +.KF +.if t .in .25i +.so fig2.pic +.if t .in -.25i +.sp 2v +.ce +Fig. 2\(emFile system data structure. +.sp +.KE +.PP +All I/O operations on files are carried out +with the aid of the corresponding i-node table entry. +The accessing of a file is a straightforward +implementation of the algorithms mentioned previously. +The user is not aware of i-nodes and i-numbers. +References to the file system are made in terms of +path names of the directory tree. +Converting a path name into an i-node table entry +is also straightforward. +Starting at some known i-node +(the root or the current directory of some process), +the next component of the path name is +searched by reading the directory. +This gives an i-number and an implied device +(that of the directory). +Thus the next i-node table entry can be accessed. +If that was the last component of the path name, +then this i-node is the result. +If not, +this i-node is the directory needed to look up +the next component of the path name, and the +algorithm is repeated. +.PP +The user process accesses the file system with +certain primitives. +The most common of these are +.UL open , +.UL create , +.UL read , +.UL write , +.UL seek , +and +.UL close . +The data structures maintained are shown in Fig. 2. +In the system data segment associated with a user, +there is room for some (usually between 10 and 50) open files. +This open file table consists of pointers that can be used to access +corresponding i-node table entries. +Associated with each of these open files is +a current I/O pointer. +This is a byte offset of +the next read/write operation on the file. +The system treats each read/write request +as random with an implied seek to the +I/O pointer. +The user usually thinks of the file as +sequential with the I/O pointer +automatically counting the number of bytes +that have been read/written from the file. +The user may, +of course, +perform random I/O by setting the I/O pointer +before reads/writes. +.PP +With file sharing, +it is necessary to allow related +processes to share a common I/O pointer +and yet have separate I/O pointers +for independent processes +that access the same file. +With these two conditions, +the I/O pointer cannot reside +in the i-node table nor can +it reside in the list of +open files for the process. +A new table +(the open file table) +was invented for the sole purpose +of holding the I/O pointer. +Processes that share the same open +file +(the result of +.UL fork s) +share a common open file table entry. +A separate open of the same file will +only share the i-node table entry, +but will have distinct open file table entries. +.PP +The main file system primitives are implemented as follows. +.UL \&open +converts a file system path name into an i-node +table entry. +A pointer to the i-node table entry is placed in a +newly created open file table entry. +A pointer to the file table entry is placed in the +system data segment for the process. +.UL \&create +first creates a new i-node entry, +writes the i-number into a directory, and +then builds the same structure as for an +.UL open . +.UL \&read +and +.UL write +just access the i-node entry as described above. +.UL \&seek +simply manipulates the I/O pointer. +No physical seeking is done. +.UL \&close +just frees the structures built by +.UL open +and +.UL create . +Reference counts are kept on the open file table entries and +the i-node table entries to free these structures after +the last reference goes away. +.UL \&unlink +simply decrements the count of the +number of directories pointing at the given i-node. +When the last reference to an i-node table entry +goes away, +if the i-node has no directories pointing to it, +then the file is removed and the i-node is freed. +This delayed removal of files prevents +problems arising from removing active files. +A file may be removed while still open. +The resulting unnamed file vanishes +when the file is closed. +This is a method of obtaining temporary files. +.PP +There is a type of unnamed +.UC FIFO +file called a +.UL pipe. +Implementation of +.UL pipe s +consists of implied +.UL seek s +before each +.UL read +or +.UL write +in order to implement +first-in-first-out. +There are also checks and synchronization +to prevent the +writer from grossly outproducing the +reader and to prevent the reader from +overtaking the writer. +.NH 2 +Mounted file systems +.PP +The file system of a +.UX +system +starts with some designated block device +formatted as described above to contain +a hierarchy. +The root of this structure is the root of +the +.UX +file system. +A second formatted block device may be +mounted +at any leaf of +the current hierarchy. +This logically extends the current hierarchy. +The implementation of +mounting +is trivial. +A mount table is maintained containing +pairs of designated leaf i-nodes and +block devices. +When converting a path name into an i-node, +a check is made to see if the new i-node is a +designated leaf. +If it is, +the i-node of the root +of the block device replaces it. +.PP +Allocation of space for a file is taken +from the free pool on the device on which the +file lives. +Thus a file system consisting of many +mounted devices does not have a common pool of +free secondary storage space. +This separation of space on different +devices is necessary to allow easy +unmounting +of a device. +.NH 2 +Other system functions +.PP +There are some other things that the system +does for the user\-a +little accounting, +a little tracing/debugging, +and a little access protection. +Most of these things are not very +well developed +because our use of the system in computing science research +does not need them. +There are some features that are missed in some +applications, for example, better inter-process communication. +.PP +The +.UX +kernel is an I/O multiplexer more than +a complete operating system. +This is as it should be. +Because of this outlook, +many features are +found in most +other operating systems that are missing from the +.UX +kernel. +For example, +the +.UX +kernel does not support +file access methods, +file disposition, +file formats, +file maximum size, +spooling, +command language, +logical records, +physical records, +assignment of logical file names, +logical file names, +more than one character set, +an operator's console, +an operator, +log-in, +or log-out. +Many of these things are symptoms rather than features. +Many of these things are implemented +in user software +using the kernel as a tool. +A good example of this is the command language. +.[ +bourne shell 1978 bstj +%Q This issue +.] +Each user may have his own command language. +Maintenance of such code is as easy as +maintaining user code. +The idea of implementing ``system'' code with general +user primitives +comes directly from +.UC MULTICS . +.[ +organick multics 1972 +.] +.LP +.[ +$LIST$ +.] diff --git a/share/doc/psd/02.implement/ref.bib b/share/doc/psd/02.implement/ref.bib new file mode 100644 index 000000000000..34140640e8bc --- /dev/null +++ b/share/doc/psd/02.implement/ref.bib @@ -0,0 +1,54 @@ +# $FreeBSD$ + +%T U\s-2NIX\s0 Time-Sharing System: The U\s-2NIX\s0 Shell +%A S. R. Bourne +%K unix bstj +%J Bell Sys. Tech. J. +%V 57 +%N 6 +%P 1971-1990 +%D 1978 + +%A R. E. Griswold +%A D. R. Hanson +%T An Overview of SL5 +%J SIGPLAN Notices +%V 12 +%N 4 +%D April 1977 +%P 40-50 + +%A E. W. Dijkstra +%T Cooperating Sequential Processes +%B Programming Languages +%E F. Genuys +%I Academic Press +%C New York +%D 1968 +%P 43-112 + +%A J. A. Hawley +%A W. B. Meyer +%T M\s-2UNIX\s0, A Multiprocessing Version of U\s-2NIX\s0 +%K munix unix +%R M.S. Thesis +%I Naval Postgraduate School +%C Monterey, Cal. +%D 1975 + +%T The U\s-2NIX\s0 Time-Sharing System +%K unix bstj +%A D. M. Ritchie +%A K. Thompson +%J Bell Sys. Tech. J. +%V 57 +%N 6 +%P 1905-1929 +%D 1978 + +%A E. I. Organick +%T The M\s-2ULTICS\s0 System +%K multics +%I M.I.T. Press +%C Cambridge, Mass. +%D 1972 diff --git a/share/doc/psd/03.iosys/Makefile b/share/doc/psd/03.iosys/Makefile new file mode 100644 index 000000000000..113bf90fd4fe --- /dev/null +++ b/share/doc/psd/03.iosys/Makefile @@ -0,0 +1,8 @@ +# @(#)Makefile 8.1 (Berkeley) 6/8/93 +# $FreeBSD$ + +VOLUME= psd/03.iosys +SRCS= iosys +MACROS= -ms + +.include <bsd.doc.mk> diff --git a/share/doc/psd/03.iosys/iosys b/share/doc/psd/03.iosys/iosys new file mode 100644 index 000000000000..1459f142a9d5 --- /dev/null +++ b/share/doc/psd/03.iosys/iosys @@ -0,0 +1,1086 @@ +.\" Copyright (C) Caldera International Inc. 2001-2002. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions are +.\" met: +.\" +.\" Redistributions of source code and documentation must retain the above +.\" copyright notice, this list of conditions and the following +.\" disclaimer. +.\" +.\" Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" +.\" All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" +.\" This product includes software developed or owned by Caldera +.\" International, Inc. Neither the name of Caldera International, Inc. +.\" nor the names of other contributors may be used to endorse or promote +.\" products derived from this software without specific prior written +.\" permission. +.\" +.\" USE OF THE SOFTWARE PROVIDED FOR UNDER THIS LICENSE BY CALDERA +.\" INTERNATIONAL, INC. AND CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR +.\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +.\" WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +.\" DISCLAIMED. IN NO EVENT SHALL CALDERA INTERNATIONAL, INC. BE LIABLE +.\" FOR ANY DIRECT, INDIRECT INCIDENTAL, SPECIAL, EXEMPLARY, OR +.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR +.\" BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, +.\" WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE +.\" OR OTHERWISE) RISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN +.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +.\" +.\" @(#)iosys 8.1 (Berkeley) 6/8/93 +.\" +.\" $FreeBSD$ +.EH 'PSD:3-%''The UNIX I/O System' +.OH 'The UNIX I/O System''PSD:3-%' +.TL +The UNIX I/O System +.AU +Dennis M. Ritchie +.AI +AT&T Bell Laboratories +Murray Hill, NJ +.PP +This paper gives an overview of the workings of the UNIX\(dg +.FS +\(dgUNIX is a Trademark of Bell Laboratories. +.FE +I/O system. +It was written with an eye toward providing +guidance to writers of device driver routines, +and is oriented more toward describing the environment +and nature of device drivers than the implementation +of that part of the file system which deals with +ordinary files. +.PP +It is assumed that the reader has a good knowledge +of the overall structure of the file system as discussed +in the paper ``The UNIX Time-sharing System.'' +A more detailed discussion +appears in +``UNIX Implementation;'' +the current document restates parts of that one, +but is still more detailed. +It is most useful in +conjunction with a copy of the system code, +since it is basically an exegesis of that code. +.SH +Device Classes +.PP +There are two classes of device: +.I block +and +.I character. +The block interface is suitable for devices +like disks, tapes, and DECtape +which work, or can work, with addressible 512-byte blocks. +Ordinary magnetic tape just barely fits in this category, +since by use of forward +and +backward spacing any block can be read, even though +blocks can be written only at the end of the tape. +Block devices can at least potentially contain a mounted +file system. +The interface to block devices is very highly structured; +the drivers for these devices share a great many routines +as well as a pool of buffers. +.PP +Character-type devices have a much +more straightforward interface, although +more work must be done by the driver itself. +.PP +Devices of both types are named by a +.I major +and a +.I minor +device number. +These numbers are generally stored as an integer +with the minor device number +in the low-order 8 bits and the major device number +in the next-higher 8 bits; +macros +.I major +and +.I minor +are available to access these numbers. +The major device number selects which driver will deal with +the device; the minor device number is not used +by the rest of the system but is passed to the +driver at appropriate times. +Typically the minor number +selects a subdevice attached to +a given controller, or one of +several similar hardware interfaces. +.PP +The major device numbers for block and character devices +are used as indices in separate tables; +they both start at 0 and therefore overlap. +.SH +Overview of I/O +.PP +The purpose of +the +.I open +and +.I creat +system calls is to set up entries in three separate +system tables. +The first of these is the +.I u_ofile +table, +which is stored in the system's per-process +data area +.I u. +This table is indexed by +the file descriptor returned by the +.I open +or +.I creat, +and is accessed during +a +.I read, +.I write, +or other operation on the open file. +An entry contains only +a pointer to the corresponding +entry of the +.I file +table, +which is a per-system data base. +There is one entry in the +.I file +table for each +instance of +.I open +or +.I creat. +This table is per-system because the same instance +of an open file must be shared among the several processes +which can result from +.I forks +after the file is opened. +A +.I file +table entry contains +flags which indicate whether the file +was open for reading or writing or is a pipe, and +a count which is used to decide when all processes +using the entry have terminated or closed the file +(so the entry can be abandoned). +There is also a 32-bit file offset +which is used to indicate where in the file the next read +or write will take place. +Finally, there is a pointer to the +entry for the file in the +.I inode +table, +which contains a copy of the file's i-node. +.PP +Certain open files can be designated ``multiplexed'' +files, and several other flags apply to such +channels. +In such a case, instead of an offset, +there is a pointer to an associated multiplex channel table. +Multiplex channels will not be discussed here. +.PP +An entry in the +.I file +table corresponds precisely to an instance of +.I open +or +.I creat; +if the same file is opened several times, +it will have several +entries in this table. +However, +there is at most one entry +in the +.I inode +table for a given file. +Also, a file may enter the +.I inode +table not only because it is open, +but also because it is the current directory +of some process or because it +is a special file containing a currently-mounted +file system. +.PP +An entry in the +.I inode +table differs somewhat from the +corresponding i-node as stored on the disk; +the modified and accessed times are not stored, +and the entry is augmented +by a flag word containing information about the entry, +a count used to determine when it may be +allowed to disappear, +and the device and i-number +whence the entry came. +Also, the several block numbers that give addressing +information for the file are expanded from +the 3-byte, compressed format used on the disk to full +.I long +quantities. +.PP +During the processing of an +.I open +or +.I creat +call for a special file, +the system always calls the device's +.I open +routine to allow for any special processing +required (rewinding a tape, turning on +the data-terminal-ready lead of a modem, etc.). +However, +the +.I close +routine is called only when the last +process closes a file, +that is, when the i-node table entry +is being deallocated. +Thus it is not feasible +for a device to maintain, or depend on, +a count of its users, although it is quite +possible to +implement an exclusive-use device which cannot +be reopened until it has been closed. +.PP +When a +.I read +or +.I write +takes place, +the user's arguments +and the +.I file +table entry are used to set up the +variables +.I u.u_base, +.I u.u_count, +and +.I u.u_offset +which respectively contain the (user) address +of the I/O target area, the byte-count for the transfer, +and the current location in the file. +If the file referred to is +a character-type special file, the appropriate read +or write routine is called; it is responsible +for transferring data and updating the +count and current location appropriately +as discussed below. +Otherwise, the current location is used to calculate +a logical block number in the file. +If the file is an ordinary file the logical block +number must be mapped (possibly using indirect blocks) +to a physical block number; a block-type +special file need not be mapped. +This mapping is performed by the +.I bmap +routine. +In any event, the resulting physical block number +is used, as discussed below, to +read or write the appropriate device. +.SH +Character Device Drivers +.PP +The +.I cdevsw +table specifies the interface routines present for +character devices. +Each device provides five routines: +open, close, read, write, and special-function +(to implement the +.I ioctl +system call). +Any of these may be missing. +If a call on the routine +should be ignored, +(e.g. +.I open +on non-exclusive devices that require no setup) +the +.I cdevsw +entry can be given as +.I nulldev; +if it should be considered an error, +(e.g. +.I write +on read-only devices) +.I nodev +is used. +For terminals, +the +.I cdevsw +structure also contains a pointer to the +.I tty +structure associated with the terminal. +.PP +The +.I open +routine is called each time the file +is opened with the full device number as argument. +The second argument is a flag which is +non-zero only if the device is to be written upon. +.PP +The +.I close +routine is called only when the file +is closed for the last time, +that is when the very last process in +which the file is open closes it. +This means it is not possible for the driver to +maintain its own count of its users. +The first argument is the device number; +the second is a flag which is non-zero +if the file was open for writing in the process which +performs the final +.I close. +.PP +When +.I write +is called, it is supplied the device +as argument. +The per-user variable +.I u.u_count +has been set to +the number of characters indicated by the user; +for character devices, this number may be 0 +initially. +.I u.u_base +is the address supplied by the user from which to start +taking characters. +The system may call the +routine internally, so the +flag +.I u.u_segflg +is supplied that indicates, +if +.I on, +that +.I u.u_base +refers to the system address space instead of +the user's. +.PP +The +.I write +routine +should copy up to +.I u.u_count +characters from the user's buffer to the device, +decrementing +.I u.u_count +for each character passed. +For most drivers, which work one character at a time, +the routine +.I "cpass( )" +is used to pick up characters +from the user's buffer. +Successive calls on it return +the characters to be written until +.I u.u_count +goes to 0 or an error occurs, +when it returns \(mi1. +.I Cpass +takes care of interrogating +.I u.u_segflg +and updating +.I u.u_count. +.PP +Write routines which want to transfer +a probably large number of characters into an internal +buffer may also use the routine +.I "iomove(buffer, offset, count, flag)" +which is faster when many characters must be moved. +.I Iomove +transfers up to +.I count +characters into the +.I buffer +starting +.I offset +bytes from the start of the buffer; +.I flag +should be +.I B_WRITE +(which is 0) in the write case. +Caution: +the caller is responsible for making sure +the count is not too large and is non-zero. +As an efficiency note, +.I iomove +is much slower if any of +.I "buffer+offset, count" +or +.I u.u_base +is odd. +.PP +The device's +.I read +routine is called under conditions similar to +.I write, +except that +.I u.u_count +is guaranteed to be non-zero. +To return characters to the user, the routine +.I "passc(c)" +is available; it takes care of housekeeping +like +.I cpass +and returns \(mi1 as the last character +specified by +.I u.u_count +is returned to the user; +before that time, 0 is returned. +.I Iomove +is also usable as with +.I write; +the flag should be +.I B_READ +but the same cautions apply. +.PP +The ``special-functions'' routine +is invoked by the +.I stty +and +.I gtty +system calls as follows: +.I "(*p) (dev, v)" +where +.I p +is a pointer to the device's routine, +.I dev +is the device number, +and +.I v +is a vector. +In the +.I gtty +case, +the device is supposed to place up to 3 words of status information +into the vector; this will be returned to the caller. +In the +.I stty +case, +.I v +is 0; +the device should take up to 3 words of +control information from +the array +.I "u.u_arg[0...2]." +.PP +Finally, each device should have appropriate interrupt-time +routines. +When an interrupt occurs, it is turned into a C-compatible call +on the devices's interrupt routine. +The interrupt-catching mechanism makes +the low-order four bits of the ``new PS'' word in the +trap vector for the interrupt available +to the interrupt handler. +This is conventionally used by drivers +which deal with multiple similar devices +to encode the minor device number. +After the interrupt has been processed, +a return from the interrupt handler will +return from the interrupt itself. +.PP +A number of subroutines are available which are useful +to character device drivers. +Most of these handlers, for example, need a place +to buffer characters in the internal interface +between their ``top half'' (read/write) +and ``bottom half'' (interrupt) routines. +For relatively low data-rate devices, the best mechanism +is the character queue maintained by the +routines +.I getc +and +.I putc. +A queue header has the structure +.DS +struct { + int c_cc; /* character count */ + char *c_cf; /* first character */ + char *c_cl; /* last character */ +} queue; +.DE +A character is placed on the end of a queue by +.I "putc(c, &queue)" +where +.I c +is the character and +.I queue +is the queue header. +The routine returns \(mi1 if there is no space +to put the character, 0 otherwise. +The first character on the queue may be retrieved +by +.I "getc(&queue)" +which returns either the (non-negative) character +or \(mi1 if the queue is empty. +.PP +Notice that the space for characters in queues is +shared among all devices in the system +and in the standard system there are only some 600 +character slots available. +Thus device handlers, +especially write routines, must take +care to avoid gobbling up excessive numbers of characters. +.PP +The other major help available +to device handlers is the sleep-wakeup mechanism. +The call +.I "sleep(event, priority)" +causes the process to wait (allowing other processes to run) +until the +.I event +occurs; +at that time, the process is marked ready-to-run +and the call will return when there is no +process with higher +.I priority. +.PP +The call +.I "wakeup(event)" +indicates that the +.I event +has happened, that is, causes processes sleeping +on the event to be awakened. +The +.I event +is an arbitrary quantity agreed upon +by the sleeper and the waker-up. +By convention, it is the address of some data area used +by the driver, which guarantees that events +are unique. +.PP +Processes sleeping on an event should not assume +that the event has really happened; +they should check that the conditions which +caused them to sleep no longer hold. +.PP +Priorities can range from 0 to 127; +a higher numerical value indicates a less-favored +scheduling situation. +A distinction is made between processes sleeping +at priority less than the parameter +.I PZERO +and those at numerically larger priorities. +The former cannot +be interrupted by signals, although it +is conceivable that it may be swapped out. +Thus it is a bad idea to sleep with +priority less than PZERO on an event which might never occur. +On the other hand, calls to +.I sleep +with larger priority +may never return if the process is terminated by +some signal in the meantime. +Incidentally, it is a gross error to call +.I sleep +in a routine called at interrupt time, since the process +which is running is almost certainly not the +process which should go to sleep. +Likewise, none of the variables in the user area +``\fIu\fB.\fR'' +should be touched, let alone changed, by an interrupt routine. +.PP +If a device driver +wishes to wait for some event for which it is inconvenient +or impossible to supply a +.I wakeup, +(for example, a device going on-line, which does not +generally cause an interrupt), +the call +.I "sleep(&lbolt, priority) +may be given. +.I Lbolt +is an external cell whose address is awakened once every 4 seconds +by the clock interrupt routine. +.PP +The routines +.I "spl4( ), spl5( ), spl6( ), spl7( )" +are available to +set the processor priority level as indicated to avoid +inconvenient interrupts from the device. +.PP +If a device needs to know about real-time intervals, +then +.I "timeout(func, arg, interval) +will be useful. +This routine arranges that after +.I interval +sixtieths of a second, the +.I func +will be called with +.I arg +as argument, in the style +.I "(*func)(arg). +Timeouts are used, for example, +to provide real-time delays after function characters +like new-line and tab in typewriter output, +and to terminate an attempt to +read the 201 Dataphone +.I dp +if there is no response within a specified number +of seconds. +Notice that the number of sixtieths of a second is limited to 32767, +since it must appear to be positive, +and that only a bounded number of timeouts +can be going on at once. +Also, the specified +.I func +is called at clock-interrupt time, so it should +conform to the requirements of interrupt routines +in general. +.SH +The Block-device Interface +.PP +Handling of block devices is mediated by a collection +of routines that manage a set of buffers containing +the images of blocks of data on the various devices. +The most important purpose of these routines is to assure +that several processes that access the same block of the same +device in multiprogrammed fashion maintain a consistent +view of the data in the block. +A secondary but still important purpose is to increase +the efficiency of the system by +keeping in-core copies of blocks that are being +accessed frequently. +The main data base for this mechanism is the +table of buffers +.I buf. +Each buffer header contains a pair of pointers +.I "(b_forw, b_back)" +which maintain a doubly-linked list +of the buffers associated with a particular +block device, and a +pair of pointers +.I "(av_forw, av_back)" +which generally maintain a doubly-linked list of blocks +which are ``free,'' that is, +eligible to be reallocated for another transaction. +Buffers that have I/O in progress +or are busy for other purposes do not appear in this list. +The buffer header +also contains the device and block number to which the +buffer refers, and a pointer to the actual storage associated with +the buffer. +There is a word count +which is the negative of the number of words +to be transferred to or from the buffer; +there is also an error byte and a residual word +count used to communicate information +from an I/O routine to its caller. +Finally, there is a flag word +with bits indicating the status of the buffer. +These flags will be discussed below. +.PP +Seven routines constitute +the most important part of the interface with the +rest of the system. +Given a device and block number, +both +.I bread +and +.I getblk +return a pointer to a buffer header for the block; +the difference is that +.I bread +is guaranteed to return a buffer actually containing the +current data for the block, +while +.I getblk +returns a buffer which contains the data in the +block only if it is already in core (whether it is +or not is indicated by the +.I B_DONE +bit; see below). +In either case the buffer, and the corresponding +device block, is made ``busy,'' +so that other processes referring to it +are obliged to wait until it becomes free. +.I Getblk +is used, for example, +when a block is about to be totally rewritten, +so that its previous contents are +not useful; +still, no other process can be allowed to refer to the block +until the new data is placed into it. +.PP +The +.I breada +routine is used to implement read-ahead. +it is logically similar to +.I bread, +but takes as an additional argument the number of +a block (on the same device) to be read asynchronously +after the specifically requested block is available. +.PP +Given a pointer to a buffer, +the +.I brelse +routine +makes the buffer again available to other processes. +It is called, for example, after +data has been extracted following a +.I bread. +There are three subtly-different write routines, +all of which take a buffer pointer as argument, +and all of which logically release the buffer for +use by others and place it on the free list. +.I Bwrite +puts the +buffer on the appropriate device queue, +waits for the write to be done, +and sets the user's error flag if required. +.I Bawrite +places the buffer on the device's queue, but does not wait +for completion, so that errors cannot be reflected directly to +the user. +.I Bdwrite +does not start any I/O operation at all, +but merely marks +the buffer so that if it happens +to be grabbed from the free list to contain +data from some other block, the data in it will +first be written +out. +.PP +.I Bwrite +is used when one wants to be sure that +I/O takes place correctly, and that +errors are reflected to the proper user; +it is used, for example, when updating i-nodes. +.I Bawrite +is useful when more overlap is desired +(because no wait is required for I/O to finish) +but when it is reasonably certain that the +write is really required. +.I Bdwrite +is used when there is doubt that the write is +needed at the moment. +For example, +.I bdwrite +is called when the last byte of a +.I write +system call falls short of the end of a +block, on the assumption that +another +.I write +will be given soon which will re-use the same block. +On the other hand, +as the end of a block is passed, +.I bawrite +is called, since probably the block will +not be accessed again soon and one might as +well start the writing process as soon as possible. +.PP +In any event, notice that the routines +.I "getblk" +and +.I bread +dedicate the given block exclusively to the +use of the caller, and make others wait, +while one of +.I "brelse, bwrite, bawrite," +or +.I bdwrite +must eventually be called to free the block for use by others. +.PP +As mentioned, each buffer header contains a flag +word which indicates the status of the buffer. +Since they provide +one important channel for information between the drivers and the +block I/O system, it is important to understand these flags. +The following names are manifest constants which +select the associated flag bits. +.IP B_READ 10 +This bit is set when the buffer is handed to the device strategy routine +(see below) to indicate a read operation. +The symbol +.I B_WRITE +is defined as 0 and does not define a flag; it is provided +as a mnemonic convenience to callers of routines like +.I swap +which have a separate argument +which indicates read or write. +.IP B_DONE 10 +This bit is set +to 0 when a block is handed to the device strategy +routine and is turned on when the operation completes, +whether normally as the result of an error. +It is also used as part of the return argument of +.I getblk +to indicate if 1 that the returned +buffer actually contains the data in the requested block. +.IP B_ERROR 10 +This bit may be set to 1 when +.I B_DONE +is set to indicate that an I/O or other error occurred. +If it is set the +.I b_error +byte of the buffer header may contain an error code +if it is non-zero. +If +.I b_error +is 0 the nature of the error is not specified. +Actually no driver at present sets +.I b_error; +the latter is provided for a future improvement +whereby a more detailed error-reporting +scheme may be implemented. +.IP B_BUSY 10 +This bit indicates that the buffer header is not on +the free list, i.e. is +dedicated to someone's exclusive use. +The buffer still remains attached to the list of +blocks associated with its device, however. +When +.I getblk +(or +.I bread, +which calls it) searches the buffer list +for a given device and finds the requested +block with this bit on, it sleeps until the bit +clears. +.IP B_PHYS 10 +This bit is set for raw I/O transactions that +need to allocate the Unibus map on an 11/70. +.IP B_MAP 10 +This bit is set on buffers that have the Unibus map allocated, +so that the +.I iodone +routine knows to deallocate the map. +.IP B_WANTED 10 +This flag is used in conjunction with the +.I B_BUSY +bit. +Before sleeping as described +just above, +.I getblk +sets this flag. +Conversely, when the block is freed and the busy bit +goes down (in +.I brelse) +a +.I wakeup +is given for the block header whenever +.I B_WANTED +is on. +This strategem avoids the overhead +of having to call +.I wakeup +every time a buffer is freed on the chance that someone +might want it. +.IP B_AGE +This bit may be set on buffers just before releasing them; if it +is on, +the buffer is placed at the head of the free list, rather than at the +tail. +It is a performance heuristic +used when the caller judges that the same block will not soon be used again. +.IP B_ASYNC 10 +This bit is set by +.I bawrite +to indicate to the appropriate device driver +that the buffer should be released when the +write has been finished, usually at interrupt time. +The difference between +.I bwrite +and +.I bawrite +is that the former starts I/O, waits until it is done, and +frees the buffer. +The latter merely sets this bit and starts I/O. +The bit indicates that +.I relse +should be called for the buffer on completion. +.IP B_DELWRI 10 +This bit is set by +.I bdwrite +before releasing the buffer. +When +.I getblk, +while searching for a free block, +discovers the bit is 1 in a buffer it would otherwise grab, +it causes the block to be written out before reusing it. +.SH +Block Device Drivers +.PP +The +.I bdevsw +table contains the names of the interface routines +and that of a table for each block device. +.PP +Just as for character devices, block device drivers may supply +an +.I open +and a +.I close +routine +called respectively on each open and on the final close +of the device. +Instead of separate read and write routines, +each block device driver has a +.I strategy +routine which is called with a pointer to a buffer +header as argument. +As discussed, the buffer header contains +a read/write flag, the core address, +the block number, a (negative) word count, +and the major and minor device number. +The role of the strategy routine +is to carry out the operation as requested by the +information in the buffer header. +When the transaction is complete the +.I B_DONE +(and possibly the +.I B_ERROR) +bits should be set. +Then if the +.I B_ASYNC +bit is set, +.I brelse +should be called; +otherwise, +.I wakeup. +In cases where the device +is capable, under error-free operation, +of transferring fewer words than requested, +the device's word-count register should be placed +in the residual count slot of +the buffer header; +otherwise, the residual count should be set to 0. +This particular mechanism is really for the benefit +of the magtape driver; +when reading this device +records shorter than requested are quite normal, +and the user should be told the actual length of the record. +.PP +Although the most usual argument +to the strategy routines +is a genuine buffer header allocated as discussed above, +all that is actually required +is that the argument be a pointer to a place containing the +appropriate information. +For example the +.I swap +routine, which manages movement +of core images to and from the swapping device, +uses the strategy routine +for this device. +Care has to be taken that +no extraneous bits get turned on in the +flag word. +.PP +The device's table specified by +.I bdevsw +has a +byte to contain an active flag and an error count, +a pair of links which constitute the +head of the chain of buffers for the device +.I "(b_forw, b_back)," +and a first and last pointer for a device queue. +Of these things, all are used solely by the device driver +itself +except for the buffer-chain pointers. +Typically the flag encodes the state of the +device, and is used at a minimum to +indicate that the device is currently engaged in +transferring information and no new command should be issued. +The error count is useful for counting retries +when errors occur. +The device queue is used to remember stacked requests; +in the simplest case it may be maintained as a first-in +first-out list. +Since buffers which have been handed over to +the strategy routines are never +on the list of free buffers, +the pointers in the buffer which maintain the free list +.I "(av_forw, av_back)" +are also used to contain the pointers +which maintain the device queues. +.PP +A couple of routines +are provided which are useful to block device drivers. +.I "iodone(bp)" +arranges that the buffer to which +.I bp +points be released or awakened, +as appropriate, +when the +strategy module has finished with the buffer, +either normally or after an error. +(In the latter case the +.I B_ERROR +bit has presumably been set.) +.PP +The routine +.I "geterror(bp)" +can be used to examine the error bit in a buffer header +and arrange that any error indication found therein is +reflected to the user. +It may be called only in the non-interrupt +part of a driver when I/O has completed +.I (B_DONE +has been set). +.SH +Raw Block-device I/O +.PP +A scheme has been set up whereby block device drivers may +provide the ability to transfer information +directly between the user's core image and the device +without the use of buffers and in blocks as large as +the caller requests. +The method involves setting up a character-type special file +corresponding to the raw device +and providing +.I read +and +.I write +routines which set up what is usually a private, +non-shared buffer header with the appropriate information +and call the device's strategy routine. +If desired, separate +.I open +and +.I close +routines may be provided but this is usually unnecessary. +A special-function routine might come in handy, especially for +magtape. +.PP +A great deal of work has to be done to generate the +``appropriate information'' +to put in the argument buffer for +the strategy module; +the worst part is to map relocated user addresses to physical addresses. +Most of this work is done by +.I "physio(strat, bp, dev, rw) +whose arguments are the name of the +strategy routine +.I strat, +the buffer pointer +.I bp, +the device number +.I dev, +and a read-write flag +.I rw +whose value is either +.I B_READ +or +.I B_WRITE. +.I Physio +makes sure that the user's base address and count are +even (because most devices work in words) +and that the core area affected is contiguous +in physical space; +it delays until the buffer is not busy, and makes it +busy while the operation is in progress; +and it sets up user error return information. diff --git a/share/doc/psd/04.uprog/Makefile b/share/doc/psd/04.uprog/Makefile new file mode 100644 index 000000000000..f149dcf27423 --- /dev/null +++ b/share/doc/psd/04.uprog/Makefile @@ -0,0 +1,8 @@ +# @(#)Makefile 8.1 (Berkeley) 6/8/93 +# $FreeBSD$ + +VOLUME= psd/04.uprog +SRCS= p.mac p0 p1 p2 p3 p4 p5 p6 p8 p9 +MACROS= -ms + +.include <bsd.doc.mk> diff --git a/share/doc/psd/04.uprog/p.mac b/share/doc/psd/04.uprog/p.mac new file mode 100644 index 000000000000..7f9295c729ac --- /dev/null +++ b/share/doc/psd/04.uprog/p.mac @@ -0,0 +1,71 @@ +.\" Copyright (C) Caldera International Inc. 2001-2002. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions are +.\" met: +.\" +.\" Redistributions of source code and documentation must retain the above +.\" copyright notice, this list of conditions and the following +.\" disclaimer. +.\" +.\" Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" +.\" All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" +.\" This product includes software developed or owned by Caldera +.\" International, Inc. Neither the name of Caldera International, Inc. +.\" nor the names of other contributors may be used to endorse or promote +.\" products derived from this software without specific prior written +.\" permission. +.\" +.\" USE OF THE SOFTWARE PROVIDED FOR UNDER THIS LICENSE BY CALDERA +.\" INTERNATIONAL, INC. AND CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR +.\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +.\" WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +.\" DISCLAIMED. IN NO EVENT SHALL CALDERA INTERNATIONAL, INC. BE LIABLE +.\" FOR ANY DIRECT, INDIRECT INCIDENTAL, SPECIAL, EXEMPLARY, OR +.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR +.\" BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, +.\" WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE +.\" OR OTHERWISE) RISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN +.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +.\" +.\" $FreeBSD$ +.\" +.\" @(#)p.mac 8.1 (Berkeley) 6/8/93 +.de UC +\&\\$3\s-1\\$1\\s0\&\\$2 +.. +.de IT +\&\\$3\fI\\$1\fR\^\&\\$2 +.. +.de UL +\%\&\\$3\f(CW\s-1\\$1\s0\fR\&\\$2 +.. +.de P1 +.DS I .5i +.nf +.ft CW +.ps \\n(PS-1 +.vs \\n(VS-1 +.. +.de P2 +.ps \\n(PS +.vs \\n(VS +.ft R +.DE +.. +.hy 14 \"2=not last lines; 4= no -xx; 8=no xx- +.am SH +.ft R +.. +.am NH +.ft R +.. +.am TL +.ft R +.. diff --git a/share/doc/psd/04.uprog/p0 b/share/doc/psd/04.uprog/p0 new file mode 100644 index 000000000000..5aaf32555f91 --- /dev/null +++ b/share/doc/psd/04.uprog/p0 @@ -0,0 +1,82 @@ +.\" Copyright (C) Caldera International Inc. 2001-2002. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions are +.\" met: +.\" +.\" Redistributions of source code and documentation must retain the above +.\" copyright notice, this list of conditions and the following +.\" disclaimer. +.\" +.\" Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" +.\" All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" +.\" This product includes software developed or owned by Caldera +.\" International, Inc. Neither the name of Caldera International, Inc. +.\" nor the names of other contributors may be used to endorse or promote +.\" products derived from this software without specific prior written +.\" permission. +.\" +.\" USE OF THE SOFTWARE PROVIDED FOR UNDER THIS LICENSE BY CALDERA +.\" INTERNATIONAL, INC. AND CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR +.\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +.\" WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +.\" DISCLAIMED. IN NO EVENT SHALL CALDERA INTERNATIONAL, INC. BE LIABLE +.\" FOR ANY DIRECT, INDIRECT INCIDENTAL, SPECIAL, EXEMPLARY, OR +.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR +.\" BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, +.\" WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE +.\" OR OTHERWISE) RISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN +.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +.\" +.\" $FreeBSD$ +.\" +.\" @(#)p0 8.1 (Berkeley) 6/8/93 +.\" +.if n .ls 1 +.\" .TM 78-1273-9 39199 39199-11 +.\" .ND October 2, 1978 +.\" .old TM 75-1273-11 October 22, 1975 +.OH 'UNIX Programming \(em Second Edition''PSD:4-%' +.EH 'PSD:4-%''UNIX Programming \(em Second Edition' +.TL +UNIX Programming \(em Second Edition +.AU "MH 2C-518" 6021 +Brian W. Kernighan +.AU "MH 2C-517" 3770 +Dennis M. Ritchie +.AI +AT&T Bell Laboratories +Murray Hill, NJ 07974 +.AB +.PP +This paper is an introduction to programming on +the +.UX +system. +The emphasis is on how to write programs that interface +to the operating system, +either directly or through the standard I/O library. +The topics discussed include +.IP " \(bu" +handling command arguments +.IP " \(bu" +rudimentary I/O; the standard input and output +.IP " \(bu" +the standard I/O library; file system access +.IP " \(bu" +low-level I/O: open, read, write, close, seek +.IP " \(bu" +processes: exec, fork, pipes +.IP " \(bu" +signals \(em interrupts, etc. +.PP +There is also an appendix which describes +the standard I/O library in detail. +.AE +.\" .CS 17 0 17 0 0 4 diff --git a/share/doc/psd/04.uprog/p1 b/share/doc/psd/04.uprog/p1 new file mode 100644 index 000000000000..848862ec7942 --- /dev/null +++ b/share/doc/psd/04.uprog/p1 @@ -0,0 +1,88 @@ +.\" Copyright (C) Caldera International Inc. 2001-2002. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions are +.\" met: +.\" +.\" Redistributions of source code and documentation must retain the above +.\" copyright notice, this list of conditions and the following +.\" disclaimer. +.\" +.\" Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" +.\" All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" +.\" This product includes software developed or owned by Caldera +.\" International, Inc. Neither the name of Caldera International, Inc. +.\" nor the names of other contributors may be used to endorse or promote +.\" products derived from this software without specific prior written +.\" permission. +.\" +.\" USE OF THE SOFTWARE PROVIDED FOR UNDER THIS LICENSE BY CALDERA +.\" INTERNATIONAL, INC. AND CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR +.\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +.\" WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +.\" DISCLAIMED. IN NO EVENT SHALL CALDERA INTERNATIONAL, INC. BE LIABLE +.\" FOR ANY DIRECT, INDIRECT INCIDENTAL, SPECIAL, EXEMPLARY, OR +.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR +.\" BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, +.\" WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE +.\" OR OTHERWISE) RISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN +.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +.\" +.\" $FreeBSD$ +.\" +.\" @(#)p1 8.1 (Berkeley) 6/8/93 +.\" +.if n .ls 2 +.if t .tr |\(or +.NH +INTRODUCTION +.PP +This paper describes how to write +programs +that interface with the +.UC UNIX +operating system in a non-trivial way. +This includes programs that use files by name, +that use pipes, +that invoke other commands as they run, +or that attempt to catch interrupts and other signals +during execution. +.PP +The document collects material which is scattered +throughout several sections of +.I +The +.UC UNIX +Programmer's Manual +.R +[1] +for Version 7 +.UC UNIX . +There is no attempt to be complete; +only generally useful material is dealt with. +It is assumed that you will be programming in C, +so you must be able to read the language +roughly up to the level of +.I +The C Programming Language +.R +[2]. +Some of the material in sections 2 through 4 +is based on +topics covered more carefully there. +You should also be familiar with +.UC UNIX +itself +at least +to the level of +.I +.UC UNIX +for Beginners +.R +[3]. diff --git a/share/doc/psd/04.uprog/p2 b/share/doc/psd/04.uprog/p2 new file mode 100644 index 000000000000..280f65cd51cd --- /dev/null +++ b/share/doc/psd/04.uprog/p2 @@ -0,0 +1,275 @@ +.\" Copyright (C) Caldera International Inc. 2001-2002. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions are +.\" met: +.\" +.\" Redistributions of source code and documentation must retain the above +.\" copyright notice, this list of conditions and the following +.\" disclaimer. +.\" +.\" Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" +.\" All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" +.\" This product includes software developed or owned by Caldera +.\" International, Inc. Neither the name of Caldera International, Inc. +.\" nor the names of other contributors may be used to endorse or promote +.\" products derived from this software without specific prior written +.\" permission. +.\" +.\" USE OF THE SOFTWARE PROVIDED FOR UNDER THIS LICENSE BY CALDERA +.\" INTERNATIONAL, INC. AND CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR +.\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +.\" WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +.\" DISCLAIMED. IN NO EVENT SHALL CALDERA INTERNATIONAL, INC. BE LIABLE +.\" FOR ANY DIRECT, INDIRECT INCIDENTAL, SPECIAL, EXEMPLARY, OR +.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR +.\" BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, +.\" WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE +.\" OR OTHERWISE) RISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN +.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +.\" +.\" $FreeBSD$ +.\" +.\" @(#)p2 8.1 (Berkeley) 6/8/93 +.\" +.NH +BASICS +.NH 2 +Program Arguments +.PP +When a C program is run as a command, +the arguments on the command line are made available +to the +function +.UL main +as an argument count +.UL argc +and an array +.UL argv +of +pointers to +character strings +that contain +the arguments. +By convention, +.UL argv[0] +is the command name itself, +so +.UL argc +is always greater than 0. +.PP +The following program illustrates the mechanism: +it simply echoes its arguments +back to the terminal. +(This is essentially the +.UL echo +command.) +.P1 +main(argc, argv) /* echo arguments */ +int argc; +char *argv[]; +{ + int i; + + for (i = 1; i < argc; i++) + printf("%s%c", argv[i], (i<argc-1) ? ' ' : '\en'); +} +.P2 +.UL argv +is a pointer to an array +whose individual elements are pointers to arrays of characters; +each is terminated by +.UL \e0 , +so they can be treated as strings. +The program starts by printing +.UL argv[1] +and loops until it has printed them all. +.PP +The argument count and the arguments +are parameters to +.UL main . +If you want to keep them around so other +routines can get at them, you must +copy them to external variables. +.NH 2 +The ``Standard Input'' and ``Standard Output'' +.PP +The simplest input mechanism is to read the ``standard input,'' +which is generally the user's terminal. +The function +.UL getchar +returns the next input character each time it is called. +A file may be substituted for the terminal by +using the +.UL < +convention: +if +.UL prog +uses +.UL getchar , +then +the command line +.P1 +prog <file +.P2 +causes +.UL prog +to read +.UL file +instead of the terminal. +.UL prog +itself need know nothing about where its input +is coming from. +This is also true if the input comes from another program via +the +.U +pipe mechanism: +.P1 +otherprog | prog +.P2 +provides the standard input for +.UL prog +from the standard output of +.UL otherprog. +.PP +.UL getchar +returns the value +.UL EOF +when it encounters the end of file +(or an error) +on whatever you are reading. +The value of +.UL EOF +is normally defined to be +.UL -1 , +but it is unwise to take any advantage +of that knowledge. +As will become clear shortly, +this value is automatically defined for you when +you compile a program, +and need not be of any concern. +.PP +Similarly, +.UL putchar(c) +puts the character +.UL c +on the ``standard output,'' +which is also by default the terminal. +The output can be captured on a file +by using +.UL > : +if +.UL prog +uses +.UL putchar , +.P1 +prog >outfile +.P2 +writes the standard output on +.UL outfile +instead of the terminal. +.UL outfile +is created if it doesn't exist; +if it already exists, its previous contents are overwritten. +And a pipe can be used: +.P1 +prog | otherprog +.P2 +puts the standard output of +.UL prog +into the standard input of +.UL otherprog. +.PP +The function +.UL printf , +which formats output in various ways, +uses +the same mechanism as +.UL putchar +does, +so calls to +.UL printf +and +.UL putchar +may be intermixed in any order; +the output will appear in the order of the calls. +.PP +Similarly, the function +.UL scanf +provides for formatted input conversion; +it will read the standard input and break it +up into strings, numbers, etc., +as desired. +.UL scanf +uses the same mechanism as +.UL getchar , +so calls to them may also be intermixed. +.PP +Many programs +read only one input and write one output; +for such programs I/O +with +.UL getchar , +.UL putchar , +.UL scanf , +and +.UL printf +may be entirely adequate, +and it is almost always enough to get started. +This is particularly true if +the +.UC UNIX +pipe facility is used to connect the output of +one program to the input of the next. +For example, the following program +strips out all ascii control characters +from its input +(except for newline and tab). +.P1 +#include <stdio.h> + +main() /* ccstrip: strip non-graphic characters */ +{ + int c; + while ((c = getchar()) != EOF) + if ((c >= ' ' && c < 0177) || c == '\et' || c == '\en') + putchar(c); + exit(0); +} +.P2 +The line +.P1 +#include <stdio.h> +.P2 +should appear at the beginning of each source file. +It causes the C compiler to read a file +.IT /usr/include/stdio.h ) ( +of +standard routines and symbols +that includes the definition of +.UL EOF . +.PP +If it is necessary to treat multiple files, +you can use +.UL cat +to collect the files for you: +.P1 +cat file1 file2 ... | ccstrip >output +.P2 +and thus avoid learning how to access files from a program. +By the way, +the call to +.UL exit +at the end is not necessary to make the program work +properly, +but it assures that any caller +of the program will see a normal termination status +(conventionally 0) +from the program when it completes. +Section 6 discusses status returns in more detail. diff --git a/share/doc/psd/04.uprog/p3 b/share/doc/psd/04.uprog/p3 new file mode 100644 index 000000000000..201c4a904af2 --- /dev/null +++ b/share/doc/psd/04.uprog/p3 @@ -0,0 +1,469 @@ +.\" Copyright (C) Caldera International Inc. 2001-2002. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions are +.\" met: +.\" +.\" Redistributions of source code and documentation must retain the above +.\" copyright notice, this list of conditions and the following +.\" disclaimer. +.\" +.\" Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" +.\" All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" +.\" This product includes software developed or owned by Caldera +.\" International, Inc. Neither the name of Caldera International, Inc. +.\" nor the names of other contributors may be used to endorse or promote +.\" products derived from this software without specific prior written +.\" permission. +.\" +.\" USE OF THE SOFTWARE PROVIDED FOR UNDER THIS LICENSE BY CALDERA +.\" INTERNATIONAL, INC. AND CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR +.\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +.\" WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +.\" DISCLAIMED. IN NO EVENT SHALL CALDERA INTERNATIONAL, INC. BE LIABLE +.\" FOR ANY DIRECT, INDIRECT INCIDENTAL, SPECIAL, EXEMPLARY, OR +.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR +.\" BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, +.\" WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE +.\" OR OTHERWISE) RISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN +.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +.\" +.\" $FreeBSD$ +.\" +.\" @(#)p3 8.1 (Berkeley) 6/8/93 +.\" +.NH +THE STANDARD I/O LIBRARY +.PP +The ``Standard I/O Library'' +is a collection of routines +intended to provide +efficient +and portable +I/O services +for most C programs. +The standard I/O library is available on each system that supports C, +so programs that confine +their system interactions +to its facilities +can be transported from one system to another essentially without change. +.PP +In this section, we will discuss the basics of the standard I/O library. +The appendix contains a more complete description of its capabilities. +.NH 2 +File Access +.PP +The programs written so far have all +read the standard input and written the standard output, +which we have assumed are magically pre-defined. +The next step +is to write a program that accesses +a file that is +.ul +not +already connected to the program. +One simple example is +.IT wc , +which counts the lines, words and characters +in a set of files. +For instance, the command +.P1 +wc x.c y.c +.P2 +prints the number of lines, words and characters +in +.UL x.c +and +.UL y.c +and the totals. +.PP +The question is how to arrange for the named files +to be read \(em +that is, how to connect the file system names +to the I/O statements which actually read the data. +.PP +The rules are simple. +Before it can be read or written +a file has to be +.ul +opened +by the standard library function +.UL fopen . +.UL fopen +takes an external name +(like +.UL x.c +or +.UL y.c ), +does some housekeeping and negotiation with the operating system, +and returns an internal name +which must be used in subsequent +reads or writes of the file. +.PP +This internal name is actually a pointer, +called a +.IT file +.IT pointer , +to a structure +which contains information about the file, +such as the location of a buffer, +the current character position in the buffer, +whether the file is being read or written, +and the like. +Users don't need to know the details, +because part of the standard I/O definitions +obtained by including +.UL stdio.h +is a structure definition called +.UL FILE . +The only declaration needed for a file pointer +is exemplified by +.P1 +FILE *fp, *fopen(); +.P2 +This says that +.UL fp +is a pointer to a +.UL FILE , +and +.UL fopen +returns a pointer to +a +.UL FILE . +.UL FILE \& ( +is a type name, like +.UL int , +not a structure tag. +.PP +The actual call to +.UL fopen +in a program +is +.P1 +fp = fopen(name, mode); +.P2 +The first argument of +.UL fopen +is the +name +of the file, +as a character string. +The second argument is the +mode, +also as a character string, +which indicates how you intend to +use the file. +The only allowable modes are +read +.UL \&"r" ), ( +write +.UL \&"w" ), ( +or append +.UL \&"a" ). ( +.PP +If a file that you open for writing or appending does not exist, +it is created +(if possible). +Opening an existing file for writing causes the old contents +to be discarded. +Trying to read a file that does not exist +is an error, +and there may be other causes of error +as well +(like trying to read a file +when you don't have permission). +If there is any error, +.UL fopen +will return the null pointer +value +.UL NULL +(which is defined as zero in +.UL stdio.h ). +.PP +The next thing needed is a way to read or write the file +once it is open. +There are several possibilities, +of which +.UL getc +and +.UL putc +are the simplest. +.UL getc +returns the next character from a file; +it needs the file pointer to tell it what file. +Thus +.P1 +c = getc(fp) +.P2 +places in +.UL c +the next character from the file referred to by +.UL fp ; +it returns +.UL EOF +when it reaches end of file. +.UL putc +is the inverse of +.UL getc : +.P1 +putc(c, fp) +.P2 +puts the character +.UL c +on the file +.UL fp +and returns +.UL c . +.UL getc +and +.UL putc +return +.UL EOF +on error. +.PP +When a program is started, three files are opened automatically, +and file pointers are provided for them. +These files are the standard input, +the standard output, +and the standard error output; +the corresponding file pointers are +called +.UL stdin , +.UL stdout , +and +.UL stderr . +Normally these are all connected to the terminal, +but +may be redirected to files or pipes as described in +Section 2.2. +.UL stdin , +.UL stdout +and +.UL stderr +are pre-defined in the I/O library +as the standard input, output and error files; +they may be used anywhere an object of type +.UL FILE\ * +can be. +They are +constants, however, +.ul +not +variables, +so don't try to assign to them. +.PP +With some of the preliminaries out of the way, +we can now write +.IT wc . +The basic design +is one that has been found +convenient for many programs: +if there are command-line arguments, they are processed in order. +If there are no arguments, the standard input +is processed. +This way the program can be used stand-alone +or as part of a larger process. +.P1 +#include <stdio.h> + +main(argc, argv) /* wc: count lines, words, chars */ +int argc; +char *argv[]; +{ + int c, i, inword; + FILE *fp, *fopen(); + long linect, wordct, charct; + long tlinect = 0, twordct = 0, tcharct = 0; + + i = 1; + fp = stdin; + do { + if (argc > 1 && (fp=fopen(argv[i], "r")) == NULL) { + fprintf(stderr, "wc: can't open %s\en", argv[i]); + continue; + } + linect = wordct = charct = inword = 0; + while ((c = getc(fp)) != EOF) { + charct++; + if (c == '\en') + linect++; + if (c == ' ' || c == '\et' || c == '\en') + inword = 0; + else if (inword == 0) { + inword = 1; + wordct++; + } + } + printf("%7ld %7ld %7ld", linect, wordct, charct); + printf(argc > 1 ? " %s\en" : "\en", argv[i]); + fclose(fp); + tlinect += linect; + twordct += wordct; + tcharct += charct; + } while (++i < argc); + if (argc > 2) + printf("%7ld %7ld %7ld total\en", tlinect, twordct, tcharct); + exit(0); +} +.P2 +The function +.UL fprintf +is identical to +.UL printf , +save that the first argument is a file pointer +that specifies the file to be +written. +.PP +The function +.UL fclose +is the inverse of +.UL fopen ; +it breaks the connection between the file pointer and the external name +that was established by +.UL fopen , +freeing the +file pointer for another file. +Since there is a limit on the number +of files +that a program may have open simultaneously, +it's a good idea to free things when they are no longer needed. +There is also another reason to call +.UL fclose +on an output file +\(em it flushes the buffer +in which +.UL putc +is collecting output. +.UL fclose \& ( +is called automatically for each open file +when a program terminates normally.) +.NH 2 +Error Handling \(em Stderr and Exit +.PP +.UL stderr +is assigned to a program in the same way that +.UL stdin +and +.UL stdout +are. +Output written on +.UL stderr +appears on the user's terminal +even if the standard output is redirected. +.IT wc +writes its diagnostics on +.UL stderr +instead of +.UL stdout +so that if one of the files can't +be accessed for some reason, +the message +finds its way to the user's terminal instead of disappearing +down a pipeline +or into an output file. +.PP +The program actually signals errors in another way, +using the function +.UL exit +to terminate program execution. +The argument of +.UL exit +is available to whatever process +called it (see Section 6), +so the success or failure +of the program can be tested by another program +that uses this one as a sub-process. +By convention, a return value of 0 +signals that all is well; +non-zero values signal abnormal situations. +.PP +.UL exit +itself +calls +.UL fclose +for each open output file, +to flush out any buffered output, +then calls +a routine named +.UL _exit . +The function +.UL _exit +causes immediate termination without any buffer flushing; +it may be called directly if desired. +.NH 2 +Miscellaneous I/O Functions +.PP +The standard I/O library provides several other I/O functions +besides those we have illustrated above. +.PP +Normally output with +.UL putc , +etc., is buffered (except to +.UL stderr ); +to force it out immediately, use +.UL fflush(fp) . +.PP +.UL fscanf +is identical to +.UL scanf , +except that its first argument is a file pointer +(as with +.UL fprintf ) +that specifies the file from which the input comes; +it returns +.UL EOF +at end of file. +.PP +The functions +.UL sscanf +and +.UL sprintf +are identical to +.UL fscanf +and +.UL fprintf , +except that the first argument names a character string +instead of a file pointer. +The conversion is done from the string +for +.UL sscanf +and into it for +.UL sprintf . +.PP +.UL fgets(buf,\ size,\ fp) +copies the next line from +.UL fp , +up to and including a newline, +into +.UL buf ; +at most +.UL size-1 +characters are copied; +it returns +.UL NULL +at end of file. +.UL fputs(buf,\ fp) +writes the string in +.UL buf +onto file +.UL fp . +.PP +The function +.UL ungetc(c,\ fp) +``pushes back'' the character +.UL c +onto the input stream +.UL fp ; +a subsequent call to +.UL getc , +.UL fscanf , +etc., +will encounter +.UL c . +Only one character of pushback per file is permitted. diff --git a/share/doc/psd/04.uprog/p4 b/share/doc/psd/04.uprog/p4 new file mode 100644 index 000000000000..fe23ac31d540 --- /dev/null +++ b/share/doc/psd/04.uprog/p4 @@ -0,0 +1,600 @@ +.\" Copyright (C) Caldera International Inc. 2001-2002. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions are +.\" met: +.\" +.\" Redistributions of source code and documentation must retain the above +.\" copyright notice, this list of conditions and the following +.\" disclaimer. +.\" +.\" Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" +.\" All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" +.\" This product includes software developed or owned by Caldera +.\" International, Inc. Neither the name of Caldera International, Inc. +.\" nor the names of other contributors may be used to endorse or promote +.\" products derived from this software without specific prior written +.\" permission. +.\" +.\" USE OF THE SOFTWARE PROVIDED FOR UNDER THIS LICENSE BY CALDERA +.\" INTERNATIONAL, INC. AND CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR +.\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +.\" WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +.\" DISCLAIMED. IN NO EVENT SHALL CALDERA INTERNATIONAL, INC. BE LIABLE +.\" FOR ANY DIRECT, INDIRECT INCIDENTAL, SPECIAL, EXEMPLARY, OR +.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR +.\" BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, +.\" WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE +.\" OR OTHERWISE) RISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN +.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +.\" +.\" $FreeBSD$ +.\" +.\" @(#)p4 8.1 (Berkeley) 6/8/93 +.\" +.NH +LOW-LEVEL I/O +.PP +This section describes the +bottom level of I/O on the +.UC UNIX +system. +The lowest level of I/O in +.UC UNIX +provides no buffering or any other services; +it is in fact a direct entry into the operating system. +You are entirely on your own, +but on the other hand, +you have the most control over what happens. +And since the calls and usage are quite simple, +this isn't as bad as it sounds. +.NH 2 +File Descriptors +.PP +In the +.UC UNIX +operating system, +all input and output is done +by reading or writing files, +because all peripheral devices, even the user's terminal, +are files in the file system. +This means that a single, homogeneous interface +handles all communication between a program and peripheral devices. +.PP +In the most general case, +before reading or writing a file, +it is necessary to inform the system +of your intent to do so, +a process called +``opening'' the file. +If you are going to write on a file, +it may also be necessary to create it. +The system checks your right to do so +(Does the file exist? +Do you have permission to access it?), +and if all is well, +returns a small positive integer +called a +.ul +file descriptor. +Whenever I/O is to be done on the file, +the file descriptor is used instead of the name to identify the file. +(This is roughly analogous to the use of +.UC READ(5,...) +and +.UC WRITE(6,...) +in Fortran.) +All +information about an open file is maintained by the system; +the user program refers to the file +only +by the file descriptor. +.PP +The file pointers discussed in section 3 +are similar in spirit to file descriptors, +but file descriptors are more fundamental. +A file pointer is a pointer to a structure that contains, +among other things, the file descriptor for the file in question. +.PP +Since input and output involving the user's terminal +are so common, +special arrangements exist to make this convenient. +When the command interpreter (the +``shell'') +runs a program, +it opens +three files, with file descriptors 0, 1, and 2, +called the standard input, +the standard output, and the standard error output. +All of these are normally connected to the terminal, +so if a program reads file descriptor 0 +and writes file descriptors 1 and 2, +it can do terminal I/O +without worrying about opening the files. +.PP +If I/O is redirected +to and from files with +.UL < +and +.UL > , +as in +.P1 +prog <infile >outfile +.P2 +the shell changes the default assignments for file descriptors +0 and 1 +from the terminal to the named files. +Similar observations hold if the input or output is associated with a pipe. +Normally file descriptor 2 remains attached to the terminal, +so error messages can go there. +In all cases, +the file assignments are changed by the shell, +not by the program. +The program does not need to know where its input +comes from nor where its output goes, +so long as it uses file 0 for input and 1 and 2 for output. +.NH 2 +Read and Write +.PP +All input and output is done by +two functions called +.UL read +and +.UL write . +For both, the first argument is a file descriptor. +The second argument is a buffer in your program where the data is to +come from or go to. +The third argument is the number of bytes to be transferred. +The calls are +.P1 +n_read = read(fd, buf, n); + +n_written = write(fd, buf, n); +.P2 +Each call returns a byte count +which is the number of bytes actually transferred. +On reading, +the number of bytes returned may be less than +the number asked for, +because fewer than +.UL n +bytes remained to be read. +(When the file is a terminal, +.UL read +normally reads only up to the next newline, +which is generally less than what was requested.) +A return value of zero bytes implies end of file, +and +.UL -1 +indicates an error of some sort. +For writing, the returned value is the number of bytes +actually written; +it is generally an error if this isn't equal +to the number supposed to be written. +.PP +The number of bytes to be read or written is quite arbitrary. +The two most common values are +1, +which means one character at a time +(``unbuffered''), +and +512, +which corresponds to a physical blocksize on many peripheral devices. +This latter size will be most efficient, +but even character at a time I/O +is not inordinately expensive. +.PP +Putting these facts together, +we can write a simple program to copy +its input to its output. +This program will copy anything to anything, +since the input and output can be redirected to any file or device. +.P1 +#define BUFSIZE 512 /* best size for PDP-11 UNIX */ + +main() /* copy input to output */ +{ + char buf[BUFSIZE]; + int n; + + while ((n = read(0, buf, BUFSIZE)) > 0) + write(1, buf, n); + exit(0); +} +.P2 +If the file size is not a multiple of +.UL BUFSIZE , +some +.UL read +will return a smaller number of bytes +to be written by +.UL write ; +the next call to +.UL read +after that +will return zero. +.PP +It is instructive to see how +.UL read +and +.UL write +can be used to construct +higher level routines like +.UL getchar , +.UL putchar , +etc. +For example, +here is a version of +.UL getchar +which does unbuffered input. +.P1 +#define CMASK 0377 /* for making char's > 0 */ + +getchar() /* unbuffered single character input */ +{ + char c; + + return((read(0, &c, 1) > 0) ? c & CMASK : EOF); +} +.P2 +.UL c +.ul +must +be declared +.UL char , +because +.UL read +accepts a character pointer. +The character being returned must be masked with +.UL 0377 +to ensure that it is positive; +otherwise sign extension may make it negative. +(The constant +.UL 0377 +is appropriate for the +.UC PDP -11 +but not necessarily for other machines.) +.PP +The second version of +.UL getchar +does input in big chunks, +and hands out the characters one at a time. +.P1 +#define CMASK 0377 /* for making char's > 0 */ +#define BUFSIZE 512 + +getchar() /* buffered version */ +{ + static char buf[BUFSIZE]; + static char *bufp = buf; + static int n = 0; + + if (n == 0) { /* buffer is empty */ + n = read(0, buf, BUFSIZE); + bufp = buf; + } + return((--n >= 0) ? *bufp++ & CMASK : EOF); +} +.P2 +.NH 2 +Open, Creat, Close, Unlink +.PP +Other than the default +standard input, output and error files, +you must explicitly open files in order to +read or write them. +There are two system entry points for this, +.UL open +and +.UL creat +[sic]. +.PP +.UL open +is rather like the +.UL fopen +discussed in the previous section, +except that instead of returning a file pointer, +it returns a file descriptor, +which is just an +.UL int . +.P1 +int fd; + +fd = open(name, rwmode); +.P2 +As with +.UL fopen , +the +.UL name +argument +is a character string corresponding to the external file name. +The access mode argument +is different, however: +.UL rwmode +is 0 for read, 1 for write, and 2 for read and write access. +.UL open +returns +.UL -1 +if any error occurs; +otherwise it returns a valid file descriptor. +.PP +It is an error to +try to +.UL open +a file that does not exist. +The entry point +.UL creat +is provided to create new files, +or to re-write old ones. +.P1 +fd = creat(name, pmode); +.P2 +returns a file descriptor +if it was able to create the file +called +.UL name , +and +.UL -1 +if not. +If the file +already exists, +.UL creat +will truncate it to zero length; +it is not an error to +.UL creat +a file that already exists. +.PP +If the file is brand new, +.UL creat +creates it with the +.ul +protection mode +specified by +the +.UL pmode +argument. +In the +.UC UNIX +file system, +there are nine bits of protection information +associated with a file, +controlling read, write and execute permission for +the owner of the file, +for the owner's group, +and for all others. +Thus a three-digit octal number +is most convenient for specifying the permissions. +For example, +0755 +specifies read, write and execute permission for the owner, +and read and execute permission for the group and everyone else. +.PP +To illustrate, +here is a simplified version of +the +.UC UNIX +utility +.IT cp , +a program which copies one file to another. +(The main simplification is that our version +copies only one file, +and does not permit the second argument +to be a directory.) +.P1 +#define NULL 0 +#define BUFSIZE 512 +#define PMODE 0644 /* RW for owner, R for group, others */ + +main(argc, argv) /* cp: copy f1 to f2 */ +int argc; +char *argv[]; +{ + int f1, f2, n; + char buf[BUFSIZE]; + + if (argc != 3) + error("Usage: cp from to", NULL); + if ((f1 = open(argv[1], 0)) == -1) + error("cp: can't open %s", argv[1]); + if ((f2 = creat(argv[2], PMODE)) == -1) + error("cp: can't create %s", argv[2]); + + while ((n = read(f1, buf, BUFSIZE)) > 0) + if (write(f2, buf, n) != n) + error("cp: write error", NULL); + exit(0); +} +.P2 +.P1 +error(s1, s2) /* print error message and die */ +char *s1, *s2; +{ + printf(s1, s2); + printf("\en"); + exit(1); +} +.P2 +.PP +As we said earlier, +there is a limit (typically 15-25) +on the number of files which a program +may have open simultaneously. +Accordingly, any program which intends to process +many files must be prepared to re-use +file descriptors. +The routine +.UL close +breaks the connection between a file descriptor +and an open file, +and frees the +file descriptor for use with some other file. +Termination of a program +via +.UL exit +or return from the main program closes all open files. +.PP +The function +.UL unlink(filename) +removes the file +.UL filename +from the file system. +.NH 2 +Random Access \(em Seek and Lseek +.PP +File I/O is normally sequential: +each +.UL read +or +.UL write +takes place at a position in the file +right after the previous one. +When necessary, however, +a file can be read or written in any arbitrary order. +The +system call +.UL lseek +provides a way to move around in +a file without actually reading +or writing: +.P1 +lseek(fd, offset, origin); +.P2 +forces the current position in the file +whose descriptor is +.UL fd +to move to position +.UL offset , +which is taken relative to the location +specified by +.UL origin . +Subsequent reading or writing will begin at that position. +.UL offset +is +a +.UL long ; +.UL fd +and +.UL origin +are +.UL int 's. +.UL origin +can be 0, 1, or 2 to specify that +.UL offset +is to be +measured from +the beginning, from the current position, or from the +end of the file respectively. +For example, +to append to a file, +seek to the end before writing: +.P1 +lseek(fd, 0L, 2); +.P2 +To get back to the beginning (``rewind''), +.P1 +lseek(fd, 0L, 0); +.P2 +Notice the +.UL 0L +argument; +it could also be written as +.UL (long)\ 0 . +.PP +With +.UL lseek , +it is possible to treat files more or less like large arrays, +at the price of slower access. +For example, the following simple function reads any number of bytes +from any arbitrary place in a file. +.P1 +get(fd, pos, buf, n) /* read n bytes from position pos */ +int fd, n; +long pos; +char *buf; +{ + lseek(fd, pos, 0); /* get to pos */ + return(read(fd, buf, n)); +} +.P2 +.PP +In pre-version 7 +.UC UNIX , +the basic entry point to the I/O system +is called +.UL seek . +.UL seek +is identical to +.UL lseek , +except that its +.UL offset +argument is an +.UL int +rather than a +.UL long . +Accordingly, +since +.UC PDP -11 +integers have only 16 bits, +the +.UL offset +specified +for +.UL seek +is limited to 65,535; +for this reason, +.UL origin +values of 3, 4, 5 cause +.UL seek +to multiply the given offset by 512 +(the number of bytes in one physical block) +and then interpret +.UL origin +as if it were 0, 1, or 2 respectively. +Thus to get to an arbitrary place in a large file +requires two seeks, first one which selects +the block, then one which +has +.UL origin +equal to 1 and moves to the desired byte within the block. +.NH 2 +Error Processing +.PP +The routines discussed in this section, +and in fact all the routines which are direct entries into the system +can incur errors. +Usually they indicate an error by returning a value of \-1. +Sometimes it is nice to know what sort of error occurred; +for this purpose all these routines, when appropriate, +leave an error number in the external cell +.UL errno . +The meanings of the various error numbers are +listed +in the introduction to Section II +of the +.I +.UC UNIX +Programmer's Manual, +.R +so your program can, for example, determine if +an attempt to open a file failed because it did not exist +or because the user lacked permission to read it. +Perhaps more commonly, +you may want to print out the +reason for failure. +The routine +.UL perror +will print a message associated with the value +of +.UL errno ; +more generally, +.UL sys\_errno +is an array of character strings which can be indexed +by +.UL errno +and printed by your program. diff --git a/share/doc/psd/04.uprog/p5 b/share/doc/psd/04.uprog/p5 new file mode 100644 index 000000000000..652641fbfe7d --- /dev/null +++ b/share/doc/psd/04.uprog/p5 @@ -0,0 +1,577 @@ +.\" Copyright (C) Caldera International Inc. 2001-2002. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions are +.\" met: +.\" +.\" Redistributions of source code and documentation must retain the above +.\" copyright notice, this list of conditions and the following +.\" disclaimer. +.\" +.\" Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" +.\" All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" +.\" This product includes software developed or owned by Caldera +.\" International, Inc. Neither the name of Caldera International, Inc. +.\" nor the names of other contributors may be used to endorse or promote +.\" products derived from this software without specific prior written +.\" permission. +.\" +.\" USE OF THE SOFTWARE PROVIDED FOR UNDER THIS LICENSE BY CALDERA +.\" INTERNATIONAL, INC. AND CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR +.\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +.\" WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +.\" DISCLAIMED. IN NO EVENT SHALL CALDERA INTERNATIONAL, INC. BE LIABLE +.\" FOR ANY DIRECT, INDIRECT INCIDENTAL, SPECIAL, EXEMPLARY, OR +.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR +.\" BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, +.\" WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE +.\" OR OTHERWISE) RISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN +.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +.\" +.\" $FreeBSD$ +.\" +.\" @(#)p5 8.1 (Berkeley) 6/8/93 +.\" +.NH +PROCESSES +.PP +It is often easier to use a program written +by someone else than to invent one's own. +This section describes how to +execute a program from within another. +.NH 2 +The ``System'' Function +.PP +The easiest way to execute a program from another +is to use +the standard library routine +.UL system . +.UL system +takes one argument, a command string exactly as typed +at the terminal +(except for the newline at the end) +and executes it. +For instance, to time-stamp the output of a program, +.P1 +main() +{ + system("date"); + /* rest of processing */ +} +.P2 +If the command string has to be built from pieces, +the in-memory formatting capabilities of +.UL sprintf +may be useful. +.PP +Remember than +.UL getc +and +.UL putc +normally buffer their input; +terminal I/O will not be properly synchronized unless +this buffering is defeated. +For output, use +.UL fflush ; +for input, see +.UL setbuf +in the appendix. +.NH 2 +Low-Level Process Creation \(em Execl and Execv +.PP +If you're not using the standard library, +or if you need finer control over what +happens, +you will have to construct calls to other programs +using the more primitive routines that the standard +library's +.UL system +routine is based on. +.PP +The most basic operation is to execute another program +.ul +without +.IT returning , +by using the routine +.UL execl . +To print the date as the last action of a running program, +use +.P1 +execl("/bin/date", "date", NULL); +.P2 +The first argument to +.UL execl +is the +.ul +file name +of the command; you have to know where it is found +in the file system. +The second argument is conventionally +the program name +(that is, the last component of the file name), +but this is seldom used except as a place-holder. +If the command takes arguments, they are strung out after +this; +the end of the list is marked by a +.UL NULL +argument. +.PP +The +.UL execl +call +overlays the existing program with +the new one, +runs that, then exits. +There is +.ul +no +return to the original program. +.PP +More realistically, +a program might fall into two or more phases +that communicate only through temporary files. +Here it is natural to make the second pass +simply an +.UL execl +call from the first. +.PP +The one exception to the rule that the original program never gets control +back occurs when there is an error, for example if the file can't be found +or is not executable. +If you don't know where +.UL date +is located, say +.P1 +execl("/bin/date", "date", NULL); +execl("/usr/bin/date", "date", NULL); +fprintf(stderr, "Someone stole 'date'\en"); +.P2 +.PP +A variant of +.UL execl +called +.UL execv +is useful when you don't know in advance how many arguments there are going to be. +The call is +.P1 +execv(filename, argp); +.P2 +where +.UL argp +is an array of pointers to the arguments; +the last pointer in the array must be +.UL NULL +so +.UL execv +can tell where the list ends. +As with +.UL execl , +.UL filename +is the file in which the program is found, and +.UL argp[0] +is the name of the program. +(This arrangement is identical to the +.UL argv +array for program arguments.) +.PP +Neither of these routines provides the niceties of normal command execution. +There is no automatic search of multiple directories \(em +you have to know precisely where the command is located. +Nor do you get the expansion of metacharacters like +.UL < , +.UL > , +.UL * , +.UL ? , +and +.UL [] +in the argument list. +If you want these, use +.UL execl +to invoke the shell +.UL sh , +which then does all the work. +Construct a string +.UL commandline +that contains the complete command as it would have been typed +at the terminal, then say +.P1 +execl("/bin/sh", "sh", "-c", commandline, NULL); +.P2 +The shell is assumed to be at a fixed place, +.UL /bin/sh . +Its argument +.UL -c +says to treat the next argument +as a whole command line, so it does just what you want. +The only problem is in constructing the right information +in +.UL commandline . +.NH 2 +Control of Processes \(em Fork and Wait +.PP +So far what we've talked about isn't really all that useful by itself. +Now we will show how to regain control after running +a program with +.UL execl +or +.UL execv . +Since these routines simply overlay the new program on the old one, +to save the old one requires that it first be split into +two copies; +one of these can be overlaid, while the other waits for the new, +overlaying program to finish. +The splitting is done by a routine called +.UL fork : +.P1 +proc_id = fork(); +.P2 +splits the program into two copies, both of which continue to run. +The only difference between the two is the value of +.UL proc_id , +the ``process id.'' +In one of these processes (the ``child''), +.UL proc_id +is zero. +In the other +(the ``parent''), +.UL proc_id +is non-zero; it is the process number of the child. +Thus the basic way to call, and return from, +another program is +.P1 +if (fork() == 0) + execl("/bin/sh", "sh", "-c", cmd, NULL); /* in child */ +.P2 +And in fact, except for handling errors, this is sufficient. +The +.UL fork +makes two copies of the program. +In the child, the value returned by +.UL fork +is zero, so it calls +.UL execl +which does the +.UL command +and then dies. +In the parent, +.UL fork +returns non-zero +so it skips the +.UL execl. +(If there is any error, +.UL fork +returns +.UL -1 ). +.PP +More often, the parent wants to wait for the child to terminate +before continuing itself. +This can be done with +the function +.UL wait : +.P1 +int status; + +if (fork() == 0) + execl(...); +wait(&status); +.P2 +This still doesn't handle any abnormal conditions, such as a failure +of the +.UL execl +or +.UL fork , +or the possibility that there might be more than one child running simultaneously. +(The +.UL wait +returns the +process id +of the terminated child, if you want to check it against the value +returned by +.UL fork .) +Finally, this fragment doesn't deal with any +funny behavior on the part of the child +(which is reported in +.UL status ). +Still, these three lines +are the heart of the standard library's +.UL system +routine, +which we'll show in a moment. +.PP +The +.UL status +returned by +.UL wait +encodes in its low-order eight bits +the system's idea of the child's termination status; +it is 0 for normal termination and non-zero to indicate +various kinds of problems. +The next higher eight bits are taken from the argument +of the call to +.UL exit +which caused a normal termination of the child process. +It is good coding practice +for all programs to return meaningful +status. +.PP +When a program is called by the shell, +the three file descriptors +0, 1, and 2 are set up pointing at the right files, +and all other possible file descriptors +are available for use. +When this program calls another one, +correct etiquette suggests making sure the same conditions +hold. +Neither +.UL fork +nor the +.UL exec +calls affects open files in any way. +If the parent is buffering output +that must come out before output from the child, +the parent must flush its buffers +before the +.UL execl . +Conversely, +if a caller buffers an input stream, +the called program will lose any information +that has been read by the caller. +.NH 2 +Pipes +.PP +A +.ul +pipe +is an I/O channel intended for use +between two cooperating processes: +one process writes into the pipe, +while the other reads. +The system looks after buffering the data and synchronizing +the two processes. +Most pipes are created by the shell, +as in +.P1 +ls | pr +.P2 +which connects the standard output of +.UL ls +to the standard input of +.UL pr . +Sometimes, however, it is most convenient +for a process to set up its own plumbing; +in this section, we will illustrate how +the pipe connection is established and used. +.PP +The system call +.UL pipe +creates a pipe. +Since a pipe is used for both reading and writing, +two file descriptors are returned; +the actual usage is like this: +.P1 +int fd[2]; + +stat = pipe(fd); +if (stat == -1) + /* there was an error ... */ +.P2 +.UL fd +is an array of two file descriptors, where +.UL fd[0] +is the read side of the pipe and +.UL fd[1] +is for writing. +These may be used in +.UL read , +.UL write +and +.UL close +calls just like any other file descriptors. +.PP +If a process reads a pipe which is empty, +it will wait until data arrives; +if a process writes into a pipe which +is too full, it will wait until the pipe empties somewhat. +If the write side of the pipe is closed, +a subsequent +.UL read +will encounter end of file. +.PP +To illustrate the use of pipes in a realistic setting, +let us write a function called +.UL popen(cmd,\ mode) , +which creates a process +.UL cmd +(just as +.UL system +does), +and returns a file descriptor that will either +read or write that process, according to +.UL mode . +That is, +the call +.P1 +fout = popen("pr", WRITE); +.P2 +creates a process that executes +the +.UL pr +command; +subsequent +.UL write +calls using the file descriptor +.UL fout +will send their data to that process +through the pipe. +.PP +.UL popen +first creates the +the pipe with a +.UL pipe +system call; +it then +.UL fork s +to create two copies of itself. +The child decides whether it is supposed to read or write, +closes the other side of the pipe, +then calls the shell (via +.UL execl ) +to run the desired process. +The parent likewise closes the end of the pipe it does not use. +These closes are necessary to make end-of-file tests work properly. +For example, if a child that intends to read +fails to close the write end of the pipe, it will never +see the end of the pipe file, just because there is one writer +potentially active. +.P1 +#include <stdio.h> + +#define READ 0 +#define WRITE 1 +#define tst(a, b) (mode == READ ? (b) : (a)) +static int popen_pid; + +popen(cmd, mode) +char *cmd; +int mode; +{ + int p[2]; + + if (pipe(p) < 0) + return(NULL); + if ((popen_pid = fork()) == 0) { + close(tst(p[WRITE], p[READ])); + close(tst(0, 1)); + dup(tst(p[READ], p[WRITE])); + close(tst(p[READ], p[WRITE])); + execl("/bin/sh", "sh", "-c", cmd, 0); + _exit(1); /* disaster has occurred if we get here */ + } + if (popen_pid == -1) + return(NULL); + close(tst(p[READ], p[WRITE])); + return(tst(p[WRITE], p[READ])); +} +.P2 +The sequence of +.UL close s +in the child +is a bit tricky. +Suppose +that the task is to create a child process that will read data from the parent. +Then the first +.UL close +closes the write side of the pipe, +leaving the read side open. +The lines +.P1 +close(tst(0, 1)); +dup(tst(p[READ], p[WRITE])); +.P2 +are the conventional way to associate the pipe descriptor +with the standard input of the child. +The +.UL close +closes file descriptor 0, +that is, the standard input. +.UL dup +is a system call that +returns a duplicate of an already open file descriptor. +File descriptors are assigned in increasing order +and the first available one is returned, +so +the effect of the +.UL dup +is to copy the file descriptor for the pipe (read side) +to file descriptor 0; +thus the read side of the pipe becomes the standard input. +(Yes, this is a bit tricky, but it's a standard idiom.) +Finally, the old read side of the pipe is closed. +.PP +A similar sequence of operations takes place +when the child process is supposed to write +from the parent instead of reading. +You may find it a useful exercise to step through that case. +.PP +The job is not quite done, +for we still need a function +.UL pclose +to close the pipe created by +.UL popen . +The main reason for using a separate function rather than +.UL close +is that it is desirable to wait for the termination of the child process. +First, the return value from +.UL pclose +indicates whether the process succeeded. +Equally important when a process creates several children +is that only a bounded number of unwaited-for children +can exist, even if some of them have terminated; +performing the +.UL wait +lays the child to rest. +Thus: +.P1 +#include <signal.h> + +pclose(fd) /* close pipe fd */ +int fd; +{ + register r, (*hstat)(), (*istat)(), (*qstat)(); + int status; + extern int popen_pid; + + close(fd); + istat = signal(SIGINT, SIG_IGN); + qstat = signal(SIGQUIT, SIG_IGN); + hstat = signal(SIGHUP, SIG_IGN); + while ((r = wait(&status)) != popen_pid && r != -1); + if (r == -1) + status = -1; + signal(SIGINT, istat); + signal(SIGQUIT, qstat); + signal(SIGHUP, hstat); + return(status); +} +.P2 +The calls to +.UL signal +make sure that no interrupts, etc., +interfere with the waiting process; +this is the topic of the next section. +.PP +The routine as written has the limitation that only one pipe may +be open at once, because of the single shared variable +.UL popen_pid ; +it really should be an array indexed by file descriptor. +A +.UL popen +function, with slightly different arguments and return value is available +as part of the standard I/O library discussed below. +As currently written, it shares the same limitation. diff --git a/share/doc/psd/04.uprog/p6 b/share/doc/psd/04.uprog/p6 new file mode 100644 index 000000000000..c323d94b64e4 --- /dev/null +++ b/share/doc/psd/04.uprog/p6 @@ -0,0 +1,361 @@ +.\" Copyright (C) Caldera International Inc. 2001-2002. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions are +.\" met: +.\" +.\" Redistributions of source code and documentation must retain the above +.\" copyright notice, this list of conditions and the following +.\" disclaimer. +.\" +.\" Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" +.\" All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" +.\" This product includes software developed or owned by Caldera +.\" International, Inc. Neither the name of Caldera International, Inc. +.\" nor the names of other contributors may be used to endorse or promote +.\" products derived from this software without specific prior written +.\" permission. +.\" +.\" USE OF THE SOFTWARE PROVIDED FOR UNDER THIS LICENSE BY CALDERA +.\" INTERNATIONAL, INC. AND CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR +.\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +.\" WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +.\" DISCLAIMED. IN NO EVENT SHALL CALDERA INTERNATIONAL, INC. BE LIABLE +.\" FOR ANY DIRECT, INDIRECT INCIDENTAL, SPECIAL, EXEMPLARY, OR +.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR +.\" BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, +.\" WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE +.\" OR OTHERWISE) RISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN +.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +.\" +.\" $FreeBSD$ +.\" +.\" @(#)p6 8.1 (Berkeley) 6/8/93 +.\" +.NH +SIGNALS \(em INTERRUPTS AND ALL THAT +.PP +This section is concerned with how to +deal gracefully with signals from +the outside world (like interrupts), and with program faults. +Since there's nothing very useful that +can be done from within C about program +faults, which arise mainly from illegal memory references +or from execution of peculiar instructions, +we'll discuss only the outside-world signals: +.IT interrupt , +which is sent when the +.UC DEL +character is typed; +.IT quit , +generated by the +.UC FS +character; +.IT hangup , +caused by hanging up the phone; +and +.IT terminate , +generated by the +.IT kill +command. +When one of these events occurs, +the signal is sent to +.IT all +processes which were started +from the corresponding terminal; +unless other arrangements have been made, +the signal +terminates the process. +In the +.IT quit +case, a core image file is written for debugging +purposes. +.PP +The routine which alters the default action +is +called +.UL signal . +It has two arguments: the first specifies the signal, and the second +specifies how to treat it. +The first argument is just a number code, but the second is the +address is either a function, or a somewhat strange code +that requests that the signal either be ignored, or that it be +given the default action. +The include file +.UL signal.h +gives names for the various arguments, and should always be included +when signals are used. +Thus +.P1 +#include <signal.h> + ... +signal(SIGINT, SIG_IGN); +.P2 +causes interrupts to be ignored, while +.P1 +signal(SIGINT, SIG_DFL); +.P2 +restores the default action of process termination. +In all cases, +.UL signal +returns the previous value of the signal. +The second argument to +.UL signal +may instead be the name of a function +(which has to be declared explicitly if +the compiler hasn't seen it already). +In this case, the named routine will be called +when the signal occurs. +Most commonly this facility is used +to allow the program to clean up +unfinished business before terminating, for example to +delete a temporary file: +.P1 +#include <signal.h> + +main() +{ + int onintr(); + + if (signal(SIGINT, SIG_IGN) != SIG_IGN) + signal(SIGINT, onintr); + + /* Process ... */ + + exit(0); +} + +onintr() +{ + unlink(tempfile); + exit(1); +} +.P2 +.PP +Why the test and the double call to +.UL signal ? +Recall that signals like interrupt are sent to +.ul +all +processes started from a particular terminal. +Accordingly, when a program is to be run +non-interactively +(started by +.UL & ), +the shell turns off interrupts for it +so it won't be stopped by interrupts intended for foreground processes. +If this program began by announcing that all interrupts were to be sent +to the +.UL onintr +routine regardless, +that would undo the shell's effort to protect it +when run in the background. +.PP +The solution, shown above, is to test the state of interrupt handling, +and to continue to ignore interrupts if they are already being ignored. +The code as written +depends on the fact that +.UL signal +returns the previous state of a particular signal. +If signals were already being ignored, the process should continue to ignore them; +otherwise, they should be caught. +.PP +A more sophisticated program may wish to intercept +an interrupt and interpret it as a request +to stop what it is doing +and return to its own command-processing loop. +Think of a text editor: +interrupting a long printout should not cause it +to terminate and lose the work +already done. +The outline of the code for this case is probably best written like this: +.P1 +#include <signal.h> +#include <setjmp.h> +jmp_buf sjbuf; + +main() +{ + int (*istat)(), onintr(); + + istat = signal(SIGINT, SIG_IGN); /* save original status */ + setjmp(sjbuf); /* save current stack position */ + if (istat != SIG_IGN) + signal(SIGINT, onintr); + + /* main processing loop */ +} +.P2 +.P1 +onintr() +{ + printf("\enInterrupt\en"); + longjmp(sjbuf); /* return to saved state */ +} +.P2 +The include file +.UL setjmp.h +declares the type +.UL jmp_buf +an object in which the state +can be saved. +.UL sjbuf +is such an object; it is an array of some sort. +The +.UL setjmp +routine then saves +the state of things. +When an interrupt occurs, +a call is forced to the +.UL onintr +routine, +which can print a message, set flags, or whatever. +.UL longjmp +takes as argument an object stored into by +.UL setjmp , +and restores control +to the location after the call to +.UL setjmp , +so control (and the stack level) will pop back +to the place in the main routine where +the signal is set up and the main loop entered. +Notice, by the way, that +the signal +gets set again after an interrupt occurs. +This is necessary; most signals are automatically +reset to their default action when they occur. +.PP +Some programs that want to detect signals simply can't be stopped +at an arbitrary point, +for example in the middle of updating a linked list. +If the routine called on occurrence of a signal +sets a flag and then +returns instead of calling +.UL exit +or +.UL longjmp , +execution will continue +at the exact point it was interrupted. +The interrupt flag can then be tested later. +.PP +There is one difficulty associated with this +approach. +Suppose the program is reading the +terminal when the interrupt is sent. +The specified routine is duly called; it sets its flag +and returns. +If it were really true, as we said +above, that ``execution resumes at the exact point it was interrupted,'' +the program would continue reading the terminal +until the user typed another line. +This behavior might well be confusing, since the user +might not know that the program is reading; +he presumably would prefer to have the signal take effect instantly. +The method chosen to resolve this difficulty +is to terminate the terminal read when execution +resumes after the signal, returning an error code +which indicates what happened. +.PP +Thus programs which catch and resume +execution after signals should be prepared for ``errors'' +which are caused by interrupted +system calls. +(The ones to watch out for are reads from a terminal, +.UL wait , +and +.UL pause .) +A program +whose +.UL onintr +program just sets +.UL intflag , +resets the interrupt signal, and returns, +should usually include code like the following when it reads +the standard input: +.P1 +if (getchar() == EOF) + if (intflag) + /* EOF caused by interrupt */ + else + /* true end-of-file */ +.P2 +.PP +A final subtlety to keep in mind becomes important +when signal-catching is combined with execution of other programs. +Suppose a program catches interrupts, and also includes +a method (like ``!'' in the editor) +whereby other programs can be executed. +Then the code should look something like this: +.P1 +if (fork() == 0) + execl(...); +signal(SIGINT, SIG_IGN); /* ignore interrupts */ +wait(&status); /* until the child is done */ +signal(SIGINT, onintr); /* restore interrupts */ +.P2 +Why is this? +Again, it's not obvious but not really difficult. +Suppose the program you call catches its own interrupts. +If you interrupt the subprogram, +it will get the signal and return to its +main loop, and probably read your terminal. +But the calling program will also pop out of +its wait for the subprogram and read your terminal. +Having two processes reading +your terminal is very unfortunate, +since the system figuratively flips a coin to decide +who should get each line of input. +A simple way out is to have the parent program +ignore interrupts until the child is done. +This reasoning is reflected in the standard I/O library function +.UL system : +.P1 +#include <signal.h> + +system(s) /* run command string s */ +char *s; +{ + int status, pid, w; + register int (*istat)(), (*qstat)(); + + if ((pid = fork()) == 0) { + execl("/bin/sh", "sh", "-c", s, 0); + _exit(127); + } + istat = signal(SIGINT, SIG_IGN); + qstat = signal(SIGQUIT, SIG_IGN); + while ((w = wait(&status)) != pid && w != -1) + ; + if (w == -1) + status = -1; + signal(SIGINT, istat); + signal(SIGQUIT, qstat); + return(status); +} +.P2 +.PP +As an aside on declarations, +the function +.UL signal +obviously has a rather strange second argument. +It is in fact a pointer to a function delivering an integer, +and this is also the type of the signal routine itself. +The two values +.UL SIG_IGN +and +.UL SIG_DFL +have the right type, but are chosen so they coincide with +no possible actual functions. +For the enthusiast, here is how they are defined for the PDP-11; +the definitions should be sufficiently ugly +and nonportable to encourage use of the include file. +.P1 +#define SIG_DFL (int (*)())0 +#define SIG_IGN (int (*)())1 +.P2 diff --git a/share/doc/psd/04.uprog/p8 b/share/doc/psd/04.uprog/p8 new file mode 100644 index 000000000000..3c7c7b4c2084 --- /dev/null +++ b/share/doc/psd/04.uprog/p8 @@ -0,0 +1,62 @@ +.\" Copyright (C) Caldera International Inc. 2001-2002. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions are +.\" met: +.\" +.\" Redistributions of source code and documentation must retain the above +.\" copyright notice, this list of conditions and the following +.\" disclaimer. +.\" +.\" Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" +.\" All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" +.\" This product includes software developed or owned by Caldera +.\" International, Inc. Neither the name of Caldera International, Inc. +.\" nor the names of other contributors may be used to endorse or promote +.\" products derived from this software without specific prior written +.\" permission. +.\" +.\" USE OF THE SOFTWARE PROVIDED FOR UNDER THIS LICENSE BY CALDERA +.\" INTERNATIONAL, INC. AND CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR +.\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +.\" WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +.\" DISCLAIMED. IN NO EVENT SHALL CALDERA INTERNATIONAL, INC. BE LIABLE +.\" FOR ANY DIRECT, INDIRECT INCIDENTAL, SPECIAL, EXEMPLARY, OR +.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR +.\" BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, +.\" WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE +.\" OR OTHERWISE) RISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN +.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +.\" +.\" $FreeBSD$ +.\" +.\" @(#)p8 8.1 (Berkeley) 6/8/93 +.\" +.SH +References +.LP +.IP [1] +K. L. Thompson and D. M. Ritchie, +.ul +The +.ul +.UC UNIX +.ul +Programmer's Manual, +Bell Laboratories, 1978. +.IP [2] +B. W. Kernighan and D. M. Ritchie, +.ul +The C Programming Language, +Prentice-Hall, Inc., 1978. +.IP [3] +B. W. Kernighan, +.UC UNIX \& `` +for Beginners \(em Second Edition.'' +Bell Laboratories, 1978. diff --git a/share/doc/psd/04.uprog/p9 b/share/doc/psd/04.uprog/p9 new file mode 100644 index 000000000000..2366444ea52e --- /dev/null +++ b/share/doc/psd/04.uprog/p9 @@ -0,0 +1,680 @@ +.\" Copyright (C) Caldera International Inc. 2001-2002. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions are +.\" met: +.\" +.\" Redistributions of source code and documentation must retain the above +.\" copyright notice, this list of conditions and the following +.\" disclaimer. +.\" +.\" Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" +.\" All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" +.\" This product includes software developed or owned by Caldera +.\" International, Inc. Neither the name of Caldera International, Inc. +.\" nor the names of other contributors may be used to endorse or promote +.\" products derived from this software without specific prior written +.\" permission. +.\" +.\" USE OF THE SOFTWARE PROVIDED FOR UNDER THIS LICENSE BY CALDERA +.\" INTERNATIONAL, INC. AND CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR +.\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +.\" WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +.\" DISCLAIMED. IN NO EVENT SHALL CALDERA INTERNATIONAL, INC. BE LIABLE +.\" FOR ANY DIRECT, INDIRECT INCIDENTAL, SPECIAL, EXEMPLARY, OR +.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR +.\" BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, +.\" WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE +.\" OR OTHERWISE) RISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN +.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +.\" +.\" $FreeBSD$ +.\" +.\" @(#)p9 8.1 (Berkeley) 6/8/93 +.\" +.sp 100 +.TL +.ft R +Appendix \(em The Standard I/O Library +.AU +D. M. Ritchie +.AI +AT&T Bell Laboratories +Murray Hill, NJ 07974 +.PP +The standard I/O library +was designed with the following goals in mind. +.IP 1. +It must be as efficient as possible, both in time and in space, +so that there will be no hesitation in using it +no matter how critical the application. +.IP 2. +It must be simple to use, and also free of the magic +numbers and mysterious calls +whose use mars the understandability and portability +of many programs using older packages. +.IP 3. +The interface provided should be applicable on all machines, +whether or not the programs which implement it are directly portable +to other systems, +or to machines other than the PDP-11 running a version of +.UC UNIX . +.SH +1. General Usage +.PP +Each program using the library must have the line +.P1 + #include <stdio.h> +.P2 +which defines certain macros and variables. +The routines are in the normal C library, +so no special library argument is needed for loading. +All names in the include file intended only for internal use begin +with an underscore +.UL _ +to reduce the possibility +of collision with a user name. +The names intended to be visible outside the package are +.IP \f3stdin\f1 10 +The name of the standard input file +.IP \f3stdout\f1 10 +The name of the standard output file +.IP \f3stderr\f1 10 +The name of the standard error file +.IP \f3EOF\f1 10 +is actually \-1, and is the value returned by +the read routines on end-of-file or error. +.IP \f3NULL\f1 10 +is a notation for the null pointer, returned by +pointer-valued functions +to indicate an error +.IP \f3FILE\f1 10 +expands to +.UL struct +.UL _iob +and is a useful +shorthand when declaring pointers +to streams. +.IP \f3BUFSIZ\f1 10 +is a number (viz. 512) +of the size suitable for an I/O buffer supplied by the user. +See +.UL setbuf , +below. +.IP \f3getc,\ getchar,\ putc,\ putchar,\ feof,\ ferror,\ f\&ileno\f1 10 +.br +are defined as macros. +Their actions are described below; +they are mentioned here +to point out that it is not possible to +redeclare them +and that they are not actually functions; +thus, for example, they may not have breakpoints set on them. +.PP +The routines in this package +offer the convenience of automatic buffer allocation +and output flushing where appropriate. +The names +.UL stdin , +.UL stdout , +and +.UL stderr +are in effect constants and may not be assigned to. +.SH +2. Calls +.nr PD .4v +.LP +.UL FILE\ *fopen(filename,\ type)\ char\ *filename,\ *type; +.nr PD 0 +.IP +.br +opens the file and, if needed, allocates a buffer for it. +.UL filename +is a character string specifying the name. +.UL type +is a character string (not a single character). +It may be +.UL \&"r" , +.UL \&"w" , +or +.UL \&"a" +to indicate +intent to read, write, or append. +The value returned is a file pointer. +If it is +.UL NULL +the attempt to open failed. +.ne 3 +.nr PD .4v +.LP +.UL FILE\ *freopen(filename,\ type,\ ioptr)\ char\ *filename,\ *type;\ FILE\ *ioptr; +.nr PD 0 +.IP +.br +The stream named by +.UL ioptr +is closed, if necessary, and then reopened +as if by +.UL fopen . +If the attempt to open fails, +.UL NULL +is returned, +otherwise +.UL ioptr , +which will now refer to the new file. +Often the reopened stream is +.UL stdin +or +.UL stdout . +.nr PD .4v +.LP +.UL int\ getc(ioptr)\ FILE\ *ioptr; +.nr PD 0 +.IP +returns the next character from the stream named by +.UL ioptr , +which is a pointer to a file such as returned by +.UL fopen , +or the name +.UL stdin . +The integer +.UL EOF +is returned on end-of-file or when +an error occurs. +The null character +.UL \e0 +is a legal character. +.nr PD .4v +.LP +.UL int\ fgetc(ioptr)\ FILE\ *ioptr; +.nr PD 0 +.IP +.br +acts like +.UL getc +but is a genuine function, +not a macro, +so it can be pointed to, passed as an argument, etc. +.nr PD .4v +.LP +.UL putc(c,\ ioptr)\ FILE\ *ioptr; +.nr PD 0 +.IP +.br +.UL putc +writes the character +.UL c +on the output stream named by +.UL ioptr , +which is a value returned from +.UL fopen +or perhaps +.UL stdout +or +.UL stderr . +The character is returned as value, +but +.UL EOF +is returned on error. +.nr PD .4v +.LP +.UL fputc(c,\ ioptr)\ FILE\ *ioptr; +.nr PD 0 +.IP +.br +acts like +.UL putc +but is a genuine +function, not a macro. +.nr PD .4v +.LP +.UL fclose(ioptr)\ FILE\ *ioptr; +.nr PD 0 +.IP +.br +The file corresponding to +.UL ioptr +is closed after any buffers are emptied. +A buffer allocated by the I/O system is freed. +.UL fclose +is automatic on normal termination of the program. +.nr PD .4v +.LP +.UL fflush(ioptr)\ FILE\ *ioptr; +.nr PD 0 +.IP +.br +Any buffered information on the (output) stream named by +.UL ioptr +is written out. +Output files are normally buffered +if and only if they are not directed to the terminal; +however, +.UL stderr +always starts off unbuffered and remains so unless +.UL setbuf +is used, or unless it is reopened. +.nr PD .4v +.LP +.UL exit(errcode); +.nr PD 0 +.IP +.br +terminates the process and returns its argument as status +to the parent. +This is a special version of the routine +which calls +.UL fflush +for each output file. +To terminate without flushing, +use +.UL _exit . +.nr PD .4v +.LP +.UL feof(ioptr)\ FILE\ *ioptr; +.nr PD 0 +.IP +.br +returns non-zero when end-of-file +has occurred on the specified input stream. +.nr PD .4v +.LP +.UL ferror(ioptr)\ FILE\ *ioptr; +.nr PD 0 +.IP +.br +returns non-zero when an error has occurred while reading +or writing the named stream. +The error indication lasts until the file has been closed. +.nr PD .4v +.LP +.UL getchar(); +.nr PD 0 +.IP +.br +is identical to +.UL getc(stdin) . +.nr PD .4v +.LP +.UL putchar(c); +.nr PD 0 +.IP +.br +is identical to +.UL putc(c,\ stdout) . +.nr PD .4v +.nr PD .4v +.ne 2 +.LP +.UL char\ *fgets(s,\ n,\ ioptr)\ char\ *s;\ FILE\ *ioptr; +.nr PD 0 +.IP +.br +reads up to +.UL n-1 +characters from the stream +.UL ioptr +into the character pointer +.UL s . +The read terminates with a newline character. +The newline character is placed in the buffer +followed by a null character. +.UL fgets +returns the first argument, +or +.UL NULL +if error or end-of-file occurred. +.nr PD .4v +.nr PD .4v +.LP +.UL fputs(s,\ ioptr)\ char\ *s;\ FILE\ *ioptr; +.nr PD 0 +.IP +.br +writes the null-terminated string (character array) +.UL s +on the stream +.UL ioptr . +No newline is appended. +No value is returned. +.nr PD .4v +.LP +.UL ungetc(c,\ ioptr)\ FILE\ *ioptr; +.nr PD 0 +.IP +.br +The argument character +.UL c +is pushed back on the input stream named by +.UL ioptr . +Only one character may be pushed back. +.ne 5 +.nr PD .4v +.LP +.UL printf(format,\ a1,\ ...)\ char\ *format; +.br +.UL fprintf(ioptr,\ format,\ a1,\ ...)\ FILE\ *ioptr;\ char\ *format; +.br +.UL sprintf(s,\ format,\ a1,\ ...)char\ *s,\ *format; +.br +.nr PD 0 +.IP +.UL printf +writes on the standard output. +.UL fprintf +writes on the named output stream. +.UL sprintf +puts characters in the character array (string) +named by +.UL s . +The specifications are as described in section +.UL printf (3) +of the +.ul +.UC UNIX +.ul +Programmer's Manual. +.nr PD .4v +.LP +.UL scanf(format,\ a1,\ ...)\ char\ *format; +.br +.UL fscanf(ioptr,\ format,\ a1,\ ...)\ FILE\ *ioptr;\ char\ *format; +.br +.UL sscanf(s,\ format,\ a1,\ ...)\ char\ *s,\ *format; +.nr PD 0 +.IP +.br +.UL scanf +reads from the standard input. +.UL fscanf +reads from the named input stream. +.UL sscanf +reads from the character string +supplied as +.UL s . +.UL scanf +reads characters, interprets +them according to a format, and stores the results in its arguments. +Each routine expects as arguments +a control string +.UL format , +and a set of arguments, +.I +each of which must be a pointer, +.R +indicating where the converted input should be stored. +.if t .sp .4v +.UL scanf +returns as its value the number of successfully matched and assigned input +items. +This can be used to decide how many input items were found. +On end of file, +.UL EOF +is returned; note that this is different +from 0, which means that the next input character does not +match what was called for in the control string. +.nr PD .4v +.LP +.UL fread(ptr,\ sizeof(*ptr),\ nitems,\ ioptr)\ FILE\ *ioptr; +.nr PD 0 +.IP +.br +reads +.UL nitems +of data beginning at +.UL ptr +from file +.UL ioptr . +No advance notification +that binary I/O is being done is required; +when, for portability reasons, +it becomes required, it will be done +by adding an additional character to the mode-string on the +.UL fopen +call. +.nr PD .4v +.LP +.UL fwrite(ptr,\ sizeof(*ptr),\ nitems,\ ioptr)\ FILE\ *ioptr; +.nr PD 0 +.IP +.br +Like +.UL fread , +but in the other direction. +.nr PD .4v +.LP +.UL rewind(ioptr)\ FILE\ *ioptr; +.nr PD 0 +.IP +.br +rewinds the stream +named by +.UL ioptr . +It is not very useful except on input, +since a rewound output file is still open only for output. +.nr PD .4v +.LP +.UL system(string)\ char\ *string; +.nr PD 0 +.IP +.br +The +.UL string +is executed by the shell as if typed at the terminal. +.nr PD .4v +.LP +.UL getw(ioptr)\ FILE\ *ioptr; +.nr PD 0 +.IP +.br +returns the next word from the input stream named by +.UL ioptr . +.UL EOF +is returned on end-of-file or error, +but since this a perfectly good +integer +.UL feof +and +.UL ferror +should be used. +A ``word'' is 16 bits on the +.UC PDP-11. +.nr PD .4v +.LP +.UL putw(w,\ ioptr)\ FILE\ *ioptr; +.nr PD 0 +.IP +.br +writes the integer +.UL w +on the named output stream. +.nr PD .4v +.LP +.UL setbuf(ioptr,\ buf)\ FILE\ *ioptr;\ char\ *buf; +.nr PD 0 +.IP +.br +.UL setbuf +may be used after a stream has been opened +but before I/O has started. +If +.UL buf +is +.UL NULL , +the stream will be unbuffered. +Otherwise the buffer supplied will be used. +It must be a character array of sufficient size: +.P1 +char buf[BUFSIZ]; +.P2 +.nr PD .4v +.LP +.UL fileno(ioptr)\ FILE\ *ioptr; +.nr PD 0 +.IP +.br +returns the integer file descriptor associated with the file. +.nr PD .4v +.LP +.UL fseek(ioptr,\ offset,\ ptrname)\ FILE\ *ioptr;\ long\ offset; +.nr PD 0 +.IP +.br +The location of the next byte in the stream +named by +.UL ioptr +is adjusted. +.UL offset +is a long integer. +If +.UL ptrname +is 0, the offset is measured from the beginning of the file; +if +.UL ptrname +is 1, the offset is measured from the current read or +write pointer; +if +.UL ptrname +is 2, the offset is measured from the end of the file. +The routine accounts properly for any buffering. +(When this routine is used on +.UC UNIX \& non- +systems, +the offset must be a value returned from +.UL ftell +and the ptrname must be 0). +.ne 3 +.nr PD .4v +.LP +.UL long\ ftell(ioptr)\ FILE\ *ioptr; +.nr PD 0 +.IP +.br +The byte offset, measured from the beginning of the file, +associated with the named stream is returned. +Any buffering is properly accounted for. +(On +.UC UNIX \& non- +systems the value of this call is useful only +for handing to +.UL fseek , +so as to position the file to the same place it was when +.UL ftell +was called.) +.nr PD .4v +.LP +.UL getpw(uid,\ buf)\ char\ *buf; +.nr PD 0 +.IP +.br +The password file is searched for the given integer user ID. +If an appropriate line is found, it is copied into +the character array +.UL buf , +and 0 is returned. +If no line is found corresponding to the user ID +then 1 is returned. +.nr PD .4v +.LP +.UL char\ *malloc(num); +.nr PD 0 +.IP +.br +allocates +.UL num +bytes. +The pointer returned is sufficiently well aligned to be usable for any purpose. +.UL NULL +is returned if no space is available. +.nr PD .4v +.LP +.UL char\ *calloc(num,\ size); +.nr PD 0 +.IP +.br +allocates space for +.UL num +items each of size +.UL size . +The space is guaranteed to be set to 0 and the pointer is +sufficiently well aligned to be usable for any purpose. +.UL NULL +is returned if no space is available . +.nr PD .4v +.LP +.UL cfree(ptr)\ char\ *ptr; +.nr PD 0 +.IP +.br +Space is returned to the pool used by +.UL calloc . +Disorder can be expected if the pointer was not obtained +from +.UL calloc . +.nr PD .4v +.LP +The following are macros whose definitions may be obtained by including +.UL <ctype.h> . +.nr PD .4v +.LP +.UL isalpha(c) +returns non-zero if the argument is alphabetic. +.nr PD .4v +.LP +.UL isupper(c) +returns non-zero if the argument is upper-case alphabetic. +.nr PD .4v +.LP +.UL islower(c) +returns non-zero if the argument is lower-case alphabetic. +.nr PD .4v +.LP +.UL isdigit(c) +returns non-zero if the argument is a digit. +.nr PD .4v +.LP +.UL isspace(c) +returns non-zero if the argument is a spacing character: +tab, newline, carriage return, vertical tab, +form feed, space. +.nr PD .4v +.LP +.UL ispunct(c) +returns non-zero if the argument is +any punctuation character, i.e., not a space, letter, +digit or control character. +.nr PD .4v +.LP +.UL isalnum(c) +returns non-zero if the argument is a letter or a digit. +.nr PD .4v +.LP +.UL isprint(c) +returns non-zero if the argument is printable \(em +a letter, digit, or punctuation character. +.nr PD .4v +.LP +.UL iscntrl(c) +returns non-zero if the argument is a control character. +.nr PD .4v +.LP +.UL isascii(c) +returns non-zero if the argument is an ascii character, i.e., less than octal 0200. +.nr PD .4v +.LP +.UL toupper(c) +returns the upper-case character corresponding to the lower-case +letter +.UL c. +.nr PD .4v +.LP +.UL tolower(c) +returns the lower-case character corresponding to the upper-case +letter +.UL c . diff --git a/share/doc/psd/05.sysman/0.t b/share/doc/psd/05.sysman/0.t new file mode 100644 index 000000000000..865e8ff521ee --- /dev/null +++ b/share/doc/psd/05.sysman/0.t @@ -0,0 +1,292 @@ +.\" Copyright (c) 1983, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)0.t 8.1 (Berkeley) 6/8/93 +.\" +.if n .ND +.TL +Berkeley Software Architecture Manual +.br +4.4BSD Edition +.AU +William Joy, Robert Fabry, +.AU +Samuel Leffler, M. Kirk McKusick, +.AU +Michael Karels +.AI +Computer Systems Research Group +Computer Science Division +Department of Electrical Engineering and Computer Science +University of California, Berkeley +Berkeley, CA 94720 +.EH 'PSD:5-%''4.4BSD Architecture Manual' +.OH '4.4BSD Architecture Manual''PSD:5-%' +.AB +.FS +* UNIX is a trademark of Bell Laboratories. +.FE +This document summarizes the facilities +provided by the 4.4BSD version of the UNIX\|* operating system. +It does not attempt to act as a tutorial for use of the system +nor does it attempt to explain or justify the design of the +system facilities. +It gives neither motivation nor implementation details, +in favor of brevity. +.PP +The first section describes the basic kernel functions +provided to a UNIX process: process naming and protection, +memory management, software interrupts, +object references (descriptors), time and statistics functions, +and resource controls. +These facilities, as well as facilities for +bootstrap, shutdown and process accounting, +are provided solely by the kernel. +.PP +The second section describes the standard system +abstractions for +files and file systems, +communication, +terminal handling, +and process control and debugging. +These facilities are implemented by the operating system or by +network server processes. +.AE +.LP +.bp +.ft B +.br +.sv 2 +.ce +TABLE OF CONTENTS +.ft R +.LP +.sp 1 +.nf +.B "Introduction." +.LP +.if t .sp .5v +.nf +.B "0. Notation and types" +.LP +.if t .sp .5v +.nf +.B "1. Kernel primitives" +.LP +.if t .sp .5v +.nf +.nf +\fB1.1. Processes and protection\fP +1.1.1. Host and process identifiers +1.1.2. Process creation and termination +1.1.3. User and group ids +1.1.4. Process groups +.LP +.nf +\fB1.2. Memory management\fP +1.2.1. Text, data and stack +1.2.2. Mapping pages +1.2.3. Page protection control +1.2.4. Giving and getting advice +1.2.5. Protection primitives +.LP +.if t .sp .5v +.nf +\fB1.3. Signals\fP +1.3.1. Overview +1.3.2. Signal types +1.3.3. Signal handlers +1.3.4. Sending signals +1.3.5. Protecting critical sections +1.3.6. Signal stacks +.LP +.if t .sp .5v +.nf +\fB1.4. Timing and statistics\fP +1.4.1. Real time +1.4.2. Interval time +.LP +.if t .sp .5v +.nf +\fB1.5. Descriptors\fP +1.5.1. The reference table +1.5.2. Descriptor properties +1.5.3. Managing descriptor references +1.5.4. Multiplexing requests +1.5.5. Descriptor wrapping +.LP +.if t .sp .5v +.nf +\fB1.6. Resource controls\fP +1.6.1. Process priorities +1.6.2. Resource utilization +1.6.3. Resource limits +.LP +.if t .sp .5v +.nf +\fB1.7. System operation support\fP +1.7.1. Bootstrap operations +1.7.2. Shutdown operations +1.7.3. Accounting +.bp +.LP +.if t .sp .5v +.sp 1 +.nf +\fB2. System facilities\fP +.LP +.if t .sp .5v +.nf +\fB2.1. Generic operations\fP +2.1.1. Read and write +2.1.2. Input/output control +2.1.3. Non-blocking and asynchronous operations +.LP +.if t .sp .5v +.nf +\fB2.2. File system\fP +2.2.1 Overview +2.2.2. Naming +2.2.3. Creation and removal +2.2.3.1. Directory creation and removal +2.2.3.2. File creation +2.2.3.3. Creating references to devices +2.2.3.4. Portal creation +2.2.3.6. File, device, and portal removal +2.2.4. Reading and modifying file attributes +2.2.5. Links and renaming +2.2.6. Extension and truncation +2.2.7. Checking accessibility +2.2.8. Locking +2.2.9. Disc quotas +.LP +.if t .sp .5v +.nf +\fB2.3. Interprocess communication\fP +2.3.1. Interprocess communication primitives +2.3.1.1.\0 Communication domains +2.3.1.2.\0 Socket types and protocols +2.3.1.3.\0 Socket creation, naming and service establishment +2.3.1.4.\0 Accepting connections +2.3.1.5.\0 Making connections +2.3.1.6.\0 Sending and receiving data +2.3.1.7.\0 Scatter/gather and exchanging access rights +2.3.1.8.\0 Using read and write with sockets +2.3.1.9.\0 Shutting down halves of full-duplex connections +2.3.1.10.\0 Socket and protocol options +2.3.2. UNIX domain +2.3.2.1. Types of sockets +2.3.2.2. Naming +2.3.2.3. Access rights transmission +2.3.3. INTERNET domain +2.3.3.1. Socket types and protocols +2.3.3.2. Socket naming +2.3.3.3. Access rights transmission +2.3.3.4. Raw access +.LP +.if t .sp .5v +.nf +\fB2.4. Terminals and devices\fP +2.4.1. Terminals +2.4.1.1. Terminal input +2.4.1.1.1 Input modes +2.4.1.1.2 Interrupt characters +2.4.1.1.3 Line editing +2.4.1.2. Terminal output +2.4.1.3. Terminal control operations +2.4.1.4. Terminal hardware support +2.4.2. Structured devices +2.4.3. Unstructured devices +.LP +.if t .sp .5v +.nf +\fB2.5. Process control and debugging\fP +.LP +.if t .sp .5v +.nf +\fBI. Summary of facilities\fP +.LP +.de sh +.ds RH \\$1 +.bp +.NH \\*(ss +\s+2\\$1\s0 +.PP +.PP +.. +.bp +.ds ss 1 +.de _d +.if t .ta .6i 2.1i 2.6i +.\" 2.94 went to 2.6, 3.64 to 3.30 +.if n .ta .84i 2.6i 3.30i +.. +.de _f +.if t .ta .5i 1.25i 2.5i 3.5i +.\" 3.5i went to 3.8i +.if n .ta .7i 1.75i 3.8i 4.8i +.. +.nr H1 -1 +.sh "Notation and types +.PP +The notation used to describe system calls is a variant of a +C language call, consisting of a prototype call followed by +declaration of parameters and results. +An additional keyword \fBresult\fP, not part of the normal C language, +is used to indicate which of the declared entities receive results. +As an example, consider the \fIread\fP call, as described in +section 2.1: +.DS +cc = read(fd, buf, nbytes); +result int cc; int fd; result char *buf; int nbytes; +.DE +The first line shows how the \fIread\fP routine is called, with +three parameters. +As shown on the second line \fIcc\fP is an integer and \fIread\fP also +returns information in the parameter \fIbuf\fP. +.PP +Description of all error conditions arising from each system call +is not provided here; they appear in the programmer's manual. +In particular, when accessed from the C language, +many calls return a characteristic \-1 value +when an error occurs, returning the error code in the global variable +\fIerrno\fP. +Other languages may present errors in different ways. +.PP +A number of system standard types are defined in the include file +.I <sys/types.h> +and used in the specifications here and in many C programs. +These include \fBcaddr_t\fP giving a memory address (typically as +a character pointer), +\fBoff_t\fP giving a file offset (typically as a long integer), +and a set of unsigned types \fBu_char\fP, \fBu_short\fP, \fBu_int\fP +and \fBu_long\fP, shorthand names for \fBunsigned char\fP, \fBunsigned +short\fP, etc. diff --git a/share/doc/psd/05.sysman/1.0.t b/share/doc/psd/05.sysman/1.0.t new file mode 100644 index 000000000000..5a465a7281cf --- /dev/null +++ b/share/doc/psd/05.sysman/1.0.t @@ -0,0 +1,56 @@ +.\" Copyright (c) 1983, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)1.0.t 8.1 (Berkeley) 6/8/93 +.\" +.ds ss 1 +.sh "Kernel primitives +.PP +The facilities available to a UNIX user process are logically +divided into two parts: kernel facilities directly implemented by +UNIX code running in the operating system, and system facilities +implemented either by the system, or in cooperation with a +\fIserver process\fP. These kernel facilities are described in +this section 1. +.PP +The facilities implemented in the kernel are those which define the +\fIUNIX virtual machine\fP in which each process runs. +Like many real machines, this virtual machine has memory management hardware, +an interrupt facility, timers and counters. The UNIX +virtual machine also allows access to files and other objects through a set of +\fIdescriptors\fP. Each descriptor resembles a device controller, +and supports a set of operations. Like devices on real machines, some +of which are internal to the machine and some of which are external, +parts of the descriptor machinery are built-in to the operating system, while +other parts are often implemented in server processes on other machines. +The facilities provided through the descriptor machinery are described in +section 2. +.ds ss 2 diff --git a/share/doc/psd/05.sysman/1.1.t b/share/doc/psd/05.sysman/1.1.t new file mode 100644 index 000000000000..223cf831e556 --- /dev/null +++ b/share/doc/psd/05.sysman/1.1.t @@ -0,0 +1,216 @@ +.\" Copyright (c) 1983, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)1.1.t 8.1 (Berkeley) 6/8/93 +.\" $FreeBSD$ +.\" +.sh "Processes and protection +.NH 3 +Host and process identifiers +.PP +Each UNIX host has associated with it a 32-bit host id, and a host +name of up to 256 characters (as defined by MAXHOSTNAMELEN in +\fI<sys/param.h>\fP). +These are set (by a privileged user) +and returned by the calls: +.DS +sethostid(hostid) +long hostid; + +hostid = gethostid(); +result long hostid; + +sethostname(name, len) +char *name; int len; + +len = gethostname(buf, buflen) +result int len; result char *buf; int buflen; +.DE +On each host runs a set of \fIprocesses\fP. +Each process is largely independent of other processes, +having its own protection domain, address space, timers, and +an independent set of references to system or user implemented objects. +.PP +Each process in a host is named by an integer +called the \fIprocess id\fP. This number is +in the range 1-30000 +and is returned by +the \fIgetpid\fP routine: +.DS +pid = getpid(); +result int pid; +.DE +On each UNIX host this identifier is guaranteed to be unique; +in a multi-host environment, the (hostid, process id) pairs are +guaranteed unique. +.NH 3 +Process creation and termination +.PP +A new process is created by making a logical duplicate of an +existing process: +.DS +pid = fork(); +result int pid; +.DE +The \fIfork\fP call returns twice, once in the parent process, where +\fIpid\fP is the process identifier of the child, +and once in the child process where \fIpid\fP is 0. +The parent-child relationship induces a hierarchical structure on +the set of processes in the system. +.PP +A process may terminate by executing an \fIexit\fP call: +.DS +exit(status) +int status; +.DE +returning 8 bits of exit status to its parent. +.PP +When a child process exits or +terminates abnormally, the parent process receives +information about any +event which caused termination of the child process. A +second call provides a non-blocking interface and may also be used +to retrieve information about resources consumed by the process during its +lifetime. +.DS +#include <sys/wait.h> + +pid = wait(astatus); +result int pid; result union wait *astatus; + +pid = wait3(astatus, options, arusage); +result int pid; result union waitstatus *astatus; +int options; result struct rusage *arusage; +.DE +.PP +A process can overlay itself with the memory image of another process, +passing the newly created process a set of parameters, using the call: +.DS +execve(name, argv, envp) +char *name, **argv, **envp; +.DE +The specified \fIname\fP must be a file which is in a format recognized +by the system, either a binary executable file or a file which causes +the execution of a specified interpreter program to process its contents. +.NH 3 +User and group ids +.PP +Each process in the system has associated with it two user-id's: +a \fIreal user id\fP and a \fIeffective user id\fP, both 16 bit +unsigned integers (type \fBuid_t\fP). +Each process has an \fIreal accounting group id\fP and an \fIeffective +accounting group id\fP and a set of +\fIaccess group id's\fP. The group id's are 16 bit unsigned integers +(type \fBgid_t\fP). +Each process may be in several different access groups, with the maximum +concurrent number of access groups a system compilation parameter, +the constant NGROUPS in the file \fI<sys/param.h>\fP, +guaranteed to be at least 8. +.PP +The real and effective user ids associated with a process are returned by: +.DS +ruid = getuid(); +result uid_t ruid; + +euid = geteuid(); +result uid_t euid; +.DE +the real and effective accounting group ids by: +.DS +rgid = getgid(); +result gid_t rgid; + +egid = getegid(); +result gid_t egid; +.DE +The access group id set is returned by a \fIgetgroups\fP call*: +.DS +ngroups = getgroups(gidsetsize, gidset); +result int ngroups; int gidsetsize; result int gidset[gidsetsize]; +.DE +.FS +* The type of the gidset array in getgroups and setgroups +remains integer for compatibility with 4.2BSD. +It may change to \fBgid_t\fP in future releases. +.FE +.PP +The user and group id's +are assigned at login time using the \fIsetreuid\fP, \fIsetregid\fP, +and \fIsetgroups\fP calls: +.DS +setreuid(ruid, euid); +int ruid, euid; + +setregid(rgid, egid); +int rgid, egid; + +setgroups(gidsetsize, gidset) +int gidsetsize; int gidset[gidsetsize]; +.DE +The \fIsetreuid\fP call sets both the real and effective user-id's, +while the \fIsetregid\fP call sets both the real +and effective accounting group id's. +Unless the caller is the super-user, \fIruid\fP +must be equal to either the current real or effective user-id, +and \fIrgid\fP equal to either the current real or effective +accounting group id. The \fIsetgroups\fP call is restricted +to the super-user. +.NH 3 +Process groups +.PP +Each process in the system is also normally associated with a \fIprocess +group\fP. The group of processes in a process group is sometimes +referred to as a \fIjob\fP and manipulated by high-level system +software (such as the shell). +The current process group of a process is returned by the +\fIgetpgrp\fP call: +.DS +pgrp = getpgrp(pid); +result int pgrp; int pid; +.DE +When a process is in a specific process group it may receive +software interrupts affecting the group, causing the group to +suspend or resume execution or to be interrupted or terminated. +In particular, a system terminal has a process group and only processes +which are in the process group of the terminal may read from the +terminal, allowing arbitration of terminals among several different jobs. +.PP +The process group associated with a process may be changed by +the \fIsetpgrp\fP call: +.DS +setpgrp(pid, pgrp); +int pid, pgrp; +.DE +Newly created processes are assigned process id's distinct from all +processes and process groups, and the same process group as their +parent. A normal (unprivileged) process may set its process group equal +to its process id. A privileged process may set the process group of any +process to any value. diff --git a/share/doc/psd/05.sysman/1.2.t b/share/doc/psd/05.sysman/1.2.t new file mode 100644 index 000000000000..8527a75f8f3e --- /dev/null +++ b/share/doc/psd/05.sysman/1.2.t @@ -0,0 +1,273 @@ +.\" Copyright (c) 1983, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)1.2.t 8.1 (Berkeley) 6/8/93 +.\" $FreeBSD$ +.\" +.sh "Memory management\(dg +.NH 3 +Text, data and stack +.PP +.FS +\(dg This section represents the interface planned for later +releases of the system. Of the calls described in this section, +only \fIsbrk\fP and \fIgetpagesize\fP are included in 4.3BSD. +.FE +Each process begins execution with three logical areas of memory +called text, data and stack. +The text area is read-only and shared, while the data and stack +areas are private to the process. Both the data and stack areas may +be extended and contracted on program request. The call +.DS +addr = sbrk(incr); +result caddr_t addr; int incr; +.DE +changes the size of the data area by \fIincr\fP bytes and +returns the new end of the data area, while +.DS +addr = sstk(incr); +result caddr_t addr; int incr; +.DE +changes the size of the stack area. +The stack area is also automatically extended as needed. +On the VAX the text and data areas are adjacent in the P0 region, +while the stack section is in the P1 region, and grows downward. +.NH 3 +Mapping pages +.PP +The system supports sharing of data between processes +by allowing pages to be mapped into memory. These mapped +pages may be \fIshared\fP with other processes or \fIprivate\fP +to the process. +Protection and sharing options are defined in \fI<sys/mman.h>\fP as: +.DS +.ta \w'#define\ \ 'u +\w'MAP_HASSEMAPHORE\ \ 'u +\w'0x0080\ \ 'u +/* protections are chosen from these bits, or-ed together */ +#define PROT_READ 0x04 /* pages can be read */ +#define PROT_WRITE 0x02 /* pages can be written */ +#define PROT_EXEC 0x01 /* pages can be executed */ +.DE +.DS +.ta \w'#define\ \ 'u +\w'MAP_HASSEMAPHORE\ \ 'u +\w'0x0080\ \ 'u +/* flags contain mapping type, sharing type and options */ +/* mapping type; choose one */ +#define MAP_FILE 0x0001 /* mapped from a file or device */ +#define MAP_ANON 0x0002 /* allocated from memory, swap space */ +#define MAP_TYPE 0x000f /* mask for type field */ +.DE +.DS +.ta \w'#define\ \ 'u +\w'MAP_HASSEMAPHORE\ \ 'u +\w'0x0080\ \ 'u +/* sharing types; choose one */ +#define MAP_SHARED 0x0010 /* share changes */ +#define MAP_PRIVATE 0x0000 /* changes are private */ +.DE +.DS +.ta \w'#define\ \ 'u +\w'MAP_HASSEMAPHORE\ \ 'u +\w'0x0080\ \ 'u +/* other flags */ +#define MAP_FIXED 0x0020 /* map addr must be exactly as requested */ +#define MAP_INHERIT 0x0040 /* region is retained after exec */ +#define MAP_HASSEMAPHORE 0x0080 /* region may contain semaphores */ +#define MAP_NOPREALLOC 0x0100 /* do not preallocate space */ +.DE +The cpu-dependent size of a page is returned by the +\fIgetpagesize\fP system call: +.DS +pagesize = getpagesize(); +result int pagesize; +.DE +.LP +The call: +.DS +maddr = mmap(addr, len, prot, flags, fd, pos); +result caddr_t maddr; caddr_t addr; int *len, prot, flags, fd; off_t pos; +.DE +causes the pages starting at \fIaddr\fP and continuing +for at most \fIlen\fP bytes to be mapped from the object represented by +descriptor \fIfd\fP, starting at byte offset \fIpos\fP. +The starting address of the region is returned; +for the convenience of the system, +it may differ from that supplied +unless the MAP_FIXED flag is given, +in which case the exact address will be used or the call will fail. +The actual amount mapped is returned in \fIlen\fP. +The \fIaddr\fP, \fIlen\fP, and \fIpos\fP parameters +must all be multiples of the pagesize. +A successful \fImmap\fP will delete any previous mapping +in the allocated address range. +The parameter \fIprot\fP specifies the accessibility +of the mapped pages. +The parameter \fIflags\fP specifies +the type of object to be mapped, +mapping options, and +whether modifications made to +this mapped copy of the page +are to be kept \fIprivate\fP, or are to be \fIshared\fP with +other references. +Possible types include MAP_FILE, +mapping a regular file or character-special device memory, +and MAP_ANON, which maps memory not associated with any specific file. +The file descriptor used for creating MAP_ANON regions is used only +for naming, and may be given as \-1 if no name +is associated with the region.\(dd +.FS +\(dd The current design does not allow a process +to specify the location of swap space. +In the future we may define an additional mapping type, MAP_SWAP, +in which the file descriptor argument specifies a file +or device to which swapping should be done. +.FE +The MAP_INHERIT flag allows a region to be inherited after an \fIexec\fP. +The MAP_HASSEMAPHORE flag allows special handling for +regions that may contain semaphores. +The MAP_NOPREALLOC flag allows processes to allocate regions whose +virtual address space, if fully allocated, +would exceed the available memory plus swap resources. +Such regions may get a SIGSEGV signal if they page fault and resources +are not available to service their request; +typically they would free up some resources via \fIunmap\fP so that +when they return from the signal the page +fault could be successfully completed. +.PP +A facility is provided to synchronize a mapped region with the file +it maps; the call +.DS +msync(addr, len); +caddr_t addr; int len; +.DE +writes any modified pages back to the filesystem and updates +the file modification time. +If \fIlen\fP is 0, all modified pages within the region containing \fIaddr\fP +will be flushed; +if \fIlen\fP is non-zero, only the pages containing \fIaddr\fP and \fIlen\fP +succeeding locations will be examined. +Any required synchronization of memory caches +will also take place at this time. +Filesystem operations on a file that is mapped for shared modifications +are unpredictable except after an \fImsync\fP. +.PP +A mapping can be removed by the call +.DS +munmap(addr, len); +caddr_t addr; int len; +.DE +This call deletes the mappings for the specified address range, +and causes further references to addresses within the range +to generate invalid memory references. +.NH 3 +Page protection control +.PP +A process can control the protection of pages using the call +.DS +mprotect(addr, len, prot); +caddr_t addr; int len, prot; +.DE +This call changes the specified pages to have protection \fIprot\fP\|. +Not all implementations will guarantee protection on a page basis; +the granularity of protection changes may be as large as an entire region. +.NH 3 +Giving and getting advice +.PP +A process that has knowledge of its memory behavior may +use the \fImadvise\fP call: +.DS +madvise(addr, len, behav); +caddr_t addr; int len, behav; +.DE +\fIBehav\fP describes expected behavior, as given +in \fI<sys/mman.h>\fP: +.DS +.ta \w'#define\ \ 'u +\w'MADV_SEQUENTIAL\ \ 'u +\w'00\ \ \ \ 'u +#define MADV_NORMAL 0 /* no further special treatment */ +#define MADV_RANDOM 1 /* expect random page references */ +#define MADV_SEQUENTIAL 2 /* expect sequential references */ +#define MADV_WILLNEED 3 /* will need these pages */ +#define MADV_DONTNEED 4 /* don't need these pages */ +#define MADV_SPACEAVAIL 5 /* insure that resources are reserved */ +.DE +Finally, a process may obtain information about whether pages are +core resident by using the call +.DS +mincore(addr, len, vec) +caddr_t addr; int len; result char *vec; +.DE +Here the current core residency of the pages is returned +in the character array \fIvec\fP, with a value of 1 meaning +that the page is in-core. +.NH 3 +Synchronization primitives +.PP +Primitives are provided for synchronization using semaphores in shared memory. +Semaphores must lie within a MAP_SHARED region with at least modes +PROT_READ and PROT_WRITE. +The MAP_HASSEMAPHORE flag must have been specified when the region was created. +To acquire a lock a process calls: +.DS +value = mset(sem, wait) +result int value; semaphore *sem; int wait; +.DE +\fIMset\fP indivisibly tests and sets the semaphore \fIsem\fP. +If the previous value is zero, the process has acquired the lock +and \fImset\fP returns true immediately. +Otherwise, if the \fIwait\fP flag is zero, +failure is returned. +If \fIwait\fP is true and the previous value is non-zero, +\fImset\fP relinquishes the processor until notified that it should retry. +.LP +To release a lock a process calls: +.DS +mclear(sem) +semaphore *sem; +.DE +\fIMclear\fP indivisibly tests and clears the semaphore \fIsem\fP. +If the ``WANT'' flag is zero in the previous value, +\fImclear\fP returns immediately. +If the ``WANT'' flag is non-zero in the previous value, +\fImclear\fP arranges for waiting processes to retry before returning. +.PP +Two routines provide services analogous to the kernel +\fIsleep\fP and \fIwakeup\fP functions interpreted in the domain of +shared memory. +A process may relinquish the processor by calling \fImsleep\fP +with a set semaphore: +.DS +msleep(sem) +semaphore *sem; +.DE +If the semaphore is still set when it is checked by the kernel, +the process will be put in a sleeping state +until some other process issues an \fImwakeup\fP for the same semaphore +within the region using the call: +.DS +mwakeup(sem) +semaphore *sem; +.DE +An \fImwakeup\fP may awaken all sleepers on the semaphore, +or may awaken only the next sleeper on a queue. diff --git a/share/doc/psd/05.sysman/1.3.t b/share/doc/psd/05.sysman/1.3.t new file mode 100644 index 000000000000..f81a18501179 --- /dev/null +++ b/share/doc/psd/05.sysman/1.3.t @@ -0,0 +1,254 @@ +.\" Copyright (c) 1983, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)1.3.t 8.1 (Berkeley) 6/8/93 +.\" +.sh "Signals +.PP +.NH 3 +Overview +.PP +The system defines a set of \fIsignals\fP that may be delivered +to a process. Signal delivery resembles the occurrence of a hardware +interrupt: the signal is blocked from further occurrence, +the current process context is saved, and a new one +is built. A process may specify +the \fIhandler\fP to which a signal is delivered, or specify that +the signal is to be \fIblocked\fP or \fIignored\fP. A process may +also specify that a +\fIdefault\fP action is to be taken when signals occur. +.PP +Some signals +will cause a process to exit when they are not caught. This +may be accompanied by creation of a \fIcore\fP image file, containing +the current memory image of the process for use in post-mortem debugging. +A process may choose to have signals delivered on a special +stack, so that sophisticated software stack manipulations are possible. +.PP +All signals have the same \fIpriority\fP. If multiple signals +are pending simultaneously, the order in which they are delivered +to a process is implementation specific. Signal routines execute +with the signal that caused their invocation \fIblocked\fP, but other +signals may yet occur. Mechanisms are provided whereby critical sections +of code may protect themselves against the occurrence of specified signals. +.NH 3 +Signal types +.PP +The signals defined by the system fall into one of +five classes: hardware conditions, +software conditions, input/output notification, process control, or +resource control. +The set of signals is defined in the file \fI<signal.h>\fP. +.PP +Hardware signals are derived from exceptional conditions which +may occur during +execution. Such signals include SIGFPE representing floating +point and other arithmetic exceptions, SIGILL for illegal instruction +execution, SIGSEGV for addresses outside the currently assigned +area of memory, and SIGBUS for accesses that violate memory +protection constraints. +Other, more cpu-specific hardware signals exist, +such as those for the various customer-reserved instructions on +the VAX (SIGIOT, SIGEMT, and SIGTRAP). +.PP +Software signals reflect interrupts generated by user request: +SIGINT for the normal interrupt signal; SIGQUIT for the more +powerful \fIquit\fP signal, that normally causes a core image +to be generated; SIGHUP and SIGTERM that cause graceful +process termination, either because a user has ``hung up'', or +by user or program request; and SIGKILL, a more powerful termination +signal which a process cannot catch or ignore. +Programs may define their own asynchronous events using SIGUSR1 +and SIGUSR2. +Other software signals (SIGALRM, SIGVTALRM, SIGPROF) +indicate the expiration of interval timers. +.PP +A process can request notification via a SIGIO signal +when input or output is possible +on a descriptor, or when a \fInon-blocking\fP operation completes. +A process may request to receive a SIGURG signal when an +urgent condition arises. +.PP +A process may be \fIstopped\fP by a signal sent to it or the members +of its process group. The SIGSTOP signal is a powerful stop +signal, because it cannot be caught. Other stop signals +SIGTSTP, SIGTTIN, and SIGTTOU are used when a user request, input +request, or output request respectively is the reason for stopping the process. +A SIGCONT signal is sent to a process when it is +continued from a stopped state. +Processes may receive notification with a SIGCHLD signal when +a child process changes state, either by stopping or by terminating. +.PP +Exceeding resource limits may cause signals to be generated. +SIGXCPU occurs when a process nears its CPU time limit and SIGXFSZ +warns that the limit on file size creation has been reached. +.NH 3 +Signal handlers +.PP +A process has a handler associated with each signal. +The handler controls the way the signal is delivered. +The call +.DS +#include <signal.h> + +._f +struct sigvec { + int (*sv_handler)(); + int sv_mask; + int sv_flags; +}; + +sigvec(signo, sv, osv) +int signo; struct sigvec *sv; result struct sigvec *osv; +.DE +assigns interrupt handler address \fIsv_handler\fP to signal \fIsigno\fP. +Each handler address +specifies either an interrupt routine for the signal, that the +signal is to be ignored, +or that a default action (usually process termination) is to occur +if the signal occurs. +The constants +SIG_IGN and SIG_DEF used as values for \fIsv_handler\fP +cause ignoring or defaulting of a condition. +The \fIsv_mask\fP value specifies the +signal mask to be used when the handler is invoked; it implicitly includes +the signal which invoked the handler. +Signal masks include one bit for each signal; +the mask for a signal \fIsigno\fP is provided by the macro +\fIsigmask\fP(\fIsigno\fP), from \fI<signal.h>\fP. +\fISv_flags\fP specifies whether system calls should be +restarted if the signal handler returns and +whether the handler should operate on the normal run-time +stack or a special signal stack (see below). If \fIosv\fP +is non-zero, the previous signal vector is returned. +.PP +When a signal condition arises for a process, the signal +is added to a set of signals pending for the process. +If the signal is not currently \fIblocked\fP by the process +then it will be delivered. The process of signal delivery +adds the signal to be delivered and those signals +specified in the associated signal +handler's \fIsv_mask\fP to a set of those \fImasked\fP +for the process, saves the current process context, +and places the process in the context of the signal +handling routine. The call is arranged so that if the signal +handling routine exits normally the signal mask will be restored +and the process will resume execution in the original context. +If the process wishes to resume in a different context, then +it must arrange to restore the signal mask itself. +.PP +The mask of \fIblocked\fP signals is independent of handlers for +signals. It delays signals from being delivered much as a +raised hardware interrupt priority level delays hardware interrupts. +Preventing an interrupt from occurring by changing the handler is analogous to +disabling a device from further interrupts. +.PP +The signal handling routine \fIsv_handler\fP is called by a C call +of the form +.DS +(*sv_handler)(signo, code, scp); +int signo; long code; struct sigcontext *scp; +.DE +The \fIsigno\fP gives the number of the signal that occurred, and +the \fIcode\fP, a word of information supplied by the hardware. +The \fIscp\fP parameter is a pointer to a machine-dependent +structure containing the information for restoring the +context before the signal. +.NH 3 +Sending signals +.PP +A process can send a signal to another process or group of processes +with the calls: +.DS +kill(pid, signo) +int pid, signo; + +killpgrp(pgrp, signo) +int pgrp, signo; +.DE +Unless the process sending the signal is privileged, +it must have the same effective user id as the process receiving the signal. +.PP +Signals are also sent implicitly from a terminal device to the +process group associated with the terminal when certain input characters +are typed. +.NH 3 +Protecting critical sections +.PP +To block a section of code against one or more signals, a \fIsigblock\fP +call may be used to add a set of signals to the existing mask, returning +the old mask: +.DS +oldmask = sigblock(mask); +result long oldmask; long mask; +.DE +The old mask can then be restored later with \fIsigsetmask\fP\|, +.DS +oldmask = sigsetmask(mask); +result long oldmask; long mask; +.DE +The \fIsigblock\fP call can be used to read the current mask +by specifying an empty \fImask\fP\|. +.PP +It is possible to check conditions with some signals blocked, +and then to pause waiting for a signal and restoring the mask, by using: +.DS +sigpause(mask); +long mask; +.DE +.NH 3 +Signal stacks +.PP +Applications that maintain complex or fixed size stacks can use +the call +.DS +._f +struct sigstack { + caddr_t ss_sp; + int ss_onstack; +}; + +sigstack(ss, oss) +struct sigstack *ss; result struct sigstack *oss; +.DE +to provide the system with a stack based at \fIss_sp\fP for delivery +of signals. The value \fIss_onstack\fP indicates whether the +process is currently on the signal stack, +a notion maintained in software by the system. +.PP +When a signal is to be delivered, the system checks whether +the process is on a signal stack. If not, then the process is switched +to the signal stack for delivery, with the return from the signal +arranged to restore the previous stack. +.PP +If the process wishes to take a non-local exit from the signal routine, +or run code from the signal stack that uses a different stack, +a \fIsigstack\fP call should be used to reset the signal stack. diff --git a/share/doc/psd/05.sysman/1.4.t b/share/doc/psd/05.sysman/1.4.t new file mode 100644 index 000000000000..a67a5cedcfa8 --- /dev/null +++ b/share/doc/psd/05.sysman/1.4.t @@ -0,0 +1,137 @@ +.\" Copyright (c) 1983, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)1.4.t 8.1 (Berkeley) 6/8/93 +.\" +.sh "Timers +.NH 3 +Real time +.PP +The system's notion of the current Greenwich time and the current time +zone is set and returned by the call by the calls: +.DS +#include <sys/time.h> + +settimeofday(tvp, tzp); +struct timeval *tp; +struct timezone *tzp; + +gettimeofday(tp, tzp); +result struct timeval *tp; +result struct timezone *tzp; +.DE +where the structures are defined in \fI<sys/time.h>\fP as: +.DS +._f +struct timeval { + long tv_sec; /* seconds since Jan 1, 1970 */ + long tv_usec; /* and microseconds */ +}; + +struct timezone { + int tz_minuteswest; /* of Greenwich */ + int tz_dsttime; /* type of dst correction to apply */ +}; +.DE +The precision of the system clock is hardware dependent. +Earlier versions of UNIX contained only a 1-second resolution version +of this call, which remains as a library routine: +.DS +time(tvsec) +result long *tvsec; +.DE +returning only the tv_sec field from the \fIgettimeofday\fP call. +.NH 3 +Interval time +.PP +The system provides each process with three interval timers, +defined in \fI<sys/time.h>\fP: +.DS +._d +#define ITIMER_REAL 0 /* real time intervals */ +#define ITIMER_VIRTUAL 1 /* virtual time intervals */ +#define ITIMER_PROF 2 /* user and system virtual time */ +.DE +The ITIMER_REAL timer decrements +in real time. It could be used by a library routine to +maintain a wakeup service queue. A SIGALRM signal is delivered +when this timer expires. +.PP +The ITIMER_VIRTUAL timer decrements in process virtual time. +It runs only when the process is executing. A SIGVTALRM signal +is delivered when it expires. +.PP +The ITIMER_PROF timer decrements both in process virtual time and when +the system is running on behalf of the process. +It is designed to be used by processes to statistically profile +their execution. +A SIGPROF signal is delivered when it expires. +.PP +A timer value is defined by the \fIitimerval\fP structure: +.DS +._f +struct itimerval { + struct timeval it_interval; /* timer interval */ + struct timeval it_value; /* current value */ +}; +.DE +and a timer is set or read by the call: +.DS +getitimer(which, value); +int which; result struct itimerval *value; + +setitimer(which, value, ovalue); +int which; struct itimerval *value; result struct itimerval *ovalue; +.DE +The third argument to \fIsetitimer\fP specifies an optional structure +to receive the previous contents of the interval timer. +A timer can be disabled by specifying a timer value of 0. +.PP +The system rounds argument timer intervals to be not less than the +resolution of its clock. This clock resolution can be determined +by loading a very small value into a timer and reading the timer back to +see what value resulted. +.PP +The \fIalarm\fP system call of earlier versions of UNIX is provided +as a library routine using the ITIMER_REAL timer. The process +profiling facilities of earlier versions of UNIX +remain because +it is not always possible to guarantee +the automatic restart of system calls after +receipt of a signal. +The \fIprofil\fP call arranges for the kernel to begin gathering +execution statistics for a process: +.DS +profil(buf, bufsize, offset, scale); +result char *buf; int bufsize, offset, scale; +.DE +This begins sampling of the program counter, with statistics maintained +in the user-provided buffer. diff --git a/share/doc/psd/05.sysman/1.5.t b/share/doc/psd/05.sysman/1.5.t new file mode 100644 index 000000000000..e642e2dfa2bd --- /dev/null +++ b/share/doc/psd/05.sysman/1.5.t @@ -0,0 +1,225 @@ +.\" Copyright (c) 1983, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)1.5.t 8.1 (Berkeley) 6/8/93 +.\" +.sh Descriptors +.PP +.NH 3 +The reference table +.PP +Each process has access to resources through +\fIdescriptors\fP. Each descriptor is a handle allowing +the process to reference objects such as files, devices +and communications links. +.PP +Rather than allowing processes direct access to descriptors, the system +introduces a level of indirection, so that descriptors may be shared +between processes. Each process has a \fIdescriptor reference table\fP, +containing pointers to the actual descriptors. The descriptors +themselves thus have multiple references, and are reference counted by the +system. +.PP +Each process has a fixed size descriptor reference table, where +the size is returned by the \fIgetdtablesize\fP call: +.DS +nds = getdtablesize(); +result int nds; +.DE +and guaranteed to be at least 20. The entries in the descriptor reference +table are referred to by small integers; for example if there +are 20 slots they are numbered 0 to 19. +.NH 3 +Descriptor properties +.PP +Each descriptor has a logical set of properties maintained +by the system and defined by its \fItype\fP. +Each type supports a set of operations; +some operations, such as reading and writing, are common to several +abstractions, while others are unique. +The generic operations applying to many of these types are described +in section 2.1. Naming contexts, files and directories are described in +section 2.2. Section 2.3 describes communications domains and sockets. +Terminals and (structured and unstructured) devices are described +in section 2.4. +.NH 3 +Managing descriptor references +.PP +A duplicate of a descriptor reference may be made by doing +.DS +new = dup(old); +result int new; int old; +.DE +returning a copy of descriptor reference \fIold\fP indistinguishable from +the original. The \fInew\fP chosen by the system will be the +smallest unused descriptor reference slot. +A copy of a descriptor reference may be made in a specific slot +by doing +.DS +dup2(old, new); +int old, new; +.DE +The \fIdup2\fP call causes the system to deallocate the descriptor reference +current occupying slot \fInew\fP, if any, replacing it with a reference +to the same descriptor as old. +This deallocation is also performed by: +.DS +close(old); +int old; +.DE +.NH 3 +Multiplexing requests +.PP +The system provides a +standard way to do +synchronous and asynchronous multiplexing of operations. +.PP +Synchronous multiplexing is performed by using the \fIselect\fP call +to examine the state of multiple descriptors simultaneously, +and to wait for state changes on those descriptors. +Sets of descriptors of interest are specified as bit masks, +as follows: +.DS +#include <sys/types.h> + +nds = select(nd, in, out, except, tvp); +result int nds; int nd; result fd_set *in, *out, *except; +struct timeval *tvp; + +FD_ZERO(&fdset); +FD_SET(fd, &fdset); +FD_CLR(fd, &fdset); +FD_ISSET(fd, &fdset); +int fs; fs_set fdset; +.DE +The \fIselect\fP call examines the descriptors +specified by the +sets \fIin\fP, \fIout\fP and \fIexcept\fP, replacing +the specified bit masks by the subsets that select true for input, +output, and exceptional conditions respectively (\fInd\fP +indicates the number of file descriptors specified by the bit masks). +If any descriptors meet the following criteria, +then the number of such descriptors is returned in \fInds\fP and the +bit masks are updated. +.if n .ds bu * +.if t .ds bu \(bu +.IP \*(bu +A descriptor selects for input if an input oriented operation +such as \fIread\fP or \fIreceive\fP is possible, or if a +connection request may be accepted (see section 2.3.1.4). +.IP \*(bu +A descriptor selects for output if an output oriented operation +such as \fIwrite\fP or \fIsend\fP is possible, or if an operation +that was ``in progress'', such as connection establishment, +has completed (see section 2.1.3). +.IP \*(bu +A descriptor selects for an exceptional condition if a condition +that would cause a SIGURG signal to be generated exists (see section 1.3.2), +or other device-specific events have occurred. +.LP +If none of the specified conditions is true, the operation +waits for one of the conditions to arise, +blocking at most the amount of time specified by \fItvp\fP. +If \fItvp\fP is given as 0, the \fIselect\fP waits indefinitely. +.PP +Options affecting I/O on a descriptor +may be read and set by the call: +.DS +._d +dopt = fcntl(d, cmd, arg) +result int dopt; int d, cmd, arg; + +/* interesting values for cmd */ +#define F_SETFL 3 /* set descriptor options */ +#define F_GETFL 4 /* get descriptor options */ +#define F_SETOWN 5 /* set descriptor owner (pid/pgrp) */ +#define F_GETOWN 6 /* get descriptor owner (pid/pgrp) */ +.DE +The F_SETFL \fIcmd\fP may be used to set a descriptor in +non-blocking I/O mode and/or enable signaling when I/O is +possible. F_SETOWN may be used to specify a process or process +group to be signaled when using the latter mode of operation +or when urgent indications arise. +.PP +Operations on non-blocking descriptors will +either complete immediately, +note an error EWOULDBLOCK, +partially complete an input or output operation returning a partial count, +or return an error EINPROGRESS noting that the requested operation is +in progress. +A descriptor which has signalling enabled will cause the specified process +and/or process group +be signaled, with a SIGIO for input, output, or in-progress +operation complete, or +a SIGURG for exceptional conditions. +.PP +For example, when writing to a terminal +using non-blocking output, +the system will accept only as much data as there is buffer space for +and return; when making a connection on a \fIsocket\fP, the operation may +return indicating that the connection establishment is ``in progress''. +The \fIselect\fP facility can be used to determine when further +output is possible on the terminal, or when the connection establishment +attempt is complete. +.NH 3 +Descriptor wrapping.\(dg +.PP +.FS +\(dg The facilities described in this section are not included +in 4.3BSD. +.FE +A user process may build descriptors of a specified type by +\fIwrapping\fP a communications channel with a system supplied protocol +translator: +.DS +new = wrap(old, proto) +result int new; int old; struct dprop *proto; +.DE +Operations on the descriptor \fIold\fP are then translated by the +system provided protocol translator into requests on the underlying +object \fIold\fP in a way defined by the protocol. +The protocols supported by the kernel may vary from system to system +and are described in the programmers manual. +.PP +Protocols may be based on communications multiplexing or a rights-passing +style of handling multiple requests made on the same object. For instance, +a protocol for implementing a file abstraction may or may not include +locally generated ``read-ahead'' requests. A protocol that provides for +read-ahead may provide higher performance but have a more difficult +implementation. +.PP +Another example is the terminal driving facilities. Normally a terminal +is associated with a communications line, and the terminal type +and standard terminal access protocol are wrapped around a synchronous +communications line and given to the user. If a virtual terminal +is required, the terminal driver can be wrapped around a communications +link, the other end of which is held by a virtual terminal protocol +interpreter. diff --git a/share/doc/psd/05.sysman/1.6.t b/share/doc/psd/05.sysman/1.6.t new file mode 100644 index 000000000000..109d2714f54a --- /dev/null +++ b/share/doc/psd/05.sysman/1.6.t @@ -0,0 +1,135 @@ +.\" Copyright (c) 1983, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)1.6.t 8.1 (Berkeley) 6/8/93 +.\" +.sh "Resource controls +.NH 3 +Process priorities +.PP +The system gives CPU scheduling priority to processes that have not used +CPU time recently. This tends to favor interactive processes and +processes that execute only for short periods. +It is possible to determine the priority currently +assigned to a process, process group, or the processes of a specified user, +or to alter this priority using the calls: +.DS +._d +#define PRIO_PROCESS 0 /* process */ +#define PRIO_PGRP 1 /* process group */ +#define PRIO_USER 2 /* user id */ + +prio = getpriority(which, who); +result int prio; int which, who; + +setpriority(which, who, prio); +int which, who, prio; +.DE +The value \fIprio\fP is in the range \-20 to 20. +The default priority is 0; lower priorities cause more +favorable execution. +The \fIgetpriority\fP call returns the highest priority (lowest numerical value) +enjoyed by any of the specified processes. +The \fIsetpriority\fP call sets the priorities of all of the +specified processes to the specified value. +Only the super-user may lower priorities. +.NH 3 +Resource utilization +.PP +The resources used by a process are returned by a \fIgetrusage\fP call, +returning information in a structure defined in \fI<sys/resource.h>\fP: +.DS +._d +#define RUSAGE_SELF 0 /* usage by this process */ +#define RUSAGE_CHILDREN -1 /* usage by all children */ + +getrusage(who, rusage) +int who; result struct rusage *rusage; + +._f +struct rusage { + struct timeval ru_utime; /* user time used */ + struct timeval ru_stime; /* system time used */ + int ru_maxrss; /* maximum core resident set size: kbytes */ + int ru_ixrss; /* integral shared memory size (kbytes*sec) */ + int ru_idrss; /* unshared data memory size */ + int ru_isrss; /* unshared stack memory size */ + int ru_minflt; /* page-reclaims */ + int ru_majflt; /* page faults */ + int ru_nswap; /* swaps */ + int ru_inblock; /* block input operations */ + int ru_oublock; /* block output operations */ + int ru_msgsnd; /* messages sent */ + int ru_msgrcv; /* messages received */ + int ru_nsignals; /* signals received */ + int ru_nvcsw; /* voluntary context switches */ + int ru_nivcsw; /* involuntary context switches */ +}; +.DE +The \fIwho\fP parameter specifies whose resource usage is to be returned. +The resources used by the current process, or by all +the terminated children of the current process may be requested. +.NH 3 +Resource limits +.PP +The resources of a process for which limits are controlled by the +kernel are defined in \fI<sys/resource.h>\fP, and controlled by the +\fIgetrlimit\fP and \fIsetrlimit\fP calls: +.DS +._d +#define RLIMIT_CPU 0 /* cpu time in milliseconds */ +#define RLIMIT_FSIZE 1 /* maximum file size */ +#define RLIMIT_DATA 2 /* maximum data segment size */ +#define RLIMIT_STACK 3 /* maximum stack segment size */ +#define RLIMIT_CORE 4 /* maximum core file size */ +#define RLIMIT_RSS 5 /* maximum resident set size */ + +#define RLIM_NLIMITS 6 + +#define RLIM_INFINITY 0x7f\&f\&f\&f\&f\&f\&f + +._f +struct rlimit { + int rlim_cur; /* current (soft) limit */ + int rlim_max; /* hard limit */ +}; + +getrlimit(resource, rlp) +int resource; result struct rlimit *rlp; + +setrlimit(resource, rlp) +int resource; struct rlimit *rlp; +.DE +.PP +Only the super-user can raise the maximum limits. +Other users may only +alter \fIrlim_cur\fP within the range from 0 to \fIrlim_max\fP +or (irreversibly) lower \fIrlim_max\fP. diff --git a/share/doc/psd/05.sysman/1.7.t b/share/doc/psd/05.sysman/1.7.t new file mode 100644 index 000000000000..09e1a027930f --- /dev/null +++ b/share/doc/psd/05.sysman/1.7.t @@ -0,0 +1,100 @@ +.\" Copyright (c) 1983, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)1.7.t 8.1 (Berkeley) 6/8/93 +.\" +.sh "System operation support +.PP +Unless noted otherwise, +the calls in this section are permitted only to a privileged user. +.NH 3 +Bootstrap operations +.PP +The call +.DS +mount(blkdev, dir, ronly); +char *blkdev, *dir; int ronly; +.DE +extends the UNIX name space. The \fImount\fP call specifies +a block device \fIblkdev\fP containing a UNIX file system +to be made available starting at \fIdir\fP. If \fIronly\fP is +set then the file system is read-only; writes to the file system +will not be permitted and access times will not be updated +when files are referenced. +\fIDir\fP is normally a name in the root directory. +.PP +The call +.DS +swapon(blkdev, size); +char *blkdev; int size; +.DE +specifies a device to be made available for paging and swapping. +.PP +.NH 3 +Shutdown operations +.PP +The call +.DS +unmount(dir); +char *dir; +.DE +unmounts the file system mounted on \fIdir\fP. +This call will succeed only if the file system is +not currently being used. +.PP +The call +.DS +sync(); +.DE +schedules input/output to clean all system buffer caches. +(This call does not require privileged status.) +.PP +The call +.DS +reboot(how) +int how; +.DE +causes a machine halt or reboot. The call may request a reboot +by specifying \fIhow\fP as RB_AUTOBOOT, or that the machine be halted +with RB_HALT. These constants are defined in \fI<sys/reboot.h>\fP. +.NH 3 +Accounting +.PP +The system optionally keeps an accounting record in a file +for each process that exits on the system. +The format of this record is beyond the scope of this document. +The accounting may be enabled to a file \fIname\fP by doing +.DS +acct(path); +char *path; +.DE +If \fIpath\fP is null, then accounting is disabled. Otherwise, +the named file becomes the accounting file. diff --git a/share/doc/psd/05.sysman/2.0.t b/share/doc/psd/05.sysman/2.0.t new file mode 100644 index 000000000000..ca44bc2a8690 --- /dev/null +++ b/share/doc/psd/05.sysman/2.0.t @@ -0,0 +1,83 @@ +.\" Copyright (c) 1983, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)2.0.t 8.1 (Berkeley) 6/8/93 +.\" +.ds ss 1 +.sh "System facilities +This section discusses the system facilities that +are not considered part of the kernel. +.PP +The system abstractions described are: +.IP "Directory contexts +.br +A directory context is a position in the UNIX file system name +space. Operations on files and other named objects in a file system are +always specified relative to such a context. +.IP "Files +.br +Files are used to store uninterpreted sequence of bytes on which +random access \fIreads\fP and \fIwrites\fP may occur. +Pages from files may also be mapped into process address space.\(dg +A directory may be read as a file. +.FS +\(dg Support for mapping files is not included in the 4.3 release. +.FE +.IP "Communications domains +.br +A communications domain represents +an interprocess communications environment, such as the communications +facilities of the UNIX system, +communications in the INTERNET, or the resource sharing protocols +and access rights of a resource sharing system on a local network. +.IP "Sockets +.br +A socket is an endpoint of communication and the focal +point for IPC in a communications domain. Sockets may be created in pairs, +or given names and used to rendezvous with other sockets +in a communications domain, accepting connections from these +sockets or exchanging messages with them. These operations model +a labeled or unlabeled communications graph, and can be used in a +wide variety of communications domains. Sockets can have different +\fItypes\fP\| to provide different semantics of communication, +increasing the flexibility of the model. +.IP "Terminals and other devices +.br +Devices include +terminals, providing input editing and interrupt generation +and output flow control and editing, magnetic tapes, +disks and other peripherals. They often support the generic +\fIread\fP and \fIwrite\fP operations as well as a number of \fIioctl\fP\|s. +.IP "Processes +.br +Process descriptors provide facilities for control and debugging of +other processes. +.ds ss 2 diff --git a/share/doc/psd/05.sysman/2.1.t b/share/doc/psd/05.sysman/2.1.t new file mode 100644 index 000000000000..ef25887ee180 --- /dev/null +++ b/share/doc/psd/05.sysman/2.1.t @@ -0,0 +1,138 @@ +.\" Copyright (c) 1983, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)2.1.t 8.1 (Berkeley) 6/8/93 +.\" +.sh "Generic operations +.PP +.PP +Many system abstractions support the +operations \fIread\fP, \fIwrite\fP and \fIioctl\fP. We describe +the basics of these common primitives here. +Similarly, the mechanisms whereby normally synchronous operations +may occur in a non-blocking or asynchronous fashion are +common to all system-defined abstractions and are described here. +.NH 3 +Read and write +.PP +The \fIread\fP and \fIwrite\fP system calls can be applied +to communications channels, files, terminals and devices. +They have the form: +.DS +cc = read(fd, buf, nbytes); +result int cc; int fd; result caddr_t buf; int nbytes; + +cc = write(fd, buf, nbytes); +result int cc; int fd; caddr_t buf; int nbytes; +.DE +The \fIread\fP call transfers as much data as possible from the +object defined by \fIfd\fP to the buffer at address \fIbuf\fP of +size \fInbytes\fP. The number of bytes transferred is +returned in \fIcc\fP, which is \-1 if a return occurred before +any data was transferred because of an error or use of non-blocking +operations. +.PP +The \fIwrite\fP call transfers data from the buffer to the +object defined by \fIfd\fP. Depending on the type of \fIfd\fP, +it is possible that the \fIwrite\fP call will accept some portion +of the provided bytes; the user should resubmit the other bytes +in a later request in this case. +Error returns because of interrupted or otherwise incomplete operations +are possible. +.PP +Scattering of data on input or gathering of data for output +is also possible using an array of input/output vector descriptors. +The type for the descriptors is defined in \fI<sys/uio.h>\fP as: +.DS +._f +struct iovec { + caddr_t iov_msg; /* base of a component */ + int iov_len; /* length of a component */ +}; +.DE +The calls using an array of descriptors are: +.DS +cc = readv(fd, iov, iovlen); +result int cc; int fd; struct iovec *iov; int iovlen; + +cc = writev(fd, iov, iovlen); +result int cc; int fd; struct iovec *iov; int iovlen; +.DE +Here \fIiovlen\fP is the count of elements in the \fIiov\fP array. +.NH 3 +Input/output control +.PP +Control operations on an object are performed by the \fIioctl\fP +operation: +.DS +ioctl(fd, request, buffer); +int fd, request; caddr_t buffer; +.DE +This operation causes the specified \fIrequest\fP to be performed +on the object \fIfd\fP. The \fIrequest\fP parameter specifies +whether the argument buffer is to be read, written, read and written, +or is not needed, and also the size of the buffer, as well as the +request. +Different descriptor types and subtypes within descriptor types +may use distinct \fIioctl\fP requests. For example, +operations on terminals control flushing of input and output +queues and setting of terminal parameters; operations on +disks cause formatting operations to occur; operations on tapes +control tape positioning. +.PP +The names for basic control operations are defined in \fI<sys/ioctl.h>\fP. +.NH 3 +Non-blocking and asynchronous operations +.PP +A process that wishes to do non-blocking operations on one of +its descriptors sets the descriptor in non-blocking mode as +described in section 1.5.4. Thereafter the \fIread\fP call will +return a specific EWOULDBLOCK error indication if there is no data to be +\fIread\fP. The process may +\fIselect\fP the associated descriptor to determine when a read is +possible. +.PP +Output attempted when a descriptor can accept less than is requested +will either accept some of the provided data, returning a shorter than normal +length, or return an error indicating that the operation would block. +More output can be performed as soon as a \fIselect\fP call indicates +the object is writeable. +.PP +Operations other than data input or output +may be performed on a descriptor in a non-blocking fashion. +These operations will return with a characteristic error indicating +that they are in progress +if they cannot complete immediately. The descriptor +may then be \fIselect\fPed for \fIwrite\fP to find out +when the operation has been completed. When \fIselect\fP indicates +the descriptor is writeable, the operation has completed. +Depending on the nature of the descriptor and the operation, +additional activity may be started or the new state may be tested. diff --git a/share/doc/psd/05.sysman/2.2.t b/share/doc/psd/05.sysman/2.2.t new file mode 100644 index 000000000000..996e9b5953b4 --- /dev/null +++ b/share/doc/psd/05.sysman/2.2.t @@ -0,0 +1,470 @@ +.\" Copyright (c) 1983, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)2.2.t 8.1 (Berkeley) 6/8/93 +.\" +.sh "File system +.NH 3 +Overview +.PP +The file system abstraction provides access to a hierarchical +file system structure. +The file system contains directories (each of which may contain +other sub-directories) as well as files and references to other +objects such as devices and inter-process communications sockets. +.PP +Each file is organized as a linear array of bytes. No record +boundaries or system related information is present in +a file. +Files may be read and written in a random-access fashion. +The user may read the data in a directory as though +it were an ordinary file to determine the names of the contained files, +but only the system may write into the directories. +The file system stores only a small amount of ownership, protection and usage +information with a file. +.NH 3 +Naming +.PP +The file system calls take \fIpath name\fP arguments. +These consist of a zero or more component \fIfile names\fP +separated by ``/\^'' characters, where each file name +is up to 255 ASCII characters excluding null and ``/\^''. +.PP +Each process always has two naming contexts: one for the +root directory of the file system and one for the +current working directory. These are used +by the system in the filename translation process. +If a path name begins with a ``/\^'', it is called +a full path name and interpreted relative to the root directory context. +If the path name does not begin with a ``/\^'' it is called +a relative path name and interpreted relative to the current directory +context. +.PP +The system limits +the total length of a path name to 1024 characters. +.PP +The file name ``..'' in each directory refers to +the parent directory of that directory. +The parent directory of the root of the file system is always that directory. +.PP +The calls +.DS +chdir(path); +char *path; + +chroot(path) +char *path; +.DE +change the current working directory and root directory context of a process. +Only the super-user can change the root directory context of a process. +.NH 3 +Creation and removal +.PP +The file system allows directories, files, special devices, +and ``portals'' to be created and removed from the file system. +.NH 4 +Directory creation and removal +.PP +A directory is created with the \fImkdir\fP system call: +.DS +mkdir(path, mode); +char *path; int mode; +.DE +where the mode is defined as for files (see below). +Directories are removed with the \fIrmdir\fP system call: +.DS +rmdir(path); +char *path; +.DE +A directory must be empty if it is to be deleted. +.NH 4 +File creation +.PP +Files are created with the \fIopen\fP system call, +.DS +fd = open(path, oflag, mode); +result int fd; char *path; int oflag, mode; +.DE +The \fIpath\fP parameter specifies the name of the +file to be created. The \fIoflag\fP parameter must +include O_CREAT from below to cause the file to be created. +Bits for \fIoflag\fP are +defined in \fI<sys/file.h>\fP: +.DS +._d +#define O_RDONLY 000 /* open for reading */ +#define O_WRONLY 001 /* open for writing */ +#define O_RDWR 002 /* open for read & write */ +#define O_NDELAY 004 /* non-blocking open */ +#define O_APPEND 010 /* append on each write */ +#define O_CREAT 01000 /* open with file create */ +#define O_TRUNC 02000 /* open with truncation */ +#define O_EXCL 04000 /* error on create if file exists */ +.DE +.PP +One of O_RDONLY, O_WRONLY and O_RDWR should be specified, +indicating what types of operations are desired to be performed +on the open file. The operations will be checked against the user's +access rights to the file before allowing the \fIopen\fP to succeed. +Specifying O_APPEND causes writes to automatically append to the +file. +The flag O_CREAT causes the file to be created if it does not +exist, owned by the current user +and the group of the containing directory. +The protection for the new file is specified in \fImode\fP. +The file mode is used as a three digit octal number. +Each digit encodes read access as 4, write access as 2 and execute +access as 1, or'ed together. The 0700 bits describe owner +access, the 070 bits describe the access rights for processes in the same +group as the file, and the 07 bits describe the access rights +for other processes. +.PP +If the open specifies to create the file with O_EXCL +and the file already exists, then the \fIopen\fP will fail +without affecting the file in any way. This provides a +simple exclusive access facility. +If the file exists but is a symbolic link, the open will fail +regardless of the existence of the file specified by the link. +.NH 4 +Creating references to devices +.PP +The file system allows entries which reference peripheral devices. +Peripherals are distinguished as \fIblock\fP or \fIcharacter\fP +devices according by their ability to support block-oriented +operations. +Devices are identified by their ``major'' and ``minor'' +device numbers. The major device number determines the kind +of peripheral it is, while the minor device number indicates +one of possibly many peripherals of that kind. +Structured devices have all operations performed internally +in ``block'' quantities while +unstructured devices often have a number of +special \fIioctl\fP operations, and may have input and output +performed in varying units. +The \fImknod\fP call creates special entries: +.DS +mknod(path, mode, dev); +char *path; int mode, dev; +.DE +where \fImode\fP is formed from the object type +and access permissions. The parameter \fIdev\fP is a configuration +dependent parameter used to identify specific character or +block I/O devices. +.NH 4 +Portal creation\(dg +.PP +.FS +\(dg The \fIportal\fP call is not implemented in 4.3BSD. +.FE +The call +.DS +fd = portal(name, server, param, dtype, protocol, domain, socktype) +result int fd; char *name, *server, *param; int dtype, protocol; +int domain, socktype; +.DE +places a \fIname\fP in the file system name space that causes connection to a +server process when the name is used. +The portal call returns an active portal in \fIfd\fP as though an +access had occurred to activate an inactive portal, as now described. +.PP +When an inactive portal is accessed, the system sets up a socket +of the specified \fIsocktype\fP in the specified communications +\fIdomain\fP (see section 2.3), and creates the \fIserver\fP process, +giving it the specified \fIparam\fP as argument to help it identify +the portal, and also giving it the newly created socket as descriptor +number 0. The accessor of the portal will create a socket in the same +\fIdomain\fP and \fIconnect\fP to the server. The user will then +\fIwrap\fP the socket in the specified \fIprotocol\fP to create an object of +the required descriptor type \fIdtype\fP and proceed with the +operation which was in progress before the portal was encountered. +.PP +While the server process holds the socket (which it received as \fIfd\fP +from the \fIportal\fP call on descriptor 0 at activation) further references +will result in connections being made to the same socket. +.NH 4 +File, device, and portal removal +.PP +A reference to a file, special device or portal may be removed with the +\fIunlink\fP call, +.DS +unlink(path); +char *path; +.DE +The caller must have write access to the directory in which +the file is located for this call to be successful. +.NH 3 +Reading and modifying file attributes +.PP +Detailed information about the attributes of a file +may be obtained with the calls: +.DS +#include <sys/stat.h> + +stat(path, stb); +char *path; result struct stat *stb; + +fstat(fd, stb); +int fd; result struct stat *stb; +.DE +The \fIstat\fP structure includes the file +type, protection, ownership, access times, +size, and a count of hard links. +If the file is a symbolic link, then the status of the link +itself (rather than the file the link references) +may be found using the \fIlstat\fP call: +.DS +lstat(path, stb); +char *path; result struct stat *stb; +.DE +.PP +Newly created files are assigned the user id of the +process that created it and the group id of the directory +in which it was created. The ownership of a file may +be changed by either of the calls +.DS +chown(path, owner, group); +char *path; int owner, group; + +fchown(fd, owner, group); +int fd, owner, group; +.DE +.PP +In addition to ownership, each file has three levels of access +protection associated with it. These levels are owner relative, +group relative, and global (all users and groups). Each level +of access has separate indicators for read permission, write +permission, and execute permission. +The protection bits associated with a file may be set by either +of the calls: +.DS +chmod(path, mode); +char *path; int mode; + +fchmod(fd, mode); +int fd, mode; +.DE +where \fImode\fP is a value indicating the new protection +of the file, as listed in section 2.2.3.2. +.PP +Finally, the access and modify times on a file may be set by the call: +.DS +utimes(path, tvp) +char *path; struct timeval *tvp[2]; +.DE +This is particularly useful when moving files between media, to +preserve relationships between the times the file was modified. +.NH 3 +Links and renaming +.PP +Links allow multiple names for a file +to exist. Links exist independently of the file linked to. +.PP +Two types of links exist, \fIhard\fP links and \fIsymbolic\fP +links. A hard link is a reference counting mechanism that +allows a file to have multiple names within the same file +system. Symbolic links cause string substitution +during the pathname interpretation process. +.PP +Hard links and symbolic links have different +properties. A hard link insures the target +file will always be accessible, even after its original +directory entry is removed; no such guarantee exists for a symbolic link. +Symbolic links can span file systems boundaries. +.PP +The following calls create a new link, named \fIpath2\fP, +to \fIpath1\fP: +.DS +link(path1, path2); +char *path1, *path2; + +symlink(path1, path2); +char *path1, *path2; +.DE +The \fIunlink\fP primitive may be used to remove +either type of link. +.PP +If a file is a symbolic link, the ``value'' of the +link may be read with the \fIreadlink\fP call, +.DS +len = readlink(path, buf, bufsize); +result int len; result char *path, *buf; int bufsize; +.DE +This call returns, in \fIbuf\fP, the null-terminated string +substituted into pathnames passing through \fIpath\fP\|. +.PP +Atomic renaming of file system resident objects is possible +with the \fIrename\fP call: +.DS +rename(oldname, newname); +char *oldname, *newname; +.DE +where both \fIoldname\fP and \fInewname\fP must be +in the same file system. +If \fInewname\fP exists and is a directory, then it must be empty. +.NH 3 +Extension and truncation +.PP +Files are created with zero length and may be extended +simply by writing or appending to them. While a file is +open the system maintains a pointer into the file +indicating the current location in the file associated with +the descriptor. This pointer may be moved about in the +file in a random access fashion. +To set the current offset into a file, the \fIlseek\fP +call may be used, +.DS +oldoffset = lseek(fd, offset, type); +result off_t oldoffset; int fd; off_t offset; int type; +.DE +where \fItype\fP is given in \fI<sys/file.h>\fP as one of: +.DS +._d +#define L_SET 0 /* set absolute file offset */ +#define L_INCR 1 /* set file offset relative to current position */ +#define L_XTND 2 /* set offset relative to end-of-file */ +.DE +The call ``lseek(fd, 0, L_INCR)'' +returns the current offset into the file. +.PP +Files may have ``holes'' in them. Holes are void areas in the +linear extent of the file where data has never been +written. These may be created by seeking to +a location in a file past the current end-of-file and writing. +Holes are treated by the system as zero valued bytes. +.PP +A file may be truncated with either of the calls: +.DS +truncate(path, length); +char *path; int length; + +ftruncate(fd, length); +int fd, length; +.DE +reducing the size of the specified file to \fIlength\fP bytes. +.NH 3 +Checking accessibility +.PP +A process running with +different real and effective user ids +may interrogate the accessibility of a file to the +real user by using +the \fIaccess\fP call: +.DS +accessible = access(path, how); +result int accessible; char *path; int how; +.DE +Here \fIhow\fP is constructed by or'ing the following bits, defined +in \fI<sys/file.h>\fP: +.DS +._d +#define F_OK 0 /* file exists */ +#define X_OK 1 /* file is executable */ +#define W_OK 2 /* file is writable */ +#define R_OK 4 /* file is readable */ +.DE +The presence or absence of advisory locks does not affect the +result of \fIaccess\fP\|. +.NH 3 +Locking +.PP +The file system provides basic facilities that allow cooperating processes +to synchronize their access to shared files. A process may +place an advisory \fIread\fP or \fIwrite\fP lock on a file, +so that other cooperating processes may avoid interfering +with the process' access. This simple mechanism +provides locking with file granularity. More granular +locking can be built using the IPC facilities to provide a lock +manager. +The system does not force processes to obey the locks; +they are of an advisory nature only. +.PP +Locking is performed after an \fIopen\fP call by applying the +\fIflock\fP primitive, +.DS +flock(fd, how); +int fd, how; +.DE +where the \fIhow\fP parameter is formed from bits defined in \fI<sys/file.h>\fP: +.DS +._d +#define LOCK_SH 1 /* shared lock */ +#define LOCK_EX 2 /* exclusive lock */ +#define LOCK_NB 4 /* don't block when locking */ +#define LOCK_UN 8 /* unlock */ +.DE +Successive lock calls may be used to increase or +decrease the level of locking. If an object is currently +locked by another process when a \fIflock\fP call is made, +the caller will be blocked until the current lock owner +releases the lock; this may be avoided by including LOCK_NB +in the \fIhow\fP parameter. +Specifying LOCK_UN removes all locks associated with the descriptor. +Advisory locks held by a process are automatically deleted when +the process terminates. +.NH 3 +Disk quotas +.PP +As an optional facility, each file system may be requested to +impose limits on a user's disk usage. +Two quantities are limited: the total amount of disk space which +a user may allocate in a file system and the total number of files +a user may create in a file system. Quotas are expressed as +\fIhard\fP limits and \fIsoft\fP limits. A hard limit is +always imposed; if a user would exceed a hard limit, the operation +which caused the resource request will fail. A soft limit results +in the user receiving a warning message, but with allocation succeeding. +Facilities are provided to turn soft limits into hard limits if a +user has exceeded a soft limit for an unreasonable period of time. +.PP +To enable disk quotas on a file system the \fIsetquota\fP call +is used: +.DS +setquota(special, file) +char *special, *file; +.DE +where \fIspecial\fP refers to a structured device file where +a mounted file system exists, and +\fIfile\fP refers to a disk quota file (residing on the file +system associated with \fIspecial\fP) from which user quotas +should be obtained. The format of the disk quota file is +implementation dependent. +.PP +To manipulate disk quotas the \fIquota\fP call is provided: +.DS +#include <sys/quota.h> + +quota(cmd, uid, arg, addr) +int cmd, uid, arg; caddr_t addr; +.DE +The indicated \fIcmd\fP is applied to the user ID \fIuid\fP. +The parameters \fIarg\fP and \fIaddr\fP are command specific. +The file \fI<sys/quota.h>\fP contains definitions pertinent to the +use of this call. diff --git a/share/doc/psd/05.sysman/2.3.t b/share/doc/psd/05.sysman/2.3.t new file mode 100644 index 000000000000..edf3e10fb52e --- /dev/null +++ b/share/doc/psd/05.sysman/2.3.t @@ -0,0 +1,413 @@ +.\" Copyright (c) 1983, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)2.3.t 8.1 (Berkeley) 6/8/93 +.\" $FreeBSD$ +.\" +.sh "Interprocess communications +.NH 3 +Interprocess communication primitives +.NH 4 +Communication domains +.PP +The system provides access to an extensible set of +communication \fIdomains\fP. A communication domain +is identified by a manifest constant defined in the +file \fI<sys/socket.h>\fP. +Important standard domains supported by the system are the ``unix'' +domain, AF_UNIX, for communication within the system, the ``Internet'' +domain for communication in the DARPA Internet, AF_INET, +and the ``NS'' domain, AF_NS, for communication +using the Xerox Network Systems protocols. +Other domains can be added to the system. +.NH 4 +Socket types and protocols +.PP +Within a domain, communication takes place between communication endpoints +known as \fIsockets\fP. Each socket has the potential to exchange +information with other sockets of an appropriate type within the domain. +.PP +Each socket has an associated +abstract type, which describes the semantics of communication using that +socket. Properties such as reliability, ordering, and prevention +of duplication of messages are determined by the type. +The basic set of socket types is defined in \fI<sys/socket.h>\fP: +.DS +/* Standard socket types */ +._d +#define SOCK_DGRAM 1 /* datagram */ +#define SOCK_STREAM 2 /* virtual circuit */ +#define SOCK_RAW 3 /* raw socket */ +#define SOCK_RDM 4 /* reliably-delivered message */ +#define SOCK_SEQPACKET 5 /* sequenced packets */ +.DE +The SOCK_DGRAM type models the semantics of datagrams in network communication: +messages may be lost or duplicated and may arrive out-of-order. +A datagram socket may send messages to and receive messages from multiple +peers. +The SOCK_RDM type models the semantics of reliable datagrams: messages +arrive unduplicated and in-order, the sender is notified if +messages are lost. +The \fIsend\fP and \fIreceive\fP operations (described below) +generate reliable/unreliable datagrams. +The SOCK_STREAM type models connection-based virtual circuits: two-way +byte streams with no record boundaries. +Connection setup is required before data communication may begin. +The SOCK_SEQPACKET type models a connection-based, +full-duplex, reliable, sequenced packet exchange; +the sender is notified if messages are lost, and messages are never +duplicated or presented out-of-order. +Users of the last two abstractions may use the facilities for +out-of-band transmission to send out-of-band data. +.PP +SOCK_RAW is used for unprocessed access to internal network layers +and interfaces; it has no specific semantics. +.PP +Other socket types can be defined. +.PP +Each socket may have a specific \fIprotocol\fP associated with it. +This protocol is used within the domain to provide the semantics +required by the socket type. +Not all socket types are supported by each domain; +support depends on the existence and the implementation +of a suitable protocol within the domain. +For example, within the ``Internet'' domain, the SOCK_DGRAM type may be +implemented by the UDP user datagram protocol, and the SOCK_STREAM +type may be implemented by the TCP transmission control protocol, while +no standard protocols to provide SOCK_RDM or SOCK_SEQPACKET sockets exist. +.NH 4 +Socket creation, naming and service establishment +.PP +Sockets may be \fIconnected\fP or \fIunconnected\fP. An unconnected +socket descriptor is obtained by the \fIsocket\fP call: +.DS +s = socket(domain, type, protocol); +result int s; int domain, type, protocol; +.DE +The socket domain and type are as described above, +and are specified using the definitions from \fI<sys/socket.h>\fP. +The protocol may be given as 0, meaning any suitable protocol. +One of several possible protocols may be selected using identifiers +obtained from a library routine, \fIgetprotobyname\fP. +.PP +An unconnected socket descriptor of a connection-oriented type +may yield a connected socket descriptor +in one of two ways: either by actively connecting to another socket, +or by becoming associated with a name in the communications domain and +\fIaccepting\fP a connection from another socket. +Datagram sockets need not establish connections before use. +.PP +To accept connections or to receive datagrams, +a socket must first have a binding +to a name (or address) within the communications domain. +Such a binding may be established by a \fIbind\fP call: +.DS +bind(s, name, namelen); +int s; struct sockaddr *name; int namelen; +.DE +Datagram sockets may have default bindings established when first +sending data if not explicitly bound earlier. +In either case, +a socket's bound name may be retrieved with a \fIgetsockname\fP call: +.DS +getsockname(s, name, namelen); +int s; result struct sockaddr *name; result int *namelen; +.DE +while the peer's name can be retrieved with \fIgetpeername\fP: +.DS +getpeername(s, name, namelen); +int s; result struct sockaddr *name; result int *namelen; +.DE +Domains may support sockets with several names. +.NH 4 +Accepting connections +.PP +Once a binding is made to a connection-oriented socket, +it is possible to \fIlisten\fP for connections: +.DS +listen(s, backlog); +int s, backlog; +.DE +The \fIbacklog\fP specifies the maximum count of connections +that can be simultaneously queued awaiting acceptance. +.PP +An \fIaccept\fP call: +.DS +t = accept(s, name, anamelen); +result int t; int s; result struct sockaddr *name; result int *anamelen; +.DE +returns a descriptor for a new, connected, socket +from the queue of pending connections on \fIs\fP. +If no new connections are queued for acceptance, +the call will wait for a connection unless non-blocking I/O has been enabled. +.NH 4 +Making connections +.PP +An active connection to a named socket is made by the \fIconnect\fP call: +.DS +connect(s, name, namelen); +int s; struct sockaddr *name; int namelen; +.DE +Although datagram sockets do not establish connections, +the \fIconnect\fP call may be used with such sockets +to create an \fIassociation\fP with the foreign address. +The address is recorded for use in future \fIsend\fP calls, +which then need not supply destination addresses. +Datagrams will be received only from that peer, +and asynchronous error reports may be received. +.PP +It is also possible to create connected pairs of sockets without +using the domain's name space to rendezvous; this is done with the +\fIsocketpair\fP call\(dg: +.FS +\(dg 4.3BSD supports \fIsocketpair\fP creation only in the ``unix'' +communication domain. +.FE +.DS +socketpair(domain, type, protocol, sv); +int domain, type, protocol; result int sv[2]; +.DE +Here the returned \fIsv\fP descriptors correspond to those obtained with +\fIaccept\fP and \fIconnect\fP. +.PP +The call +.DS +pipe(pv) +result int pv[2]; +.DE +creates a pair of SOCK_STREAM sockets in the UNIX domain, +with pv[0] only writable and pv[1] only readable. +.NH 4 +Sending and receiving data +.PP +Messages may be sent from a socket by: +.DS +cc = sendto(s, buf, len, flags, to, tolen); +result int cc; int s; caddr_t buf; int len, flags; caddr_t to; int tolen; +.DE +if the socket is not connected or: +.DS +cc = send(s, buf, len, flags); +result int cc; int s; caddr_t buf; int len, flags; +.DE +if the socket is connected. +The corresponding receive primitives are: +.DS +msglen = recvfrom(s, buf, len, flags, from, fromlenaddr); +result int msglen; int s; result caddr_t buf; int len, flags; +result caddr_t from; result int *fromlenaddr; +.DE +and +.DS +msglen = recv(s, buf, len, flags); +result int msglen; int s; result caddr_t buf; int len, flags; +.DE +.PP +In the unconnected case, +the parameters \fIto\fP and \fItolen\fP +specify the destination or source of the message, while +the \fIfrom\fP parameter stores the source of the message, +and \fI*fromlenaddr\fP initially gives the size of the \fIfrom\fP +buffer and is updated to reflect the true length of the \fIfrom\fP +address. +.PP +All calls cause the message to be received in or sent from +the message buffer of length \fIlen\fP bytes, starting at address \fIbuf\fP. +The \fIflags\fP specify +peeking at a message without reading it or sending or receiving +high-priority out-of-band messages, as follows: +.DS +._d +#define MSG_PEEK 0x1 /* peek at incoming message */ +#define MSG_OOB 0x2 /* process out-of-band data */ +.DE +.NH 4 +Scatter/gather and exchanging access rights +.PP +It is possible scatter and gather data and to exchange access rights +with messages. When either of these operations is involved, +the number of parameters to the call becomes large. +Thus the system defines a message header structure, in \fI<sys/socket.h>\fP, +which can be +used to conveniently contain the parameters to the calls: +.DS +.if t .ta .5i 1.25i 2i 2.7i +.if n ._f +struct msghdr { + caddr_t msg_name; /* optional address */ + int msg_namelen; /* size of address */ + struct iov *msg_iov; /* scatter/gather array */ + int msg_iovlen; /* # elements in msg_iov */ + caddr_t msg_accrights; /* access rights sent/received */ + int msg_accrightslen; /* size of msg_accrights */ +}; +.DE +Here \fImsg_name\fP and \fImsg_namelen\fP specify the source or destination +address if the socket is unconnected; \fImsg_name\fP may be given as +a null pointer if no names are desired or required. +The \fImsg_iov\fP and \fImsg_iovlen\fP describe the scatter/gather +locations, as described in section 2.1.3. +Access rights to be sent along with the message are specified +in \fImsg_accrights\fP, which has length \fImsg_accrightslen\fP. +In the ``unix'' domain these are an array of integer descriptors, +taken from the sending process and duplicated in the receiver. +.PP +This structure is used in the operations \fIsendmsg\fP and \fIrecvmsg\fP: +.DS +sendmsg(s, msg, flags); +int s; struct msghdr *msg; int flags; + +msglen = recvmsg(s, msg, flags); +result int msglen; int s; result struct msghdr *msg; int flags; +.DE +.NH 4 +Using read and write with sockets +.PP +The normal UNIX \fIread\fP and \fIwrite\fP calls may be +applied to connected sockets and translated into \fIsend\fP and \fIreceive\fP +calls from or to a single area of memory and discarding any rights +received. A process may operate on a virtual circuit socket, a terminal +or a file with blocking or non-blocking input/output +operations without distinguishing the descriptor type. +.NH 4 +Shutting down halves of full-duplex connections +.PP +A process that has a full-duplex socket such as a virtual circuit +and no longer wishes to read from or write to this socket can +give the call: +.DS +shutdown(s, direction); +int s, direction; +.DE +where \fIdirection\fP is 0 to not read further, 1 to not +write further, or 2 to completely shut the connection down. +If the underlying protocol supports unidirectional or bidirectional shutdown, +this indication will be passed to the peer. +For example, a shutdown for writing might produce an end-of-file +condition at the remote end. +.NH 4 +Socket and protocol options +.PP +Sockets, and their underlying communication protocols, may +support \fIoptions\fP. These options may be used to manipulate +implementation- or protocol-specific facilities. +The \fIgetsockopt\fP +and \fIsetsockopt\fP calls are used to control options: +.DS +getsockopt(s, level, optname, optval, optlen) +int s, level, optname; result caddr_t optval; result int *optlen; + +setsockopt(s, level, optname, optval, optlen) +int s, level, optname; caddr_t optval; int optlen; +.DE +The option \fIoptname\fP is interpreted at the indicated +protocol \fIlevel\fP for socket \fIs\fP. If a value is specified +with \fIoptval\fP and \fIoptlen\fP, it is interpreted by +the software operating at the specified \fIlevel\fP. The \fIlevel\fP +SOL_SOCKET is reserved to indicate options maintained +by the socket facilities. Other \fIlevel\fP values indicate +a particular protocol which is to act on the option request; +these values are normally interpreted as a ``protocol number''. +.NH 3 +UNIX domain +.PP +This section describes briefly the properties of the UNIX communications +domain. +.NH 4 +Types of sockets +.PP +In the UNIX domain, +the SOCK_STREAM abstraction provides pipe-like +facilities, while SOCK_DGRAM provides (usually) +reliable message-style communications. +.NH 4 +Naming +.PP +Socket names are strings and may appear in the UNIX file +system name space through portals\(dg. +.FS +\(dg The 4.3BSD implementation of the UNIX domain embeds +bound sockets in the UNIX file system name space; +this may change in future releases. +.FE +.NH 4 +Access rights transmission +.PP +The ability to pass UNIX descriptors with messages in this domain +allows migration of service within the system and allows +user processes to be used in building system facilities. +.NH 3 +INTERNET domain +.PP +This section describes briefly how the Internet domain is +mapped to the model described in this section. More +information will be found in the document describing the +network implementation in 4.3BSD. +.NH 4 +Socket types and protocols +.PP +SOCK_STREAM is supported by the Internet TCP protocol; +SOCK_DGRAM by the UDP protocol. +Each is layered atop the transport-level Internet Protocol (IP). +The Internet Control Message Protocol is implemented atop/beside IP +and is accessible via a raw socket. +The SOCK_SEQPACKET +has no direct Internet family analogue; a protocol +based on one from the XEROX NS family and layered on +top of IP could be implemented to fill this gap. +.NH 4 +Socket naming +.PP +Sockets in the Internet domain have names composed of the 32 bit +Internet address, and a 16 bit port number. +Options may be used to +provide IP source routing or security options. +The 32-bit address is composed of network and host parts; +the network part is variable in size and is frequency encoded. +The host part may optionally be interpreted as a subnet field +plus the host on subnet; this is enabled by setting a network address +mask at boot time. +.NH 4 +Access rights transmission +.PP +No access rights transmission facilities are provided in the Internet domain. +.NH 4 +Raw access +.PP +The Internet domain allows the super-user access to the raw facilities +of IP. +These interfaces are modeled as SOCK_RAW sockets. +Each raw socket is associated with one IP protocol number, +and receives all traffic received for that protocol. +This allows administrative and debugging +functions to occur, +and enables user-level implementations of special-purpose protocols +such as inter-gateway routing protocols. diff --git a/share/doc/psd/05.sysman/2.4.t b/share/doc/psd/05.sysman/2.4.t new file mode 100644 index 000000000000..cd7dcb987900 --- /dev/null +++ b/share/doc/psd/05.sysman/2.4.t @@ -0,0 +1,174 @@ +.\" Copyright (c) 1983, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)2.4.t 8.1 (Berkeley) 6/8/93 +.\" +.sh "Terminals and Devices +.NH 3 +Terminals +.PP +Terminals support \fIread\fP and \fIwrite\fP I/O operations, +as well as a collection of terminal specific \fIioctl\fP operations, +to control input character interpretation and editing, +and output format and delays. +.NH 4 +Terminal input +.PP +Terminals are handled according to the underlying communication +characteristics such as baud rate and required delays, +and a set of software parameters. +.NH 5 +Input modes +.PP +A terminal is in one of three possible modes: \fIraw\fP, \fIcbreak\fP, +or \fIcooked\fP. +In raw mode all input is passed through to the +reading process immediately and without interpretation. +In cbreak mode, the handler interprets input only by looking +for characters that cause interrupts or output flow control; +all other characters are made available as in raw mode. +In cooked mode, input +is processed to provide standard line-oriented local editing functions, +and input is presented on a line-by-line basis. +.NH 5 +Interrupt characters +.PP +Interrupt characters are interpreted by the terminal handler only in +cbreak and cooked modes, and +cause a software interrupt to be sent to all processes in the process +group associated with the terminal. Interrupt characters exist +to send SIGINT +and SIGQUIT signals, +and to stop a process group +with the SIGTSTP signal either immediately, or when +all input up to the stop character has been read. +.NH 5 +Line editing +.PP +When the terminal is in cooked mode, editing of an input line +is performed. Editing facilities allow deletion of the previous +character or word, or deletion of the current input line. +In addition, a special character may be used to reprint the current +input line after some number of editing operations have been applied. +.PP +Certain other characters are interpreted specially when a process is +in cooked mode. The \fIend of line\fP character determines +the end of an input record. The \fIend of file\fP character simulates +an end of file occurrence on terminal input. Flow control is provided +by \fIstop output\fP and \fIstart output\fP control characters. Output +may be flushed with the \fIflush output\fP character; and a \fIliteral +character\fP may be used to force literal input of the immediately +following character in the input line. +.PP +Input characters may be echoed to the terminal as they are received. +Non-graphic ASCII input characters may be echoed as a two-character +printable representation, ``^character.'' +.NH 4 +Terminal output +.PP +On output, the terminal handler provides some simple formatting services. +These include converting the carriage return character to the +two character return-linefeed sequence, +inserting delays after certain standard control characters, +expanding tabs, and providing translations +for upper-case only terminals. +.NH 4 +Terminal control operations +.PP +When a terminal is first opened it is initialized to a standard +state and configured with a set of standard control, editing, +and interrupt characters. A process +may alter this configuration with certain +control operations, specifying parameters in a standard structure:\(dg +.FS +\(dg The control interface described here is an internal interface only +in 4.3BSD. Future releases will probably use a modified interface +based on currently-proposed standards. +.FE +.DS +._f +struct ttymode { + short tt_ispeed; /* input speed */ + int tt_iflags; /* input flags */ + short tt_ospeed; /* output speed */ + int tt_oflags; /* output flags */ +}; +.DE +and ``special characters'' are specified with the +\fIttychars\fP structure, +.DS +._f +struct ttychars { + char tc_erasec; /* erase char */ + char tc_killc; /* erase line */ + char tc_intrc; /* interrupt */ + char tc_quitc; /* quit */ + char tc_startc; /* start output */ + char tc_stopc; /* stop output */ + char tc_eofc; /* end-of-file */ + char tc_brkc; /* input delimiter (like nl) */ + char tc_suspc; /* stop process signal */ + char tc_dsuspc; /* delayed stop process signal */ + char tc_rprntc; /* reprint line */ + char tc_flushc; /* flush output (toggles) */ + char tc_werasc; /* word erase */ + char tc_lnextc; /* literal next character */ +}; +.DE +.NH 4 +Terminal hardware support +.PP +The terminal handler allows a user to access basic +hardware related functions; e.g. line speed, +modem control, parity, and stop bits. A special signal, +SIGHUP, is automatically +sent to processes in a terminal's process +group when a carrier transition is detected. This is +normally associated with a user hanging up on a modem +controlled terminal line. +.NH 3 +Structured devices +.PP +Structures devices are typified by disks and magnetic +tapes, but may represent any random-access device. +The system performs read-modify-write type buffering actions on block +devices to allow them to be read and written in a totally random +access fashion like ordinary files. +File systems are normally created in block devices. +.NH 3 +Unstructured devices +.PP +Unstructured devices are those devices which +do not support block structure. Familiar unstructured devices +are raw communications lines (with +no terminal handler), raster plotters, magnetic tape and disks unfettered +by buffering and permitting large block input/output and positioning +and formatting commands. diff --git a/share/doc/psd/05.sysman/2.5.t b/share/doc/psd/05.sysman/2.5.t new file mode 100644 index 000000000000..109eb6a46ef8 --- /dev/null +++ b/share/doc/psd/05.sysman/2.5.t @@ -0,0 +1,39 @@ +.\" Copyright (c) 1983, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)2.5.t 8.1 (Berkeley) 6/8/93 +.\" +.sh "Process and kernel descriptors +.PP +The status of the facilities in this section is still under discussion. +The \fIptrace\fP facility of earlier UNIX systems +remains in 4.3BSD. +Planned enhancements would allow a descriptor-based process control facility. diff --git a/share/doc/psd/05.sysman/Makefile b/share/doc/psd/05.sysman/Makefile new file mode 100644 index 000000000000..2c0ec7b8c9d3 --- /dev/null +++ b/share/doc/psd/05.sysman/Makefile @@ -0,0 +1,10 @@ +# From: @(#)Makefile 8.1 (Berkeley) 6/8/93 +# $FreeBSD$ + +VOLUME= psd/05.sysman +SRCS= 0.t 1.0.t 1.1.t 1.2.t 1.3.t 1.4.t 1.5.t 1.6.t 1.7.t \ + 2.0.t 2.1.t 2.2.t 2.3.t 2.4.t 2.5.t a.t +MACROS= -ms +USE_TBL= + +.include <bsd.doc.mk> diff --git a/share/doc/psd/05.sysman/a.t b/share/doc/psd/05.sysman/a.t new file mode 100644 index 000000000000..dd9cfd991d05 --- /dev/null +++ b/share/doc/psd/05.sysman/a.t @@ -0,0 +1,235 @@ +.\" Copyright (c) 1983, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)a.t 8.1 (Berkeley) 6/8/93 +.\" +.ds RH Summary of facilities +.bp +.SH +\s+2I. Summary of facilities\s0 +.PP +.de h +.br +.if n .ne 8 +\fB\\$1 \\$2\fP +.br +.. +.nr H1 0 +.NH +Kernel primitives +.LP +.h 1.1. "Process naming and protection +.in +5 +.TS +lw(1.6i) aw(3i). +sethostid set UNIX host id +gethostid get UNIX host id +sethostname set UNIX host name +gethostname get UNIX host name +getpid get process id +fork create new process +exit terminate a process +execve execute a different process +getuid get user id +geteuid get effective user id +setreuid set real and effective user id's +getgid get accounting group id +getegid get effective accounting group id +getgroups get access group set +setregid set real and effective group id's +setgroups set access group set +getpgrp get process group +setpgrp set process group +.TE +.in -5 +.h 1.2 "Memory management +.in +5 +.TS +lw(1.6i) aw(3i). +<sys/mman.h> memory management definitions +sbrk change data section size +sstk\(dg change stack section size +getpagesize get memory page size +mmap\(dg map pages of memory +msync\(dg flush modified mapped pages to filesystem +munmap\(dg unmap memory +mprotect\(dg change protection of pages +madvise\(dg give memory management advice +mincore\(dg determine core residency of pages +msleep\(dg sleep on a lock +mwakeup\(dg wakeup process sleeping on a lock +.TE +.FS +\(dg Not supported in 4.3BSD. +.FE +.in -5 +.h 1.3 "Signals +.in +5 +.TS +lw(1.6i) aw(3i). +<signal.h> signal definitions +sigvec set handler for signal +kill send signal to process +killpgrp send signal to process group +sigblock block set of signals +sigsetmask restore set of blocked signals +sigpause wait for signals +sigstack set software stack for signals +.TE +.in -5 +.h 1.4 "Timing and statistics +.in +5 +.TS +lw(1.6i) aw(3i). +<sys/time.h> time-related definitions +gettimeofday get current time and timezone +settimeofday set current time and timezone +getitimer read an interval timer +setitimer get and set an interval timer +profil profile process +.TE +.in -5 +.h 1.5 "Descriptors +.in +5 +.TS +lw(1.6i) aw(3i). +getdtablesize descriptor reference table size +dup duplicate descriptor +dup2 duplicate to specified index +close close descriptor +select multiplex input/output +fcntl control descriptor options +wrap\(dg wrap descriptor with protocol +.TE +.FS +\(dg Not supported in 4.3BSD. +.FE +.in -5 +.h 1.6 "Resource controls +.in +5 +.TS +lw(1.6i) aw(3i). +<sys/resource.h> resource-related definitions +getpriority get process priority +setpriority set process priority +getrusage get resource usage +getrlimit get resource limitations +setrlimit set resource limitations +.TE +.in -5 +.h 1.7 "System operation support +.in +5 +.TS +lw(1.6i) aw(3i). +mount mount a device file system +swapon add a swap device +umount umount a file system +sync flush system caches +reboot reboot a machine +acct specify accounting file +.TE +.in -5 +.NH +System facilities +.LP +.h 2.1 "Generic operations +.in +5 +.TS +lw(1.6i) aw(3i). +read read data +write write data +<sys/uio.h> scatter-gather related definitions +readv scattered data input +writev gathered data output +<sys/ioctl.h> standard control operations +ioctl device control operation +.TE +.in -5 +.h 2.2 "File system +.PP +Operations marked with a * exist in two forms: as shown, +operating on a file name, and operating on a file descriptor, +when the name is preceded with a ``f''. +.in +5 +.TS +lw(1.6i) aw(3i). +<sys/file.h> file system definitions +chdir change directory +chroot change root directory +mkdir make a directory +rmdir remove a directory +open open a new or existing file +mknod make a special file +portal\(dg make a portal entry +unlink remove a link +stat* return status for a file +lstat returned status of link +chown* change owner +chmod* change mode +utimes change access/modify times +link make a hard link +symlink make a symbolic link +readlink read contents of symbolic link +rename change name of file +lseek reposition within file +truncate* truncate file +access determine accessibility +flock lock a file +.TE +.in -5 +.h 2.3 "Communications +.in +5 +.TS +lw(1.6i) aw(3i). +<sys/socket.h> standard definitions +socket create socket +bind bind socket to name +getsockname get socket name +listen allow queuing of connections +accept accept a connection +connect connect to peer socket +socketpair create pair of connected sockets +sendto send data to named socket +send send data to connected socket +recvfrom receive data on unconnected socket +recv receive data on connected socket +sendmsg send gathered data and/or rights +recvmsg receive scattered data and/or rights +shutdown partially close full-duplex connection +getsockopt get socket option +setsockopt set socket option +.TE +.in -5 +.h 2.4 "Terminals, block and character devices +.in +5 +.in -5 +.h 2.5 "Processes and kernel hooks +.in +5 diff --git a/share/doc/psd/05.sysman/spell.ok b/share/doc/psd/05.sysman/spell.ok new file mode 100644 index 000000000000..b0cbd9ce9cf5 --- /dev/null +++ b/share/doc/psd/05.sysman/spell.ok @@ -0,0 +1,332 @@ +AF +ANON +AUTOBOOT +Behav +CLR +DEF +DGRAM +DONTNEED +Datagram +Datagrams +EINPROGRESS +EWOULDBLOCK +EXCL +FD +FSIZE +Fabry +GETFL +GETOWN +HASSEMAPHORE +HASSEMPHORE +IGN +INCR +INET +IP +IPC +ISSET +ITIMER +Karels +Leffler +MADV +MAXHOSTNAMELEN +MSG +Manual''PS1:6 +McKusick +Mclear +Mset +NB +NDELAY +NGROUPS +NLIMITS +NOEXTEND +NS +OOB +PGRP +PRIO +PROT +PS1:6 +RB +RDM +RDONLY +RDWR +RH +RLIM +RLIMIT +RSS +RUSAGE +SEQPACKET +SETFL +SETOWN +SIG +SIGALRM +SIGBUS +SIGCHLD +SIGCONT +SIGEMT +SIGFPE +SIGHUP +SIGILL +SIGINT +SIGIO +SIGIOT +SIGKILL +SIGPROF +SIGQUIT +SIGSEGV +SIGSTOP +SIGTERM +SIGTRAP +SIGTSTP +SIGTTIN +SIGTTOU +SIGURG +SIGUSR1 +SIGUSR2 +SIGVTALRM +SIGXCPU +SIGXFSZ +Sem +Sv +TCP +TRUNC +UDP +VAX +WILLNEED +WRONLY +XTND +accessor +accrights +accrightslen +addr +anamelen +arg +argv +arusage +astatus +behav +blkdev +brkc +bu +buf +buflen +bufsize +caddr +cbreak +chroot +cmd +datagram +datagrams +dev +dopt +dprop +ds +dst +dsttime +dsuspc +dtype +dup2 +egid +envp +eofc +erasec +errno +euid +fchmod +fchown +fcntl +fd +fdset +file.h +filename +filesystem +flushc +fromlenaddr +fs +fstat +ftruncate +getdtablesize +getegid +geteuid +getgid +getgroups +gethostid +gethostname +getitimer +getpagesize +getpeername +getpriority +getprotobyname +getrlimit +getrusage +getsockname +getsockopt +gettimeofday +gid +gidset +gidsetsize +hostid +idrss +iflags +inblock +incr +intrc +ioctl.h +iov +iovec +iovlen +ispeed +isrss +itimerval +ixrss +kbytes +killc +killpgrp +len +linefeed +lnextc +lstat +maddr +madvise +majflt +maxrss +mclear +mincore +minflt +minuteswest +mman.h +mmap +mprotect +mremap +mset +msg +msghdr +msglen +msgrcv +msgsnd +msleep +msync +munmap +mwakeup +namelen +nbytes +nd +nds +newname +ngroups +nivcsw +nl +nsignals +nswap +nvcsw +oflag +oflags +oldmask +oldname +oldoffset +onstack +optlen +optname +optval +or'ed +or'ing +ospeed +oss +osv +oublock +ovalue +pagesize +param +param.h +path1 +path2 +pathname +pathnames +pgrp +pid +pos +prio +prot +proto +pv +quitc +quota.h +readlink +readv +reboot.h +recv +recvfrom +recvmsg +resource.h +rgid +rlim +rlimit +rlp +ronly +rprntc +ru +ruid +rusage +sbrk +scp +sem +sendmsg +sendto +setgroups +sethostid +sethostname +setitimer +setpriority +setquota +setregid +setreuid +setrlimit +setsockopt +settimeofday +sigblock +sigcontext +sigmask +signal.h +signo +sigpause +sigsetmask +sigstack +sigvec +sockaddr +socket.h +socketpair +socktype +sp +ss +sstk +startc +stat.h +stb +stopc +suspc +sv +sw +symlink +ta +time.h +timeval +timezone +tolen +tt +ttychars +ttymode +tv +tvp +tvsec +types.h +tz +tzp +uid +uio.h +umount +usec +vec +wait.h +waitstatus +werasc +writeable +writev diff --git a/share/doc/psd/06.Clang/Clang.ms b/share/doc/psd/06.Clang/Clang.ms new file mode 100644 index 000000000000..639591349e82 --- /dev/null +++ b/share/doc/psd/06.Clang/Clang.ms @@ -0,0 +1,4575 @@ +.\" Copyright (C) Caldera International Inc. 2001-2002. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions are +.\" met: +.\" +.\" Redistributions of source code and documentation must retain the above +.\" copyright notice, this list of conditions and the following +.\" disclaimer. +.\" +.\" Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" +.\" All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" +.\" This product includes software developed or owned by Caldera +.\" International, Inc. Neither the name of Caldera International, Inc. +.\" nor the names of other contributors may be used to endorse or promote +.\" products derived from this software without specific prior written +.\" permission. +.\" +.\" USE OF THE SOFTWARE PROVIDED FOR UNDER THIS LICENSE BY CALDERA +.\" INTERNATIONAL, INC. AND CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR +.\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +.\" WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +.\" DISCLAIMED. IN NO EVENT SHALL CALDERA INTERNATIONAL, INC. BE LIABLE +.\" FOR ANY DIRECT, INDIRECT INCIDENTAL, SPECIAL, EXEMPLARY, OR +.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR +.\" BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, +.\" WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE +.\" OR OTHERWISE) RISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN +.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +.\" +.\" @(#)Clang.ms 8.1 (Berkeley) 6/8/93 +.\" +.\" $FreeBSD$ +.nr Cl 2 +.TL +The C Programming Language - Reference Manual +.AU +Dennis M. Ritchie +.AI +AT&T Bell Laboratories +Murray Hill, NJ 07974 +.PP +This manual is a reprint, with updates to the current C standard, from +\fIThe C Programming Language\fR, +by Brian W. Kernighan and Dennis M. Ritchie, Prentice-Hall, Inc., 1978. +.PP +\fBThis document is of historical interest only. Do not use it as a reference +for modern implementations of C.\fP +.EH 'PSD:6-%''The C Programming Language - Reference Manual' +.OH 'The C Programming Language - Reference Manual''PSD:6-%' +.NH 1 +Introduction +.PP +This manual describes the C language on the DEC PDP-11\(dg, the DEC VAX-11, +.FS +.LP +\(dg DEC PDP-11, and DEC VAX-11 are trademarks of Digital Equipment Corporation. +.LP +\(dd 3B 20 is a trademark of AT&T. +.FE +and the AT&T 3B 20\(dd. +Where differences exist, it concentrates on the VAX, but tries to point +out implementation-dependent details. With few exceptions, these dependencies +follow directly from the underlying properties of the hardware; the various +compilers are generally quite compatible. +.NH 1 +Lexical Conventions +.PP +There are six classes of tokens\ -\ +identifiers, keywords, constants, strings, operators, and other separators. +Blanks, tabs, new\(hylines, +and comments (collectively, ``white space'') as described below +are ignored except as they serve to separate +tokens. +Some white space is required to separate +otherwise adjacent identifiers, +keywords, and constants. +.PP +If the input stream has been parsed into tokens +up to a given character, the next token is taken +to include the longest string of characters +which could possibly constitute a token. +.NH 2 +Comments +.PP +The characters +.B +/* +.R +introduce a comment which terminates +with the characters +\fB\(**/\fR. +Comments do not nest. +.NH 2 +Identifiers (Names) +.PP +An identifier is a sequence of letters and digits. +The first character must be a letter. +The underscore +(\fB_\fR) +counts as a letter. +Uppercase and lowercase letters +are different. +Although there is no limit on the length of a name, +only initial characters are significant: at least +eight characters of a non-external name, and perhaps +fewer for external names. +Moreover, some implementations may collapse case +distinctions for external names. +The external name sizes include: +.DS +.TS +l l. +PDP-11 7 characters, 2 cases +VAX-11 >100 characters, 2 cases +AT&T 3B 20 >100 characters, 2 cases +.TE +.fi +.DE +.NH 2 +Keywords +.PP +The following identifiers are reserved for use +as keywords and may not be used otherwise: +.DS +.ta 0.8i 1.6i 2.4i 3.2i 4.0i +\fBauto do for return typedef +break double goto short union +case else if sizeof unsigned +char enum int static void +continue external long struct while +default float register switch\fR +.ta 0.5i +.DE +.PP +Some implementations also reserve the words +.B +fortran, asm, gfloat, hfloat +.R +and +.B quad +.R +.NH 2 +Constants +.PP +There are several kinds +of constants. +Each has a type; an introduction to types is given in ``NAMES.'' +Hardware characteristics that affect sizes are summarized in +``Hardware Characteristics'' under ``LEXICAL CONVENTIONS.'' +.NH 3 +Integer Constants +.br +.PP +An integer constant consisting of a sequence of digits +is taken +to be octal if it begins with +.B +0 +.R +(digit zero). +An octal constant consists of the digits \fB0\fR through \fB7\fR only. +A sequence of digits preceded by +.B +0x +.R +or +.B +0X +.R +(digit zero) is taken to be a hexadecimal integer. +The hexadecimal digits include +.B +a +.R +or +.B +A +.R +through +.B +f +.R +or +.B +F +.R +with values 10 through 15. +Otherwise, the integer constant is taken to be decimal. +A decimal constant whose value exceeds the largest +signed machine integer is taken to be +\fBlong\fR; +an octal or hex constant which exceeds the largest unsigned machine integer +is likewise taken to be +.B +long\fR. +.R +Otherwise, integer constants are \fBint\fR. +.NH 3 +Explicit Long Constants +.br +.PP +A decimal, octal, or hexadecimal integer constant immediately followed +by +.B +l +.R +(letter ell) +or +.B +L +.R +is a long constant. +As discussed below, +on some machines +integer and long values may be considered identical. +.NH 3 +Character Constants +.br +.PP +A character constant is a character enclosed in single quotes, +as in '\fBx\fR'. +The value of a character constant is the numerical value of the +character in the machine's character set. +.PP +Certain nongraphic characters, +the single quote +(\fB'\fR) +and the backslash +(\fB\e\fR), +may be represented according to the following table +of escape sequences: +.DS +.TS +l l l. +new\(hyline NL (LF) \en +horizontal tab HT \et +vertical tab VT \ev +backspace BS \eb +carriage return CR \er +form feed FF \ef +backslash \e \e\e +single quote ' \e' +bit pattern \fIddd\fR\^ \e\fIddd\fR\^ +.TE +.DE +.PP +The escape +\e\fIddd\fR +consists of the backslash followed by 1, 2, or 3 octal digits +which are taken to specify the value of the +desired character. +A special case of this construction is +.B +\e0 +.R +(not followed +by a digit), which indicates the character +.B +NUL\fR. +.R +If the character following a backslash is not one +of those specified, the +behavior is undefined. +A new-line character is illegal in a character constant. +The type of a character constant is \fBint\fR. +.NH 3 +Floating Constants +.br +.PP +A floating constant consists of +an integer part, a decimal point, a fraction part, +an +.B +e +.R +or +\fBE\fR, +and an optionally signed integer exponent. +The integer and fraction parts both consist of a sequence +of digits. +Either the integer part or the fraction +part (not both) may be missing. +Either the decimal point or +the +.B +e +.R +and the exponent (not both) may be missing. +Every floating constant has type \fBdouble\fR. +.NH 3 +Enumeration Constants +.br +.PP +Names declared as enumerators +(see ``Structure, Union, and Enumeration Declarations'' under +``DECLARATIONS'') +have type \fBint\fR. +.NH 2 +Strings +.PP +A string is a sequence of characters surrounded by +double quotes, +as in +\fB"..."\fR. +A string has type +``array of \fBchar\fR'' and storage class +\fBstatic\fR +(see ``NAMES'') +and is initialized with +the given characters. +The compiler places +a null byte +(\fB\e0\fR) +at the end of each string so that programs +which scan the string can +find its end. +In a string, the double quote character +(\fB"\fR) +must be preceded by +a +\fB\e\fR; +in addition, the same escapes as described for character +constants may be used. +.PP +A +.B +\e +.R +and +the immediately following new\(hyline are ignored. +All strings, even when written identically, are distinct. +.NH 2 +Hardware Characteristics +.PP +The following figure summarize +certain hardware properties that vary from machine to machine. +.DS +.TS +center box; +c cfB s cfB s cfB s +c c s c s c s +l | l1 lp8 | l1 lp8 | l1 lp8. + DEC PDP\-11 DEC VAX-11 AT&T 3B + (ASCII) (ASCII) (ASCII) +.sp +_ +char 8 bits 8 bits 8bits +int 16 32 32 +short 16 16 16 +long 32 32 32 +float 32 32 32 +double 64 64 64 +float range \(+-10 \(+-38 \(+-10 \(+-38 \(+-10 \(+-38 +\^ \^ \^ \^ +double range \(+-10 \(+-38 \(+-10 \(+-38 \(+-10 \(+-308 +\^ \^ \^ \^ +.TE +.\" .FG 4 4 1 "DEC PDP-11 HARDWARE CHARACTERISTICS" +.DE +.PP +.NH 1 +Syntax Notation +.PP +Syntactic categories are indicated by +.I +italic +.R +type +and literal words and characters +in +\fBbold\fR +type. +Alternative categories are listed on separate lines. +An optional terminal or nonterminal symbol is +indicated by the subscript ``opt,'' so that +.DS +{ \fIexpression\v'0.5'\s-2opt\s0\v'-0.5'\fR } +.DE +.LP +indicates an optional expression enclosed in braces. +The syntax is summarized in ``SYNTAX SUMMARY''. +.NH 1 +Names +.PP +The C language bases the interpretation of an +identifier upon two attributes of the identifier \(mi its +.I +storage class +.R +and its +.I +type\fR. +The storage class determines the location and lifetime +of the storage associated with an identifier; +the type determines +the meaning of the values +found in the identifier's storage. +.NH 2 +Storage Class +.PP +.\" The original text had borrowed BL, LI and LE from the mm macros. +.\" That way madness lies. +There are four declarable storage classes: +.RS +.br +\(bu Automatic +.br +\(bu Static +.br +\(bu External +.br +\(bu Register. +.RE +.PP +Automatic variables are local to each invocation of +a block (see ``Compound Statement or Block'' in +``STATEMENTS'') and are discarded upon exit from the block. +Static variables are local to a block but retain +their values upon reentry to a block even after control +has left the block. +External variables exist and retain their values throughout +the execution of the entire program and +may be used for communication between +functions, even separately compiled functions. +Register variables are (if possible) stored in the fast registers +of the machine; like automatic +variables, they are local to each block and disappear on exit from the block. +.NH 2 +Type +.PP +The C language supports several +fundamental +types of objects. +Objects declared as characters +(\fBchar\fR) +are large enough to store any member of the implementation's +character set. +If a genuine character from that character set is +stored in a \fBchar\fR variable, +its value is equivalent to the integer code for that character. +Other quantities may be stored into character variables, but +the implementation is machine dependent. +In particular, \fBchar\fR may be signed or unsigned by default. +.PP +Up to three sizes of integer, declared +.B +short +.R +\fBint\fR, +\fBint\fR, +and +.B +long +.R +\fBint\fR, +are available. +Longer integers provide no less storage than shorter ones, +but the implementation may make either short integers or long integers, +or both, equivalent to plain integers. +``Plain'' integers have the natural size suggested +by the host machine architecture. +The other sizes are provided to meet special needs. +.PP +The properties of \fBenum\fR types (see ``Structure, Union, and Enumeration Declarations'' +under ``DECLARATIONS'') +are identical to those of +some integer types. +The implementation may use the range of values to +determine how to allocate storage. +.PP +Unsigned +integers, declared +.B +unsigned, +.R +obey the laws of arithmetic modulo +2\v'-0.5'\fIn\fR\v'0.5' +where \fIn\fR is the number of bits in the representation. +(On the +PDP-11, +unsigned long quantities are not supported.) +.PP +Single-precision floating point +(\fBfloat\fR) +and double precision floating point +(\fBdouble\fR) +may be synonymous in some implementations. +.PP +Because objects of the foregoing types can usefully be interpreted +as numbers, they will be referred to as +.I +arithmetic +.R +types. +\fBChar\fR, +.B +int +.R +of all sizes whether \fBunsigned\fR or not, and +.B +enum +.R +will collectively be called +.I +integral +.R +types. +The +.B +float +.R +and +.B +double +.R +types will collectively be called +.I +floating +.R +types. +.PP +The +.B +void +.R +type +specifies an empty set of values. +It is used as the type returned by functions that +generate no value. +.PP +Besides the fundamental arithmetic types, there is a +conceptually infinite class of derived types constructed +from the fundamental types in the following ways: +.IP \fIArrays\fR +of objects of most types +.IP \fIFunctions\fR +which return objects of a given type +.IP \fIPointers\fR +to objects of a given type +.IP \fIStructures\fR +containing a sequence of objects of various types +.IP \fIUnions\fR +capable of containing any one of several objects of various types. +.LP +In general these methods +of constructing objects can +be applied recursively. +.NH 1 +Objects and Lvalues +.PP +An +.I +object +.R +is a manipulatable region of storage. +An +.I +lvalue +.R +is an expression referring to an object. +An obvious example of an lvalue +expression is an identifier. +There are operators which yield lvalues: +for example, +if +.B +E +.R +is an expression of pointer type, then +.B +\(**E +.R +is an lvalue +expression referring to the object to which +.B +E +.R +points. +The name ``lvalue'' comes from the assignment expression +.B +E1\ =\ E2 +.R +in which the left operand +.B +E1 +.R +must be +an lvalue expression. +The discussion of each operator +below indicates whether it expects lvalue operands and whether it +yields an lvalue. +.NH 1 +Conversions +.PP +A number of operators may, depending on their operands, +cause conversion of the value of an operand from one type to another. +This part explains the result to be expected from such +conversions. +The conversions demanded by most ordinary operators are summarized under +``Arithmetic Conversions.'' +The summary will be supplemented +as required by the discussion +of each operator. +.NH 2 +Characters and Integers +.PP +A character or a short integer may be used wherever an +integer may be used. +In all cases +the value is converted to an integer. +Conversion of a shorter integer +to a longer preserves sign. +Whether or not sign-extension occurs for characters is machine +dependent, but it is guaranteed that a member of the +standard character set is non-negative. +Of the machines treated here, +only the +PDP-11 +and +VAX-11 +sign-extend. +On these machines, +.B +char +.R +variables range in value from +\(mi128 to 127. +The more explicit type +.B +unsigned +.R +.B +char +.R +forces the values to range from 0 to 255. +.PP +On machines that treat characters as signed, +the characters of the +ASCII +set are all non-negative. +However, a character constant specified +with an octal escape suffers sign extension +and may appear negative; +for example, +\fB\'\e377\'\fR +\fRhas the value +.B +\(mi1\fR. +.PP +When a longer integer is converted to a shorter +integer +or to a +.B +char, +.R +it is truncated on the left. +Excess bits are simply discarded. +.NH 2 +Float and Double +.PP +All floating arithmetic in C is carried out in double precision. +Whenever a +.B +float +.R +appears in an expression it is lengthened to +.B +double +.R +by zero padding its fraction. +When a +.B +double +.R +must be +converted to +\fBfloat\fR, +for example by an assignment, +the +.B +double +.R +is rounded before +truncation to +.B +float +.R +length. +This result is undefined if it cannot be represented as a float. +On the VAX, the compiler can be directed to use single precision for expressions +containing only float and integer operands. +.NH 2 +Floating and Integral +.PP +Conversions of floating values to integral type +are rather machine dependent. +In particular, the direction of truncation of negative numbers +varies. +The result is undefined if +it will not fit in the space provided. +.PP +Conversions of integral values to floating type +are well behaved. +Some loss of accuracy occurs +if the destination lacks sufficient bits. +.NH 2 +Pointers and Integers +.PP +An expression of integral type may be added to or subtracted from +a pointer; in such a case, +the first is converted as +specified in the discussion of the addition operator. +Two pointers to objects of the same type may be subtracted; +in this case, the result is converted to an integer +as specified in the discussion of the subtraction +operator. +.NH 2 +Unsigned +.PP +Whenever an unsigned integer and a plain integer +are combined, the plain integer is converted to unsigned +and the result is unsigned. +The value +is the least unsigned integer congruent to the signed +integer (modulo 2\v'-0.3'\s-2wordsize\s+2\v'0.3'). +In a 2's complement representation, +this conversion is conceptual; and there is no actual change in the +bit pattern. +.PP +When an unsigned \fBshort\fR integer is converted to +\fBlong\fR, +the value of the result is the same numerically as that of the +unsigned integer. +Thus the conversion amounts to padding with zeros on the left. +.NH 2 +Arithmetic Conversions +.PP +A great many operators cause conversions +and yield result types in a similar way. +This pattern will be called the ``usual arithmetic conversions.'' +.IP 1. +First, any operands of type +.B +char +.R +or +.B +short +.R +are converted to +\fBint\fR, +and any operands of type \fBunsigned char\fR +or \fBunsigned short\fR are converted +to \fBunsigned int\fR. +.IP 2. +Then, if either operand is +.B +double, +.R +the other is converted to +.B +double +.R +and that is the type of the result. +.IP 3. +Otherwise, if either operand is \fBunsigned long\fR, +the other is converted to \fBunsigned long\fR and that +is the type of the result. +.IP 4. +Otherwise, if either operand is +\fBlong\fR, +the other is converted to +.B +long +.R +and that is the type of the result. +.IP 5. +Otherwise, if one operand is \fBlong\fR, and +the other is \fBunsigned int\fR, they are both +converted to \fBunsigned long\fR and that is +the type of the result. +.IP 6. +Otherwise, if either operand is +.B +unsigned, +.R +the other is converted to +.B +unsigned +.R +and that is the type of the result. +.IP 7. +Otherwise, both operands must be +\fBint\fR, +and that is the type of the result. +.LP +.NH 2 +Void +.PP +The (nonexistent) value of a +.B +void +.R +object may not be used in any way, +and neither explicit nor implicit conversion may be applied. +Because a void expression denotes a nonexistent value, +such an expression may be used only +as an expression statement +(see ``Expression Statement'' under ``STATEMENTS'') +or as the left operand +of a comma expression (see ``Comma Operator'' under ``EXPRESSIONS''). +.PP +An expression may be converted to +type +.B +void +.R +by use of a cast. +For example, this makes explicit the discarding of the value +of a function call used as an expression statement. +.NH 1 +Expressions +.PP +The precedence of expression operators is the same +as the order of the major +subsections of this section, highest precedence first. +Thus, for example, the expressions referred to as the operands of +.B +\(pl +.R +(see ``Additive Operators'') +are those expressions defined under ``Primary Expressions'', +``Unary Operators'', and ``Multiplicative Operators''. +Within each subpart, the operators have the same +precedence. +Left- or right-associativity is specified +in each subsection for the operators +discussed therein. +The precedence and associativity of all the expression +operators are summarized in the +grammar of ``SYNTAX SUMMARY''. +.PP +Otherwise, the order of evaluation of expressions +is undefined. In particular, the compiler +considers itself free to +compute subexpressions in the order it believes +most efficient +even if the subexpressions +involve side effects. +The order in which subexpression evaluation takes place is unspecified. +Expressions involving a commutative and associative +operator +(\fB\(**,\fR +\fB\(pl\fR, +\fB&\fR, +\fB|\fR, +\fB^\fR) +may be rearranged arbitrarily even in the presence +of parentheses; +to force a particular order of evaluation, +an explicit temporary must be used. +.PP +The handling of overflow and divide check +in expression evaluation +is undefined. +Most existing implementations of C ignore integer overflows; +treatment of +division by 0 and all floating-point exceptions +varies between machines and is usually +adjustable by a library function. +.NH 2 +Primary Expressions +.PP +Primary expressions +involving \fB\.\fR, +\fB\(mi>\fR, +subscripting, and function calls +group left to right. +.DS +\fIprimary-expression: + identifier + constant + string + ( expression ) + primary-expression [ expression ] + primary-expression ( expression-list\v'0.5'\s-2opt\s0\v'-0.5' ) + primary-expression . identifier + primary-expression \(mi> identifier\fR +.DE +.DS +\fIexpression-list: + expression + expression-list , expression\fR +.DE +.PP +An identifier is a primary expression provided it has been +suitably declared as discussed below. +Its type is specified by its declaration. +If the type of the identifier is ``array of .\|.\|.'', +then the value of the identifier expression +is a pointer +to the first object in the array; and the +type of the expression is +``pointer to .\|.\|.''. +Moreover, an array identifier is not an lvalue +expression. +Likewise, an identifier which is declared +``function returning .\|.\|.'', +when used except in the function-name position +of a call, is converted to ``pointer to function returning .\|.\|.''. +.PP +A +constant is a primary expression. +Its type may be +\fBint\fR, +\fBlong\fR, +or +.B +double +.R +depending on its form. +Character constants have type +.B +int +.R +and floating constants have type +.B +double\fR. +.R +.PP +A string is a primary expression. +Its type is originally ``array of +\fBchar\fR'', +but following +the same rule given above for identifiers, +this is modified to ``pointer to +\fBchar\fR'' and +the +result is a pointer to the first character +in the string. +(There is an exception in certain initializers; +see ``Initialization'' under ``DECLARATIONS.'') +.PP +A parenthesized expression is a primary expression +whose type and value are identical +to those of the unadorned expression. +The presence of parentheses does +not affect whether the expression is an +lvalue. +.PP +A primary expression followed by an expression in square +brackets is a primary expression. +The intuitive meaning is that of a subscript. +Usually, the primary expression has type ``pointer to .\|.\|.'', +the subscript expression is +\fBint\fR, +and the type of the result is ``\|.\|.\|.\|''. +The expression +.B +E1[E2] +.R +is +identical (by definition) to +.B +\(**((E1)\(plE2))\fR. +All the clues +needed to understand +this notation are contained in this subpart together +with the discussions +in ``Unary Operators'' and ``Additive Operators'' on identifiers, +.B +\(** +.R +and +.B +\(pl +.R +respectively. +The implications are summarized under ``Arrays, Pointers, and Subscripting'' +under ``TYPES REVISITED.'' +.PP +A function call is a primary expression followed by parentheses +containing a possibly +empty, comma-separated list of expressions +which constitute the actual arguments to the +function. +The primary expression must be of type ``function returning .\|.\|.,'' +and the result of the function call is of type ``\|.\|.\|.\|''. +As indicated +below, a hitherto unseen identifier followed +immediately by a left parenthesis +is contextually declared +to represent a function returning +an integer; +thus in the most common case, integer-valued functions +need not be declared. +.PP +Any actual arguments of type +.B +float +.R +are +converted to +.B +double +.R +before the call. +Any of type +.B +char +.R +or +.B +short +.R +are converted to +.B +int\fR. +.R +Array names are converted to pointers. +No other conversions are performed automatically; +in particular, the compiler does not compare +the types of actual arguments with those of formal +arguments. +If conversion is needed, use a cast; +see ``Unary Operators'' and ``Type Names'' under +``DECLARATIONS.'' +.PP +In preparing for the call to a function, +a copy is made of each actual parameter. +Thus, all argument passing in C is strictly by value. +A function may +change the values of its formal parameters, but +these changes cannot affect the values +of the actual parameters. +It is possible +to pass a pointer on the understanding +that the function may change the value +of the object to which the pointer points. +An array name is a pointer expression. +The order of evaluation of arguments is undefined by the language; +take note that the various compilers differ. +Recursive calls to any +function are permitted. +.PP +A primary expression followed by a dot followed by an identifier +is an expression. +The first expression must be a structure or a union, and the identifier +must name a member of the structure or union. +The value is the named member of the structure or union, and it is +an lvalue if the first expression is an lvalue. +.PP +A primary expression followed by an arrow (built from +.B +\(mi +.R +and +.B +> +.R +) +followed by an identifier +is an expression. +The first expression must be a pointer to a structure or a union +and the identifier must name a member of that structure or union. +The result is an lvalue referring to the named member +of the structure or union +to which the pointer expression points. +Thus the expression +.B +E1\(mi>MOS +.R +is the same as +.B +(\(**E1).MOS\fR. +.R +Structures and unions are discussed in +``Structure, Union, and Enumeration Declarations'' under +``DECLARATIONS.'' +.NH 2 +Unary Operators +.PP +Expressions with unary operators +group right to left. +.tr ~~ +.DS +\fIunary-expression: + \(** expression + & lvalue + \(mi expression + ! expression + \s+2~\s0 expression + \(pl\(pl lvalue + \(mi\(milvalue + lvalue \(pl\(pl + lvalue \(mi\(mi + ( type-name ) expression\fR + sizeof\fI expression\fR + sizeof\fI ( type-name )\fR +.DE +.PP +The unary +.B +\(** +.R +operator +means +.I +indirection +.R +; +the expression must be a pointer, and the result +is an lvalue referring to the object to +which the expression points. +If the type of the expression is ``pointer to .\|.\|.,'' +the type of the result is ``\|.\|.\|.\|''. +.PP +The result of the unary +.B +& +.R +operator is a pointer +to the object referred to by the +lvalue. +If the type of the lvalue is ``\|.\|.\|.\|'', +the type of the result is ``pointer to .\|.\|.''. +.PP +The result +of the unary +.B +\(mi +.R +operator +is the negative of its operand. +The usual arithmetic conversions are performed. +The negative of an unsigned quantity is computed by +subtracting its value from +2\v'-0.5'\fIn\fR\^\v'0.5' where \fIn\fR\^ is the number of bits in +the corresponding signed type. +.sp +.tr ~~ +There is no unary +.B +\(pl +.R +operator. +.PP +The result of the logical negation operator +.B +! +.R +is one if the value of its operand is zero, zero if the value of its +operand is nonzero. +The type of the result is +.B +int\fR. +.R +It is applicable to any arithmetic type +or to pointers. +.PP +The +.B +\s+2~\s0 +.R +operator yields the one's complement of its operand. +The usual arithmetic conversions are performed. +The type of the operand must be integral. +.PP +The object referred to by the lvalue operand of prefix +.B +\(pl\(pl +.R +is incremented. +The value is the new value of the operand +but is not an lvalue. +The expression +.B +\(pl\(plx +.R +is equivalent to +\fBx=x\(pl1\fR. +See the discussions ``Additive Operators'' and ``Assignment +Operators'' for information on conversions. +.PP +The lvalue operand of prefix +.B +\(mi\(mi +.R +is decremented +analogously to the +prefix +.B +\(pl\(pl +.R +operator. +.PP +When postfix +.B +\(pl\(pl +.R +is applied to an lvalue, +the result is the value of the object referred to by the lvalue. +After the result is noted, the object +is incremented in the same +manner as for the prefix +.B +\(pl\(pl +.R +operator. +The type of the result is the same as the type of the lvalue expression. +.PP +When postfix +.B +\(mi\(mi +.R +is applied to an lvalue, +the result is the value of the object referred to by the lvalue. +After the result is noted, the object +is decremented in the manner as for the prefix +.B +\(mi\(mi +.R +operator. +The type of the result is the same as the type of the lvalue +expression. +.PP +An expression preceded by the parenthesized name of a data type +causes conversion of the value of the expression to the named type. +This construction is called a +.I +cast\fR. +.R +Type names are described in ``Type Names'' under ``Declarations.'' +.PP +The +.B +sizeof +.R +operator yields the size +in bytes of its operand. +(A +.I +byte +.R +is undefined by the language +except in terms of the value of +.B +sizeof\fR. +.R +However, in all existing implementations, +a byte is the space required to hold a +\fBchar.\fR) +When applied to an array, the result is the total +number of bytes in the array. +The size is determined from +the declarations of +the objects in the expression. +This expression is semantically an +.B +unsigned +.R +constant and may +be used anywhere a constant is required. +Its major use is in communication with routines +like storage allocators and I/O systems. +.PP +The +.B +sizeof +.R +operator +may also be applied to a parenthesized type name. +In that case it yields the size in bytes of an object +of the indicated type. +.PP +The construction +\fBsizeof(\fItype\|\fR\^)\fR\^ +is taken to be a unit, +so the expression +\fBsizeof(\fItype\|\fB)-2\fR +is the same as +\fB(sizeof(\fItype\|\fB))-2\fR. +.NH 2 +Multiplicative Operators +.PP +The multiplicative operators +\fB\(**\fR, +\fB/\fR, +and +.B +% +.R +group left to right. +The usual arithmetic conversions are performed. +.DS +\fImultiplicative expression: + expression \(** expression + expression / expression + expression % expression\fR +.DE +.PP +The binary +.B +\(** +.R +operator indicates multiplication. +The +.B +\(** +.R +operator is associative, +and expressions with several multiplications at the same +level may be rearranged by the compiler. +The binary +.B +/ +.R +operator indicates division. +.PP +The binary +.B +% +.R +operator yields the remainder +from the division of the first expression by the second. +The operands must be integral. +.PP +When positive integers are divided, truncation is toward 0; +but the form of truncation is machine-dependent +if either operand is negative. +On all machines covered by this manual, +the remainder has the same sign as the dividend. +It is always true that +.B +(a/b)\(**b\ \(pl a%b +.R +is equal to +.B +a +.R +(if +.B +b +.R +is not 0). +.NH 2 +Additive Operators +.PP +The additive operators +.B +\(pl +.R +and +.B +\(mi +.R +group left to right. +The usual arithmetic conversions are performed. +There are some additional type possibilities for each operator. +.DS +\fIadditive-expression: + expression \(pl expression + expression \(mi expression\fR +.DE +.PP +The result of the +.B +\(pl +.R +operator is the sum of the operands. +A pointer to an object in an array and +a value of any integral type +may be added. +The latter is in all cases converted to +an address offset +by multiplying it +by the length of the object to which the +pointer points. +The result is a pointer +of the same type as the original pointer +which points to another object in the same array, +appropriately offset from the original object. +Thus if +.B +P +.R +is a pointer +to an object in an array, the expression +.B +P\(pl1 +.R +is a pointer +to the next object in the array. +No further type combinations are allowed for pointers. +.PP +The +.B +\(pl +.R +operator is associative, +and expressions with several additions at the same level may +be rearranged by the compiler. +.PP +The result of the +.B +\(mi +.R +operator is the difference of the operands. +The usual arithmetic conversions are performed. +Additionally, +a value of any integral type +may be subtracted from a pointer, +and then the same conversions for addition apply. +.PP +If two pointers to objects of the same type are subtracted, +the result is converted +(by division by the length of the object) +to an +.B +int +.R +representing the number of +objects separating +the pointed-to objects. +This conversion will in general give unexpected +results unless the pointers point +to objects in the same array, since pointers, even +to objects of the same type, do not necessarily differ +by a multiple of the object length. +.NH 2 +Shift Operators +.PP +The shift operators +.B +<< +.R +and +.B +>> +.R +group left to right. +Both perform the usual arithmetic conversions on their operands, +each of which must be integral. +Then the right operand is converted to +\fBint\fR; +the type of the result is that of the left operand. +The result is undefined if the right operand is negative +or greater than or equal to the length of the object in bits. +On the VAX a negative right operand is interpreted as reversing +the direction of the shift. +.DS +\fIshift-expression: + expression << expression + expression >> expression\fR +.DE +.PP +The value of +.B +E1<<E2 +.R +is +.B +E1 +.R +(interpreted as a bit +pattern) left-shifted +.B +E2 +.R +bits. +Vacated bits are 0 filled. +The value of +.B +E1>>E2 +.R +is +.B +E1 +.R +right-shifted +.B +E2 +.R +bit positions. +The right shift is guaranteed to be logical +(0 fill) +if +.B +E1 +.R +is +\fBunsigned\fR; +otherwise, it may be +arithmetic. +.NH 2 +Relational Operators +.PP +The relational operators group left to right. +.DS +\fIrelational-expression: + expression < expression + expression > expression + expression <= expression + expression >= expression\fR +.DE +.PP +The operators +.B +< +.R +(less than), +.B +> +.R +(greater than), \fB<=\fR +(less than +or equal to), and +.B +>= +.R +(greater than or equal to) +all yield 0 if the specified relation is false +and 1 if it is true. +The type of the result is +.B +int\fR. +The usual arithmetic conversions are performed. +Two pointers may be compared; +the result depends on the relative locations in the address space +of the pointed-to objects. +Pointer comparison is portable only when the pointers point to objects +in the same array. +.NH 2 +Equality Operators +.PP +.DS +\fIequality-expression: + expression == expression + expression != expression\fR +.DE +.PP +The +.B +== +.R +(equal to) and the +.B +!= +.R +(not equal to) operators +are exactly analogous to the relational +operators except for their lower +precedence. +(Thus +.B +a<b\ ==\ c<d +.R +is 1 whenever +.B +a<b +.R +and +.B +c<d +.R +have the same truth value). +.PP +A pointer may be compared to an integer +only if the +integer is the constant 0. +A pointer to which 0 has been assigned is guaranteed +not to point to any object +and will appear to be equal to 0. +In conventional usage, such a pointer is considered to be null. +.NH 2 +Bitwise \s-1AND\s0 Operator +.PP +.DS +\fIand-expression: + expression & expression\fR +.DE +.PP +The +.B +& +.R +operator is associative, +and expressions involving +.B +& +.R +may be rearranged. +The usual arithmetic conversions are performed. +The result is the bitwise +AND +function of the operands. +The operator applies only to integral +operands. +.NH 2 +Bitwise Exclusive \s-1OR\s0 Operator +.DS +\fIexclusive-or-expression: + expression ^ expression\fR +.DE +.PP +The +.B +^ +.R +operator is associative, +and expressions involving +.B +^ +.R +may be rearranged. +The usual arithmetic conversions are performed; +the result is +the bitwise exclusive +OR +function of +the operands. +The operator applies only to integral +operands. +.NH 2 +Bitwise Inclusive \s-1OR\s0 Operator +.DS +\fIinclusive-or-expression: + expression | expression\fR +.DE +.PP +The +.B +| +.R +operator is associative, +and expressions involving +.B +| +.R +may be rearranged. +The usual arithmetic conversions are performed; +the result is the bitwise inclusive +OR +function of its operands. +The operator applies only to integral +operands. +.NH 2 +Logical \s-1AND\s0 Operator +.DS +\fIlogical-and-expression: + expression && expression\fR +.DE +.PP +The +.B +&& +.R +operator groups left to right. +It returns 1 if both its operands +evaluate to nonzero, 0 otherwise. +Unlike +\fB&\fR, +.B +&& +.R +guarantees left to right +evaluation; moreover, the second operand is not evaluated +if the first operand is 0. +.PP +The operands need not have the same type, but each +must have one of the fundamental +types or be a pointer. +The result is always +.B +int\fR. +.R +.NH 2 +Logical \s-1OR\s0 Operator +.DS +\fIlogical-or-expression: + expression || expression\fR +.DE +.PP +The +.B +|| +.R +operator groups left to right. +It returns 1 if either of its operands +evaluates to nonzero, 0 otherwise. +Unlike +\fB|\fR, +.B +|| +.R +guarantees left to right evaluation; moreover, +the second operand is not evaluated +if the value of the first operand is nonzero. +.PP +The operands need not have the same type, but each +must +have one of the fundamental types +or be a pointer. +The result is always +.B +int\fR. +.R +.NH 2 +Conditional Operator +.DS +\fIconditional-expression: + expression ? expression : expression\fR +.DE +.PP +Conditional expressions group right to left. +The first expression is evaluated; +and if it is nonzero, the result is the value of the +second expression, otherwise that of third expression. +If possible, the usual arithmetic conversions are performed +to bring the second and third expressions to a common type. +If both are structures or unions of the same type, +the result has the type of the structure or union. +If both pointers are of the same type, +the result has the common type. +Otherwise, one must be a pointer and the other the constant 0, +and the result has the type of the pointer. +Only one of the second and third +expressions is evaluated. +.NH 2 +Assignment Operators +.PP +There are a number of assignment operators, +all of which group right to left. +All require an lvalue as their left operand, +and the type of an assignment expression is that +of its left operand. +The value is the value stored in the +left operand after the assignment has taken place. +The two parts of a compound assignment operator are separate +tokens. +.DS +\fIassignment-expression: + lvalue = expression + lvalue \(pl= expression + lvalue \(mi= expression + lvalue \(**= expression + lvalue /= expression + lvalue %= expression + lvalue >>= expression + lvalue <<= expression + lvalue &= expression + lvalue ^= expression + lvalue |= expression\fR +.DE +.PP +In the simple assignment with +\fB=\fR, +the value of the expression replaces that of the object +referred +to by the lvalue. +If both operands have arithmetic type, +the right operand is converted to the type of the left +preparatory to the assignment. +Second, both operands may be structures or unions of the same type. +Finally, if the left operand is a pointer, the right operand must in general be a pointer +of the same type. +However, the constant 0 may be assigned to a pointer; +it is guaranteed that this value will produce a null +pointer distinguishable from a pointer to any object. +.PP +The behavior of an expression +of the form +\fBE1\fR\^ \fIop\fR\^ = \fBE2\fR\^ +may be inferred by +taking it as equivalent to +\fBE1 = E1 \fIop\fR\^ (\fBE2\fR\^); +however, +.B +E1 +.R +is evaluated only once. +In +.B +\(pl= +.R +and +\fB\(mi=\fR, +the left operand may be a pointer; in which case, the (integral) right +operand is converted as explained +in ``Additive Operators.'' +All right operands and all nonpointer left operands must +have arithmetic type. +.NH 2 +Comma Operator +.DS +\fIcomma-expression: + expression , expression\fR +.DE +.PP +A pair of expressions separated by a comma is evaluated +left to right, and the value of the left expression is +discarded. +The type and value of the result are the +type and value of the right operand. +This operator groups left to right. +In contexts where comma is given a special meaning, +e.g., in lists of actual arguments +to functions (see ``Primary Expressions'') and lists +of initializers (see ``Initialization'' under ``DECLARATIONS''), +the comma operator as described in this subpart +can only appear in parentheses. For example, +.DS +\fBf(a, (t=3, t\(pl2), c)\fR +.DE +.LP +has three arguments, the second of which has the value 5. +.NH 1 +Declarations +.PP +Declarations are used to specify the interpretation +which C gives to each identifier; they do not necessarily +reserve storage associated with the identifier. +Declarations have the form +.DS +\fIdeclaration: + decl-specifiers declarator-list\v'0.5'\s-2opt\s0\v'-0.5' ;\fR +.DE +.PP +The declarators in the declarator-list +contain the identifiers being declared. +The decl-specifiers +consist of a sequence of type and storage class specifiers. +.DS +\fIdecl-specifiers: + type-specifier decl-specifiers\v'0.5'\s-2opt\s0\v'-0.5' + sc-specifier decl-specifiers\v'0.5'\s-2opt\s0\v'-0.5'\fR +.DE +.PP +The list must be self-consistent in a way described below. +.NH 2 +Storage Class Specifiers +.PP +The sc-specifiers are: +.DS +\fIsc-specifier:\fB + auto + static + extern + register + typedef\fR +.DE +.PP +The +.B +typedef +.R +specifier does not reserve storage +and is called a ``storage class specifier'' only for syntactic convenience. +See ``Typedef'' for more information. +The meanings of the various storage classes were discussed in ``Names.'' +.PP +The +\fBauto\fR, +\fBstatic\fR, +and +.B +register +.R +declarations also serve as definitions +in that they cause an appropriate amount of storage to be reserved. +In the +.B +extern +.R +case, +there must be an external definition (see ``External Definitions'') +for the given identifiers +somewhere outside the function in which they are declared. +.PP +A +.B +register +.R +declaration is best thought of as an +.B +auto +.R +declaration, together with a hint to the compiler +that the variables declared will be heavily used. +Only the first few +such declarations in each function are effective. +Moreover, only variables of certain types will be stored in registers; +on the +PDP-11, +they are +.B +int +.R +or pointer. +One other restriction applies to register variables: +the address-of operator +.B +& +.R +cannot be applied to them. +Smaller, faster programs can be expected if register declarations +are used appropriately, +but future improvements in code generation +may render them unnecessary. +.PP +At most, one sc-specifier may be given in a declaration. +If the sc-specifier is missing from a declaration, it +is taken to be +.B +auto +.R +inside a function, +.B +extern +.R +outside. +Exception: +functions are never +automatic. +.NH 2 +Type Specifiers +.PP +The type-specifiers are +.DS +\fItype-specifier: + struct-or-union-specifier + typedef-name + enum-specifier +basic-type-specifier: + basic-type + basic-type basic-type-specifiers +basic-type:\fB + char + short + int + long + unsigned + float + double + void\fR +.DE +.PP +At most one of the words \fBlong\fR or \fBshort\fR +may be specified in conjunction with \fBint\fR; +the meaning is the same as if \fBint\fR were not mentioned. +The word \fBlong\fR may be specified in conjunction with +\fBfloat\fR; +the meaning is the same as \fBdouble\fR. +The word \fBunsigned\fR may be specified alone, or +in conjunction with \fBint\fR or any of its short +or long varieties, or with \fBchar\fR. +.PP +Otherwise, at most on type-specifier may be +given in a declaration. +In particular, adjectival use of \fBlong\fR, +\fBshort\fR, or \fBunsigned\fR is not permitted +with \fBtypedef\fR names. +If the type-specifier is missing from a declaration, +it is taken to be \fBint\fR. +.PP +Specifiers for structures, unions, and enumerations are discussed in +``Structure, Union, and Enumeration Declarations.'' +Declarations with +.B +typedef +.R +names are discussed in ``Typedef.'' +.NH 2 +Declarators +.PP +The declarator-list appearing in a declaration +is a comma-separated sequence of declarators, +each of which may have an initializer. +.DS +\fIdeclarator-list: + init-declarator + init-declarator , declarator-list +.DE +.DS +\fIinit-declarator: + declarator initializer\v'0.5'\s-2opt\s0\v'-0.5'\fR +.DE +.PP +Initializers are discussed in ``Initialization''. +The specifiers in the declaration +indicate the type and storage class of the objects to which the +declarators refer. +Declarators have the syntax: +.DS +\fIdeclarator: + identifier + ( declarator ) + \(** declarator + declarator () + declarator [ constant-expression\v'0.5'\s-2opt\s0\v'-0.5' ]\fR +.DE +.PP +The grouping is +the same as in expressions. +.NH 2 +Meaning of Declarators +.PP +Each declarator is taken to be +an assertion that when a construction of +the same form as the declarator appears in an expression, +it yields an object of the indicated +type and storage class. +.PP +Each declarator contains exactly one identifier; it is this identifier that +is declared. +If an unadorned identifier appears +as a declarator, then it has the type +indicated by the specifier heading the declaration. +.PP +A declarator in parentheses is identical to the unadorned declarator, +but the binding of complex declarators may be altered by parentheses. +See the examples below. +.PP +Now imagine a declaration +.DS +\fBT D1\fR +.DE +.LP +where +.B +T +.R +is a type-specifier (like +\fBint\fR, +etc.) +and +.B +D1 +.R +is a declarator. +Suppose this declaration makes the identifier have type +``\|.\|.\|.\| +.B +T +.R +,'' +where the ``\|.\|.\|.\|'' is empty if +.B +D1 +.R +is just a plain identifier +(so that the type of +.B +x +.R +in +\fB`int x''\fR +is just +\fBint\fR). +Then if +.B +D1 +.R +has the form +.DS +\fB\(**D\fR +.DE +.LP +the type of the contained identifier is +``\|.\|.\|.\| pointer to +.B +T +.R +\&.'' +.PP +If +.B +D1 +.R +has the form +.DS +\fBD\|(\|\|)\|\fR +.DE +.LP +then the contained identifier has the type +``\|.\|.\|. function returning +\fBT\fR.'' +.LP +If +.B +D1 +.R +has the form +.DS +\fBD\|[\|\fIconstant-expression\fB\|]\fR +.DE +.LP +or +.DS +\fBD\|[\|]\|\fR +.DE +.LP +then the contained identifier has type +``\|.\|.\|.\| array of +\fBT\fR.'' +In the first case, the constant +expression +is an expression +whose value is determinable at compile time +, whose type is +.B +int\fR, +and whose value is positive. +(Constant expressions are defined precisely in ``Constant Expressions.'') +When several ``array of'' specifications are adjacent, a multidimensional +array is created; +the constant expressions which specify the bounds +of the arrays may be missing only for the first member of the sequence. +This elision is useful when the array is external +and the actual definition, which allocates storage, +is given elsewhere. +The first constant expression may also be omitted +when the declarator is followed by initialization. +In this case the size is calculated from the number +of initial elements supplied. +.PP +An array may be constructed from one of the basic types, from a pointer, +from a structure or union, +or from another array (to generate a multidimensional array). +.PP +Not all the possibilities +allowed by the syntax above are actually +permitted. +The restrictions are as follows: +functions may not return +arrays or functions +although they may return pointers; +there are no arrays of functions although +there may be arrays of pointers to functions. +Likewise, a structure or union may not contain a function; +but it may contain a pointer to a function. +.PP +As an example, the declaration +.DS +\fBint i, \(**ip, f(), \(**fip(), (\(**pfi)();\fR +.DE +.LP +declares an integer +\fBi\fR, +a pointer +.B +ip +.R +to an integer, +a function +.B +f +.R +returning an integer, +a function +.B +fip +.R +returning a pointer to an integer, +and a pointer +.B +pfi +.R +to a function which +returns an integer. +It is especially useful to compare the last two. +The binding of +.B +\(**fip() +.R +is +.B +\(**(fip())\fR. +.R +The declaration suggests, +and the same construction in an expression +requires, the calling of a function +.B +fip\fR. +.R +Using indirection through the (pointer) result +to yield an integer. +In the declarator +\fB(\(**pfi)()\fR, +the extra parentheses are necessary, as they are also +in an expression, to indicate that indirection through +a pointer to a function yields a function, which is then called; +it returns an integer. +.PP +As another example, +.DS +\fBfloat fa[17], \(**afp[17];\fR +.DE +.LP +declares an array of +.B +float +.R +numbers and an array of +pointers to +.B +float +.R +numbers. +Finally, +.DS +\fBstatic int x3d[3][5][7];\fR +.DE +.LP +declares a static 3-dimensional array of integers, +with rank 3\(mu5\(mu7. +In complete detail, +.B +x3d +.R +is an array of three items; +each item is an array of five arrays; +each of the latter arrays is an array of seven +integers. +Any of the expressions +\fBx3d\fR, +\fBx3d[i]\fR, +\fBx3d[i][j]\fR, +.B +x3d[i][j][k] +.R +may reasonably appear in an expression. +The first three have type ``array'' +and the last has type +.B +int\fR. +.R +.NH 2 +Structure and Union Declarations +.PP +A structure +is an object consisting of a sequence of named members. +Each member may have any type. +A union is an object which may, at a given time, contain any one +of several members. +Structure and union specifiers have the same form. +.DS +\fIstruct-or-union-specifier: + struct-or-union { struct-decl-list } + struct-or-union identifier { struct-decl-list } + struct-or-union identifier +.DE +.DS +\fIstruct-or-union:\fB + struct + union\fR +.DE +.PP +The +struct-decl-list +.ne 4 +is a sequence of declarations for the members of the structure or union: +.DS +\fIstruct-decl-list: + struct-declaration + struct-declaration struct-decl-list +.DE +.DS +\fIstruct-declaration: + type-specifier struct-declarator-list ; +.DE +.DS +\fIstruct-declarator-list: + struct-declarator + struct-declarator , struct-declarator-list\fR +.DE +.PP +In the usual case, a struct-declarator is just a declarator +for a member of a structure or union. +A structure member may also consist of a specified number of bits. +Such a member is also called a +.I +field ; +.R +its length, +a non-negative constant expression, +is set off from the field name by a colon. +.DS +\fIstruct-declarator: + declarator + declarator : constant-expression + : constant-expression\fR +.DE +.PP +Within a structure, the objects declared +have addresses which increase as the declarations +are read left to right. +Each nonfield member of a structure +begins on an addressing boundary appropriate +to its type; +therefore, there may +be unnamed holes in a structure. +Field members are packed into machine integers; +they do not straddle words. +A field which does not fit into the space remaining in a word +is put into the next word. +No field may be wider than a word. +.PP +Fields are assigned right to left +on the +PDP-11 +and +VAX-11, +left to right on the 3B 20. +.PP +A struct-declarator with no declarator, only a colon and a width, +indicates an unnamed field useful for padding to conform +to externally-imposed layouts. +As a special case, a field with a width of 0 +specifies alignment of the next field at an implementation dependent boundary. +.PP +The language does not restrict the types of things that +are declared as fields, +but implementations are not required to support any but +integer fields. +Moreover, +even +.B +int +.R +fields may be considered to be unsigned. +On the +PDP-11, +fields are not signed and have only integer values; +on the +VAX-11, +fields declared with +.B +int +.R +are treated as containing a sign. +For these reasons, +it is strongly recommended that fields be declared as +.B +unsigned\fR. +.R +In all implementations, +there are no arrays of fields, +and the address-of operator +.B +& +.R +may not be applied to them, so that there are no pointers to +fields. +.PP +A union may be thought of as a structure all of whose members +begin at offset 0 and whose size is sufficient to contain +any of its members. +At most, one of the members can be stored in a union +at any time. +.PP +A structure or union specifier of the second form, that is, one of +.DS + \fBstruct \fIidentifier { struct-decl-list \fR} + \fBunion \fIidentifier { struct-decl-list \fR} +.DE +.LP +declares the identifier to be the +.I +structure tag +.R +(or union tag) +of the structure specified by the list. +A subsequent declaration may then use +the third form of specifier, one of +.DS + \fBstruct \fIidentifier\fR + \fBunion \fIidentifier\fR +.DE +.PP +Structure tags allow definition of self-referential +structures. Structure tags also +permit the long part of the declaration to be +given once and used several times. +It is illegal to declare a structure or union +which contains an instance of +itself, but a structure or union may contain a pointer to an instance of itself. +.PP +The third form of a structure or union specifier may be +used prior to a declaration which gives the complete specification +of the structure or union in situations in which the size +of the structure or union is unnecessary. +The size is unnecessary in two situations: when a +pointer to a structure or union is being declared and +when a \fBtypedef\fR name is declared to be a synonym +for a structure or union. +This, for example, allows the declaration of a pair +of structures which contain pointers to each other. +.PP +The names of members and tags do not conflict +with each other or with ordinary variables. +A particular name may not be used twice +in the same structure, +but the same name may be used in several different structures in the same scope. +.PP +A simple but important example of a structure declaration is +the following binary tree structure: +.DS +\fBstruct tnode +{ + char tword[20]; + int count; + struct tnode \(**left; + struct tnode \(**right; +};\fR +.DE +.LP +which contains an array of 20 characters, an integer, and two pointers +to similar structures. +Once this declaration has been given, the +declaration +.DS +\fBstruct tnode s, \(**sp;\fR +.DE +.LP +declares +.B +s +.R +to be a structure of the given sort +and +.B +sp +.R +to be a pointer to a structure +of the given sort. +With these declarations, the expression +.DS +\fBsp->count\fR +.DE +.LP +refers to the +.B +count +.R +field of the structure to which +.B +sp +.R +points; +.DS +\fBs.left\fR +.DE +.LP +refers to the left subtree pointer +of the structure +\fBs\fR; +and +.DS +\fBs.right->tword[0]\fR +.DE +.LP +refers to the first character of the +.B +tword +.R +member of the right subtree of +.B +s\fR. +.R +.PP +.NH 2 +Enumeration Declarations +.PP +Enumeration variables and constants have integral type. +.DS +\fIenum-specifier:\fB + enum\fI { enum-list \fR}\fB + enum \fIidentifier { enum-list \fR}\fB + enum \fIidentifier +.sp +enum-list: + enumerator + enum-list , enumerator +.sp +enumerator: + identifier + identifier = constant-expression\fR +.DE +.PP +The identifiers in an enum-list are declared as constants +and may appear wherever constants are required. +If no enumerators with +.B += +.R +appear, then the values of the +corresponding constants begin at 0 and increase by 1 as the declaration is +read from left to right. +An enumerator with +.B += +.R +gives the associated identifier the value +indicated; subsequent identifiers continue the progression from the assigned value. +.PP +The names of enumerators in the same scope must all be distinct +from each other and from those of ordinary variables. +.PP +The role of the identifier in the enum-specifier +is entirely analogous to that of the structure tag +in a struct-specifier; it names a particular enumeration. +For example, +.DS L +\fBenum color { chartreuse, burgundy, claret=20, winedark }; +\&... +enum color *cp, col; +\&... +col = claret; +cp = &col; +\&... +if (*cp == burgundy) ...\fR +.DE +.LP +makes +.B +color +.R +the enumeration-tag of a type describing various colors, +and then declares +.B +cp +.R +as a pointer to an object of that type, +and +.B +col +.R +as an object of that type. +The possible values are drawn from the set {0,1,20,21}. +.NH 2 +Initialization +.PP +A declarator may specify an initial value for the +identifier being declared. +The initializer is preceded by +.B += +.R +and +consists of an expression or a list of values nested in braces. +.DS +\fIinitializer: + = expression + = { initializer-list } + = { initializer-list , } +.DE +.DS +\fIinitializer-list: + expression + initializer-list , initializer-list\fR + { \fIinitializer-list \fR} + { \fIinitializer-list\fR , } +.DE +.PP +All the expressions in an initializer +for a static or external variable must be constant +expressions, which are described in ``CONSTANT EXPRESSIONS'', +or expressions which reduce to the address of a previously +declared variable, possibly offset by a constant expression. +Automatic or register variables may be initialized by arbitrary +expressions involving constants and previously declared variables and functions. +.PP +Static and external variables that are not initialized are +guaranteed to start off as zero. +Automatic and register variables that are not initialized +are guaranteed to start off as garbage. +.PP +When an initializer applies to a +.I +scalar +.R +(a pointer or an object of arithmetic type), +it consists of a single expression, perhaps in braces. +The initial value of the object is taken from +the expression; the same conversions as for assignment are performed. +.PP +When the declared variable is an +.I +aggregate +.R +(a structure or array), +the initializer consists of a brace-enclosed, comma-separated list of +initializers for the members of the aggregate +written in increasing subscript or member order. +If the aggregate contains subaggregates, this rule +applies recursively to the members of the aggregate. +If there are fewer initializers in the list than there are members of the aggregate, +then the aggregate is padded with zeros. +It is not permitted to initialize unions or automatic aggregates. +.PP +Braces may in some cases be omitted. +If the initializer begins with a left brace, then +the succeeding comma-separated list of initializers initializes +the members of the aggregate; +it is erroneous for there to be more initializers than members. +If, however, the initializer does not begin with a left brace, +then only enough elements from the list are taken to account +for the members of the aggregate; any remaining members +are left to initialize the next member of the aggregate of which +the current aggregate is a part. +.PP +A final abbreviation allows a +.B +char +.R +array to be initialized by a string. +In this case successive characters of the string +initialize the members of the array. +.PP +For example, +.DS +\fBint x[] = { 1, 3, 5 };\fR +.DE +.LP +declares and initializes +.B +x +.R +as a one-dimensional array which has three members, since no size was specified +and there are three initializers. +.DS +\fBfloat y[4][3] = +{ + { 1, 3, 5 }, + { 2, 4, 6 }, + { 3, 5, 7 }, +};\fR +.DE +.LP +is a completely-bracketed initialization: +1, 3, and 5 initialize the first row of +the array +\fBy[0]\fR, +namely +\fBy[0][0]\fR, +\fBy[0][1]\fR, +and +.B +y[0][2]\fR. +.R +Likewise, the next two lines initialize +.B +y[1] +.R +and +.B +y[2]\fR. +.R +The initializer ends early and therefore +.B +y[3] +.R +is initialized with 0. +Precisely, the same effect could have been achieved by +.DS +\fBfloat y[4][3] = +{ + 1, 3, 5, 2, 4, 6, 3, 5, 7 +};\fR +.DE +.PP +The initializer for +.B +y +.R +begins with a left brace but that for +.B +y[0] +.R +does not; +therefore, three elements from the list are used. +Likewise, the next three are taken successively for +.B +y[1] +.R +and +.B +y[2]\fR. +.R +Also, +.DS +\fBfloat y[4][3] = +{ + { 1 }, { 2 }, { 3 }, { 4 } +};\fR +.DE +.LP +initializes the first column of +.B +y +.R +(regarded as a two-dimensional array) +and leaves the rest 0. +.PP +Finally, +.DS +\fBchar msg[] = "Syntax error on line %s\en";\fR +.DE +.LP +shows a character array whose members are initialized +with a string. +.NH 2 +Type Names +.PP +In two contexts (to specify type conversions explicitly +by means of a cast +and as an argument of +\fBsizeof\fR), +it is desired to supply the name of a data type. +This is accomplished using a ``type name'', which in essence +is a declaration for an object of that type which omits the name of +the object. +.DS +\fItype-name: + type-specifier abstract-declarator +.DE +.DS +\fIabstract-declarator: + empty + ( abstract-declarator ) + \(** abstract-declarator + abstract-declarator () + abstract-declarator\fR\^ [ \fIconstant-expression\v'0.5'\s-2opt\s0\v'-0.5' \fR\^] +.DE +.PP +To avoid ambiguity, +in the construction +.DS + \fI( abstract-declarator \fR) +.DE +.LP +the +abstract-declarator +is required to be nonempty. +Under this restriction, +it is possible to identify uniquely the location in the abstract-declarator +where the identifier would appear if the construction were a declarator +in a declaration. +The named type is then the same as the type of the +hypothetical identifier. +For example, +.DS +\fBint +int \(** +int \(**[3] +int (\(**)[3] +int \(**() +int (\(**)() +int (\(**[3])()\fR +.DE +.LP +name respectively the types ``integer,'' ``pointer to integer,'' +``array of three pointers to integers,'' +``pointer to an array of three integers,'' +``function returning pointer to integer,'' +``pointer to function returning an integer,'' +and ``array of three pointers to functions returning an integer.'' +.NH 2 +Typedef +.PP +Declarations whose ``storage class'' is +.B +typedef +.R +do not define storage but instead +define identifiers which can be used later +as if they were type keywords naming fundamental +or derived types. +.DS +\fItypedef-name:\fR + \fIidentifier\fR +.DE +.PP +Within the scope of a declaration involving +\fBtypedef\fR, +each identifier appearing as part of +any declarator therein becomes syntactically +equivalent to the type keyword +naming the type +associated with the identifier +in the way described in ``Meaning of Declarators.'' +For example, +after +.DS +\fBtypedef int MILES, \(**KLICKSP; +typedef struct { double re, im; } complex;\fR +.DE +.LP +the constructions +.DS +\fBMILES distance; +extern KLICKSP metricp; +complex z, \(**zp;\fR +.DE +.LP +are all legal declarations; the type of +.B +distance +.R +is +\fBint\fR, +that of +.B +metricp +.R +is ``pointer to \fBint\fR, '' +and that of +.B +z +.R +is the specified structure. +The +.B +zp +.R +is a pointer to such a structure. +.PP +The +.B +typedef +.R +does not introduce brand-new types, only synonyms for +types which could be specified in another way. +Thus +in the example above +.B +distance +.R +is considered to have exactly the same type as +any other +.B +int +.R +object. +.NH 1 +Statements +.PP +Except as indicated, statements are executed in sequence. +.NH 2 +Expression Statement +.PP +Most statements are expression statements, which have +the form +.DS +\fIexpression \fR; +.DE +.PP +Usually expression statements are assignments or function +calls. +.NH 2 +Compound Statement or Block +.PP +So that several statements can be used where one is expected, +the compound statement (also, and equivalently, called ``block'') is provided: +.DS +\fIcompound-statement: + { declaration-list\v'0.5'\s-2opt\s0\v'-0.5' statement-list\v'0.5'\s-2opt\s0\v'-0.5' } +.DE +.DS +\fIdeclaration-list: + declaration + declaration declaration-list +.DE +.DS +\fIstatement-list: + statement + statement statement-list\fR +.DE +.PP +If any of the identifiers +in the declaration-list were previously declared, +the outer declaration is pushed down for the duration of the block, +after which it resumes its force. +.PP +Any initializations of +.B +auto +.R +or +.B +register +.R +variables are performed each time the block is entered at the top. +It is currently possible +(but a bad practice) +to transfer into a block; +in that case the initializations are not performed. +Initializations of +.B +static +.R +variables are performed only once when the program +begins execution. +Inside a block, +.B +extern +.R +declarations do not reserve storage +so initialization is not permitted. +.NH 2 +Conditional Statement +.PP +The two forms of the conditional statement are +.DS +\fBif\fR\^ ( \fIexpression\fR\^ ) \fIstatement\fR\^ +\fBif\fR\^ ( \fIexpression\fR\^ ) \fIstatement \fBelse \fIstatement\fR\^ +.DE +.PP +In both cases, the expression is evaluated; +and if it is nonzero, the first substatement +is executed. +In the second case, the second substatement is executed +if the expression is 0. +The ``else'' ambiguity is resolved by connecting +an +.B +else +.R +with the last encountered +\fBelse\fR-less +.B +if\fR. +.R +.NH 2 +While Statement +.PP +The +.B +while +.R +statement has the form +.DS +\fBwhile\fR\^ ( \fIexpression\fR\^ ) \fIstatement\fR\^ +.DE +.PP +The substatement is executed repeatedly +so long as the value of the +expression remains nonzero. +The test takes place before each execution of the +statement. +.NH 2 +Do Statement +.PP +The +.B +do +.R +statement has the form +.DS +\fBdo \fIstatement \fBwhile\fR\^ ( \fIexpression \fR\^) ; +.DE +.PP +The substatement is executed repeatedly until +the value of the expression becomes 0. +The test takes place after each execution of the +statement. +.NH 2 +For Statement +.PP +The +.B +for +.R +statement has the form: +.DS +\fBfor\fI ( exp-1\v'0.5'\s-2opt\s0\v'-0.5' ; exp-2\v'0.5'\s-2opt\s0\v'-0.5' ; exp-3\v'0.5'\s-2opt\s0\v'-0.5' ) statement\fR +.DE +.PP +.sp +Except for the behavior of \fBcontinue\fR, +this statement is equivalent to +.DS +\fIexp-1 \fR; +\fBwhile\fR\^ ( \fIexp-2\ ) \fR\^ +{ + \fIstatement + exp-3 ;\fR +} +.DE +.PP +Thus the first expression specifies initialization +for the loop; the second specifies +a test, made before each iteration, such +that the loop is exited when the expression becomes +0. +The third expression often specifies an incrementing +that is performed after each iteration. +.PP +Any or all of the expressions may be dropped. +A missing +.I +exp-2 +.R +makes the +implied +.B +while +.R +clause equivalent to +\fBwhile(1)\fR; +other missing expressions are simply +dropped from the expansion above. +.NH 2 +Switch Statement +.PP +The +.B +switch +.R +statement causes control to be transferred +to one of several statements depending on +the value of an expression. +It has the form +.DS +\fBswitch\fR\^ ( \fIexpression\fR\^ ) \fIstatement\fR\^ +.DE +.PP +The usual arithmetic conversion is performed on the +expression, but the result must be +.B +int\fR. +.R +The statement is typically compound. +Any statement within the statement +may be labeled with one or more case prefixes +as follows: +.DS +\fBcase \fIconstant-expression \fR: +.DE +.LP +where the constant +expression +must be +.B +int\fR. +.R +No two of the case constants in the same switch +may have the same value. +Constant expressions are precisely defined in ``CONSTANT EXPRESSIONS.'' +.PP +There may also be at most one statement prefix of the +form +.DS +\fBdefault :\fR +.DE +.PP +When the +.B +switch +.R +statement is executed, its expression +is evaluated and compared with each case constant. +If one of the case constants is +equal to the value of the expression, +control is passed to the statement +following the matched case prefix. +If no case constant matches the expression +and if there is a +\fBdefault\fR, +prefix, control +passes to the prefixed +statement. +If no case matches and if there is no +\fBdefault\fR, +then +none of the statements in the +switch is executed. +.PP +The prefixes +.B +case +.R +and +.B +default +.R +do not alter the flow of control, +which continues unimpeded across such prefixes. +To exit from a switch, see +``Break Statement.'' +.PP +Usually, the statement that is the subject of a switch is compound. +Declarations may appear at the head of this +statement, +but +initializations of automatic or register variables +are ineffective. +.NH 2 +Break Statement +.PP +The statement +.DS +\fBbreak ;\fR +.DE +.LP +causes termination of the smallest enclosing +\fBwhile\fR, +\fBdo\fR, +\fBfor\fR, +or +\fBswitch\fR +statement; +control passes to the +statement following the terminated statement. +.NH 2 +Continue Statement +.PP +The statement +.DS +\fBcontinue ;\fR +.DE +.LP +causes control to pass to the loop-continuation portion of the +smallest enclosing +\fBwhile\fR, +\fBdo\fR, +or +\fBfor\fR +statement; that is to the end of the loop. +More precisely, in each of the statements +.DS +.TS +lw(2i) lw(2i) lw(2i). +\fBwhile (\|.\|.\|.\|) { do { for (\|.\|.\|.\|) {\fR + \fIstatement ; statement ; statement ;\fR + \fBcontin: ; contin: ; contin: ; +} } while (...); }\fR +.TE +.DE +.LP +a +.B +continue +.R +is equivalent to +.B +goto\ contin\fR. +.R +(Following the +.B +contin: +.R +is a null statement, see ``Null Statement''.) +.NH 2 +Return Statement +.PP +A function returns to its caller by means of +the +.B +return +.R +statement which has one of the +forms +.DS +\fBreturn ; +return \fIexpression \fR; +.DE +.PP +In the first case, the returned value is undefined. +In the second case, the value of the expression +is returned to the caller +of the function. +If required, the expression is converted, +as if by assignment, to the type of +function in which it appears. +Flowing off the end of a function is +equivalent to a return with no returned value. +The expression may be parenthesized. +.NH 2 +Goto Statement +.PP +Control may be transferred unconditionally by means of +the statement +.DS +\fBgoto \fIidentifier \fR; +.DE +.PP +The identifier must be a label +(see ``Labeled Statement'') +located in the current function. +.NH 2 +Labeled Statement +.PP +Any statement may be preceded by +label prefixes of the form +.DS +\fIidentifier \fR: +.DE +.LP +which serve to declare the identifier +as a label. +The only use of a label is as a target of a +.B +goto\fR. +.R +The scope of a label is the current function, +excluding any subblocks in which the same identifier has been redeclared. +See ``SCOPE RULES.'' +.NH 2 +Null Statement +.PP +The null statement has the form +.DS + \fB;\fR +.DE +.PP +A null statement is useful to carry a label just before the +.B +} +.R +of a compound statement or to supply a null +body to a looping statement such as +.B +while\fR. +.R +.NH 1 +External Definitions +.PP +A C program consists of a sequence of external definitions. +An external definition declares an identifier to +have storage class +.B +extern +.R +(by default) +or perhaps +\fBstatic\fR, +and +a specified type. +The type-specifier (see ``Type Specifiers'' in +``DECLARATIONS'') may also be empty, in which +case the type is taken to be +.B +int\fR. +.R +The scope of external definitions persists to the end +of the file in which they are declared just as the effect +of declarations persists to the end of a block. +The syntax of external definitions is the same +as that of all declarations except that +only at this level may the code for functions be given. +.NH 2 +External Function Definitions +.PP +Function definitions have the form +.DS +\fIfunction-definition: + decl-specifiers\v'0.5'\s-2opt\s0\v'-0.5' function-declarator function-body\fR +.DE +.PP +The only sc-specifiers +allowed +among the decl-specifiers +are +.B +extern +.R +or +\fBstatic\fR; +see ``Scope of Externals'' in +``SCOPE RULES'' for the distinction between them. +A function declarator is similar to a declarator +for a ``function returning .\|.\|.\|'' except that +it lists the formal parameters of +the function being defined. +.DS +\fIfunction-declarator: + declarator ( parameter-list\v'0.5'\s-2opt\s0\v'-0.5' ) +.DE +.DS +\fIparameter-list: + identifier + identifier , parameter-list\fR +.DE +.PP +The function-body +has the form +.DS +\fIfunction-body: + declaration-list\v'0.5'\s-2opt\s0\v'-0.5' compound-statement\fR +.DE +.PP +The identifiers in the parameter list, and only those identifiers, +may be declared in the declaration list. +Any identifiers whose type is not given are taken to be +.B +int\fR. +.R +The only storage class which may be specified is +\fBregister\fR; +if it is specified, the corresponding actual parameter +will be copied, if possible, into a register +at the outset of the function. +.PP +A simple example of a complete function definition is +.DS +\fBint max(a, b, c) + int a, b, c; +{ + int m; +.sp + m = (a > b) ? a : b; + return((m > c) ? m : c); +}\fR +.DE +.PP +Here +.B +int +.R +is the type-specifier; +.B +max(a,\ b,\ c) +.R +is the function-declarator; +.B +int\ a,\ b,\ c; +.R +is the declaration-list for +the formal +parameters; +\fB{\ ...\ }\fR +is the +block giving the code for the statement. +.PP +The C program converts all +.B +float +.R +actual parameters +to +\fBdouble\fR, +so formal parameters declared +.B +float +.R +have their declaration adjusted to read +.B +double\fR. +.R +All \fBchar\fR and \fBshort\fR formal parameter +declarations are similarly adjusted +to read \fBint\fR. +Also, since a reference to an array in any context +(in particular as an actual parameter) +is taken to mean +a pointer to the first element of the array, +declarations of formal parameters declared ``array of .\|.\|.\|'' +are adjusted to read ``pointer to .\|.\|.\|.'' +.NH 2 +External Data Definitions +.PP +An external data definition has the form +.DS +\fIdata-definition: + declaration\fR +.DE +.PP +The storage class of such data may be +.B +extern +.R +(which is the default) +or +.B +static +.R +but not +.B +auto +.R +or +\fBregister\fR. +.NH 1 +Scope Rules +.PP +A C program need not all +be compiled at the same time. The source text of the +program +may be kept in several files, and precompiled +routines may be loaded from +libraries. +Communication among the functions of a program +may be carried out both through explicit calls +and through manipulation of external data. +.PP +Therefore, there are two kinds of scopes to consider: +first, what may be called the +.UL lexical +.UL scope +of an identifier, which is essentially the +region of a program during which it may +be used without drawing ``undefined identifier'' +diagnostics; +and second, the scope +associated with external identifiers, +which is characterized by the rule +that references to the same external +identifier are references to the same object. +.NH 2 +Lexical Scope +.PP +The lexical scope of identifiers declared in external definitions +persists from the definition through +the end of the source file +in which they appear. +The lexical scope of identifiers which are formal parameters +persists through the function with which they are +associated. +The lexical scope of identifiers declared at the head of a block +persists until the end of the block. +The lexical scope of labels is the whole of the +function in which they appear. +.PP +In all cases, however, +if an identifier is explicitly declared at the head of a block, +including the block constituting a function, +any declaration of that identifier outside the block +is suspended until the end of the block. +.PP +Remember also (see ``Structure, Union, and Enumeration Declarations'' in +``DECLARATIONS'') that tags, identifiers associated with +ordinary variables, +and identities associated with structure and union members +form three disjoint classes +which do not conflict. +Members and tags follow the same scope rules +as other identifiers. +The \fBenum\fR constants are in the same +class as ordinary variables and follow the same scope rules. +The +.B +typedef +.R +names are in the same class as ordinary identifiers. +They may be redeclared in inner blocks, but an explicit +type must be given in the inner declaration: +.DS +\fBtypedef float distance; +\&... +{ + auto int distance; + ...\fR +} +.DE +.PP +The +.B +int +.R +must be present in the second declaration, +or it would be taken to be +a declaration with no declarators and type +.B +distance\fR. +.R +.NH 2 +Scope of Externals +.PP +If a function refers to an identifier declared to be +\fBextern\fR, +then somewhere among the files or libraries +constituting the complete program +there must be at least one external definition +for the identifier. +All functions in a given program which refer to the same +external identifier refer to the same object, +so care must be taken that the type and size +specified in the definition +are compatible with those specified +by each function which references the data. +.PP +It is illegal to explicitly initialize any external +identifier more than once in the set of files and libraries +comprising a multi-file program. +It is legal to have more than one data definition +for any external non-function identifier; +explicit use of \fBextern\fR does not +change the meaning of an external declaration. +.PP +In restricted environments, the use of the \fBextern\fR +storage class takes on an additional meaning. +In these environments, the explicit appearance of the +\fBextern\fR keyword in external data declarations of +identities without initialization indicates that +the storage for the identifiers is allocated elsewhere, +either in this file or another file. +It is required that there be exactly one definition of +each external identifier (without \fBextern\fR) +in the set of files and libraries +comprising a mult-file program. +.PP +Identifiers declared +.B +static +.R +at the top level in external definitions +are not visible in other files. +Functions may be declared +.B +static\fR. +.R +.nr Hu 1 +.NH 1 +Compiler Control Lines +.PP +The C compiler contains a preprocessor capable +of macro substitution, conditional compilation, +and inclusion of named files. +Lines beginning with +.B +# +.R +communicate +with this preprocessor. +There may be any number of blanks and horizontal tabs +between the \fB#\fR and the directive. +These lines have syntax independent of the rest of the language; +they may appear anywhere and have effect which lasts (independent of +scope) until the end of the source program file. +.nr Hu 1 +.NH 2 +Token Replacement +.PP +A compiler-control line of the form +.DS +\fB#define \fIidentifier token-string\v'0.5'\s-2opt\s0\v'-0.5'\fR +.DE +.LP +causes the preprocessor to replace subsequent instances +of the identifier with the given string of tokens. +Semicolons in or at the end of the token-string are part of that string. +A line of the form +.DS +\fB#define \fIidentifier(identifier, ... )token-string\v'0.5'\s-2opt\s0\v'-0.5'\fR +.DE +.LP +where there is no space between the first identifier +and the +\fB(\fR, +is a macro definition with arguments. +There may be zero or more formal parameters. +Subsequent instances of the first identifier followed +by a +\fB(\fR, +a sequence of tokens delimited by commas, and a +\fB)\fR +are replaced +by the token string in the definition. +Each occurrence of an identifier mentioned in the formal parameter list +of the definition is replaced by the corresponding token string from the call. +The actual arguments in the call are token strings separated by commas; +however, commas in quoted strings or protected by +parentheses do not separate arguments. +The number of formal and actual parameters must be the same. +Strings and character constants in the token-string are scanned +for formal parameters, but +strings and character constants in the rest of the program are +not scanned for defined identifiers +to replacement. +.PP +In both forms the replacement string is rescanned for more +defined identifiers. +In both forms +a long definition may be continued on another line +by writing +.B +\e +.R +at the end of the line to be continued. +.PP +This facility is most valuable for definition of ``manifest constants,'' +as in +.DS +\fB#define TABSIZE 100 +.sp +int table\|[\|TABSIZE\|]\|;\fR +.DE +.PP +A control line of the form +.DS +\fB#undef \fIidentifier\fR +.DE +.LP +causes the +identifier's preprocessor definition (if any) to be forgotten. +.PP +If a \fB#define\fRd identifier is the subject of a subsequent +\fB#define\fR with no intervening \fB#undef\fR, then +the two token-strings are compared textually. +If the two token-strings are not identical +(all white space is considered as equivalent), then +the identifier is considered to be redefined. +.nr Hu 1 +.NH 2 +File Inclusion +.PP +A compiler control line of +the form +.DS +\fB#include\fI "filename\|\fR" +.DE +.LP +causes the replacement of that +line by the entire contents of the file +.I +filename\fR. +.R +The named file is searched for first in the directory +of the file containing the \fB#include\fR, +and then in a sequence of specified or standard places. +Alternatively, a control line of the form +.DS +\fB#include\fI <filename\|\fR> +.DE +.LP +searches only the specified or standard places +and not the directory of the \fB#include\fR. +(How the places are specified is not part of the language.) +.PP +\fB#include\fRs +may be nested. +.nr Hu 1 +.NH 2 +Conditional Compilation +.PP +A compiler control line of the form +.DS +\fB#if \fIrestricted-constant-expression\fR +.DE +.LP +checks whether the restricted-constant expression evaluates to nonzero. +(Constant expressions are discussed in ``CONSTANT EXPRESSIONS''; +the following additional restrictions apply here: +the constant expression may not contain +.B +sizeof +.R +casts, or an enumeration constant.) +.PP +A restricted constant expression may also contain the +additional unary expression +.PP +\fBdefined \fIidentifier\fR +.LP +or +.PP +\fBdefined( \fIidentifier )\fR +.LP +which evaluates to one if the identifier is currently +defined in the preprocessor and zero if it is not. +.PP +All currently defined identifiers in restricted-constant-expressions +are replaced by their token-strings (except those identifiers +modified by \fBdefined\fR) just as in normal text. +The restricted constant expression will be evaluated only +after all expressions have finished. +During this evaluation, all undefined (to the procedure) +identifiers evaluate to zero. +.PP +A control line of the form +.DS +\fB#ifdef \fIidentifier\fR +.DE +.LP +checks whether the identifier is currently defined +in the preprocessor; i.e., whether it has been the +subject of a +.B +#define +.R +control line. +It is equivalent to \fB#ifdef(\fIidentifier\fB)\fR. +A control line of the form +.DS +\fB#ifndef \fIidentifier\fR +.DE +.LP +checks whether the identifier is currently undefined +in the preprocessor. +It is equivalent to +.DS +\fB#if !\|defined(\fIidentifier\fB)\fR. +.DE +.PP +All three forms are followed by an arbitrary number of lines, +possibly containing a control line +.DS +\fB#else\fR +.DE +.LP +and then by a control line +.DS +\fB#endif\fR +.DE +.PP +If the checked condition is true, +then any lines +between +.B +#else +.R +and +.B +#endif +.R +are ignored. +If the checked condition is false, then any lines between +the test and a +.B +#else +.R +or, lacking a +\fB#else\fR, +the +.B +#endif +.R +are ignored. +.PP +These constructions may be nested. +.nr Hu 1 +.NH 2 +Line Control +.PP +For the benefit of other preprocessors which generate C programs, +a line of the form +.DS +\fB#line \fIconstant "filename\fR" +.DE +.LP +causes the compiler to believe, for purposes of error +diagnostics, +that the line number of the next source line is given by the constant and the current input +file is named by "\fIfilename\fR". +If "\fIfilename\fR" is absent, the remembered file name does not change. +.nr Hu 1 +.NH 1 +Implicit Declarations +.PP +It is not always necessary to specify +both the storage class and the type +of identifiers in a declaration. +The storage class is supplied by +the context in external definitions +and in declarations of formal parameters +and structure members. +In a declaration inside a function, +if a storage class but no type +is given, the identifier is assumed +to be +\fBint\fR; +if a type but no storage class is indicated, +the identifier is assumed to +be +.B +auto\fR. +.R +An exception to the latter rule is made for +functions because +.B +auto +.R +functions do not exist. +If the type of an identifier is ``function returning .\|.\|.\|,'' +it is implicitly declared to be +.B +extern\fR. +.R +.PP +In an expression, an identifier +followed by +.B +( +.R +and not already declared +is contextually +declared to be ``function returning +.B +int\fR.'' +.nr Hu 1 +.NH 1 +Types Revisited +.PP +This part summarizes the operations +which can be performed on objects of certain types. +.nr Hu 1 +.NH 2 +Structures and Unions +.PP +Structures and unions may be assigned, passed as arguments to functions, +and returned by functions. +Other plausible operators, such as equality comparison +and structure casts, +are not implemented. +.PP +In a reference +to a structure or union member, the +name on the right +of the \fB->\fR or the \fB.\fR +must specify a member of the aggregate +named or pointed to by the expression +on the left. +In general, a member of a union may not be inspected +unless the value of the union has been assigned using that same member. +However, one special guarantee is made by the language in order +to simplify the use of unions: +if a union contains several structures that share a common initial sequence +and if the union currently contains one of these structures, +it is permitted to inspect the common initial part of any of +the contained structures. +For example, the following is a legal fragment: +.DS +\fBunion +{ + struct + { + int type; + } n; + struct + { + int type; + int intnode; + } ni; + struct + { + int type; + float floatnode; + } nf; +} u; +\&... +u.nf.type = FLOAT; +u.nf.floatnode = 3.14; +\&... +if (u.n.type == FLOAT) + ... sin(u.nf.floatnode) ...\fR +.DE +.PP +.nr Hu 1 +.NH 2 +Functions +.PP +There are only two things that +can be done with a function \fBm\fR, +call it or take its address. +If the name of a function appears in an +expression not in the function-name position of a call, +a pointer to the function is generated. +Thus, to pass one function to another, one +might say +.DS +\fBint f(); +\&... +g(f);\fR +.DE +.PP +.ne 8 +Then the definition of +.B +g +.R +might read +.DS +\fBg(funcp) + int (\(**funcp)(); +{ + ... + (\(**funcp)(); + ... +}\fR +.DE +.PP +Notice that +.B +f +.R +must be declared +explicitly in the calling routine since its appearance +in +.B +g(f) +.R +was not followed by +.B +(. +.R +.nr Hu 1 +.NH 2 +Arrays, Pointers, and Subscripting +.PP +Every time an identifier of array type appears +in an expression, it is converted into a pointer +to the first member of the array. +Because of this conversion, arrays are not +lvalues. +By definition, the subscript operator +.B +[] +.R +is interpreted +in such a way that +.B +E1[E2] +.R +is identical to +.B +\(**((E1)\(plE2))\fR. +.R +Because of the conversion rules +which apply to +\fB\(pl\fR, +if +.B +E1 +.R +is an array and +.B +E2 +.R +an integer, +then +.B +E1[E2] +.R +refers to the +.B +E2-th +.R +member of +.B +E1\fR. +.R +Therefore, +despite its asymmetric +appearance, subscripting is a commutative operation. +.PP +A consistent rule is followed in the case of +multidimensional arrays. +If +.B +E +.R +is an +\fIn\fR-dimensional +array +of rank +i\(muj\(mu...\(muk, +then +.B +E +.R +appearing in an expression is converted to +a pointer to an (n-1)-dimensional +array with rank +j\(mu...\(muk. +If the +.B +\(** +.R +operator, either explicitly +or implicitly as a result of subscripting, +is applied to this pointer, +the result is the pointed-to (n-1)-dimensional array, +which itself is immediately converted into a pointer. +.PP +For example, consider +.DS +\fBint x[3][5];\fR +.DE +.PP +Here +.B +x +.R +is a 3\(mu5 array of integers. +When +.B +x +.R +appears in an expression, it is converted +to a pointer to (the first of three) 5-membered arrays of integers. +In the expression +\fBx[i]\fR, +which is equivalent to +\fB\(**(x\(pli)\fR, +.B +x +.R +is first converted to a pointer as described; +then +.B +i +.R +is converted to the type of +\fBx\fR, +which involves multiplying +.B +i +.R +by the +length the object to which the pointer points, +namely 5-integer objects. +The results are added and indirection applied to +yield an array (of five integers) which in turn is converted to +a pointer to the first of the integers. +If there is another subscript, the same argument applies +again; this time the result is an integer. +.PP +Arrays in C are stored +row-wise (last subscript varies fastest) +and the first subscript in the declaration helps determine +the amount of storage consumed by an array. +Arrays play no other part in subscript calculations. +.nr Hu 1 +.NH 2 +Explicit Pointer Conversions +.PP +Certain conversions involving pointers are permitted +but have implementation-dependent aspects. +They are all specified by means of an explicit type-conversion +operator, see ``Unary Operators'' under``EXPRESSIONS'' and +``Type Names''under ``DECLARATIONS.'' +.PP +A pointer may be converted to any of the integral types large +enough to hold it. +Whether an +.B +int +.R +or +.B +long +.R +is required is machine dependent. +The mapping function is also machine dependent but is intended +to be unsurprising to those who know the addressing structure +of the machine. +Details for some particular machines are given below. +.PP +An object of integral type may be explicitly converted to a pointer. +The mapping always carries an integer converted from a pointer back to the same pointer +but is otherwise machine dependent. +.PP +A pointer to one type may be converted to a pointer to another type. +The resulting pointer may cause addressing exceptions +upon use if +the subject pointer does not refer to an object suitably aligned in storage. +It is guaranteed that +a pointer to an object of a given size may be converted to a pointer to an object +of a smaller size +and back again without change. +.PP +For example, +a storage-allocation routine +might accept a size (in bytes) +of an object to allocate, and return a +.B +char +.R +pointer; +it might be used in this way. +.DS +\fBextern char \(**malloc(); +double \(**dp; +.sp +dp = (double \(**) malloc(sizeof(double)); +\(**dp = 22.0 / 7.0;\fR +.DE +.PP +The +.B +alloc +.R +must ensure (in a machine-dependent way) +that its return value is suitable for conversion to a pointer to +\fBdouble\fR; +then the +.I +use +.R +of the function is portable. +.PP +The pointer +representation on the +PDP-11 +corresponds to a 16-bit integer and +measures bytes. +The +.B +char\fR's +have no alignment requirements; everything else must have an even address. +.PP +On the +VAX-11, +pointers are 32 bits long and measure bytes. +Elementary objects are aligned on a boundary equal to their +length, except that +.B +double +.R +quantities need be aligned only on even 4-byte boundaries. +Aggregates are aligned on the strictest boundary required by +any of their constituents. +.PP +The 3B 20 computer has 24-bit pointers placed into 32-bit quantities. +Most objects are +aligned on 4-byte boundaries. \fBShort\fRs are aligned in all cases on +2-byte boundaries. Arrays of characters, all structures, +\fBint\fR\^s, \fBlong\fR\^s, \fBfloat\fR\^s, and \fBdouble\fR\^s are aligned on 4-byte +boundaries; but structure members may be packed tighter. +.nr Hu 1 +.NH 2 +CONSTANT EXPRESSIONS +.PP +In several places C requires expressions that evaluate to +a constant: +after +\fBcase\fR, +as array bounds, and in initializers. +In the first two cases, the expression can +involve only integer constants, character constants, +casts to integral types, +enumeration constants, +and +.B +sizeof +.R +expressions, possibly +connected by the binary operators +.ne 10 +.DS +\(pl \(mi \(** / % & | ^ << >> == != < > <= >= && || +.DE +.LP +or by the unary operators +.DS +\(mi \s+2~\s0 +.DE +.LP +or by the ternary operator +.DS +?: +.DE +.PP +Parentheses can be used for grouping +but not for function calls. +.PP +More latitude is permitted for initializers; +besides constant expressions as discussed above, +one can also use floating constants +and arbitrary casts and +can also apply the unary +.B +& +.R +operator to external or static objects +and to external or static arrays subscripted +with a constant expression. +The unary +.B +& +.R +can also +be applied implicitly +by appearance of unsubscripted arrays and functions. +The basic rule is that initializers must +evaluate either to a constant or to the address +of a previously declared external or static object plus or minus a constant. +.nr Hu 1 +.NH 1 +Portability Considerations +.PP +Certain parts of C are inherently machine dependent. +The following list of potential trouble spots +is not meant to be all-inclusive +but to point out the main ones. +.PP +Purely hardware issues like +word size and the properties of floating point arithmetic and integer division +have proven in practice to be not much of a problem. +Other facets of the hardware are reflected +in differing implementations. +Some of these, +particularly sign extension +(converting a negative character into a negative integer) +and the order in which bytes are placed in a word, +are nuisances that must be carefully watched. +Most of the others are only minor problems. +.PP +The number of +.B +register +.R +variables that can actually be placed in registers +varies from machine to machine +as does the set of valid types. +Nonetheless, the compilers all do things properly for their own machine; +excess or invalid +.B +register +.R +declarations are ignored. +.PP +Some difficulties arise only when +dubious coding practices are used. +It is exceedingly unwise to write programs +that depend +on any of these properties. +.PP +The order of evaluation of function arguments +is not specified by the language. +The order in which side effects take place +is also unspecified. +.PP +Since character constants are really objects of type +\fBint\fR, +multicharacter character constants may be permitted. +The specific implementation +is very machine dependent +because the order in which characters +are assigned to a word +varies from one machine to another. +.PP +Fields are assigned to words and characters to integers right to left +on some machines +and left to right on other machines. +These differences are invisible to isolated programs +that do not indulge in type punning (e.g., +by converting an +.B +int +.R +pointer to a +.B +char +.R +pointer and inspecting the pointed-to storage) +but must be accounted for when conforming to externally-imposed +storage layouts. +.nr Hu 1 +.NH 1 +Syntax Summary +.PP +This summary of C syntax is intended more for aiding comprehension +than as an exact statement of the language. +.nr Hu 1 +.ne 18 +.NH 2 +Expressions +.PP +The basic expressions are: +.tr ~~ +.DS + \fIexpression: + primary + \(** expression\fR + &\fIlvalue + \(mi expression + ! expression + \s+2~\s0 expression + \(pl\(pl lvalue + \(mi\(milvalue + lvalue \(pl\(pl + lvalue \(mi\(mi + \fBsizeof\fI expression + \fBsizeof (\fItype-name\fB)\fI + ( type-name ) expression + expression binop expression + expression ? expression : expression + lvalue asgnop expression + expression , expression +.DE +.DS + \fIprimary: + identifier + constant + string + ( expression ) + primary ( expression-list\v'0.5'\s-2opt\s0\v'-0.5' ) + primary [ expression ] + primary . identifier + primary \(mi identifier +.DE +.DS + \fIlvalue: + identifier + primary [ expression ] + lvalue . identifier + primary \(mi identifier + \(** expression + ( lvalue )\fR +.DE +.PP +.PP +The primary-expression operators +.DS + () [] . \(mi +.tr ~~ +.DE +.LP +have highest priority and group left to right. +The unary operators +.DS + \(** & \(mi ! \s+2~\s0 \(pl\(pl \(mi\(mi \fBsizeof\fI ( type-name \fR) +.DE +.LP +have priority below the primary operators +but higher than any binary operator +and group right to left. +Binary operators +group left to right; they have priority +decreasing +as indicated below. +.DS + \fIbinop:\fR + \(** / % + \(pl \(mi + >> << + < > <= >= + == != + & + ^ + | + && + || +.DE +The conditional operator groups right to left. +.PP +Assignment operators all have the same +priority and all group right to left. +.DS + \fIasgnop:\fR + = \(pl= \(mi= \(**= /= %= >>= <<= &= ^= |= +.DE +.PP +The comma operator has the lowest priority and groups left to right. +.nr Hu 1 +.NH 2 +Declarations +.PP +.DS + \fIdeclaration: + decl-specifiers init-declarator-list\v'0.5'\s-2opt\s0\v'-0.5' ; +.DE +.DS + \fIdecl-specifiers: + type-specifier decl-specifiers\v'0.5'\s-2opt\s0\v'-0.5' + sc-specifier decl-specifiers\v'0.5'\s-2opt\s0\v'-0.5' +.DE +.DS + \fIsc-specifier:\fB + auto + static + extern + register + typedef +.DE +.DS + \fItype-specifier: + struct-or-union-specifier + typedef-name + enum-specifier + basic-type-specifier: + basic-type + basic-type basic-type-specifiers + basic-type:\fB + char + short + int + long + unsigned + float + double + void\fR +.DE +.DS +\fIenum-specifier:\fB + enum\fI { enum-list }\fB + enum \fIidentifier { enum-list }\fB + enum \fIidentifier +.DE +.DS + \fIenum-list: + enumerator + enum-list , enumerator +.DE +.DS + \fIenumerator: + identifier + identifier = constant-expression +.DE +.DS + \fIinit-declarator-list: + init-declarator + init-declarator , init-declarator-list +.DE +.DS + \fIinit-declarator: + declarator initializer\v'0.5'\s-2opt\s0\v'-0.5' +.DE +.DS + \fIdeclarator: + identifier + ( declarator ) + \(** declarator + declarator () + declarator [ constant-expression\v'0.5'\s-2opt\s0\v'-0.5' ] +.DE +.DS + \fIstruct-or-union-specifier:\fB + struct\fI { struct-decl-list }\fB + struct \fIidentifier { struct-decl-list }\fB + struct \fIidentifier\fB + union { \fIstruct-decl-list }\fB + union \fIidentifier { struct-decl-list }\fB + union \fIidentifier +.DE +.DS + \fIstruct-decl-list: + struct-declaration + struct-declaration struct-decl-list +.DE +.DS + \fIstruct-declaration: + type-specifier struct-declarator-list ; +.DE +.DS + \fIstruct-declarator-list: + struct-declarator + struct-declarator , struct-declarator-list +.DE +.DS + \fIstruct-declarator: + declarator + declarator : constant-expression + : constant-expression +.DE +.DS + \fIinitializer: + = expression + = { initializer-list } + = { initializer-list , } +.DE +.DS + \fIinitializer-list: + expression + initializer-list , initializer-list + { initializer-list } + { initializer-list , } +.DE +.DS + \fItype-name: + type-specifier abstract-declarator +.DE +.DS + \fIabstract-declarator: + empty + ( abstract-declarator ) + \(** abstract-declarator + abstract-declarator () + abstract-declarator [ constant-expression\v'0.5'\s-2opt\s0\v'-0.5' ] +.DE +.DS + \fItypedef-name: + identifier +.nr Hu 1 +.DE +.NH 2 +Statements +.PP +.DS + \fIcompound-statement: + { declaration-list\v'0.5'\s-2opt\s0\v'-0.5' statement-list\v'0.5'\s-2opt\s0\v'-0.5' } +.DE +.DS + \fIdeclaration-list: + declaration + declaration declaration-list +.DE +.DS + \fIstatement-list: + statement + statement statement-list +.DE +.DS + \fIstatement: + compound-statement + expression ; + \fBif\fI ( expression ) statement + \fBif\fI ( expression ) statement \fBelse\fI statement + \fBwhile\fI ( expression ) statement + \fBdo\fI statement \fBwhile\fI ( expression ) ; + \fBfor\fI (exp\v'0.3'\s-2opt\s0\v'-0.3'\fB;\fIexp\v'0.3'\s-2opt\s0\v'-0.3'\fB;\fIexp\v'0.3'\s-2opt\s0\v'-0.3'\fI) statement + \fBswitch\fI ( expression ) statement + \fBcase\fI constant-expression : statement + \fBdefault\fI : statement + \fBbreak ; + continue ; + return ; + return\fI expression ; + \fBgoto\fI identifier ; + identifier : statement + ;\fR +.nr Hu 1 +.DE +.NH 2 +External definitions +.PP +.DS + \fIprogram: + external-definition + external-definition program +.DE +.DS + \fIexternal-definition: + function-definition + data-definition +.DE +.DS + \fIfunction-definition: + decl-specifier\v'0.5'\s-2opt\s0\v'-0.5' function-declarator function-body +.DE +.DS + \fIfunction-declarator: + declarator ( parameter-list\v'0.5'\s-2opt\s0\v'-0.5' ) +.DE +.DS + \fIparameter-list: + identifier + identifier , parameter-list +.DE +.DS + \fIfunction-body: + declaration-list\v'0.5'\s-2opt\s0\v'-0.5' compound-statement +.DE +.DS + \fIdata-definition: + \fBextern\fI declaration\fB ; + \fBstatic\fI declaration\fB ; +.DE +.NH +Preprocessor +.DS + \fB#define\fI identifier token-string\v'0.3'\s-2opt\s0\v'-0.3'\fB + \fB#define\fI identifier\fB(\fIidentifier\fB,...)\fItoken-string\v'0.5'\s-2opt\s0\v'-0.5'\fB + \fB#undef\fI identifier\fB + \fB#include "\fIfilename\|\fB" + #include <\fIfilename\|\fB> + \fB#if\fI restricted-constant-expression\fB + \fB#ifdef\fI identifier\fB + \fB#ifndef\fI identifier\fB + \fB#else + \fB#endif + \fB#line\fI constant \fB"\fIfilename\|\fB" +.sp 5 +.DE +.\" .TC 2 1 3 0 diff --git a/share/doc/psd/06.Clang/Makefile b/share/doc/psd/06.Clang/Makefile new file mode 100644 index 000000000000..877a97cd7073 --- /dev/null +++ b/share/doc/psd/06.Clang/Makefile @@ -0,0 +1,9 @@ +# @(#)Makefile 8.1 (Berkeley) 6/8/93 +# $FreeBSD$ + +VOLUME= psd/06.Clang +SRCS= Clang.ms +MACROS= -ms +USE_TBL= + +.include <bsd.doc.mk> diff --git a/share/doc/psd/12.make/Makefile b/share/doc/psd/12.make/Makefile new file mode 100644 index 000000000000..b36568352384 --- /dev/null +++ b/share/doc/psd/12.make/Makefile @@ -0,0 +1,8 @@ +# From: @(#)Makefile 8.1 (Berkeley) 8/14/93 +# $FreeBSD$ + +VOLUME= psd/12.make +SRCS= stubs tutorial.ms +MACROS= -ms + +.include <bsd.doc.mk> diff --git a/share/doc/psd/12.make/stubs b/share/doc/psd/12.make/stubs new file mode 100644 index 000000000000..39d8defe2e14 --- /dev/null +++ b/share/doc/psd/12.make/stubs @@ -0,0 +1,9 @@ +.\" $FreeBSD$ +.\" +.de Ix +.. +.de Rd +.. +.de Rm +.. +.if n .ftr CR R diff --git a/share/doc/psd/12.make/tutorial.ms b/share/doc/psd/12.make/tutorial.ms new file mode 100644 index 000000000000..320d5dfc3c4b --- /dev/null +++ b/share/doc/psd/12.make/tutorial.ms @@ -0,0 +1,3747 @@ +.\" Copyright (c) 1988, 1989 by Adam de Boor +.\" Copyright (c) 1989 by Berkeley Softworks +.\" Copyright (c) 1988, 1989, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" This code is derived from software contributed to Berkeley by +.\" Adam de Boor. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)tutorial.ms 8.1 (Berkeley) 8/18/93 +.\" $FreeBSD$ +.\" +.EH 'PSD:12-%''PMake \*- A Tutorial' +.OH 'PMake \*- A Tutorial''PSD:12-%' +.\" xH is a macro to provide numbered headers that are automatically stuffed +.\" into a table-of-contents, properly indented, etc. If the first argument +.\" is numeric, it is taken as the depth for numbering (as for .NH), else +.\" the default (1) is assumed. +.\" +.\" @P The initial paragraph distance. +.\" @Q The piece of section number to increment (or 0 if none given) +.\" @R Section header. +.\" @S Indent for toc entry +.\" @T Argument to NH (can't use @Q b/c giving 0 to NH resets the counter) +.de xH +.NH \\$1 +\\$2 \\$3 \\$4 \\$5 \\$6 \\$7 \\$8 \\$9 +.nr PD .1v +.XS \\n% +.ta 0.6i +\\*(SN \\$2 \\$3 \\$4 \\$5 \\$6 \\$7 \\$8 \\$9 +.XE +.nr PD .3v +.. +.ig +.\" CW is used to place a string in fixed-width or switch to a +.\" fixed-width font. +.\" C is a typewriter font for a laserwriter. Use something else if +.\" you don't have one... +.de CW +.ie !\\n(.$ .ft S +.el \&\\$3\fS\\$1\fP\\$2 +.. +.\" Anything I put in a display I want to be in fixed-width +.am DS +.CW +.. +.\" The stuff in .No produces a little stop sign in the left margin +.\" that says NOTE in it. Unfortunately, it does cause a break, but +.\" hey. Can't have everything. In case you're wondering how I came +.\" up with such weird commands, they came from running grn on a +.\" gremlin file... +.de No +.br +.ne 0.5i +.po -0.5i +.br +.mk +.nr g3 \\n(.f +.nr g4 \\n(.s +.ig ft +.sp -1 +.\" .st cf +\D's -1u'\D't 5u' +.sp -1 +\h'50u'\D'l 71u 0u'\D'l 50u 50u'\D'l 0u 71u'\D'l -50u 50u'\D'l -71u 0u'\D'l -50u -50u'\D'l 0u -71u'\D'l 50u -50u' +.sp -1 +\D't 3u' +.sp -1 +.sp 7u +\h'53u'\D'p 14 68u 0u 46u 46u 0u 68u -46u 46u -68u 0u -47u -46u 0u -68u 47u -46u' +.sp -1 +.ft R +.ps 6 +.nr g8 \\n(.d +.ds g9 "NOTE +.sp 74u +\h'85u'\v'0.85n'\h-\w\\*(g9u/2u\&\\*(g9 +.sp |\\n(g8u +.sp 166u +.ig br +\D't 3u'\D's -1u' +.br +.po +.rt +.ft \\n(g3 +.ps \\n(g4 +.. +.de Bp +.ie !\\n(.$ .IP \(bu 2 +.el .IP "\&" 2 +.. +.po +.3i +.TL +PMake \*- A Tutorial +.AU +Adam de Boor +.AI +Berkeley Softworks +2150 Shattuck Ave, Penthouse +Berkeley, CA 94704 +adam@bsw.uu.net +\&...!uunet!bsw!adam +.FS +Permission to use, copy, modify, and distribute this software and its +documentation for any purpose and without fee is hereby granted, +provided that the above copyright notice appears in all copies. +The University of California, Berkeley Softworks, and Adam de Boor make no +representations about the suitability of this software for any +purpose. It is provided "as is" without express or implied warranty. +.FE +.PP +.xH 1 Introduction +.LP +PMake is a program for creating other programs, or anything else you +can think of for it to do. The basic idea behind PMake is that, for +any given system, be it a program or a document or whatever, there +will be some files that depend on the state of other files (on when +they were last modified). PMake takes these dependencies, which you +must specify, and uses them to build whatever it is you want it to +build. +.LP +PMake is almost fully-compatible with Make, with which you may already +be familiar. PMake's most important feature is its ability to run +several different jobs at once, making the creation of systems +considerably faster. It also has a great deal more functionality than +Make. Throughout the text, whenever something is mentioned that is an +important difference between PMake and Make (i.e. something that will +cause a makefile to fail if you don't do something about it), or is +simply important, it will be flagged with a little sign in the left +margin, like this: +.No +.LP +This tutorial is divided into three main sections corresponding to basic, +intermediate and advanced PMake usage. If you already know Make well, +you will only need to skim chapter 2 (there are some aspects of +PMake that I consider basic to its use that didn't exist in Make). +Things in chapter 3 make life much easier, while those in chapter 4 +are strictly for those who know what they are doing. Chapter 5 has +definitions for the jargon I use and chapter 6 contains possible +solutions to the problems presented throughout the tutorial. +.xH 1 The Basics of PMake +.LP +PMake takes as input a file that tells a) which files depend on which +other files to be complete and b) what to do about files that are +``out-of-date.'' This file is known as a ``makefile'' and is usually +.Ix 0 def makefile +kept in the top-most directory of the system to be built. While you +can call the makefile anything you want, PMake will look for +.CW Makefile +and +.CW makefile +(in that order) in the current directory if you don't tell it +otherwise. +.Ix 0 def makefile default +To specify a different makefile, use the +.B \-f +flag (e.g. +.CW "pmake -f program.mk" ''). `` +.Ix 0 ref flags -f +.Ix 0 ref makefile other +.LP +A makefile has four different types of lines in it: +.RS +.IP \(bu 2 +File dependency specifications +.IP \(bu 2 +Creation commands +.IP \(bu 2 +Variable assignments +.IP \(bu 2 +Comments, include statements and conditional directives +.RE +.LP +Any line may be continued over multiple lines by ending it with a +backslash. +.Ix 0 def "continuation line" +The backslash, following newline and any initial whitespace +on the following line are compressed into a single space before the +input line is examined by PMake. +.xH 2 Dependency Lines +.LP +As mentioned in the introduction, in any system, there are +dependencies between the files that make up the system. For instance, +in a program made up of several C source files and one header file, +the C files will need to be re-compiled should the header file be +changed. For a document of several chapters and one macro file, the +chapters will need to be reprocessed if any of the macros changes. +.Ix 0 def "dependency" +These are dependencies and are specified by means of dependency lines in +the makefile. +.LP +.Ix 0 def "dependency line" +On a dependency line, there are targets and sources, separated by a +one- or two-character operator. +The targets ``depend'' on the sources and are usually created from +them. +.Ix 0 def target +.Ix 0 def source +.Ix 0 ref operator +Any number of targets and sources may be specified on a dependency line. +All the targets in the line are made to depend on all the sources. +Targets and sources need not be actual files, but every source must be +either an actual file or another target in the makefile. +If you run out of room, use a backslash at the end of the line to continue onto +the next one. +.LP +Any file may be a target and any file may be a source, but the +relationship between the two (or however many) is determined by the +``operator'' that separates them. +.Ix 0 def operator +Three types of operators exist: one specifies that the datedness of a +target is determined by the state of its sources, while another +specifies other files (the sources) that need to be dealt with before +the target can be re-created. The third operator is very similar to +the first, with the additional condition that the target is +out-of-date if it has no sources. These operations are represented by +the colon, the exclamation point and the double-colon, respectively, and are +mutually exclusive. Their exact semantics are as follows: +.IP ":" +.Ix 0 def operator colon +.Ix 0 def : +If a colon is used, a target on the line is considered to be +``out-of-date'' (and in need of creation) if +.RS +.IP \(bu 2 +any of the sources has been modified more recently than the target, or +.IP \(bu 2 +the target doesn't exist. +.RE +.Ix 0 def out-of-date +.IP "\&" +Under this operation, steps will be taken to re-create the target only +if it is found to be out-of-date by using these two rules. +.IP "!" +.Ix 0 def operator force +.Ix 0 def ! +If an exclamation point is used, the target will always be re-created, +but this will not happen until all of its sources have been examined +and re-created, if necessary. +.IP "::" +.Ix 0 def operator double-colon +.Ix 0 def :: +If a double-colon is used, a target is out-of-date if: +.RS +.IP \(bu 2 +any of the sources has been modified more recently than the target, or +.IP \(bu 2 +the target doesn't exist, or +.IP \(bu 2 +the target has no sources. +.RE +.IP "\&" +If the target is out-of-date according to these rules, it will be re-created. +This operator also does something else to the targets, but I'll go +into that in the next section (``Shell Commands''). +.LP +Enough words, now for an example. Take that C program I mentioned +earlier. Say there are three C files +.CW a.c , ( +.CW b.c +and +.CW c.c ) +each of which +includes the file +.CW defs.h . +The dependencies between the files could then be expressed as follows: +.DS +program : a.o b.o c.o +a.o b.o c.o : defs.h +a.o : a.c +b.o : b.c +c.o : c.c +.DE +.LP +You may be wondering at this point, where +.CW a.o , +.CW b.o +and +.CW c.o +came in and why +.I they +depend on +.CW defs.h +and the C files don't. The reason is quite simple: +.CW program +cannot be made by linking together .c files \*- it must be +made from .o files. Likewise, if you change +.CW defs.h , +it isn't the .c files that need to be re-created, it's the .o files. +If you think of dependencies in these terms \*- which files (targets) +need to be created from which files (sources) \*- you should have no problems. +.LP +An important thing to notice about the above example, is that all the +\&.o files appear as targets on more than one line. This is perfectly +all right: the target is made to depend on all the sources mentioned +on all the dependency lines. E.g. +.CW a.o +depends on both +.CW defs.h +and +.CW a.c . +.Ix 0 ref dependency +.No +.LP +The order of the dependency lines in the makefile is +important: the first target on the first dependency line in the +makefile will be the one that gets made if you don't say otherwise. +That's why +.CW program +comes first in the example makefile, above. +.LP +Both targets and sources may contain the standard C-Shell wildcard +characters +.CW { , ( +.CW } , +.CW * , +.CW ? , +.CW [ , +and +.CW ] ), +but the non-curly-brace ones may only appear in the final component +(the file portion) of the target or source. The characters mean the +following things: +.IP \fB{}\fP +These enclose a comma-separated list of options and cause the pattern +to be expanded once for each element of the list. Each expansion +contains a different element. For example, +.CW src/{whiffle,beep,fish}.c +expands to the three words +.CW src/whiffle.c , +.CW src/beep.c , +and +.CW src/fish.c . +These braces may be nested and, unlike the other wildcard characters, +the resulting words need not be actual files. All other wildcard +characters are expanded using the files that exist when PMake is +started. +.IP \fB*\fP +This matches zero or more characters of any sort. +.CW src/*.c +will expand to the same three words as above as long as +.CW src +contains those three files (and no other files that end in +.CW .c ). +.IP \fB?\fP +Matches any single character. +.IP \fB[]\fP +This is known as a character class and contains either a list of +single characters, or a series of character ranges +.CW a-z , ( +for example means all characters between a and z), or both. It matches +any single character contained in the list. E.g. +.CW [A-Za-z] +will match all letters, while +.CW [0123456789] +will match all numbers. +.xH 2 Shell Commands +.LP +``Isn't that nice,'' you say to yourself, ``but how are files +actually `re-created,' as he likes to spell it?'' +The re-creation is accomplished by commands you place in the makefile. +These commands are passed to the Bourne shell (better known as +``/bin/sh'') to be executed and are +.Ix 0 ref shell +.Ix 0 ref re-creation +.Ix 0 ref update +expected to do what's necessary to update the target file (PMake +doesn't actually check to see if the target was created. It just +assumes it's there). +.Ix 0 ref target +.LP +Shell commands in a makefile look a lot like shell commands you would +type at a terminal, with one important exception: each command in a +makefile +.I must +be preceded by at least one tab. +.LP +Each target has associated with it a shell script made up of +one or more of these shell commands. The creation script for a target +should immediately follow the dependency line for that target. While +any given target may appear on more than one dependency line, only one +of these dependency lines may be followed by a creation script, unless +the `::' operator was used on the dependency line. +.Ix 0 ref operator double-colon +.Ix 0 ref :: +.No +.LP +If the double-colon was used, each dependency line for the target +may be followed by a shell script. That script will only be executed +if the target on the associated dependency line is out-of-date with +respect to the sources on that line, according to the rules I gave +earlier. +I'll give you a good example of this later on. +.LP +To expand on the earlier makefile, you might add commands as follows: +.DS +program : a.o b.o c.o + cc a.o b.o c.o \-o program +a.o b.o c.o : defs.h +a.o : a.c + cc \-c a.c +b.o : b.c + cc \-c b.c +c.o : c.c + cc \-c c.c +.DE +.LP +Something you should remember when writing a makefile is, the +commands will be executed if the +.I target +on the dependency line is out-of-date, not the sources. +.Ix 0 ref target +.Ix 0 ref source +.Ix 0 ref out-of-date +In this example, the command +.CW "cc \-c a.c" '' `` +will be executed if +.CW a.o +is out-of-date. Because of the `:' operator, +.Ix 0 ref : +.Ix 0 ref operator colon +this means that should +.CW a.c +.I or +.CW defs.h +have been modified more recently than +.CW a.o , +the command will be executed +.CW a.o "\&" ( +will be considered out-of-date). +.Ix 0 ref out-of-date +.LP +Remember how I said the only difference between a makefile shell +command and a regular shell command was the leading tab? I lied. There +is another way in which makefile commands differ from regular ones. +The first two characters after the initial whitespace are treated +specially. +If they are any combination of `@' and `\-', they cause PMake to do +different things. +.LP +In most cases, shell commands are printed before they're +actually executed. This is to keep you informed of what's going on. If +an `@' appears, however, this echoing is suppressed. In the case of an +.CW echo +command, say +.CW "echo Linking index" ,'' `` +it would be +rather silly to see +.DS +echo Linking index +Linking index +.DE +.LP +so PMake allows you to place an `@' before the command +.CW "@echo Linking index" '') (`` +to prevent the command from being printed. +.LP +The other special character is the `\-'. In case you didn't know, +shell commands finish with a certain ``exit status.'' This status is +made available by the operating system to whatever program invoked the +command. Normally this status will be 0 if everything went ok and +non-zero if something went wrong. For this reason, PMake will consider +an error to have occurred if one of the shells it invokes returns a non-zero +status. When it detects an error, PMake's usual action is to abort +whatever it's doing and exit with a non-zero status itself (any other +targets that were being created will continue being made, but nothing +new will be started. PMake will exit after the last job finishes). +This behavior can be altered, however, by placing a `\-' at the front +of a command +.CW "\-mv index index.old" ''), (`` +certain command-line arguments, +or doing other things, to be detailed later. In such +a case, the non-zero status is simply ignored and PMake keeps chugging +along. +.No +.LP +Because all the commands are given to a single shell to execute, such +things as setting shell variables, changing directories, etc., last +beyond the command in which they are found. This also allows shell +compound commands (like +.CW for +loops) to be entered in a natural manner. +Since this could cause problems for some makefiles that depend on +each command being executed by a single shell, PMake has a +.B \-B +.Ix 0 ref compatibility +.Ix 0 ref flags -B +flag (it stands for backwards-compatible) that forces each command to +be given to a separate shell. It also does several other things, all +of which I discourage since they are now old-fashioned.\|.\|.\|. +.No +.LP +A target's shell script is fed to the shell on its (the shell's) input stream. +This means that any commands, such as +.CW ci +that need to get input from the terminal won't work right \*- they'll +get the shell's input, something they probably won't find to their +liking. A simple way around this is to give a command like this: +.DS +ci $(SRCS) < /dev/tty +.DE +This would force the program's input to come from the terminal. If you +can't do this for some reason, your only other alternative is to use +PMake in its fullest compatibility mode. See +.B Compatibility +in chapter 4. +.Ix 0 ref compatibility +.LP +.xH 2 Variables +.LP +PMake, like Make before it, has the ability to save text in variables +to be recalled later at your convenience. Variables in PMake are used +much like variables in the shell and, by tradition, consist of +all upper-case letters (you don't +.I have +to use all upper-case letters. +In fact there's nothing to stop you from calling a variable +.CW @^&$%$ . +Just tradition). Variables are assigned-to using lines of the form +.Ix 0 def variable assignment +.DS +VARIABLE = value +.DE +.Ix 0 def variable assignment +appended-to by +.DS +VARIABLE += value +.DE +.Ix 0 def variable appending +.Ix 0 def variable assignment appended +.Ix 0 def += +conditionally assigned-to (if the variable isn't already defined) by +.DS +VARIABLE ?= value +.DE +.Ix 0 def variable assignment conditional +.Ix 0 def ?= +and assigned-to with expansion (i.e. the value is expanded (see below) +before being assigned to the variable\*-useful for placing a value at +the beginning of a variable, or other things) by +.DS +VARIABLE := value +.DE +.Ix 0 def variable assignment expanded +.Ix 0 def := +.LP +Any whitespace before +.I value +is stripped off. When appending, a space is placed between the old +value and the stuff being appended. +.LP +The final way a variable may be assigned to is using +.DS +VARIABLE != shell-command +.DE +.Ix 0 def variable assignment shell-output +.Ix 0 def != +In this case, +.I shell-command +has all its variables expanded (see below) and is passed off to a +shell to execute. The output of the shell is then placed in the +variable. Any newlines (other than the final one) are replaced by +spaces before the assignment is made. This is typically used to find +the current directory via a line like: +.DS +CWD != pwd +.DE +.LP +.B Note: +this is intended to be used to execute commands that produce small amounts +of output (e.g. ``pwd''). The implementation is less than intelligent and will +likely freeze if you execute something that produces thousands of +bytes of output (8 Kb is the limit on many UNIX systems). +.LP +The value of a variable may be retrieved by enclosing the variable +name in parentheses or curly braces and preceding the whole thing +with a dollar sign. +.LP +For example, to set the variable CFLAGS to the string +.CW "\-I/sprite/src/lib/libc \-O" ,'' `` +you would place a line +.DS +CFLAGS = \-I/sprite/src/lib/libc \-O +.DE +in the makefile and use the word +.CW "$(CFLAGS)" +wherever you would like the string +.CW "\-I/sprite/src/lib/libc \-O" +to appear. This is called variable expansion. +.Ix 0 def variable expansion +.No +.LP +Unlike Make, PMake will not expand a variable unless it knows +the variable exists. E.g. if you have a +.CW "${i}" +in a shell command and you have not assigned a value to the variable +.CW i +(the empty string is considered a value, by the way), where Make would have +substituted the empty string, PMake will leave the +.CW "${i}" +alone. +To keep PMake from substituting for a variable it knows, precede the +dollar sign with another dollar sign. +(e.g. to pass +.CW "${HOME}" +to the shell, use +.CW "$${HOME}" ). +This causes PMake, in effect, to expand the +.CW $ +macro, which expands to a single +.CW $ . +For compatibility, Make's style of variable expansion will be used +if you invoke PMake with any of the compatibility flags (\c +.B \-V , +.B \-B +or +.B \-M . +The +.B \-V +flag alters just the variable expansion). +.Ix 0 ref flags -V +.Ix 0 ref flags -B +.Ix 0 ref flags -M +.Ix 0 ref compatibility +.LP +.Ix 0 ref variable expansion +There are two different times at which variable expansion occurs: +When parsing a dependency line, the expansion occurs immediately +upon reading the line. If any variable used on a dependency line is +undefined, PMake will print a message and exit. +Variables in shell commands are expanded when the command is +executed. +Variables used inside another variable are expanded whenever the outer +variable is expanded (the expansion of an inner variable has no effect +on the outer variable. I.e. if the outer variable is used on a dependency +line and in a shell command, and the inner variable changes value +between when the dependency line is read and the shell command is +executed, two different values will be substituted for the outer +variable). +.Ix 0 def variable types +.LP +Variables come in four flavors, though they are all expanded the same +and all look about the same. They are (in order of expanding scope): +.RS +.IP \(bu 2 +Local variables. +.Ix 0 ref variable local +.IP \(bu 2 +Command-line variables. +.Ix 0 ref variable command-line +.IP \(bu 2 +Global variables. +.Ix 0 ref variable global +.IP \(bu 2 +Environment variables. +.Ix 0 ref variable environment +.RE +.LP +The classification of variables doesn't matter much, except that the +classes are searched from the top (local) to the bottom (environment) +when looking up a variable. The first one found wins. +.xH 3 Local Variables +.LP +.Ix 0 def variable local +Each target can have as many as seven local variables. These are +variables that are only ``visible'' within that target's shell script +and contain such things as the target's name, all of its sources (from +all its dependency lines), those sources that were out-of-date, etc. +Four local variables are defined for all targets. They are: +.RS +.IP ".TARGET" +.Ix 0 def variable local .TARGET +.Ix 0 def .TARGET +The name of the target. +.IP ".OODATE" +.Ix 0 def variable local .OODATE +.Ix 0 def .OODATE +The list of the sources for the target that were considered out-of-date. +The order in the list is not guaranteed to be the same as the order in +which the dependencies were given. +.IP ".ALLSRC" +.Ix 0 def variable local .ALLSRC +.Ix 0 def .ALLSRC +The list of all sources for this target in the order in which they +were given. +.IP ".PREFIX" +.Ix 0 def variable local .PREFIX +.Ix 0 def .PREFIX +The target without its suffix and without any leading path. E.g. for +the target +.CW ../../lib/compat/fsRead.c , +this variable would contain +.CW fsRead . +.RE +.LP +Three other local variables are set only for certain targets under +special circumstances. These are the ``.IMPSRC,'' +.Ix 0 ref variable local .IMPSRC +.Ix 0 ref .IMPSRC +``.ARCHIVE,'' +.Ix 0 ref variable local .ARCHIVE +.Ix 0 ref .ARCHIVE +and ``.MEMBER'' +.Ix 0 ref variable local .MEMBER +.Ix 0 ref .MEMBER +variables. When they are set and how they are used is described later. +.LP +Four of these variables may be used in sources as well as in shell +scripts. +.Ix 0 def "dynamic source" +.Ix 0 def source dynamic +These are ``.TARGET'', ``.PREFIX'', ``.ARCHIVE'' and ``.MEMBER''. The +variables in the sources are expanded once for each target on the +dependency line, providing what is known as a ``dynamic source,'' +.Rd 0 +allowing you to specify several dependency lines at once. For example, +.DS +$(OBJS) : $(.PREFIX).c +.DE +will create a dependency between each object file and its +corresponding C source file. +.xH 3 Command-line Variables +.LP +.Ix 0 def variable command-line +Command-line variables are set when PMake is first invoked by giving a +variable assignment as one of the arguments. For example, +.DS +pmake "CFLAGS = -I/sprite/src/lib/libc -O" +.DE +would make +.CW CFLAGS +be a command-line variable with the given value. Any assignments to +.CW CFLAGS +in the makefile will have no effect, because once it +is set, there is (almost) nothing you can do to change a command-line +variable (the search order, you see). Command-line variables may be +set using any of the four assignment operators, though only +.CW = +and +.CW ?= +behave as you would expect them to, mostly because assignments to +command-line variables are performed before the makefile is read, thus +the values set in the makefile are unavailable at the time. +.CW += +.Ix 0 ref += +.Ix 0 ref variable assignment appended +is the same as +.CW = , +because the old value of the variable is sought only in the scope in +which the assignment is taking place (for reasons of efficiency that I +won't get into here). +.CW := +and +.CW ?= +.Ix 0 ref := +.Ix 0 ref ?= +.Ix 0 ref variable assignment expanded +.Ix 0 ref variable assignment conditional +will work if the only variables used are in the environment. +.CW != +is sort of pointless to use from the command line, since the same +effect can no doubt be accomplished using the shell's own command +substitution mechanisms (backquotes and all that). +.xH 3 Global Variables +.LP +.Ix 0 def variable global +Global variables are those set or appended-to in the makefile. +There are two classes of global variables: those you set and those PMake sets. +As I said before, the ones you set can have any name you want them to have, +except they may not contain a colon or an exclamation point. +The variables PMake sets (almost) always begin with a +period and always contain upper-case letters, only. The variables are +as follows: +.RS +.IP .PMAKE +.Ix 0 def variable global .PMAKE +.Ix 0 def .PMAKE +.Ix 0 def variable global MAKE +.Ix 0 def MAKE +The name by which PMake was invoked is stored in this variable. For +compatibility, the name is also stored in the MAKE variable. +.IP .MAKEFLAGS +.Ix 0 def variable global .MAKEFLAGS +.Ix 0 def .MAKEFLAGS variable +.Ix 0 def variable global MFLAGS +.Ix 0 def MFLAGS +All the relevant flags with which PMake was invoked. This does not +include such things as +.B \-f +or variable assignments. Again for compatibility, this value is stored +in the MFLAGS variable as well. +.RE +.LP +Two other variables, ``.INCLUDES'' and ``.LIBS,'' are covered in the +section on special targets in chapter 3. +.Ix 0 ref variable global .INCLUDES +.Ix 0 ref variable global .LIBS +.LP +Global variables may be deleted using lines of the form: +.Ix 0 def #undef +.Ix 0 def variable deletion +.DS +#undef \fIvariable\fP +.DE +The +.CW # ' ` +must be the first character on the line. Note that this may only be +done on global variables. +.xH 3 Environment Variables +.LP +.Ix 0 def variable environment +Environment variables are passed by the shell that invoked PMake and +are given by PMake to each shell it invokes. They are expanded like +any other variable, but they cannot be altered in any way. +.LP +One special environment variable, +.CW PMAKE , +.Ix 0 def variable environment PMAKE +is examined by PMake for command-line flags, variable assignments, +etc., it should always use. This variable is examined before the +actual arguments to PMake are. In addition, all flags given to PMake, +either through the +.CW PMAKE +variable or on the command line, are placed in this environment +variable and exported to each shell PMake executes. Thus recursive +invocations of PMake automatically receive the same flags as the +top-most one. +.LP +Using all these variables, you can compress the sample makefile even more: +.DS +OBJS = a.o b.o c.o +program : $(OBJS) + cc $(.ALLSRC) \-o $(.TARGET) +$(OBJS) : defs.h +a.o : a.c + cc \-c a.c +b.o : b.c + cc \-c b.c +c.o : c.c + cc \-c c.c +.DE +.Ix 0 ref variable local .ALLSRC +.Ix 0 ref .ALLSRC +.Ix 0 ref variable local .TARGET +.Ix 0 ref .TARGET +.Rd 3 +.xH 2 Comments +.LP +.Ix 0 def comments +Comments in a makefile start with a `#' character and extend to the +end of the line. They may appear +anywhere you want them, except in a shell command (though the shell +will treat it as a comment, too). If, for some reason, you need to use the `#' +in a variable or on a dependency line, put a backslash in front of it. +PMake will compress the two into a single `#' (Note: this isn't true +if PMake is operating in full-compatibility mode). +.Ix 0 ref flags -M +.Ix 0 ref compatibility +.xH 2 Parallelism +.No +.LP +PMake was specifically designed to re-create several targets at once, +when possible. You do not have to do anything special to cause this to +happen (unless PMake was configured to not act in parallel, in which +case you will have to make use of the +.B \-L +and +.B \-J +flags (see below)), +.Ix 0 ref flags -L +.Ix 0 ref flags -J +but you do have to be careful at times. +.LP +There are several problems you are likely to encounter. One is +that some makefiles (and programs) are written in such a way that it is +impossible for two targets to be made at once. The program +.CW xstr , +for example, +always modifies the files +.CW strings +and +.CW x.c . +There is no way to change it. Thus you cannot run two of them at once +without something being trashed. Similarly, if you have commands +in the makefile that always send output to the same file, you will not +be able to make more than one target at once unless you change the +file you use. You can, for instance, add a +.CW $$$$ +to the end of the file name to tack on the process ID of the shell +executing the command (each +.CW $$ +expands to a single +.CW $ , +thus giving you the shell variable +.CW $$ ). +Since only one shell is used for all the +commands, you'll get the same file name for each command in the +script. +.LP +The other problem comes from improperly-specified dependencies that +worked in Make because of its sequential, depth-first way of examining +them. While I don't want to go into depth on how PMake +works (look in chapter 4 if you're interested), I will warn you that +files in two different ``levels'' of the dependency tree may be +examined in a different order in PMake than they were in Make. For +example, given the makefile +.DS +a : b c +b : d +.DE +PMake will examine the targets in the order +.CW c , +.CW d , +.CW b , +.CW a . +If the makefile's author expected PMake to abort before making +.CW c +if an error occurred while making +.CW b , +or if +.CW b +needed to exist before +.CW c +was made, +s/he will be sorely disappointed. The dependencies are +incomplete, since in both these cases, +.CW c +would depend on +.CW b . +So watch out. +.LP +Another problem you may face is that, while PMake is set up to handle the +output from multiple jobs in a graceful fashion, the same is not so for input. +It has no way to regulate input to different jobs, +so if you use the redirection from +.CW /dev/tty +I mentioned earlier, you must be careful not to run two of the jobs at once. +.xH 2 Writing and Debugging a Makefile +.LP +Now you know most of what's in a makefile, what do you do next? There +are two choices: (1) use one of the uncommonly-available makefile +generators or (2) write your own makefile (I leave out the third choice of +ignoring PMake and doing everything by hand as being beyond the bounds +of common sense). +.LP +When faced with the writing of a makefile, it is usually best to start +from first principles: just what +.I are +you trying to do? What do you want the makefile finally to produce? +.LP +To begin with a somewhat traditional example, let's say you need to +write a makefile to create a program, +.CW expr , +that takes standard infix expressions and converts them to prefix form (for +no readily apparent reason). You've got three source files, in C, that +make up the program: +.CW main.c , +.CW parse.c , +and +.CW output.c . +Harking back to my pithy advice about dependency lines, you write the +first line of the file: +.DS +expr : main.o parse.o output.o +.DE +because you remember +.CW expr +is made from +.CW .o +files, not +.CW .c +files. Similarly for the +.CW .o +files you produce the lines: +.DS +main.o : main.c +parse.o : parse.c +output.o : output.c +main.o parse.o output.o : defs.h +.DE +.LP +Great. You've now got the dependencies specified. What you need now is +commands. These commands, remember, must produce the target on the +dependency line, usually by using the sources you've listed. +You remember about local variables? Good, so it should come +to you as no surprise when you write +.DS +expr : main.o parse.o output.o + cc -o $(.TARGET) $(.ALLSRC) +.DE +Why use the variables? If your program grows to produce postfix +expressions too (which, of course, requires a name change or two), it +is one fewer place you have to change the file. You cannot do this for +the object files, however, because they depend on their corresponding +source files +.I and +.CW defs.h , +thus if you said +.DS + cc -c $(.ALLSRC) +.DE +you'd get (for +.CW main.o ): +.DS + cc -c main.c defs.h +.DE +which is wrong. So you round out the makefile with these lines: +.DS +main.o : main.c + cc -c main.c +parse.o : parse.c + cc -c parse.c +output.o : output.c + cc -c output.c +.DE +.LP +The makefile is now complete and will, in fact, create the program you +want it to without unnecessary compilations or excessive typing on +your part. There are two things wrong with it, however (aside from it +being altogether too long, something I'll address in chapter 3): +.IP 1) +The string +.CW "main.o parse.o output.o" '' `` +is repeated twice, necessitating two changes when you add postfix +(you were planning on that, weren't you?). This is in direct violation +of de Boor's First Rule of writing makefiles: +.QP +.I +Anything that needs to be written more than once +should be placed in a variable. +.IP "\&" +I cannot emphasize this enough as being very important to the +maintenance of a makefile and its program. +.IP 2) +There is no way to alter the way compilations are performed short of +editing the makefile and making the change in all places. This is evil +and violates de Boor's Second Rule, which follows directly from the +first: +.QP +.I +Any flags or programs used inside a makefile should be placed in a variable so +they may be changed, temporarily or permanently, with the greatest ease. +.LP +The makefile should more properly read: +.DS +OBJS = main.o parse.o output.o +expr : $(OBJS) + $(CC) $(CFLAGS) -o $(.TARGET) $(.ALLSRC) +main.o : main.c + $(CC) $(CFLAGS) -c main.c +parse.o : parse.c + $(CC) $(CFLAGS) -c parse.c +output.o : output.c + $(CC) $(CFLAGS) -c output.c +$(OBJS) : defs.h +.DE +Alternatively, if you like the idea of dynamic sources mentioned in +section 2.3.1, +.Rm 0 2.3.1 +.Rd 4 +.Ix 0 ref "dynamic source" +.Ix 0 ref source dynamic +you could write it like this: +.DS +OBJS = main.o parse.o output.o +expr : $(OBJS) + $(CC) $(CFLAGS) -o $(.TARGET) $(.ALLSRC) +$(OBJS) : $(.PREFIX).c defs.h + $(CC) $(CFLAGS) -c $(.PREFIX).c +.DE +These two rules and examples lead to de Boor's First Corollary: +.QP +.I +Variables are your friends. +.LP +Once you've written the makefile comes the sometimes-difficult task of +.Ix 0 ref debugging +making sure the darn thing works. Your most helpful tool to make sure +the makefile is at least syntactically correct is the +.B \-n +.Ix 0 ref flags -n +flag, which allows you to see if PMake will choke on the makefile. The +second thing the +.B \-n +flag lets you do is see what PMake would do without it actually doing +it, thus you can make sure the right commands would be executed were +you to give PMake its head. +.LP +When you find your makefile isn't behaving as you hoped, the first +question that comes to mind (after ``What time is it, anyway?'') is +``Why not?'' In answering this, two flags will serve you well: +.CW "-d m" '' `` +.Ix 0 ref flags -d +and +.CW "-p 2" .'' `` +.Ix 0 ref flags -p +The first causes PMake to tell you as it examines each target in the +makefile and indicate why it is deciding whatever it is deciding. You +can then use the information printed for other targets to see where +you went wrong. The +.CW "-p 2" '' `` +flag makes PMake print out its internal state when it is done, +allowing you to see that you forgot to make that one chapter depend on +that file of macros you just got a new version of. The output from +.CW "-p 2" '' `` +is intended to resemble closely a real makefile, but with additional +information provided and with variables expanded in those commands +PMake actually printed or executed. +.LP +Something to be especially careful about is circular dependencies. +.Ix 0 def dependency circular +E.g. +.DS +a : b +b : c d +d : a +.DE +In this case, because of how PMake works, +.CW c +is the only thing PMake will examine, because +.CW d +and +.CW a +will effectively fall off the edge of the universe, making it +impossible to examine +.CW b +(or them, for that matter). +PMake will tell you (if run in its normal mode) all the targets +involved in any cycle it looked at (i.e. if you have two cycles in the +graph (naughty, naughty), but only try to make a target in one of +them, PMake will only tell you about that one. You'll have to try to +make the other to find the second cycle). When run as Make, it will +only print the first target in the cycle. +.xH 2 Invoking PMake +.LP +.Ix 0 ref flags +.Ix 0 ref arguments +.Ix 0 ref usage +PMake comes with a wide variety of flags to choose from. +They may appear in any order, interspersed with command-line variable +assignments and targets to create. +The flags are as follows: +.IP "\fB\-d\fP \fIwhat\fP" +.Ix 0 def flags -d +.Ix 0 ref debugging +This causes PMake to spew out debugging information that +may prove useful to you. If you can't +figure out why PMake is doing what it's doing, you might try using +this flag. The +.I what +parameter is a string of single characters that tell PMake what +aspects you are interested in. Most of what I describe will make +little sense to you, unless you've dealt with Make before. Just +remember where this table is and come back to it as you read on. +The characters and the information they produce are as follows: +.RS +.IP a +Archive searching and caching. +.IP c +Conditional evaluation. +.IP d +The searching and caching of directories. +.IP j +Various snippets of information related to the running of the multiple +shells. Not particularly interesting. +.IP m +The making of each target: what target is being examined; when it was +last modified; whether it is out-of-date; etc. +.IP p +Makefile parsing. +.IP r +Remote execution. +.IP s +The application of suffix-transformation rules. (See chapter 3) +.IP t +The maintenance of the list of targets. +.IP v +Variable assignment. +.RE +.IP "\&" +Of these all, the +.CW m +and +.CW s +letters will be most useful to you. +If the +.B \-d +is the final argument or the argument from which it would get these +key letters (see below for a note about which argument would be used) +begins with a +.B \- , +all of these debugging flags will be set, resulting in massive amounts +of output. +.IP "\fB\-f\fP \fImakefile\fP" +.Ix 0 def flags -f +Specify a makefile to read different from the standard makefiles +.CW Makefile "\&" ( +or +.CW makefile ). +.Ix 0 ref makefile default +.Ix 0 ref makefile other +If +.I makefile +is ``\-'', PMake uses the standard input. This is useful for making +quick and dirty makefiles.\|.\|. +.Ix 0 ref makefile "quick and dirty" +.IP \fB\-h\fP +.Ix 0 def flags -h +Prints out a summary of the various flags PMake accepts. It can also +be used to find out what level of concurrency was compiled into the +version of PMake you are using (look at +.B \-J +and +.B \-L ) +and various other information on how PMake was configured. +.Ix 0 ref configuration +.Ix 0 ref makefilesystem +.IP \fB\-i\fP +.Ix 0 def flags -i +If you give this flag, PMake will ignore non-zero status returned +by any of its shells. It's like placing a `\-' before all the commands +in the makefile. +.IP \fB\-k\fP +.Ix 0 def flags -k +This is similar to +.B \-i +in that it allows PMake to continue when it sees an error, but unlike +.B \-i , +where PMake continues blithely as if nothing went wrong, +.B \-k +causes it to recognize the error and only continue work on those +things that don't depend on the target, either directly or indirectly (through +depending on something that depends on it), whose creation returned the error. +The `k' is for ``keep going''.\|.\|. +.Ix 0 ref target +.IP \fB\-l\fP +.Ix 0 def flags -l +PMake has the ability to lock a directory against other +people executing it in the same directory (by means of a file called +``LOCK.make'' that it creates and checks for in the directory). This +is a Good Thing because two people doing the same thing in the same place +can be disastrous for the final product (too many cooks and all that). +Whether this locking is the default is up to your system +administrator. If locking is on, +.B \-l +will turn it off, and vice versa. Note that this locking will not +prevent \fIyou\fP from invoking PMake twice in the same place \*- if +you own the lock file, PMake will warn you about it but continue to execute. +.IP "\fB\-m\fP \fIdirectory\fP" +.Ix 0 def flags -m +Tells PMake another place to search for included makefiles via the <...> +style. Several +.B \-m +options can be given to form a search path. If this construct is used the +default system makefile search path is completely overridden. +To be explained in chapter 3, section 3.2. +.Rm 2 3.2 +.IP \fB\-n\fP +.Ix 0 def flags -n +This flag tells PMake not to execute the commands needed to update the +out-of-date targets in the makefile. Rather, PMake will simply print +the commands it would have executed and exit. This is particularly +useful for checking the correctness of a makefile. If PMake doesn't do +what you expect it to, it's a good chance the makefile is wrong. +.IP "\fB\-p\fP \fInumber\fP" +.Ix 0 def flags -p +.Ix 0 ref debugging +This causes PMake to print its input in a reasonable form, though +not necessarily one that would make immediate sense to anyone but me. The +.I number +is a bitwise-or of 1 and 2 where 1 means it should print the input +before doing any processing and 2 says it should print it after +everything has been re-created. Thus +.CW "\-p 3" +would print it twice\*-once before processing and once after (you +might find the difference between the two interesting). This is mostly +useful to me, but you may find it informative in some bizarre circumstances. +.IP \fB\-q\fP +.Ix 0 def flags -q +If you give PMake this flag, it will not try to re-create anything. It +will just see if anything is out-of-date and exit non-zero if so. +.IP \fB\-r\fP +.Ix 0 def flags -r +When PMake starts up, it reads a default makefile that tells it what +sort of system it's on and gives it some idea of what to do if you +don't tell it anything. I'll tell you about it in chapter 3. If you +give this flag, PMake won't read the default makefile. +.IP \fB\-s\fP +.Ix 0 def flags -s +This causes PMake to not print commands before they're executed. It +is the equivalent of putting an `@' before every command in the +makefile. +.IP \fB\-t\fP +.Ix 0 def flags -t +Rather than try to re-create a target, PMake will simply ``touch'' it +so as to make it appear up-to-date. If the target didn't exist before, +it will when PMake finishes, but if the target did exist, it will +appear to have been updated. +.IP \fB\-v\fP +.Ix 0 def flags -v +This is a mixed-compatibility flag intended to mimic the System V +version of Make. It is the same as giving +.B \-B , +and +.B \-V +as well as turning off directory locking. Targets can still be created +in parallel, however. This is the mode PMake will enter if it is +invoked either as +.CW smake '' `` +or +.CW vmake ''. `` +.IP \fB\-x\fP +.Ix 0 def flags -x +This tells PMake it's ok to export jobs to other machines, if they're +available. It is used when running in Make mode, as exporting in this +mode tends to make things run slower than if the commands were just +executed locally. +.IP \fB\-B\fP +.Ix 0 ref compatibility +.Ix 0 def flags -B +Forces PMake to be as backwards-compatible with Make as possible while +still being itself. +This includes: +.RS +.IP \(bu 2 +Executing one shell per shell command +.IP \(bu 2 +Expanding anything that looks even vaguely like a variable, with the +empty string replacing any variable PMake doesn't know. +.IP \(bu 2 +Refusing to allow you to escape a `#' with a backslash. +.IP \(bu 2 +Permitting undefined variables on dependency lines and conditionals +(see below). Normally this causes PMake to abort. +.RE +.IP \fB\-C\fP +.Ix 0 def flags -C +This nullifies any and all compatibility mode flags you may have given +or implied up to the time the +.B \-C +is encountered. It is useful mostly in a makefile that you wrote for PMake +to avoid bad things happening when someone runs PMake as +.CW make '' `` +or has things set in the environment that tell it to be compatible. +.B \-C +is +.I not +placed in the +.CW PMAKE +environment variable or the +.CW .MAKEFLAGS +or +.CW MFLAGS +global variables. +.Ix 0 ref variable environment PMAKE +.Ix 0 ref variable global .MAKEFLAGS +.Ix 0 ref variable global MFLAGS +.Ix 0 ref .MAKEFLAGS variable +.Ix 0 ref MFLAGS +.IP "\fB\-D\fP \fIvariable\fP" +.Ix 0 def flags -D +Allows you to define a variable to have +.CW 1 '' `` +as its value. The variable is a global variable, not a command-line +variable. This is useful mostly for people who are used to the C +compiler arguments and those using conditionals, which I'll get into +in section 4.3 +.Rm 1 4.3 +.IP "\fB\-I\fP \fIdirectory\fP" +.Ix 0 def flags -I +Tells PMake another place to search for included makefiles. Yet +another thing to be explained in chapter 3 (section 3.2, to be +precise). +.Rm 2 3.2 +.IP "\fB\-J\fP \fInumber\fP" +.Ix 0 def flags -J +Gives the absolute maximum number of targets to create at once on both +local and remote machines. +.IP "\fB\-L\fP \fInumber\fP" +.Ix 0 def flags -L +This specifies the maximum number of targets to create on the local +machine at once. This may be 0, though you should be wary of doing +this, as PMake may hang until a remote machine becomes available, if +one is not available when it is started. +.IP \fB\-M\fP +.Ix 0 ref compatibility +.Ix 0 def flags -M +This is the flag that provides absolute, complete, full compatibility +with Make. It still allows you to use all but a few of the features of +PMake, but it is non-parallel. This is the mode PMake enters if you +call it +.CW make .'' `` +.IP \fB\-P\fP +.Ix 0 def flags -P +.Ix 0 ref "output control" +When creating targets in parallel, several shells are executing at +once, each wanting to write its own two cent's-worth to the screen. +This output must be captured by PMake in some way in order to prevent +the screen from being filled with garbage even more indecipherable +than you usually see. PMake has two ways of doing this, one of which +provides for much cleaner output and a clear separation between the +output of different jobs, the other of which provides a more immediate +response so one can tell what is really happening. The former is done +by notifying you when the creation of a target starts, capturing the +output and transferring it to the screen all at once when the job +finishes. The latter is done by catching the output of the shell (and +its children) and buffering it until an entire line is received, then +printing that line preceded by an indication of which job produced +the output. Since I prefer this second method, it is the one used by +default. The first method will be used if you give the +.B \-P +flag to PMake. +.IP \fB\-V\fP +.Ix 0 def flags -V +As mentioned before, the +.B \-V +flag tells PMake to use Make's style of expanding variables, +substituting the empty string for any variable it doesn't know. +.IP \fB\-W\fP +.Ix 0 def flags -W +There are several times when PMake will print a message at you that is +only a warning, i.e. it can continue to work in spite of your having +done something silly (such as forgotten a leading tab for a shell +command). Sometimes you are well aware of silly things you have done +and would like PMake to stop bothering you. This flag tells it to shut +up about anything non-fatal. +.IP \fB\-X\fP +.Ix 0 def flags -X +This flag causes PMake to not attempt to export any jobs to another +machine. +.LP +Several flags may follow a single `\-'. Those flags that require +arguments take them from successive parameters. E.g. +.DS +pmake -fDnI server.mk DEBUG /chip2/X/server/include +.DE +will cause PMake to read +.CW server.mk +as the input makefile, define the variable +.CW DEBUG +as a global variable and look for included makefiles in the directory +.CW /chip2/X/server/include . +.xH 2 Summary +.LP +A makefile is made of four types of lines: +.RS +.IP \(bu 2 +Dependency lines +.IP \(bu 2 +Creation commands +.IP \(bu 2 +Variable assignments +.IP \(bu 2 +Comments, include statements and conditional directives +.RE +.LP +A dependency line is a list of one or more targets, an operator +.CW : ', (` +.CW :: ', ` +or +.CW ! '), ` +and a list of zero or more sources. Sources may contain wildcards and +certain local variables. +.LP +A creation command is a regular shell command preceded by a tab. In +addition, if the first two characters after the tab (and other +whitespace) are a combination of +.CW @ ' ` +or +.CW - ', ` +PMake will cause the command to not be printed (if the character is +.CW @ ') ` +or errors from it to be ignored (if +.CW - '). ` +A blank line, dependency line or variable assignment terminates a +creation script. There may be only one creation script for each target +with a +.CW : ' ` +or +.CW ! ' ` +operator. +.LP +Variables are places to store text. They may be unconditionally +assigned-to using the +.CW = ' ` +.Ix 0 ref = +.Ix 0 ref variable assignment +operator, appended-to using the +.CW += ' ` +.Ix 0 ref += +.Ix 0 ref variable assignment appended +operator, conditionally (if the variable is undefined) assigned-to +with the +.CW ?= ' ` +.Ix 0 ref ?= +.Ix 0 ref variable assignment conditional +operator, and assigned-to with variable expansion with the +.CW := ' ` +.Ix 0 ref := +.Ix 0 ref variable assignment expanded +operator. The output of a shell command may be assigned to a variable +using the +.CW != ' ` +.Ix 0 ref != +.Ix 0 ref variable assignment shell-output +operator. Variables may be expanded (their value inserted) by enclosing +their name in parentheses or curly braces, preceded by a dollar sign. +A dollar sign may be escaped with another dollar sign. Variables are +not expanded if PMake doesn't know about them. There are seven local +variables: +.CW .TARGET , +.CW .ALLSRC , +.CW .OODATE , +.CW .PREFIX , +.CW .IMPSRC , +.CW .ARCHIVE , +and +.CW .MEMBER . +Four of them +.CW .TARGET , ( +.CW .PREFIX , +.CW .ARCHIVE , +and +.CW .MEMBER ) +may be used to specify ``dynamic sources.'' +.Ix 0 ref "dynamic source" +.Ix 0 ref source dynamic +Variables are good. Know them. Love them. Live them. +.LP +Debugging of makefiles is best accomplished using the +.B \-n , +.B "\-d m" , +and +.B "\-p 2" +flags. +.xH 2 Exercises +.ce +\s+4\fBTBA\fP\s0 +.xH 1 Short-cuts and Other Nice Things +.LP +Based on what I've told you so far, you may have gotten the impression +that PMake is just a way of storing away commands and making sure you +don't forget to compile something. Good. That's just what it is. +However, the ways I've described have been inelegant, at best, and +painful, at worst. +This chapter contains things that make the +writing of makefiles easier and the makefiles themselves shorter and +easier to modify (and, occasionally, simpler). In this chapter, I +assume you are somewhat more +familiar with Sprite (or UNIX, if that's what you're using) than I did +in chapter 2, just so you're on your toes. +So without further ado... +.xH 2 Transformation Rules +.LP +As you know, a file's name consists of two parts: a base name, which +gives some hint as to the contents of the file, and a suffix, which +usually indicates the format of the file. +Over the years, as +.UX +has developed, +naming conventions, with regard to suffixes, have also developed that have +become almost as incontrovertible as Law. E.g. a file ending in +.CW .c +is assumed to contain C source code; one with a +.CW .o +suffix is assumed to be a compiled, relocatable object file that may +be linked into any program; a file with a +.CW .ms +suffix is usually a text file to be processed by Troff with the \-ms +macro package, and so on. +One of the best aspects of both Make and PMake comes from their +understanding of how the suffix of a file pertains to its contents and +their ability to do things with a file based solely on its suffix. This +ability comes from something known as a transformation rule. A +transformation rule specifies how to change a file with one suffix +into a file with another suffix. +.LP +A transformation rule looks much like a dependency line, except the +target is made of two known suffixes stuck together. Suffixes are made +known to PMake by placing them as sources on a dependency line whose +target is the special target +.CW .SUFFIXES . +E.g. +.DS +\&.SUFFIXES : .o .c +\&.c.o : + $(CC) $(CFLAGS) -c $(.IMPSRC) +.DE +The creation script attached to the target is used to transform a file with +the first suffix (in this case, +.CW .c ) +into a file with the second suffix (here, +.CW .o ). +In addition, the target inherits whatever attributes have been applied +to the transformation rule. +The simple rule given above says that to transform a C source file +into an object file, you compile it using +.CW cc +with the +.CW \-c +flag. +This rule is taken straight from the system makefile. Many +transformation rules (and suffixes) are defined there, and I refer you +to it for more examples (type +.CW "pmake -h" '' `` +to find out where it is). +.LP +There are several things to note about the transformation rule given +above: +.RS +.IP 1) +The +.CW .IMPSRC +variable. +.Ix 0 def variable local .IMPSRC +.Ix 0 def .IMPSRC +This variable is set to the ``implied source'' (the file from which +the target is being created; the one with the first suffix), which, in this +case, is the .c file. +.IP 2) +The +.CW CFLAGS +variable. Almost all of the transformation rules in the system +makefile are set up using variables that you can alter in your +makefile to tailor the rule to your needs. In this case, if you want +all your C files to be compiled with the +.B \-g +flag, to provide information for +.CW dbx , +you would set the +.CW CFLAGS +variable to contain +.CW -g +.CW "CFLAGS = -g" '') (`` +and PMake would take care of the rest. +.RE +.LP +To give you a quick example, the makefile in 2.3.4 +.Rm 3 2.3.4 +could be changed to this: +.DS +OBJS = a.o b.o c.o +program : $(OBJS) + $(CC) -o $(.TARGET) $(.ALLSRC) +$(OBJS) : defs.h +.DE +The transformation rule I gave above takes the place of the 6 lines\** +.FS +This is also somewhat cleaner, I think, than the dynamic source +solution presented in 2.6 +.FE +.Rm 4 2.6 +.DS +a.o : a.c + cc -c a.c +b.o : b.c + cc -c b.c +c.o : c.c + cc -c c.c +.DE +.LP +Now you may be wondering about the dependency between the +.CW .o +and +.CW .c +files \*- it's not mentioned anywhere in the new makefile. This is +because it isn't needed: one of the effects of applying a +transformation rule is the target comes to depend on the implied +source. That's why it's called the implied +.I source . +.LP +For a more detailed example. Say you have a makefile like this: +.DS +a.out : a.o b.o + $(CC) $(.ALLSRC) +.DE +and a directory set up like this: +.DS +total 4 +-rw-rw-r-- 1 deboor 34 Sep 7 00:43 Makefile +-rw-rw-r-- 1 deboor 119 Oct 3 19:39 a.c +-rw-rw-r-- 1 deboor 201 Sep 7 00:43 a.o +-rw-rw-r-- 1 deboor 69 Sep 7 00:43 b.c +.DE +While just typing +.CW pmake '' `` +will do the right thing, it's much more informative to type +.CW "pmake -d s" ''. `` +This will show you what PMake is up to as it processes the files. In +this case, PMake prints the following: +.DS +Suff_FindDeps (a.out) + using existing source a.o + applying .o -> .out to "a.o" +Suff_FindDeps (a.o) + trying a.c...got it + applying .c -> .o to "a.c" +Suff_FindDeps (b.o) + trying b.c...got it + applying .c -> .o to "b.c" +Suff_FindDeps (a.c) + trying a.y...not there + trying a.l...not there + trying a.c,v...not there + trying a.y,v...not there + trying a.l,v...not there +Suff_FindDeps (b.c) + trying b.y...not there + trying b.l...not there + trying b.c,v...not there + trying b.y,v...not there + trying b.l,v...not there +--- a.o --- +cc -c a.c +--- b.o --- +cc -c b.c +--- a.out --- +cc a.o b.o +.DE +.LP +.CW Suff_FindDeps +is the name of a function in PMake that is called to check for implied +sources for a target using transformation rules. +The transformations it tries are, naturally +enough, limited to the ones that have been defined (a transformation +may be defined multiple times, by the way, but only the most recent +one will be used). You will notice, however, that there is a definite +order to the suffixes that are tried. This order is set by the +relative positions of the suffixes on the +.CW .SUFFIXES +line \*- the earlier a suffix appears, the earlier it is checked as +the source of a transformation. Once a suffix has been defined, the +only way to change its position in the pecking order is to remove all +the suffixes (by having a +.CW .SUFFIXES +dependency line with no sources) and redefine them in the order you +want. (Previously-defined transformation rules will be automatically +redefined as the suffixes they involve are re-entered.) +.LP +Another way to affect the search order is to make the dependency +explicit. In the above example, +.CW a.out +depends on +.CW a.o +and +.CW b.o . +Since a transformation exists from +.CW .o +to +.CW .out , +PMake uses that, as indicated by the +.CW "using existing source a.o" '' `` +message. +.LP +The search for a transformation starts from the suffix of the target +and continues through all the defined transformations, in the order +dictated by the suffix ranking, until an existing file with the same +base (the target name minus the suffix and any leading directories) is +found. At that point, one or more transformation rules will have been +found to change the one existing file into the target. +.LP +For example, ignoring what's in the system makefile for now, say you +have a makefile like this: +.DS +\&.SUFFIXES : .out .o .c .y .l +\&.l.c : + lex $(.IMPSRC) + mv lex.yy.c $(.TARGET) +\&.y.c : + yacc $(.IMPSRC) + mv y.tab.c $(.TARGET) +\&.c.o : + cc -c $(.IMPSRC) +\&.o.out : + cc -o $(.TARGET) $(.IMPSRC) +.DE +and the single file +.CW jive.l . +If you were to type +.CW "pmake -rd ms jive.out" ,'' `` +you would get the following output for +.CW jive.out : +.DS +Suff_FindDeps (jive.out) + trying jive.o...not there + trying jive.c...not there + trying jive.y...not there + trying jive.l...got it + applying .l -> .c to "jive.l" + applying .c -> .o to "jive.c" + applying .o -> .out to "jive.o" +.DE +and this is why: PMake starts with the target +.CW jive.out , +figures out its suffix +.CW .out ) ( +and looks for things it can transform to a +.CW .out +file. In this case, it only finds +.CW .o , +so it looks for the file +.CW jive.o . +It fails to find it, so it looks for transformations into a +.CW .o +file. Again it has only one choice: +.CW .c . +So it looks for +.CW jive.c +and, as you know, fails to find it. At this point it has two choices: +it can create the +.CW .c +file from either a +.CW .y +file or a +.CW .l +file. Since +.CW .y +came first on the +.CW .SUFFIXES +line, it checks for +.CW jive.y +first, but can't find it, so it looks for +.CW jive.l +and, lo and behold, there it is. +At this point, it has defined a transformation path as follows: +.CW .l +\(-> +.CW .c +\(-> +.CW .o +\(-> +.CW .out +and applies the transformation rules accordingly. For completeness, +and to give you a better idea of what PMake actually did with this +three-step transformation, this is what PMake printed for the rest of +the process: +.DS +Suff_FindDeps (jive.o) + using existing source jive.c + applying .c -> .o to "jive.c" +Suff_FindDeps (jive.c) + using existing source jive.l + applying .l -> .c to "jive.l" +Suff_FindDeps (jive.l) +Examining jive.l...modified 17:16:01 Oct 4, 1987...up-to-date +Examining jive.c...non-existent...out-of-date +--- jive.c --- +lex jive.l +\&.\|.\|. meaningless lex output deleted .\|.\|. +mv lex.yy.c jive.c +Examining jive.o...non-existent...out-of-date +--- jive.o --- +cc -c jive.c +Examining jive.out...non-existent...out-of-date +--- jive.out --- +cc -o jive.out jive.o +.DE +.LP +One final question remains: what does PMake do with targets that have +no known suffix? PMake simply pretends it actually has a known suffix +and searches for transformations accordingly. +The suffix it chooses is the source for the +.CW .NULL +.Ix 0 ref .NULL +target mentioned later. In the system makefile, +.CW .out +is chosen as the ``null suffix'' +.Ix 0 def suffix null +.Ix 0 def "null suffix" +because most people use PMake to create programs. You are, however, +free and welcome to change it to a suffix of your own choosing. +The null suffix is ignored, however, when PMake is in compatibility +mode (see chapter 4). +.xH 2 Including Other Makefiles +.Ix 0 def makefile inclusion +.Rd 2 +.LP +Just as for programs, it is often useful to extract certain parts of a +makefile into another file and just include it in other makefiles +somehow. Many compilers allow you say something like +.DS +#include "defs.h" +.DE +to include the contents of +.CW defs.h +in the source file. PMake allows you to do the same thing for +makefiles, with the added ability to use variables in the filenames. +An include directive in a makefile looks either like this: +.DS +#include <file> +.DE +or this +.DS +#include "file" +.DE +The difference between the two is where PMake searches for the file: +the first way, PMake will look for +the file only in the system makefile directory (or directories) +(to find out what that directory is, give PMake the +.B \-h +flag). +.Ix 0 ref flags -h +The system makefile directory search path can be overridden via the +.B \-m +option. +.Ix 0 ref flags -m +For files in double-quotes, the search is more complex: +.RS +.IP 1) +The directory of the makefile that's including the file. +.IP 2) +The current directory (the one in which you invoked PMake). +.IP 3) +The directories given by you using +.B \-I +flags, in the order in which you gave them. +.IP 4) +Directories given by +.CW .PATH +dependency lines (see chapter 4). +.IP 5) +The system makefile directory. +.RE +.LP +in that order. +.LP +You are free to use PMake variables in the filename\*-PMake will +expand them before searching for the file. You must specify the +searching method with either angle brackets or double-quotes +.I outside +of a variable expansion. I.e. the following +.DS +SYSTEM = <command.mk> + +#include $(SYSTEM) +.DE +won't work. +.xH 2 Saving Commands +.LP +.Ix 0 def ... +There may come a time when you will want to save certain commands to +be executed when everything else is done. For instance: you're +making several different libraries at one time and you want to create the +members in parallel. Problem is, +.CW ranlib +is another one of those programs that can't be run more than once in +the same directory at the same time (each one creates a file called +.CW __.SYMDEF +into which it stuffs information for the linker to use. Two of them +running at once will overwrite each other's file and the result will +be garbage for both parties). You might want a way to save the ranlib +commands til the end so they can be run one after the other, thus +keeping them from trashing each other's file. PMake allows you to do +this by inserting an ellipsis (``.\|.\|.'') as a command between +commands to be run at once and those to be run later. +.LP +So for the +.CW ranlib +case above, you might do this: +.Rd 5 +.DS +lib1.a : $(LIB1OBJS) + rm -f $(.TARGET) + ar cr $(.TARGET) $(.ALLSRC) + ... + ranlib $(.TARGET) + +lib2.a : $(LIB2OBJS) + rm -f $(.TARGET) + ar cr $(.TARGET) $(.ALLSRC) + ... + ranlib $(.TARGET) +.DE +.Ix 0 ref variable local .TARGET +.Ix 0 ref variable local .ALLSRC +This would save both +.DS +ranlib $(.TARGET) +.DE +commands until the end, when they would run one after the other +(using the correct value for the +.CW .TARGET +variable, of course). +.LP +Commands saved in this manner are only executed if PMake manages to +re-create everything without an error. +.xH 2 Target Attributes +.LP +PMake allows you to give attributes to targets by means of special +sources. Like everything else PMake uses, these sources begin with a +period and are made up of all upper-case letters. There are various +reasons for using them, and I will try to give examples for most of +them. Others you'll have to find uses for yourself. Think of it as ``an +exercise for the reader.'' By placing one (or more) of these as a source on a +dependency line, you are ``marking the target(s) with that +attribute.'' That's just the way I phrase it, so you know. +.LP +Any attributes given as sources for a transformation rule are applied +to the target of the transformation rule when the rule is applied. +.Ix 0 def attributes +.Ix 0 ref source +.Ix 0 ref target +.nr pw 12 +.IP .DONTCARE \n(pw +.Ix 0 def attributes .DONTCARE +.Ix 0 def .DONTCARE +If a target is marked with this attribute and PMake can't figure out +how to create it, it will ignore this fact and assume the file isn't +really needed or actually exists and PMake just can't find it. This may prove +wrong, but the error will be noted later on, not when PMake tries to create +the target so marked. This attribute also prevents PMake from +attempting to touch the target if it is given the +.B \-t +flag. +.Ix 0 ref flags -t +.IP .EXEC \n(pw +.Ix 0 def attributes .EXEC +.Ix 0 def .EXEC +This attribute causes its shell script to be executed while having no +effect on targets that depend on it. This makes the target into a sort +of subroutine. An example. Say you have some LISP files that need to +be compiled and loaded into a LISP process. To do this, you echo LISP +commands into a file and execute a LISP with this file as its input +when everything's done. Say also that you have to load other files +from another system before you can compile your files and further, +that you don't want to go through the loading and dumping unless one +of +.I your +files has changed. Your makefile might look a little bit +like this (remember, this is an educational example, and don't worry +about the +.CW COMPILE +rule, all will soon become clear, grasshopper): +.DS +system : init a.fasl b.fasl c.fasl + for i in $(.ALLSRC); + do + echo -n '(load "' >> input + echo -n ${i} >> input + echo '")' >> input + done + echo '(dump "$(.TARGET)")' >> input + lisp < input + +a.fasl : a.l init COMPILE +b.fasl : b.l init COMPILE +c.fasl : c.l init COMPILE +COMPILE : .USE + echo '(compile "$(.ALLSRC)")' >> input +init : .EXEC + echo '(load-system)' > input +.DE +.Ix 0 ref .USE +.Ix 0 ref attributes .USE +.Ix 0 ref variable local .ALLSRC +.IP "\&" +.CW .EXEC +sources, don't appear in the local variables of targets that depend on +them (nor are they touched if PMake is given the +.B \-t +flag). +.Ix 0 ref flags -t +Note that all the rules, not just that for +.CW system , +include +.CW init +as a source. This is because none of the other targets can be made +until +.CW init +has been made, thus they depend on it. +.IP .EXPORT \n(pw +.Ix 0 def attributes .EXPORT +.Ix 0 def .EXPORT +This is used to mark those targets whose creation should be sent to +another machine if at all possible. This may be used by some +exportation schemes if the exportation is expensive. You should ask +your system administrator if it is necessary. +.IP .EXPORTSAME \n(pw +.Ix 0 def attributes .EXPORTSAME +.Ix 0 def .EXPORTSAME +Tells the export system that the job should be exported to a machine +of the same architecture as the current one. Certain operations (e.g. +running text through +.CW nroff ) +can be performed the same on any architecture (CPU and +operating system type), while others (e.g. compiling a program with +.CW cc ) +must be performed on a machine with the same architecture. Not all +export systems will support this attribute. +.IP .IGNORE \n(pw +.Ix 0 def attributes .IGNORE +.Ix 0 def .IGNORE attribute +Giving a target the +.CW .IGNORE +attribute causes PMake to ignore errors from any of the target's commands, as +if they all had `\-' before them. +.IP .INVISIBLE \n(pw +.Ix 0 def attributes .INVISIBLE +.Ix 0 def .INVISIBLE +This allows you to specify one target as a source for another without +the one affecting the other's local variables. Useful if, say, you +have a makefile that creates two programs, one of which is used to +create the other, so it must exist before the other is created. You +could say +.DS +prog1 : $(PROG1OBJS) prog2 MAKEINSTALL +prog2 : $(PROG2OBJS) .INVISIBLE MAKEINSTALL +.DE +where +.CW MAKEINSTALL +is some complex .USE rule (see below) that depends on the +.Ix 0 ref .USE +.CW .ALLSRC +variable containing the right things. Without the +.CW .INVISIBLE +attribute for +.CW prog2 , +the +.CW MAKEINSTALL +rule couldn't be applied. This is not as useful as it should be, and +the semantics may change (or the whole thing go away) in the +not-too-distant future. +.IP .JOIN \n(pw +.Ix 0 def attributes .JOIN +.Ix 0 def .JOIN +This is another way to avoid performing some operations in parallel +while permitting everything else to be done so. Specifically it +forces the target's shell script to be executed only if one or more of the +sources was out-of-date. In addition, the target's name, +in both its +.CW .TARGET +variable and all the local variables of any target that depends on it, +is replaced by the value of its +.CW .ALLSRC +variable. +As an example, suppose you have a program that has four libraries that +compile in the same directory along with, and at the same time as, the +program. You again have the problem with +.CW ranlib +that I mentioned earlier, only this time it's more severe: you +can't just put the ranlib off to the end since the program +will need those libraries before it can be re-created. You can do +something like this: +.DS +program : $(OBJS) libraries + cc -o $(.TARGET) $(.ALLSRC) + +libraries : lib1.a lib2.a lib3.a lib4.a .JOIN + ranlib $(.OODATE) +.DE +.Ix 0 ref variable local .TARGET +.Ix 0 ref variable local .ALLSRC +.Ix 0 ref variable local .OODATE +.Ix 0 ref .TARGET +.Ix 0 ref .ALLSRC +.Ix 0 ref .OODATE +In this case, PMake will re-create the +.CW $(OBJS) +as necessary, along with +.CW lib1.a , +.CW lib2.a , +.CW lib3.a +and +.CW lib4.a . +It will then execute +.CW ranlib +on any library that was changed and set +.CW program 's +.CW .ALLSRC +variable to contain what's in +.CW $(OBJS) +followed by +.CW "lib1.a lib2.a lib3.a lib4.a" .'' `` +In case you're wondering, it's called +.CW .JOIN +because it joins together different threads of the ``input graph'' at +the target marked with the attribute. +Another aspect of the .JOIN attribute is it keeps the target from +being created if the +.B \-t +flag was given. +.Ix 0 ref flags -t +.IP .MAKE \n(pw +.Ix 0 def attributes .MAKE +.Ix 0 def .MAKE +The +.CW .MAKE +attribute marks its target as being a recursive invocation of PMake. +This forces PMake to execute the script associated with the target (if +it's out-of-date) even if you gave the +.B \-n +or +.B \-t +flag. By doing this, you can start at the top of a system and type +.DS +pmake -n +.DE +and have it descend the directory tree (if your makefiles are set up +correctly), printing what it would have executed if you hadn't +included the +.B \-n +flag. +.IP .NOEXPORT \n(pw +.Ix 0 def attributes .NOEXPORT +.Ix 0 def .NOEXPORT attribute +If possible, PMake will attempt to export the creation of all targets to +another machine (this depends on how PMake was configured). Sometimes, +the creation is so simple, it is pointless to send it to another +machine. If you give the target the +.CW .NOEXPORT +attribute, it will be run locally, even if you've given PMake the +.B "\-L 0" +flag. +.IP .NOTMAIN \n(pw +.Ix 0 def attributes .NOTMAIN +.Ix 0 def .NOTMAIN +Normally, if you do not specify a target to make in any other way, +PMake will take the first target on the first dependency line of a +makefile as the target to create. That target is known as the ``Main +Target'' and is labeled as such if you print the dependencies out +using the +.B \-p +flag. +.Ix 0 ref flags -p +Giving a target this attribute tells PMake that the target is +definitely +.I not +the Main Target. +This allows you to place targets in an included makefile and +have PMake create something else by default. +.IP .PRECIOUS \n(pw +.Ix 0 def attributes .PRECIOUS +.Ix 0 def .PRECIOUS attribute +When PMake is interrupted (you type control-C at the keyboard), it +will attempt to clean up after itself by removing any half-made +targets. If a target has the +.CW .PRECIOUS +attribute, however, PMake will leave it alone. An additional side +effect of the `::' operator is to mark the targets as +.CW .PRECIOUS . +.Ix 0 ref operator double-colon +.Ix 0 ref :: +.IP .SILENT \n(pw +.Ix 0 def attributes .SILENT +.Ix 0 def .SILENT attribute +Marking a target with this attribute keeps its commands from being +printed when they're executed, just as if they had an `@' in front of them. +.IP .USE \n(pw +.Ix 0 def attributes .USE +.Ix 0 def .USE +By giving a target this attribute, you turn it into PMake's equivalent +of a macro. When the target is used as a source for another target, +the other target acquires the commands, sources and attributes (except +.CW .USE ) +of the source. +If the target already has commands, the +.CW .USE +target's commands are added to the end. If more than one .USE-marked +source is given to a target, the rules are applied sequentially. +.IP "\&" \n(pw +The typical .USE rule (as I call them) will use the sources of the +target to which it is applied (as stored in the +.CW .ALLSRC +variable for the target) as its ``arguments,'' if you will. +For example, you probably noticed that the commands for creating +.CW lib1.a +and +.CW lib2.a +in the example in section 3.3 +.Rm 5 3.3 +were exactly the same. You can use the +.CW .USE +attribute to eliminate the repetition, like so: +.DS +lib1.a : $(LIB1OBJS) MAKELIB +lib2.a : $(LIB2OBJS) MAKELIB + +MAKELIB : .USE + rm -f $(.TARGET) + ar cr $(.TARGET) $(.ALLSRC) + ... + ranlib $(.TARGET) +.DE +.Ix 0 ref variable local .TARGET +.Ix 0 ref variable local .ALLSRC +.IP "\&" \n(pw +Several system makefiles (not to be confused with The System Makefile) +make use of these .USE rules to make your +life easier (they're in the default, system makefile directory...take a look). +Note that the .USE rule source itself +.CW MAKELIB ) ( +does not appear in any of the targets's local variables. +There is no limit to the number of times I could use the +.CW MAKELIB +rule. If there were more libraries, I could continue with +.CW "lib3.a : $(LIB3OBJS) MAKELIB" '' `` +and so on and so forth. +.xH 2 Special Targets +.LP +As there were in Make, so there are certain targets that have special +meaning to PMake. When you use one on a dependency line, it is the +only target that may appear on the left-hand-side of the operator. +.Ix 0 ref target +.Ix 0 ref operator +As for the attributes and variables, all the special targets +begin with a period and consist of upper-case letters only. +I won't describe them all in detail because some of them are rather +complex and I'll describe them in more detail than you'll want in +chapter 4. +The targets are as follows: +.nr pw 10 +.IP .BEGIN \n(pw +.Ix 0 def .BEGIN +Any commands attached to this target are executed before anything else +is done. You can use it for any initialization that needs doing. +.IP .DEFAULT \n(pw +.Ix 0 def .DEFAULT +This is sort of a .USE rule for any target (that was used only as a +source) that PMake can't figure out any other way to create. It's only +``sort of'' a .USE rule because only the shell script attached to the +.CW .DEFAULT +target is used. The +.CW .IMPSRC +variable of a target that inherits +.CW .DEFAULT 's +commands is set to the target's own name. +.Ix 0 ref .IMPSRC +.Ix 0 ref variable local .IMPSRC +.IP .END \n(pw +.Ix 0 def .END +This serves a function similar to +.CW .BEGIN , +in that commands attached to it are executed once everything has been +re-created (so long as no errors occurred). It also serves the extra +function of being a place on which PMake can hang commands you put off +to the end. Thus the script for this target will be executed before +any of the commands you save with the ``.\|.\|.''. +.Ix 0 ref ... +.IP .EXPORT \n(pw +The sources for this target are passed to the exportation system compiled +into PMake. Some systems will use these sources to configure +themselves. You should ask your system administrator about this. +.IP .IGNORE \n(pw +.Ix 0 def .IGNORE target +.Ix 0 ref .IGNORE attribute +.Ix 0 ref attributes .IGNORE +This target marks each of its sources with the +.CW .IGNORE +attribute. If you don't give it any sources, then it is like +giving the +.B \-i +flag when you invoke PMake \*- errors are ignored for all commands. +.Ix 0 ref flags -i +.IP .INCLUDES \n(pw +.Ix 0 def .INCLUDES target +.Ix 0 def variable global .INCLUDES +.Ix 0 def .INCLUDES variable +The sources for this target are taken to be suffixes that indicate a +file that can be included in a program source file. +The suffix must have already been declared with +.CW .SUFFIXES +(see below). +Any suffix so marked will have the directories on its search path +(see +.CW .PATH , +below) placed in the +.CW .INCLUDES +variable, each preceded by a +.B \-I +flag. This variable can then be used as an argument for the compiler +in the normal fashion. The +.CW .h +suffix is already marked in this way in the system makefile. +.Ix 0 ref makefilesystem +E.g. if you have +.DS +\&.SUFFIXES : .bitmap +\&.PATH.bitmap : /usr/local/X/lib/bitmaps +\&.INCLUDES : .bitmap +.DE +PMake will place +.CW "-I/usr/local/X/lib/bitmaps" '' `` +in the +.CW .INCLUDES +variable and you can then say +.DS +cc $(.INCLUDES) -c xprogram.c +.DE +(Note: the +.CW .INCLUDES +variable is not actually filled in until the entire makefile has been read.) +.IP .INTERRUPT \n(pw +.Ix 0 def .INTERRUPT +When PMake is interrupted, +it will execute the commands in the script for this target, if it +exists. +.IP .LIBS \n(pw +.Ix 0 def .LIBS target +.Ix 0 def .LIBS variable +.Ix 0 def variable global .LIBS +This does for libraries what +.CW .INCLUDES +does for include files, except the flag used is +.B \-L , +as required by those linkers that allow you to tell them where to find +libraries. The variable used is +.CW .LIBS . +Be forewarned that PMake may not have been compiled to do this if the +linker on your system doesn't accept the +.B \-L +flag, though the +.CW .LIBS +variable will always be defined once the makefile has been read. +.IP .MAIN \n(pw +.Ix 0 def .MAIN +If you didn't give a target (or targets) to create when you invoked +PMake, it will take the sources of this target as the targets to +create. +.IP .MAKEFLAGS \n(pw +.Ix 0 def .MAKEFLAGS target +This target provides a way for you to always specify flags for PMake +when the makefile is used. The flags are just as they would be typed +to the shell (except you can't use shell variables unless they're in +the environment), +though the +.B \-f +and +.B \-r +flags have no effect. +.IP .NULL \n(pw +.Ix 0 def .NULL +.Ix 0 ref suffix null +.Ix 0 ref "null suffix" +This allows you to specify what suffix PMake should pretend a file has +if, in fact, it has no known suffix. Only one suffix may be so +designated. The last source on the dependency line is the suffix that +is used (you should, however, only give one suffix.\|.\|.). +.IP .PATH \n(pw +.Ix 0 def .PATH +If you give sources for this target, PMake will take them as +directories in which to search for files it cannot find in the current +directory. If you give no sources, it will clear out any directories +added to the search path before. Since the effects of this all get +very complex, I'll leave it til chapter four to give you a complete +explanation. +.IP .PATH\fIsuffix\fP \n(pw +.Ix 0 ref .PATH +This does a similar thing to +.CW .PATH , +but it does it only for files with the given suffix. The suffix must +have been defined already. Look at +.B "Search Paths" +(section 4.1) +.Rm 6 4.1 +for more information. +.IP .PRECIOUS \n(pw +.Ix 0 def .PRECIOUS target +.Ix 0 ref .PRECIOUS attribute +.Ix 0 ref attributes .PRECIOUS +Similar to +.CW .IGNORE , +this gives the +.CW .PRECIOUS +attribute to each source on the dependency line, unless there are no +sources, in which case the +.CW .PRECIOUS +attribute is given to every target in the file. +.IP .RECURSIVE \n(pw +.Ix 0 def .RECURSIVE +.Ix 0 ref attributes .MAKE +.Ix 0 ref .MAKE +This target applies the +.CW .MAKE +attribute to all its sources. It does nothing if you don't give it any sources. +.IP .SHELL \n(pw +.Ix 0 def .SHELL +PMake is not constrained to only using the Bourne shell to execute +the commands you put in the makefile. You can tell it some other shell +to use with this target. Check out +.B "A Shell is a Shell is a Shell" +(section 4.4) +.Rm 7 4.4 +for more information. +.IP .SILENT \n(pw +.Ix 0 def .SILENT target +.Ix 0 ref .SILENT attribute +.Ix 0 ref attributes .SILENT +When you use +.CW .SILENT +as a target, it applies the +.CW .SILENT +attribute to each of its sources. If there are no sources on the +dependency line, then it is as if you gave PMake the +.B \-s +flag and no commands will be echoed. +.IP .SUFFIXES \n(pw +.Ix 0 def .SUFFIXES +This is used to give new file suffixes for PMake to handle. Each +source is a suffix PMake should recognize. If you give a +.CW .SUFFIXES +dependency line with no sources, PMake will forget about all the +suffixes it knew (this also nukes the null suffix). +For those targets that need to have suffixes defined, this is how you do it. +.LP +In addition to these targets, a line of the form +.DS +\fIattribute\fP : \fIsources\fP +.DE +applies the +.I attribute +to all the targets listed as +.I sources . +.xH 2 Modifying Variable Expansion +.LP +.Ix 0 def variable expansion modified +.Ix 0 ref variable expansion +.Ix 0 def variable modifiers +Variables need not always be expanded verbatim. PMake defines several +modifiers that may be applied to a variable's value before it is +expanded. You apply a modifier by placing it after the variable name +with a colon between the two, like so: +.DS +${\fIVARIABLE\fP:\fImodifier\fP} +.DE +Each modifier is a single character followed by something specific to +the modifier itself. +You may apply as many modifiers as you want \*- each one is applied to +the result of the previous and is separated from the previous by +another colon. +.LP +There are seven ways to modify a variable's expansion, most of which +come from the C shell variable modification characters: +.RS +.IP "M\fIpattern\fP" +.Ix 0 def :M +.Ix 0 def modifier match +This is used to select only those words (a word is a series of +characters that are neither spaces nor tabs) that match the given +.I pattern . +The pattern is a wildcard pattern like that used by the shell, where +.CW * +means 0 or more characters of any sort; +.CW ? +is any single character; +.CW [abcd] +matches any single character that is either `a', `b', `c' or `d' +(there may be any number of characters between the brackets); +.CW [0-9] +matches any single character that is between `0' and `9' (i.e. any +digit. This form may be freely mixed with the other bracket form), and +`\\' is used to escape any of the characters `*', `?', `[' or `:', +leaving them as regular characters to match themselves in a word. +For example, the system makefile +.CW <makedepend.mk> +uses +.CW "$(CFLAGS:M-[ID]*)" '' `` +to extract all the +.CW \-I +and +.CW \-D +flags that would be passed to the C compiler. This allows it to +properly locate include files and generate the correct dependencies. +.IP "N\fIpattern\fP" +.Ix 0 def :N +.Ix 0 def modifier nomatch +This is identical to +.CW :M +except it substitutes all words that don't match the given pattern. +.IP "S/\fIsearch-string\fP/\fIreplacement-string\fP/[g]" +.Ix 0 def :S +.Ix 0 def modifier substitute +Causes the first occurrence of +.I search-string +in the variable to be replaced by +.I replacement-string , +unless the +.CW g +flag is given at the end, in which case all occurrences of the string +are replaced. The substitution is performed on each word in the +variable in turn. If +.I search-string +begins with a +.CW ^ , +the string must match starting at the beginning of the word. If +.I search-string +ends with a +.CW $ , +the string must match to the end of the word (these two may be +combined to force an exact match). If a backslash precedes these two +characters, however, they lose their special meaning. Variable +expansion also occurs in the normal fashion inside both the +.I search-string +and the +.I replacement-string , +.B except +that a backslash is used to prevent the expansion of a +.CW $ , +not another dollar sign, as is usual. +Note that +.I search-string +is just a string, not a pattern, so none of the usual +regular-expression/wildcard characters have any special meaning save +.CW ^ +and +.CW $ . +In the replacement string, +the +.CW & +character is replaced by the +.I search-string +unless it is preceded by a backslash. +You are allowed to use any character except +colon or exclamation point to separate the two strings. This so-called +delimiter character may be placed in either string by preceding it +with a backslash. +.IP T +.Ix 0 def :T +.Ix 0 def modifier tail +Replaces each word in the variable expansion by its last +component (its ``tail''). For example, given +.DS +OBJS = ../lib/a.o b /usr/lib/libm.a +TAILS = $(OBJS:T) +.DE +the variable +.CW TAILS +would expand to +.CW "a.o b libm.a" .'' `` +.IP H +.Ix 0 def :H +.Ix 0 def modifier head +This is similar to +.CW :T , +except that every word is replaced by everything but the tail (the +``head''). Using the same definition of +.CW OBJS , +the string +.CW "$(OBJS:H)" '' `` +would expand to +.CW "../lib /usr/lib" .'' `` +Note that the final slash on the heads is removed and +anything without a head is replaced by the empty string. +.IP E +.Ix 0 def :E +.Ix 0 def modifier extension +.Ix 0 def modifier suffix +.Ix 0 ref suffix "variable modifier" +.CW :E +replaces each word by its suffix (``extension''). So +.CW "$(OBJS:E)" '' `` +would give you +.CW ".o .a" .'' `` +.IP R +.Ix 0 def :R +.Ix 0 def modifier root +.Ix 0 def modifier base +This replaces each word by everything but the suffix (the ``root'' of +the word). +.CW "$(OBJS:R)" '' `` +expands to `` +.CW "../lib/a b /usr/lib/libm" .'' +.RE +.LP +In addition, the System V style of substitution is also supported. +This looks like: +.DS +$(\fIVARIABLE\fP:\fIsearch-string\fP=\fIreplacement\fP) +.DE +It must be the last modifier in the chain. The search is anchored at +the end of each word, so only suffixes or whole words may be replaced. +.xH 2 More on Debugging +.xH 2 More Exercises +.IP (3.1) +You've got a set programs, each of which is created from its own +assembly-language source file (suffix +.CW .asm ). +Each program can be assembled into two versions, one with error-checking +code assembled in and one without. You could assemble them into files +with different suffixes +.CW .eobj \& ( +and +.CW .obj , +for instance), but your linker only understands files that end in +.CW .obj . +To top it all off, the final executables +.I must +have the suffix +.CW .exe . +How can you still use transformation rules to make your life easier +(Hint: assume the error-checking versions have +.CW ec +tacked onto their prefix)? +.IP (3.2) +Assume, for a moment or two, you want to perform a sort of +``indirection'' by placing the name of a variable into another one, +then you want to get the value of the first by expanding the second +somehow. Unfortunately, PMake doesn't allow constructs like +.DS I +$($(FOO)) +.DE +What do you do? Hint: no further variable expansion is performed after +modifiers are applied, thus if you cause a $ to occur in the +expansion, that's what will be in the result. +.xH 1 PMake for Gods +.LP +This chapter is devoted to those facilities in PMake that allow you to +do a great deal in a makefile with very little work, as well as do +some things you couldn't do in Make without a great deal of work (and +perhaps the use of other programs). The problem with these features, +is they must be handled with care, or you will end up with a mess. +.LP +Once more, I assume a greater familiarity with +.UX +or Sprite than I did in the previous two chapters. +.xH 2 Search Paths +.Rd 6 +.LP +PMake supports the dispersal of files into multiple directories by +allowing you to specify places to look for sources with +.CW .PATH +targets in the makefile. The directories you give as sources for these +targets make up a ``search path.'' Only those files used exclusively +as sources are actually sought on a search path, the assumption being +that anything listed as a target in the makefile can be created by the +makefile and thus should be in the current directory. +.LP +There are two types of search paths +in PMake: one is used for all types of files (including included +makefiles) and is specified with a plain +.CW .PATH +target (e.g. +.CW ".PATH : RCS" ''), `` +while the other is specific to a certain type of file, as indicated by +the file's suffix. A specific search path is indicated by immediately following +the +.CW .PATH +with the suffix of the file. For instance +.DS +\&.PATH.h : /sprite/lib/include /sprite/att/lib/include +.DE +would tell PMake to look in the directories +.CW /sprite/lib/include +and +.CW /sprite/att/lib/include +for any files whose suffix is +.CW .h . +.LP +The current directory is always consulted first to see if a file +exists. Only if it cannot be found there are the directories in the +specific search path, followed by those in the general search path, +consulted. +.LP +A search path is also used when expanding wildcard characters. If the +pattern has a recognizable suffix on it, the path for that suffix will +be used for the expansion. Otherwise the default search path is employed. +.LP +When a file is found in some directory other than the current one, all +local variables that would have contained the target's name +.CW .ALLSRC , ( +and +.CW .IMPSRC ) +will instead contain the path to the file, as found by PMake. +Thus if you have a file +.CW ../lib/mumble.c +and a makefile +.DS +\&.PATH.c : ../lib +mumble : mumble.c + $(CC) -o $(.TARGET) $(.ALLSRC) +.DE +the command executed to create +.CW mumble +would be +.CW "cc -o mumble ../lib/mumble.c" .'' `` +(As an aside, the command in this case isn't strictly necessary, since +it will be found using transformation rules if it isn't given. This is because +.CW .out +is the null suffix by default and a transformation exists from +.CW .c +to +.CW .out . +Just thought I'd throw that in.) +.LP +If a file exists in two directories on the same search path, the file +in the first directory on the path will be the one PMake uses. So if +you have a large system spread over many directories, it would behoove +you to follow a naming convention that avoids such conflicts. +.LP +Something you should know about the way search paths are implemented +is that each directory is read, and its contents cached, exactly once +\&\*- when it is first encountered \*- so any changes to the +directories while PMake is running will not be noted when searching +for implicit sources, nor will they be found when PMake attempts to +discover when the file was last modified, unless the file was created in the +current directory. While people have suggested that PMake should read +the directories each time, my experience suggests that the caching seldom +causes problems. In addition, not caching the directories slows things +down enormously because of PMake's attempts to apply transformation +rules through non-existent files \*- the number of extra file-system +searches is truly staggering, especially if many files without +suffixes are used and the null suffix isn't changed from +.CW .out . +.xH 2 Archives and Libraries +.LP +.UX +and Sprite allow you to merge files into an archive using the +.CW ar +command. Further, if the files are relocatable object files, you can +run +.CW ranlib +on the archive and get yourself a library that you can link into any +program you want. The main problem with archives is they double the +space you need to store the archived files, since there's one copy in +the archive and one copy out by itself. The problem with libraries is +you usually think of them as +.CW -lm +rather than +.CW /usr/lib/libm.a +and the linker thinks they're out-of-date if you so much as look at +them. +.LP +PMake solves the problem with archives by allowing you to tell it to +examine the files in the archives (so you can remove the individual +files without having to regenerate them later). To handle the problem +with libraries, PMake adds an additional way of deciding if a library +is out-of-date: +.IP \(bu 2 +If the table of contents is older than the library, or is missing, the +library is out-of-date. +.LP +A library is any target that looks like +.CW \-l name'' `` +or that ends in a suffix that was marked as a library using the +.CW .LIBS +target. +.CW .a +is so marked in the system makefile. +.LP +Members of an archive are specified as +``\fIarchive\fP(\fImember\fP[ \fImember\fP...])''. +Thus +.CW libdix.a(window.o) '' ``' +specifies the file +.CW window.o +in the archive +.CW libdix.a . +You may also use wildcards to specify the members of the archive. Just +remember that most the wildcard characters will only find +.I existing +files. +.LP +A file that is a member of an archive is treated specially. If the +file doesn't exist, but it is in the archive, the modification time +recorded in the archive is used for the file when determining if the +file is out-of-date. When figuring out how to make an archived member target +(not the file itself, but the file in the archive \*- the +\fIarchive\fP(\fImember\fP) target), special care is +taken with the transformation rules, as follows: +.IP \(bu 2 +\&\fIarchive\fP(\fImember\fP) is made to depend on \fImember\fP. +.IP \(bu 2 +The transformation from the \fImember\fP's suffix to the +\fIarchive\fP's suffix is applied to the \fIarchive\fP(\fImember\fP) target. +.IP \(bu 2 +The \fIarchive\fP(\fImember\fP)'s +.CW .TARGET +variable is set to the name of the \fImember\fP if \fImember\fP is +actually a target, or the path to the member file if \fImember\fP is +only a source. +.IP \(bu 2 +The +.CW .ARCHIVE +variable for the \fIarchive\fP(\fImember\fP) target is set to the name +of the \fIarchive\fP. +.Ix 0 def variable local .ARCHIVE +.Ix 0 def .ARCHIVE +.IP \(bu 2 +The +.CW .MEMBER +variable is set to the actual string inside the parentheses. In most +cases, this will be the same as the +.CW .TARGET +variable. +.Ix 0 def variable local .MEMBER +.Ix 0 def .MEMBER +.IP \(bu 2 +The \fIarchive\fP(\fImember\fP)'s place in the local variables of the +targets that depend on it is taken by the value of its +.CW .TARGET +variable. +.LP +Thus, a program library could be created with the following makefile: +.DS +\&.o.a : + ... + rm -f $(.TARGET:T) +OBJS = obj1.o obj2.o obj3.o +libprog.a : libprog.a($(OBJS)) + ar cru $(.TARGET) $(.OODATE) + ranlib $(.TARGET) +.DE +This will cause the three object files to be compiled (if the +corresponding source files were modified after the object file or, if +that doesn't exist, the archived object file), the out-of-date ones +archived in +.CW libprog.a , +a table of contents placed in the archive and the newly-archived +object files to be removed. +.LP +All this is used in the +.CW makelib.mk +system makefile to create a single library with ease. This makefile +looks like this: +.DS +.SM +# +# Rules for making libraries. The object files that make up the library +# are removed once they are archived. +# +# To make several libraries in parallel, you should define the variable +# "many_libraries". This will serialize the invocations of ranlib. +# +# To use, do something like this: +# +# OBJECTS = <files in the library> +# +# fish.a: fish.a($(OBJECTS)) MAKELIB +# +# + +#ifndef _MAKELIB_MK +_MAKELIB_MK = + +#include <po.mk> + +\&.po.a .o.a : + ... + rm -f $(.MEMBER) + +ARFLAGS ?= crl + +# +# Re-archive the out-of-date members and recreate the library's table of +# contents using ranlib. If many_libraries is defined, put the ranlib +# off til the end so many libraries can be made at once. +# +MAKELIB : .USE .PRECIOUS + ar $(ARFLAGS) $(.TARGET) $(.OODATE) +#ifndef no_ranlib +# ifdef many_libraries + ... +# endif many_libraries + ranlib $(.TARGET) +#endif no_ranlib + +#endif _MAKELIB_MK +.DE +.xH 2 On the Condition... +.Rd 1 +.LP +Like the C compiler before it, PMake allows you to configure the makefile, +based on the current environment, using conditional statements. A +conditional looks like this: +.DS +#if \fIboolean expression\fP +\fIlines\fP +#elif \fIanother boolean expression\fP +\fImore lines\fP +#else +\fIstill more lines\fP +#endif +.DE +They may be nested to a maximum depth of 30 and may occur anywhere +(except in a comment, of course). The +.CW # '' `` +must the very first character on the line. +.LP +Each +.I "boolean expression" +is made up of terms that look like function calls, the standard C +boolean operators +.CW && , +.CW || , +and +.CW ! , +and the standard relational operators +.CW == , +.CW != , +.CW > , +.CW >= , +.CW < , +and +.CW <= , +with +.CW == +and +.CW != +being overloaded to allow string comparisons as well. +.CW && +represents logical AND; +.CW || +is logical OR and +.CW ! +is logical NOT. The arithmetic and string operators take precedence +over all three of these operators, while NOT takes precedence over +AND, which takes precedence over OR. This precedence may be +overridden with parentheses, and an expression may be parenthesized to +your heart's content. Each term looks like a call on one of four +functions: +.nr pw 9 +.Ix 0 def make +.Ix 0 def conditional make +.Ix 0 def if make +.IP make \n(pw +The syntax is +.CW make( \fItarget\fP\c +.CW ) +where +.I target +is a target in the makefile. This is true if the given target was +specified on the command line, or as the source for a +.CW .MAIN +target (note that the sources for +.CW .MAIN +are only used if no targets were given on the command line). +.IP defined \n(pw +.Ix 0 def defined +.Ix 0 def conditional defined +.Ix 0 def if defined +The syntax is +.CW defined( \fIvariable\fP\c +.CW ) +and is true if +.I variable +is defined. Certain variables are defined in the system makefile that +identify the system on which PMake is being run. +.IP exists \n(pw +.Ix 0 def exists +.Ix 0 def conditional exists +.Ix 0 def if exists +The syntax is +.CW exists( \fIfile\fP\c +.CW ) +and is true if the file can be found on the global search path (i.e. +that defined by +.CW .PATH +targets, not by +.CW .PATH \fIsuffix\fP +targets). +.IP empty \n(pw +.Ix 0 def empty +.Ix 0 def conditional empty +.Ix 0 def if empty +This syntax is much like the others, except the string inside the +parentheses is of the same form as you would put between parentheses +when expanding a variable, complete with modifiers and everything. The +function returns true if the resulting string is empty (NOTE: an undefined +variable in this context will cause at the very least a warning +message about a malformed conditional, and at the worst will cause the +process to stop once it has read the makefile. If you want to check +for a variable being defined or empty, use the expression +.CW !defined( \fIvar\fP\c `` +.CW ") || empty(" \fIvar\fP\c +.CW ) '' +as the definition of +.CW || +will prevent the +.CW empty() +from being evaluated and causing an error, if the variable is +undefined). This can be used to see if a variable contains a given +word, for example: +.DS +#if !empty(\fIvar\fP:M\fIword\fP) +.DE +.LP +The arithmetic and string operators may only be used to test the value +of a variable. The lefthand side must contain the variable expansion, +while the righthand side contains either a string, enclosed in +double-quotes, or a number. The standard C numeric conventions (except +for specifying an octal number) apply to both sides. E.g. +.DS +#if $(OS) == 4.3 + +#if $(MACHINE) == "sun3" + +#if $(LOAD_ADDR) < 0xc000 +.DE +are all valid conditionals. In addition, the numeric value of a +variable can be tested as a boolean as follows: +.DS +#if $(LOAD) +.DE +would see if +.CW LOAD +contains a non-zero value and +.DS +#if !$(LOAD) +.DE +would test if +.CW LOAD +contains a zero value. +.LP +In addition to the bare +.CW #if ,'' `` +there are other forms that apply one of the first two functions to each +term. They are as follows: +.DS + ifdef \fRdefined\fP + ifndef \fR!defined\fP + ifmake \fRmake\fP + ifnmake \fR!make\fP +.DE +There are also the ``else if'' forms: +.CW elif , +.CW elifdef , +.CW elifndef , +.CW elifmake , +and +.CW elifnmake . +.LP +For instance, if you wish to create two versions of a program, one of which +is optimized (the production version) and the other of which is for debugging +(has symbols for dbx), you have two choices: you can create two +makefiles, one of which uses the +.CW \-g +flag for the compilation, while the other uses the +.CW \-O +flag, or you can use another target (call it +.CW debug ) +to create the debug version. The construct below will take care of +this for you. I have also made it so defining the variable +.CW DEBUG +(say with +.CW "pmake -D DEBUG" ) +will also cause the debug version to be made. +.DS +#if defined(DEBUG) || make(debug) +CFLAGS += -g +#else +CFLAGS += -O +#endif +.DE +There are, of course, problems with this approach. The most glaring +annoyance is that if you want to go from making a debug version to +making a production version, you have to remove all the object files, +or you will get some optimized and some debug versions in the same +program. Another annoyance is you have to be careful not to make two +targets that ``conflict'' because of some conditionals in the +makefile. For instance +.DS +#if make(print) +FORMATTER = ditroff -Plaser_printer +#endif +#if make(draft) +FORMATTER = nroff -Pdot_matrix_printer +#endif +.DE +would wreak havoc if you tried +.CW "pmake draft print" '' `` +since you would use the same formatter for each target. As I said, +this all gets somewhat complicated. +.xH 2 A Shell is a Shell is a Shell +.Rd 7 +.LP +In normal operation, the Bourne Shell (better known as +.CW sh '') `` +is used to execute the commands to re-create targets. PMake also allows you +to specify a different shell for it to use when executing these +commands. There are several things PMake must know about the shell you +wish to use. These things are specified as the sources for the +.CW .SHELL +.Ix 0 ref .SHELL +.Ix 0 ref target .SHELL +target by keyword, as follows: +.IP "\fBpath=\fP\fIpath\fP" +PMake needs to know where the shell actually resides, so it can +execute it. If you specify this and nothing else, PMake will use the +last component of the path and look in its table of the shells it +knows and use the specification it finds, if any. Use this if you just +want to use a different version of the Bourne or C Shell (yes, PMake knows +how to use the C Shell too). +.IP "\fBname=\fP\fIname\fP" +This is the name by which the shell is to be known. It is a single +word and, if no other keywords are specified (other than +.B path ), +it is the name by which PMake attempts to find a specification for +it (as mentioned above). You can use this if you would just rather use +the C Shell than the Bourne Shell +.CW ".SHELL: name=csh" '' (`` +will do it). +.IP "\fBquiet=\fP\fIecho-off command\fP" +As mentioned before, PMake actually controls whether commands are +printed by introducing commands into the shell's input stream. This +keyword, and the next two, control what those commands are. The +.B quiet +keyword is the command used to turn echoing off. Once it is turned +off, echoing is expected to remain off until the echo-on command is given. +.IP "\fBecho=\fP\fIecho-on command\fP" +The command PMake should give to turn echoing back on again. +.IP "\fBfilter=\fP\fIprinted echo-off command\fP" +Many shells will echo the echo-off command when it is given. This +keyword tells PMake in what format the shell actually prints the +echo-off command. Wherever PMake sees this string in the shell's +output, it will delete it and any following whitespace, up to and +including the next newline. See the example at the end of this section +for more details. +.IP "\fBechoFlag=\fP\fIflag to turn echoing on\fP" +Unless a target has been marked +.CW .SILENT , +PMake wants to start the shell running with echoing on. To do this, it +passes this flag to the shell as one of its arguments. If either this +or the next flag begins with a `\-', the flags will be passed to the +shell as separate arguments. Otherwise, the two will be concatenated +(if they are used at the same time, of course). +.IP "\fBerrFlag=\fP\fIflag to turn error checking on\fP" +Likewise, unless a target is marked +.CW .IGNORE , +PMake wishes error-checking to be on from the very start. To this end, +it will pass this flag to the shell as an argument. The same rules for +an initial `\-' apply as for the +.B echoFlag . +.IP "\fBcheck=\fP\fIcommand to turn error checking on\fP" +Just as for echo-control, error-control is achieved by inserting +commands into the shell's input stream. This is the command to make +the shell check for errors. It also serves another purpose if the +shell doesn't have error-control as commands, but I'll get into that +in a minute. Again, once error checking has been turned on, it is +expected to remain on until it is turned off again. +.IP "\fBignore=\fP\fIcommand to turn error checking off\fP" +This is the command PMake uses to turn error checking off. It has +another use if the shell doesn't do error-control, but I'll tell you +about that.\|.\|.\|now. +.IP "\fBhasErrCtl=\fP\fIyes or no\fP" +This takes a value that is either +.B yes +or +.B no . +Now you might think that the existence of the +.B check +and +.B ignore +keywords would be enough to tell PMake if the shell can do +error-control, but you'd be wrong. If +.B hasErrCtl +is +.B yes , +PMake uses the check and ignore commands in a straight-forward manner. +If this is +.B no , +however, their use is rather different. In this case, the check +command is used as a template, in which the string +.B %s +is replaced by the command that's about to be executed, to produce a +command for the shell that will echo the command to be executed. The +ignore command is also used as a template, again with +.B %s +replaced by the command to be executed, to produce a command that will +execute the command to be executed and ignore any error it returns. +When these strings are used as templates, you must provide newline(s) +.CW \en '') (`` +in the appropriate place(s). +.LP +The strings that follow these keywords may be enclosed in single or +double quotes (the quotes will be stripped off) and may contain the +usual C backslash-characters (\en is newline, \er is return, \eb is +backspace, \e' escapes a single-quote inside single-quotes, \e" +escapes a double-quote inside double-quotes). Now for an example. +.LP +This is actually the contents of the +.CW <shx.mk> +system makefile, and causes PMake to use the Bourne Shell in such a +way that each command is printed as it is executed. That is, if more +than one command is given on a line, each will be printed separately. +Similarly, each time the body of a loop is executed, the commands +within that loop will be printed, etc. The specification runs like +this: +.DS +# +# This is a shell specification to have the Bourne shell echo +# the commands just before executing them, rather than when it reads +# them. Useful if you want to see how variables are being expanded, etc. +# +\&.SHELL : path=/bin/sh \e + quiet="set -" \e + echo="set -x" \e + filter="+ set - " \e + echoFlag=x \e + errFlag=e \e + hasErrCtl=yes \e + check="set -e" \e + ignore="set +e" +.DE +.LP +It tells PMake the following: +.Bp +The shell is located in the file +.CW /bin/sh . +It need not tell PMake that the name of the shell is +.CW sh +as PMake can figure that out for itself (it's the last component of +the path). +.Bp +The command to stop echoing is +.CW "set -" . +.Bp +The command to start echoing is +.CW "set -x" . +.Bp +When the echo off command is executed, the shell will print +.CW "+ set - " +(The `+' comes from using the +.CW \-x +flag (rather than the +.CW \-v +flag PMake usually uses)). PMake will remove all occurrences of this +string from the output, so you don't notice extra commands you didn't +put there. +.Bp +The flag the Bourne Shell will take to start echoing in this way is +the +.CW \-x +flag. The Bourne Shell will only take its flag arguments concatenated +as its first argument, so neither this nor the +.B errFlag +specification begins with a \-. +.Bp +The flag to use to turn error-checking on from the start is +.CW \-e . +.Bp +The shell can turn error-checking on and off, and the commands to do +so are +.CW "set +e" +and +.CW "set -e" , +respectively. +.LP +I should note that this specification is for Bourne Shells that are +not part of Berkeley +.UX , +as shells from Berkeley don't do error control. You can get a similar +effect, however, by changing the last three lines to be: +.DS + hasErrCtl=no \e + check="echo \e"+ %s\e"\en" \e + ignore="sh -c '%s || exit 0\en" +.DE +.LP +This will cause PMake to execute the two commands +.DS +echo "+ \fIcmd\fP" +sh -c '\fIcmd\fP || true' +.DE +for each command for which errors are to be ignored. (In case you are +wondering, the thing for +.CW ignore +tells the shell to execute another shell without error checking on and +always exit 0, since the +.B || +causes the +.CW "exit 0" +to be executed only if the first command exited non-zero, and if the +first command exited zero, the shell will also exit zero, since that's +the last command it executed). +.xH 2 Compatibility +.Ix 0 ref compatibility +.LP +There are three (well, 3 \(12) levels of backwards-compatibility built +into PMake. Most makefiles will need none at all. Some may need a +little bit of work to operate correctly when run in parallel. Each +level encompasses the previous levels (e.g. +.B \-B +(one shell per command) implies +.B \-V ) +The three levels are described in the following three sections. +.xH 3 DEFCON 3 \*- Variable Expansion +.Ix 0 ref compatibility +.LP +As noted before, PMake will not expand a variable unless it knows of a +value for it. This can cause problems for makefiles that expect to +leave variables undefined except in special circumstances (e.g. if +more flags need to be passed to the C compiler or the output from a +text processor should be sent to a different printer). If the +variables are enclosed in curly braces +.CW ${PRINTER} ''), (`` +the shell will let them pass. If they are enclosed in parentheses, +however, the shell will declare a syntax error and the make will come +to a grinding halt. +.LP +You have two choices: change the makefile to define the variables +(their values can be overridden on the command line, since that's +where they would have been set if you used Make, anyway) or always give the +.B \-V +flag (this can be done with the +.CW .MAKEFLAGS +target, if you want). +.xH 3 DEFCON 2 \*- The Number of the Beast +.Ix 0 ref compatibility +.LP +Then there are the makefiles that expect certain commands, such as +changing to a different directory, to not affect other commands in a +target's creation script. You can solve this is either by going +back to executing one shell per command (which is what the +.B \-B +flag forces PMake to do), which slows the process down a good bit and +requires you to use semicolons and escaped newlines for shell constructs, or +by changing the makefile to execute the offending command(s) in a subshell +(by placing the line inside parentheses), like so: +.DS +install :: .MAKE + (cd src; $(.PMAKE) install) + (cd lib; $(.PMAKE) install) + (cd man; $(.PMAKE) install) +.DE +.Ix 0 ref operator double-colon +.Ix 0 ref variable global .PMAKE +.Ix 0 ref .PMAKE +.Ix 0 ref .MAKE +.Ix 0 ref attribute .MAKE +This will always execute the three makes (even if the +.B \-n +flag was given) because of the combination of the ``::'' operator and +the +.CW .MAKE +attribute. Each command will change to the proper directory to perform +the install, leaving the main shell in the directory in which it started. +.xH 3 "DEFCON 1 \*- Imitation is the Not the Highest Form of Flattery" +.Ix 0 ref compatibility +.LP +The final category of makefile is the one where every command requires +input, the dependencies are incompletely specified, or you simply +cannot create more than one target at a time, as mentioned earlier. In +addition, you may not have the time or desire to upgrade the makefile +to run smoothly with PMake. If you are the conservative sort, this is +the compatibility mode for you. It is entered either by giving PMake +the +.B \-M +flag (for Make), or by executing PMake as +.CW make .'' `` +In either case, PMake performs things exactly like Make (while still +supporting most of the nice new features PMake provides). This +includes: +.IP \(bu 2 +No parallel execution. +.IP \(bu 2 +Targets are made in the exact order specified by the makefile. The +sources for each target are made in strict left-to-right order, etc. +.IP \(bu 2 +A single Bourne shell is used to execute each command, thus the +shell's +.CW $$ +variable is useless, changing directories doesn't work across command +lines, etc. +.IP \(bu 2 +If no special characters exist in a command line, PMake will break the +command into words itself and execute the command directly, without +executing a shell first. The characters that cause PMake to execute a +shell are: +.CW # , +.CW = , +.CW | , +.CW ^ , +.CW ( , +.CW ) , +.CW { , +.CW } , +.CW ; , +.CW & , +.CW < , +.CW > , +.CW * , +.CW ? , +.CW [ , +.CW ] , +.CW : , +.CW $ , +.CW ` , +and +.CW \e . +You should notice that these are all the characters that are given +special meaning by the shell (except +.CW ' +and +.CW " , +which PMake deals with all by its lonesome). +.IP \(bu 2 +The use of the null suffix is turned off. +.Ix 0 ref "null suffix" +.Ix 0 ref suffix null +.xH 2 The Way Things Work +.LP +When PMake reads the makefile, it parses sources and targets into +nodes in a graph. The graph is directed only in the sense that PMake +knows which way is up. Each node contains not only links to all its +parents and children (the nodes that depend on it and those on which +it depends, respectively), but also a count of the number of its +children that have already been processed. +.LP +The most important thing to know about how PMake uses this graph is +that the traversal is breadth-first and occurs in two passes. +.LP +After PMake has parsed the makefile, it begins with the nodes the user +has told it to make (either on the command line, or via a +.CW .MAIN +target, or by the target being the first in the file not labeled with +the +.CW .NOTMAIN +attribute) placed in a queue. It continues to take the node off the +front of the queue, mark it as something that needs to be made, pass +the node to +.CW Suff_FindDeps +(mentioned earlier) to find any implicit sources for the node, and +place all the node's children that have yet to be marked at the end of +the queue. If any of the children is a +.CW .USE +rule, its attributes are applied to the parent, then its commands are +appended to the parent's list of commands and its children are linked +to its parent. The parent's unmade children counter is then decremented +(since the +.CW .USE +node has been processed). You will note that this allows a +.CW .USE +node to have children that are +.CW .USE +nodes and the rules will be applied in sequence. +If the node has no children, it is placed at the end of +another queue to be examined in the second pass. This process +continues until the first queue is empty. +.LP +At this point, all the leaves of the graph are in the examination +queue. PMake removes the node at the head of the queue and sees if it +is out-of-date. If it is, it is passed to a function that will execute +the commands for the node asynchronously. When the commands have +completed, all the node's parents have their unmade children counter +decremented and, if the counter is then 0, they are placed on the +examination queue. Likewise, if the node is up-to-date. Only those +parents that were marked on the downward pass are processed in this +way. Thus PMake traverses the graph back up to the nodes the user +instructed it to create. When the examination queue is empty and no +shells are running to create a target, PMake is finished. +.LP +Once all targets have been processed, PMake executes the commands +attached to the +.CW .END +target, either explicitly or through the use of an ellipsis in a shell +script. If there were no errors during the entire process but there +are still some targets unmade (PMake keeps a running count of how many +targets are left to be made), there is a cycle in the graph. PMake does +a depth-first traversal of the graph to find all the targets that +weren't made and prints them out one by one. +.xH 1 Answers to Exercises +.IP (3.1) +This is something of a trick question, for which I apologize. The +trick comes from the UNIX definition of a suffix, which PMake doesn't +necessarily share. You will have noticed that all the suffixes used in +this tutorial (and in UNIX in general) begin with a period +.CW .ms , ( +.CW .c , +etc.). Now, PMake's idea of a suffix is more like English's: it's the +characters at the end of a word. With this in mind, one possible +.Ix 0 def suffix +solution to this problem goes as follows: +.DS I +\&.SUFFIXES : ec.exe .exe ec.obj .obj .asm +ec.objec.exe .obj.exe : + link -o $(.TARGET) $(.IMPSRC) +\&.asmec.obj : + asm -o $(.TARGET) -DDO_ERROR_CHECKING $(.IMPSRC) +\&.asm.obj : + asm -o $(.TARGET) $(.IMPSRC) +.DE +.IP (3.2) +The trick to this one lies in the ``:='' variable-assignment operator +and the ``:S'' variable-expansion modifier. +.Ix 0 ref variable assignment expanded +.Ix 0 ref variable expansion modified +.Ix 0 ref modifier substitute +.Ix 0 ref :S +.Ix 0 ref := +Basically what you want is to take the pointer variable, so to speak, +and transform it into an invocation of the variable at which it +points. You might try something like +.DS I +$(PTR:S/^/\e$(/:S/$/)) +.DE +which places +.CW $( '' `` +at the front of the variable name and +.CW ) '' `` +at the end, thus transforming +.CW VAR ,'' `` +for example, into +.CW $(VAR) ,'' `` +which is just what we want. Unfortunately (as you know if you've tried +it), since, as it says in the hint, PMake does no further substitution +on the result of a modified expansion, that's \fIall\fP you get. The +solution is to make use of ``:='' to place that string into yet +another variable, then invoke the other variable directly: +.DS I +*PTR := $(PTR:S/^/\e$(/:S/$/)/) +.DE +You can then use +.CW $(*PTR) '' `` +to your heart's content. +.de Gp +.XP +\&\fB\\$1:\fP +.. +.xH 1 Glossary of Jargon +.Gp "attribute" +A property given to a target that causes PMake to treat it differently. +.Gp "command script" +The lines immediately following a dependency line that specify +commands to execute to create each of the targets on the dependency +line. Each line in the command script must begin with a tab. +.Gp "command-line variable" +A variable defined in an argument when PMake is first executed. +Overrides all assignments to the same variable name in the makefile. +.Gp "conditional" +A construct much like that used in C that allows a makefile to be +configured on the fly based on the local environment, or on what is being +made by that invocation of PMake. +.Gp "creation script" +Commands used to create a target. See ``command script.'' +.Gp "dependency" +The relationship between a source and a target. This comes in three +flavors, as indicated by the operator between the target and the +source. `:' gives a straight time-wise dependency (if the target is +older than the source, the target is out-of-date), while `!' provides +simply an ordering and always considers the target out-of-date. `::' +is much like `:', save it creates multiple instances of a target each +of which depends on its own list of sources. +.Gp "dynamic source" +This refers to a source that has a local variable invocation in it. It +allows a single dependency line to specify a different source for each +target on the line. +.Gp "global variable" +Any variable defined in a makefile. Takes precedence over variables +defined in the environment, but not over command-line or local variables. +.Gp "input graph" +What PMake constructs from a makefile. Consists of nodes made of the +targets in the makefile, and the links between them (the +dependencies). The links are directed (from source to target) and +there may not be any cycles (loops) in the graph. +.Gp "local variable" +A variable defined by PMake visible only in a target's shell script. +There are seven local variables, not all of which are defined for +every target: +.CW .TARGET , +.CW .ALLSRC , +.CW .OODATE , +.CW .PREFIX , +.CW .IMPSRC , +.CW .ARCHIVE , +and +.CW .MEMBER . +.CW .TARGET , +.CW .PREFIX , +.CW .ARCHIVE , +and +.CW .MEMBER +may be used on dependency lines to create ``dynamic sources.'' +.Gp "makefile" +A file that describes how a system is built. If you don't know what it +is after reading this tutorial.\|.\|.\|. +.Gp "modifier" +A letter, following a colon, used to alter how a variable is expanded. +It has no effect on the variable itself. +.Gp "operator" +What separates a source from a target (on a dependency line) and specifies +the relationship between the two. There are three: +.CW : ', ` +.CW :: ', ` +and +.CW ! '. ` +.Gp "search path" +A list of directories in which a file should be sought. PMake's view +of the contents of directories in a search path does not change once +the makefile has been read. A file is sought on a search path only if +it is exclusively a source. +.Gp "shell" +A program to which commands are passed in order to create targets. +.Gp "source" +Anything to the right of an operator on a dependency line. Targets on +the dependency line are usually created from the sources. +.Gp "special target" +A target that causes PMake to do special things when it's encountered. +.Gp "suffix" +The tail end of a file name. Usually begins with a period, +.CW .c +or +.CW .ms , +e.g. +.Gp "target" +A word to the left of the operator on a dependency line. More +generally, any file that PMake might create. A file may be (and often +is) both a target and a source (what it is depends on how PMake is +looking at it at the time \*- sort of like the wave/particle duality +of light, you know). +.Gp "transformation rule" +A special construct in a makefile that specifies how to create a file +of one type from a file of another, as indicated by their suffixes. +.Gp "variable expansion" +The process of substituting the value of a variable for a reference to +it. Expansion may be altered by means of modifiers. +.Gp "variable" +A place in which to store text that may be retrieved later. Also used +to define the local environment. Conditionals exist that test whether +a variable is defined or not. +.bp +.\" Output table of contents last, with an entry for the index, making +.\" sure to save and restore the last real page number for the index... +.nr @n \n(PN+1 +.\" We are not generating an index +.\" .XS \n(@n +.\" Index +.\" .XE +.nr %% \n% +.PX +.nr % \n(%% diff --git a/share/doc/psd/13.rcs/Makefile b/share/doc/psd/13.rcs/Makefile new file mode 100644 index 000000000000..ed497da4d8db --- /dev/null +++ b/share/doc/psd/13.rcs/Makefile @@ -0,0 +1,6 @@ +# $FreeBSD$ + +SUBDIR= rcs rcs_func + +.include <bsd.subdir.mk> + diff --git a/share/doc/psd/13.rcs/Makefile.inc b/share/doc/psd/13.rcs/Makefile.inc new file mode 100644 index 000000000000..666dbd862ff2 --- /dev/null +++ b/share/doc/psd/13.rcs/Makefile.inc @@ -0,0 +1,5 @@ +# $FreeBSD$ + +VOLUME= psd/13.rcs +MACROS= -ms +SRCDIR= ${.CURDIR}/../../../../../gnu/usr.bin/rcs/doc diff --git a/share/doc/psd/13.rcs/rcs/Makefile b/share/doc/psd/13.rcs/rcs/Makefile new file mode 100644 index 000000000000..6d94aed2bb44 --- /dev/null +++ b/share/doc/psd/13.rcs/rcs/Makefile @@ -0,0 +1,7 @@ +# $FreeBSD$ + +SRCS= rcs.ms +USE_PIC= +USE_TBL= + +.include <bsd.doc.mk> diff --git a/share/doc/psd/13.rcs/rcs_func/Makefile b/share/doc/psd/13.rcs/rcs_func/Makefile new file mode 100644 index 000000000000..09e5a9b226c2 --- /dev/null +++ b/share/doc/psd/13.rcs/rcs_func/Makefile @@ -0,0 +1,6 @@ +# $FreeBSD$ + +DOC= rcs_func +SRCS= rcs_func.ms + +.include <bsd.doc.mk> diff --git a/share/doc/psd/15.yacc/Makefile b/share/doc/psd/15.yacc/Makefile new file mode 100644 index 000000000000..e713ef267682 --- /dev/null +++ b/share/doc/psd/15.yacc/Makefile @@ -0,0 +1,16 @@ +# @(#)Makefile 8.1 (Berkeley) 8/14/93 +# $FreeBSD$ + +VOLUME= psd/15.yacc +SRCS= stubs ss_ ss0 ss1 ss2 ss3 ss4 ss5 ss6 ss7 ss8 ss9 \ + ss10 ss11 ssa ssb ssc ssd +EXTRA= ref.bib +MACROS= -ms +USE_REFER= +CLEANFILES= stubs + +stubs: + @(echo .R1; echo database ${.CURDIR}/ref.bib; \ + echo accumulate; echo .R2) > ${.TARGET} + +.include <bsd.doc.mk> diff --git a/share/doc/psd/15.yacc/ref.bib b/share/doc/psd/15.yacc/ref.bib new file mode 100644 index 000000000000..a1364f608f51 --- /dev/null +++ b/share/doc/psd/15.yacc/ref.bib @@ -0,0 +1,71 @@ +# $FreeBSD$ + +%T The C Programming Language +%A B. W. Kernighan +%A D. M. Ritchie +%I Prentice-Hall +%C Englewood Cliffs, New Jersey +%D 1978 + +%T LR Parsing +%A A. V. Aho +%A S. C. Johnson +%J Comp. Surveys +%V 6 +%N 2 +%P 99-124 +%D June 1974 + +%T Deterministic Parsing of Ambiguous Grammars +%A A. V. Aho +%A S. C. Johnson +%A J. D. Ullman +%J Comm. Assoc. Comp. Mach. +%K acm cacm +%V 18 +%N 8 +%P 441-452 +%D August 1975 + +%A A. V. Aho +%A J. D. Ullman +%T Principles of Compiler Design +%I Addison-Wesley +%C Reading, Mass. +%D 1977 + +%R Comp. Sci. Tech. Rep. No. 65 +%K CSTR +%A S. C. Johnson +%T Lint, a C Program Checker +%D December 1977 +%O updated version TM 78-1273-3 +%D 1978 + +%T A Portable Compiler: Theory and Practice +%A S. C. Johnson +%J Proc. 5th ACM Symp. on Principles of Programming Languages +%P 97-104 +%D January 1978 + +%K cstr +%R Comp. Sci. Tech. Rep. No. 17 +%I Bell Laboratories +%C Murray Hill, New Jersey +%A B. W. Kernighan +%A L. L. Cherry +%T A System for Typesetting Mathematics +%d May 1974, revised April 1977 +%J Comm. Assoc. Comp. Mach. +%K acm cacm +%V 18 +%P 151-157 +%D March 1975 + +%K CSTR +%R Comp. Sci. Tech. Rep. No. 39 +%I Bell Laboratories +%C Murray Hill, New Jersey +%A M. E. Lesk +%T Lex \(em A Lexical Analyzer Generator +%D October 1975 diff --git a/share/doc/psd/15.yacc/ss0 b/share/doc/psd/15.yacc/ss0 new file mode 100644 index 000000000000..223e90fbac54 --- /dev/null +++ b/share/doc/psd/15.yacc/ss0 @@ -0,0 +1,238 @@ +.\" Copyright (C) Caldera International Inc. 2001-2002. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions are +.\" met: +.\" +.\" Redistributions of source code and documentation must retain the above +.\" copyright notice, this list of conditions and the following +.\" disclaimer. +.\" +.\" Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" +.\" All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" +.\" This product includes software developed or owned by Caldera +.\" International, Inc. Neither the name of Caldera International, Inc. +.\" nor the names of other contributors may be used to endorse or promote +.\" products derived from this software without specific prior written +.\" permission. +.\" +.\" USE OF THE SOFTWARE PROVIDED FOR UNDER THIS LICENSE BY CALDERA +.\" INTERNATIONAL, INC. AND CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR +.\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +.\" WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +.\" DISCLAIMED. IN NO EVENT SHALL CALDERA INTERNATIONAL, INC. BE LIABLE +.\" FOR ANY DIRECT, INDIRECT INCIDENTAL, SPECIAL, EXEMPLARY, OR +.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR +.\" BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, +.\" WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE +.\" OR OTHERWISE) RISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN +.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +.\" +.\" @(#)ss0 8.1 (Berkeley) 6/8/93 +.\" +.\" $FreeBSD$ +.SH +0: Introduction +.PP +Yacc provides a general tool for imposing structure on the input to a computer program. +The Yacc user prepares a +specification of the input process; this includes rules +describing the input structure, code to be invoked when these +rules are recognized, and a low-level routine to do the +basic input. +Yacc then generates a function to control the input process. +This function, called a +.I parser , +calls the user-supplied low-level input routine +(the +.I "lexical analyzer" ) +to pick up the basic items +(called +.I tokens ) +from the input stream. +These tokens are organized according to the input structure rules, +called +.I "grammar rules" \|; +when one of these rules has been recognized, +then user code supplied for this rule, an +.I action , +is invoked; actions have the ability to return values and +make use of the values of other actions. +.PP +Yacc is written in a portable dialect of C +.[ +Ritchie Kernighan Language Prentice +.] +and the actions, and output subroutine, are in C as well. +Moreover, many of the syntactic conventions of Yacc follow C. +.PP +The heart of the input specification is a collection of grammar rules. +Each rule describes an allowable structure and gives it a name. +For example, one grammar rule might be +.DS +date : month\_name day \',\' year ; +.DE +Here, +.I date , +.I month\_name , +.I day , +and +.I year +represent structures of interest in the input process; +presumably, +.I month\_name , +.I day , +and +.I year +are defined elsewhere. +The comma ``,'' is enclosed in single quotes; this implies that the +comma is to appear literally in the input. +The colon and semicolon merely serve as punctuation in the rule, and have +no significance in controlling the input. +Thus, with proper definitions, the input +.DS +July 4, 1776 +.DE +might be matched by the above rule. +.PP +An important part of the input process is carried out by the +lexical analyzer. +This user routine reads the input stream, recognizing the lower level structures, +and communicates these tokens +to the parser. +For historical reasons, a structure recognized by the lexical analyzer is called a +.I "terminal symbol" , +while the structure recognized by the parser is called a +.I "nonterminal symbol" . +To avoid confusion, terminal symbols will usually be referred to as +.I tokens . +.PP +There is considerable leeway in deciding whether to recognize structures using the lexical +analyzer or grammar rules. +For example, the rules +.DS +month\_name : \'J\' \'a\' \'n\' ; +month\_name : \'F\' \'e\' \'b\' ; + + . . . + +month\_name : \'D\' \'e\' \'c\' ; +.DE +might be used in the above example. +The lexical analyzer would only need to recognize individual letters, and +.I month\_name +would be a nonterminal symbol. +Such low-level rules tend to waste time and space, and may +complicate the specification beyond Yacc's ability to deal with it. +Usually, the lexical analyzer would +recognize the month names, +and return an indication that a +.I month\_name +was seen; in this case, +.I month\_name +would be a token. +.PP +Literal characters such as ``,'' must also be passed through the lexical +analyzer, and are also considered tokens. +.PP +Specification files are very flexible. +It is realively easy to add to the above example the rule +.DS +date : month \'/\' day \'/\' year ; +.DE +allowing +.DS +7 / 4 / 1776 +.DE +as a synonym for +.DS +July 4, 1776 +.DE +In most cases, this new rule could be ``slipped in'' to a working system with minimal effort, +and little danger of disrupting existing input. +.PP +The input being read may not conform to the +specifications. +These input errors are detected as early as is theoretically possible with a +left-to-right scan; +thus, not only is the chance of reading and computing with bad +input data substantially reduced, but the bad data can usually be quickly found. +Error handling, +provided as part of the input specifications, +permits the reentry of bad data, +or the continuation of the input process after skipping over the bad data. +.PP +In some cases, Yacc fails to produce a parser when given a set of +specifications. +For example, the specifications may be self contradictory, or they may +require a more powerful recognition mechanism than that available to Yacc. +The former cases represent design errors; +the latter cases +can often be corrected +by making +the lexical analyzer +more powerful, or by rewriting some of the grammar rules. +While Yacc cannot handle all possible specifications, its power +compares favorably with similar systems; +moreover, the +constructions which are difficult for Yacc to handle are +also frequently difficult for human beings to handle. +Some users have reported that the discipline of formulating valid +Yacc specifications for their input revealed errors of +conception or design early in the program development. +.PP +The theory underlying Yacc has been described elsewhere. +.[ +Aho Johnson Surveys LR Parsing +.] +.[ +Aho Johnson Ullman Ambiguous Grammars +.] +.[ +Aho Ullman Principles Compiler Design +.] +Yacc has been extensively used in numerous practical applications, +including +.I lint , +.[ +Johnson Lint +.] +the Portable C Compiler, +.[ +Johnson Portable Compiler Theory +.] +and a system for typesetting mathematics. +.[ +Kernighan Cherry typesetting system CACM +.] +.PP +The next several sections describe the +basic process of preparing a Yacc specification; +Section 1 describes the preparation of grammar rules, +Section 2 the preparation of the user supplied actions associated with these rules, +and Section 3 the preparation of lexical analyzers. +Section 4 describes the operation of the parser. +Section 5 discusses various reasons why Yacc may be unable to produce a +parser from a specification, and what to do about it. +Section 6 describes a simple mechanism for +handling operator precedences in arithmetic expressions. +Section 7 discusses error detection and recovery. +Section 8 discusses the operating environment and special features +of the parsers Yacc produces. +Section 9 gives some suggestions which should improve the +style and efficiency of the specifications. +Section 10 discusses some advanced topics, and Section 11 gives +acknowledgements. +Appendix A has a brief example, and Appendix B gives a +summary of the Yacc input syntax. +Appendix C gives an example using some of the more advanced +features of Yacc, and, finally, +Appendix D describes mechanisms and syntax +no longer actively supported, but +provided for historical continuity with older versions of Yacc. diff --git a/share/doc/psd/15.yacc/ss1 b/share/doc/psd/15.yacc/ss1 new file mode 100644 index 000000000000..f9369fb4bf5d --- /dev/null +++ b/share/doc/psd/15.yacc/ss1 @@ -0,0 +1,175 @@ +.\" Copyright (C) Caldera International Inc. 2001-2002. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions are +.\" met: +.\" +.\" Redistributions of source code and documentation must retain the above +.\" copyright notice, this list of conditions and the following +.\" disclaimer. +.\" +.\" Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" +.\" All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" +.\" This product includes software developed or owned by Caldera +.\" International, Inc. Neither the name of Caldera International, Inc. +.\" nor the names of other contributors may be used to endorse or promote +.\" products derived from this software without specific prior written +.\" permission. +.\" +.\" USE OF THE SOFTWARE PROVIDED FOR UNDER THIS LICENSE BY CALDERA +.\" INTERNATIONAL, INC. AND CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR +.\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +.\" WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +.\" DISCLAIMED. IN NO EVENT SHALL CALDERA INTERNATIONAL, INC. BE LIABLE +.\" FOR ANY DIRECT, INDIRECT INCIDENTAL, SPECIAL, EXEMPLARY, OR +.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR +.\" BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, +.\" WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE +.\" OR OTHERWISE) RISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN +.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +.\" +.\" @(#)ss1 8.1 (Berkeley) 6/8/93 +.\" +.\" $FreeBSD$ +.tr *\(** +.tr |\(or +.SH +1: Basic Specifications +.PP +Names refer to either tokens or nonterminal symbols. +Yacc requires +token names to be declared as such. +In addition, for reasons discussed in Section 3, it is often desirable +to include the lexical analyzer as part of the specification file; +it may be useful to include other programs as well. +Thus, every specification file consists of three sections: +the +.I declarations , +.I "(grammar) rules" , +and +.I programs . +The sections are separated by double percent ``%%'' marks. +(The percent ``%'' is generally used in Yacc specifications as an escape character.) +.PP +In other words, a full specification file looks like +.DS +declarations +%% +rules +%% +programs +.DE +.PP +The declaration section may be empty. +Moreover, if the programs section is omitted, the second %% mark may be omitted also; +thus, the smallest legal Yacc specification is +.DS +%% +rules +.DE +.PP +Blanks, tabs, and newlines are ignored except +that they may not appear in names or multi-character reserved symbols. +Comments may appear wherever a name is legal; they are enclosed +in /* . . . */, as in C and PL/I. +.PP +The rules section is made up of one or more grammar rules. +A grammar rule has the form: +.DS +A : BODY ; +.DE +A represents a nonterminal name, and BODY represents a sequence of zero or more names and literals. +The colon and the semicolon are Yacc punctuation. +.PP +Names may be of arbitrary length, and may be made up of letters, dot ``.'', underscore ``\_'', and +non-initial digits. +Upper and lower case letters are distinct. +The names used in the body of a grammar rule may represent tokens or nonterminal symbols. +.PP +A literal consists of a character enclosed in single quotes ``\'''. +As in C, the backslash ``\e'' is an escape character within literals, and all the C escapes +are recognized. +Thus +.DS +\'\en\' newline +\'\er\' return +\'\e\'\' single quote ``\''' +\'\e\e\' backslash ``\e'' +\'\et\' tab +\'\eb\' backspace +\'\ef\' form feed +\'\exxx\' ``xxx'' in octal +.DE +For a number of technical reasons, the +\s-2NUL\s0 +character (\'\e0\' or 0) should never +be used in grammar rules. +.PP +If there are several grammar rules with the same left hand side, the vertical bar ``|'' +can be used to avoid rewriting the left hand side. +In addition, +the semicolon at the end of a rule can be dropped before a vertical bar. +Thus the grammar rules +.DS +A : B C D ; +A : E F ; +A : G ; +.DE +can be given to Yacc as +.DS +A : B C D + | E F + | G + ; +.DE +It is not necessary that all grammar rules with the same left side appear together in the grammar rules section, +although it makes the input much more readable, and easier to change. +.PP +If a nonterminal symbol matches the empty string, this can be indicated in the obvious way: +.DS +empty : ; +.DE +.PP +Names representing tokens must be declared; this is most simply done by writing +.DS +%token name1 name2 . . . +.DE +in the declarations section. +(See Sections 3 , 5, and 6 for much more discussion). +Every name not defined in the declarations section is assumed to represent a nonterminal symbol. +Every nonterminal symbol must appear on the left side of at least one rule. +.PP +Of all the nonterminal symbols, one, called the +.I "start symbol" , +has particular importance. +The parser is designed to recognize the start symbol; thus, +this symbol represents the largest, +most general structure described by the grammar rules. +By default, +the start symbol is taken to be the left hand side of the first +grammar rule in the rules section. +It is possible, and in fact desirable, to declare the start +symbol explicitly in the declarations section using the %start keyword: +.DS +%start symbol +.DE +.PP +The end of the input to the parser is signaled by a special token, called the +.I endmarker . +If the tokens up to, but not including, the endmarker form a structure +which matches the start symbol, the parser function returns to its caller +after the endmarker is seen; it +.I accepts +the input. +If the endmarker is seen in any other context, it is an error. +.PP +It is the job of the user-supplied lexical analyzer +to return the endmarker when appropriate; see section 3, below. +Usually the endmarker represents some reasonably obvious +I/O status, such as ``end-of-file'' or ``end-of-record''. diff --git a/share/doc/psd/15.yacc/ss10 b/share/doc/psd/15.yacc/ss10 new file mode 100644 index 000000000000..f6f1702de803 --- /dev/null +++ b/share/doc/psd/15.yacc/ss10 @@ -0,0 +1,221 @@ +.\" Copyright (C) Caldera International Inc. 2001-2002. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions are +.\" met: +.\" +.\" Redistributions of source code and documentation must retain the above +.\" copyright notice, this list of conditions and the following +.\" disclaimer. +.\" +.\" Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" +.\" All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" +.\" This product includes software developed or owned by Caldera +.\" International, Inc. Neither the name of Caldera International, Inc. +.\" nor the names of other contributors may be used to endorse or promote +.\" products derived from this software without specific prior written +.\" permission. +.\" +.\" USE OF THE SOFTWARE PROVIDED FOR UNDER THIS LICENSE BY CALDERA +.\" INTERNATIONAL, INC. AND CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR +.\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +.\" WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +.\" DISCLAIMED. IN NO EVENT SHALL CALDERA INTERNATIONAL, INC. BE LIABLE +.\" FOR ANY DIRECT, INDIRECT INCIDENTAL, SPECIAL, EXEMPLARY, OR +.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR +.\" BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, +.\" WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE +.\" OR OTHERWISE) RISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN +.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +.\" +.\" @(#)ssA 8.1 (Berkeley) 6/8/93 +.\" +.\" $FreeBSD$ +.SH +10: Advanced Topics +.PP +This section discusses a number of advanced features +of Yacc. +.SH +Simulating Error and Accept in Actions +.PP +The parsing actions of error and accept can be simulated +in an action by use of macros YYACCEPT and YYERROR. +YYACCEPT causes +.I yyparse +to return the value 0; +YYERROR causes +the parser to behave as if the current input symbol +had been a syntax error; +.I yyerror +is called, and error recovery takes place. +These mechanisms can be used to simulate parsers +with multiple endmarkers or context-sensitive syntax checking. +.SH +Accessing Values in Enclosing Rules. +.PP +An action may refer to values +returned by actions to the left of the current rule. +The mechanism is simply the same as with ordinary actions, +a dollar sign followed by a digit, but in this case the +digit may be 0 or negative. +Consider +.DS +sent : adj noun verb adj noun + { \fIlook at the sentence\fR . . . } + ; + +adj : THE { $$ = THE; } + | YOUNG { $$ = YOUNG; } + . . . + ; + +noun : DOG + { $$ = DOG; } + | CRONE + { if( $0 == YOUNG ){ + printf( "what?\en" ); + } + $$ = CRONE; + } + ; + . . . +.DE +In the action following the word CRONE, a check is made that the +preceding token shifted was not YOUNG. +Obviously, this is only possible when a great deal is known about +what might precede the symbol +.I noun +in the input. +There is also a distinctly unstructured flavor about this. +Nevertheless, at times this mechanism will save a great +deal of trouble, especially when a few combinations are to +be excluded from an otherwise regular structure. +.SH +Support for Arbitrary Value Types +.PP +By default, the values returned by actions and the lexical analyzer are integers. +Yacc can also support +values of other types, including structures. +In addition, Yacc keeps track of the types, and inserts +appropriate union member names so that the resulting parser will +be strictly type checked. +The Yacc value stack (see Section 4) +is declared to be a +.I union +of the various types of values desired. +The user declares the union, and associates union member names +to each token and nonterminal symbol having a value. +When the value is referenced through a $$ or $n construction, +Yacc will automatically insert the appropriate union name, so that +no unwanted conversions will take place. +In addition, type checking commands such as +.I Lint\| +.[ +Johnson Lint Checker 1273 +.] +will be far more silent. +.PP +There are three mechanisms used to provide for this typing. +First, there is a way of defining the union; this must be +done by the user since other programs, notably the lexical analyzer, +must know about the union member names. +Second, there is a way of associating a union member name with tokens +and nonterminals. +Finally, there is a mechanism for describing the type of those +few values where Yacc can not easily determine the type. +.PP +To declare the union, the user includes in the declaration section: +.DS +%union { + body of union ... + } +.DE +This declares the Yacc value stack, +and the external variables +.I yylval +and +.I yyval , +to have type equal to this union. +If Yacc was invoked with the +.B \-d +option, the union declaration +is copied onto the +.I y.tab.h +file. +Alternatively, +the union may be declared in a header file, and a typedef +used to define the variable YYSTYPE to represent +this union. +Thus, the header file might also have said: +.DS +typedef union { + body of union ... + } YYSTYPE; +.DE +The header file must be included in the declarations +section, by use of %{ and %}. +.PP +Once YYSTYPE is defined, +the union member names must be associated +with the various terminal and nonterminal names. +The construction +.DS +< name > +.DE +is used to indicate a union member name. +If this follows +one of the +keywords %token, +%left, %right, and %nonassoc, +the union member name is associated with the tokens listed. +Thus, saying +.DS +%left <optype> \'+\' \'\-\' +.DE +will cause any reference to values returned by these two tokens to be +tagged with +the union member name +.I optype . +Another keyword, %type, is +used similarly to associate +union member names with nonterminals. +Thus, one might say +.DS +%type <nodetype> expr stat +.DE +.PP +There remain a couple of cases where these mechanisms are insufficient. +If there is an action within a rule, the value returned +by this action has no +.I "a priori" +type. +Similarly, reference to left context values (such as $0 \- see the +previous subsection ) leaves Yacc with no easy way of knowing the type. +In this case, a type can be imposed on the reference by inserting +a union member name, between < and >, immediately after +the first $. +An example of this usage is +.DS +rule : aaa { $<intval>$ = 3; } bbb + { fun( $<intval>2, $<other>0 ); } + ; +.DE +This syntax has little to recommend it, but the situation arises rarely. +.PP +A sample specification is given in Appendix C. +The facilities in this subsection are not triggered until they are used: +in particular, the use of %type will turn on these mechanisms. +When they are used, there is a fairly strict level of checking. +For example, use of $n or $$ to refer to something with no defined type +is diagnosed. +If these facilities are not triggered, the Yacc value stack is used to +hold +.I int' s, +as was true historically. diff --git a/share/doc/psd/15.yacc/ss11 b/share/doc/psd/15.yacc/ss11 new file mode 100644 index 000000000000..c16bb52b9422 --- /dev/null +++ b/share/doc/psd/15.yacc/ss11 @@ -0,0 +1,63 @@ +.\" Copyright (C) Caldera International Inc. 2001-2002. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions are +.\" met: +.\" +.\" Redistributions of source code and documentation must retain the above +.\" copyright notice, this list of conditions and the following +.\" disclaimer. +.\" +.\" Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" +.\" All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" +.\" This product includes software developed or owned by Caldera +.\" International, Inc. Neither the name of Caldera International, Inc. +.\" nor the names of other contributors may be used to endorse or promote +.\" products derived from this software without specific prior written +.\" permission. +.\" +.\" USE OF THE SOFTWARE PROVIDED FOR UNDER THIS LICENSE BY CALDERA +.\" INTERNATIONAL, INC. AND CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR +.\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +.\" WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +.\" DISCLAIMED. IN NO EVENT SHALL CALDERA INTERNATIONAL, INC. BE LIABLE +.\" FOR ANY DIRECT, INDIRECT INCIDENTAL, SPECIAL, EXEMPLARY, OR +.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR +.\" BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, +.\" WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE +.\" OR OTHERWISE) RISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN +.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +.\" +.\" @(#)ssB 8.1 (Berkeley) 6/8/93 +.\" +.\" $FreeBSD$ +.SH +11: Acknowledgements +.PP +Yacc owes much to a +most stimulating collection of users, who have goaded +me beyond my inclination, and frequently beyond my +ability, in their endless search for ``one more feature''. +Their irritating unwillingness to learn how to +do things my way has usually led to my doing things their way; +most of the time, they have been right. +B. W. Kernighan, P. J. Plauger, S. I. Feldman, C. Imagna, +M. E. Lesk, +and A. Snyder will recognize some of their ideas in the current version +of Yacc. +C. B. Haley contributed to the error recovery algorithm. +D. M. Ritchie, B. W. Kernighan, and M. O. Harris helped translate this document into English. +Al Aho also deserves special credit for bringing +the mountain to Mohammed, and other favors. +.\" .SG "MH-1273-SCJ-unix" +.bp +.[ +$LIST$ +.] +.bp diff --git a/share/doc/psd/15.yacc/ss2 b/share/doc/psd/15.yacc/ss2 new file mode 100644 index 000000000000..f1fdb4407581 --- /dev/null +++ b/share/doc/psd/15.yacc/ss2 @@ -0,0 +1,190 @@ +.\" Copyright (C) Caldera International Inc. 2001-2002. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions are +.\" met: +.\" +.\" Redistributions of source code and documentation must retain the above +.\" copyright notice, this list of conditions and the following +.\" disclaimer. +.\" +.\" Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" +.\" All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" +.\" This product includes software developed or owned by Caldera +.\" International, Inc. Neither the name of Caldera International, Inc. +.\" nor the names of other contributors may be used to endorse or promote +.\" products derived from this software without specific prior written +.\" permission. +.\" +.\" USE OF THE SOFTWARE PROVIDED FOR UNDER THIS LICENSE BY CALDERA +.\" INTERNATIONAL, INC. AND CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR +.\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +.\" WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +.\" DISCLAIMED. IN NO EVENT SHALL CALDERA INTERNATIONAL, INC. BE LIABLE +.\" FOR ANY DIRECT, INDIRECT INCIDENTAL, SPECIAL, EXEMPLARY, OR +.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR +.\" BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, +.\" WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE +.\" OR OTHERWISE) RISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN +.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +.\" +.\" @(#)ss2 8.1 (Berkeley) 6/8/93 +.\" +.\" $FreeBSD$ +.SH +2: Actions +.PP +With each grammar rule, the user may associate actions to be performed each time +the rule is recognized in the input process. +These actions may return values, and may obtain the values returned by previous +actions. +Moreover, the lexical analyzer can return values +for tokens, if desired. +.PP +An action is an arbitrary C statement, and as such can do +input and output, call subprograms, and alter +external vectors and variables. +An action is specified by +one or more statements, enclosed in curly braces ``{'' and ``}''. +For example, +.DS +A : \'(\' B \')\' + { hello( 1, "abc" ); } +.DE +and +.DS +XXX : YYY ZZZ + { printf("a message\en"); + flag = 25; } +.DE +are grammar rules with actions. +.PP +To facilitate easy communication between the actions and the parser, the action statements are altered +slightly. +The symbol ``dollar sign'' ``$'' is used as a signal to Yacc in this context. +.PP +To return a value, the action normally sets the +pseudo-variable ``$$'' to some value. +For example, an action that does nothing but return the value 1 is +.DS + { $$ = 1; } +.DE +.PP +To obtain the values returned by previous actions and the lexical analyzer, the +action may use the pseudo-variables $1, $2, . . ., +which refer to the values returned by the +components of the right side of a rule, reading from left to right. +Thus, if the rule is +.DS +A : B C D ; +.DE +for example, then $2 has the value returned by C, and $3 the value returned by D. +.PP +As a more concrete example, consider the rule +.DS +expr : \'(\' expr \')\' ; +.DE +The value returned by this rule is usually the value of the +.I expr +in parentheses. +This can be indicated by +.DS +expr : \'(\' expr \')\' { $$ = $2 ; } +.DE +.PP +By default, the value of a rule is the value of the first element in it ($1). +Thus, grammar rules of the form +.DS +A : B ; +.DE +frequently need not have an explicit action. +.PP +In the examples above, all the actions came at the end of their rules. +Sometimes, it is desirable to get control before a rule is fully parsed. +Yacc permits an action to be written in the middle of a rule as well +as at the end. +This rule is assumed to return a value, accessible +.\" XXX What does this mean? Nobody seems to understand it. +.\" through the usual \$ mechanism by the actions to +through the usual mechanism by the actions to +the right of it. +In turn, it may access the values +returned by the symbols to its left. +Thus, in the rule +.DS +A : B + { $$ = 1; } + C + { x = $2; y = $3; } + ; +.DE +the effect is to set +.I x +to 1, and +.I y +to the value returned by C. +.PP +Actions that do not terminate a rule are actually +handled by Yacc by manufacturing a new nonterminal +symbol name, and a new rule matching this +name to the empty string. +The interior action is the action triggered off by recognizing +this added rule. +Yacc actually treats the above example as if +it had been written: +.DS +$ACT : /* empty */ + { $$ = 1; } + ; + +A : B $ACT C + { x = $2; y = $3; } + ; +.DE +.PP +In many applications, output is not done directly by the actions; +rather, a data structure, such as a parse tree, is constructed in memory, +and transformations are applied to it before output is generated. +Parse trees are particularly easy to +construct, given routines to build and maintain the tree +structure desired. +For example, suppose there is a C function +.I node , +written so that the call +.DS +node( L, n1, n2 ) +.DE +creates a node with label L, and descendants n1 and n2, and returns the index of +the newly created node. +Then parse tree can be built by supplying actions such as: +.DS +expr : expr \'+\' expr + { $$ = node( \'+\', $1, $3 ); } +.DE +in the specification. +.PP +The user may define other variables to be used by the actions. +Declarations and definitions can appear in +the declarations section, +enclosed in the marks ``%{'' and ``%}''. +These declarations and definitions have global scope, +so they are known to the action statements and the lexical analyzer. +For example, +.DS +%{ int variable = 0; %} +.DE +could be placed in the declarations section, +making +.I variable +accessible to all of the actions. +The Yacc parser uses only names beginning in ``yy''; +the user should avoid such names. +.PP +In these examples, all the values are integers: a discussion of +values of other types will be found in Section 10. diff --git a/share/doc/psd/15.yacc/ss3 b/share/doc/psd/15.yacc/ss3 new file mode 100644 index 000000000000..fa06acb75b09 --- /dev/null +++ b/share/doc/psd/15.yacc/ss3 @@ -0,0 +1,141 @@ +.\" Copyright (C) Caldera International Inc. 2001-2002. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions are +.\" met: +.\" +.\" Redistributions of source code and documentation must retain the above +.\" copyright notice, this list of conditions and the following +.\" disclaimer. +.\" +.\" Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" +.\" All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" +.\" This product includes software developed or owned by Caldera +.\" International, Inc. Neither the name of Caldera International, Inc. +.\" nor the names of other contributors may be used to endorse or promote +.\" products derived from this software without specific prior written +.\" permission. +.\" +.\" USE OF THE SOFTWARE PROVIDED FOR UNDER THIS LICENSE BY CALDERA +.\" INTERNATIONAL, INC. AND CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR +.\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +.\" WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +.\" DISCLAIMED. IN NO EVENT SHALL CALDERA INTERNATIONAL, INC. BE LIABLE +.\" FOR ANY DIRECT, INDIRECT INCIDENTAL, SPECIAL, EXEMPLARY, OR +.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR +.\" BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, +.\" WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE +.\" OR OTHERWISE) RISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN +.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +.\" +.\" @(#)ss3 8.1 (Berkeley) 6/8/93 +.\" +.\" $FreeBSD$ +.SH +3: Lexical Analysis +.PP +The user must supply a lexical analyzer to read the input stream and communicate tokens +(with values, if desired) to the parser. +The lexical analyzer is an integer-valued function called +.I yylex . +The function returns an integer, the +.I "token number" , +representing the kind of token read. +If there is a value associated with that token, it should be assigned +to the external variable +.I yylval . +.PP +The parser and the lexical analyzer must agree on these token numbers in order for +communication between them to take place. +The numbers may be chosen by Yacc, or chosen by the user. +In either case, the ``# define'' mechanism of C is used to allow the lexical analyzer +to return these numbers symbolically. +For example, suppose that the token name DIGIT has been defined in the declarations section of the +Yacc specification file. +The relevant portion of the lexical analyzer might look like: +.DS +yylex(){ + extern int yylval; + int c; + . . . + c = getchar(); + . . . + switch( c ) { + . . . + case \'0\': + case \'1\': + . . . + case \'9\': + yylval = c\-\'0\'; + return( DIGIT ); + . . . + } + . . . +.DE +.PP +The intent is to return a token number of DIGIT, and a value equal to the numerical value of the +digit. +Provided that the lexical analyzer code is placed in the programs section of the specification file, +the identifier DIGIT will be defined as the token number associated +with the token DIGIT. +.PP +This mechanism leads to clear, +easily modified lexical analyzers; the only pitfall is the need +to avoid using any token names in the grammar that are reserved +or significant in C or the parser; for example, the use of +token names +.I if +or +.I while +will almost certainly cause severe +difficulties when the lexical analyzer is compiled. +The token name +.I error +is reserved for error handling, and should not be used naively +(see Section 7). +.PP +As mentioned above, the token numbers may be chosen by Yacc or by the user. +In the default situation, the numbers are chosen by Yacc. +The default token number for a literal +character is the numerical value of the character in the local character set. +Other names are assigned token numbers +starting at 257. +.PP +To assign a token number to a token (including literals), +the first appearance of the token name or literal +.I +in the declarations section +.R +can be immediately followed by +a nonnegative integer. +This integer is taken to be the token number of the name or literal. +Names and literals not defined by this mechanism retain their default definition. +It is important that all token numbers be distinct. +.PP +For historical reasons, the endmarker must have token +number 0 or negative. +This token number cannot be redefined by the user; thus, all +lexical analyzers should be prepared to return 0 or negative as a token number +upon reaching the end of their input. +.PP +A very useful tool for constructing lexical analyzers is +the +.I Lex +program developed by Mike Lesk. +.[ +Lesk Lex +.] +These lexical analyzers are designed to work in close +harmony with Yacc parsers. +The specifications for these lexical analyzers +use regular expressions instead of grammar rules. +Lex can be easily used to produce quite complicated lexical analyzers, +but there remain some languages (such as FORTRAN) which do not +fit any theoretical framework, and whose lexical analyzers +must be crafted by hand. diff --git a/share/doc/psd/15.yacc/ss4 b/share/doc/psd/15.yacc/ss4 new file mode 100644 index 000000000000..e548d53cfc7d --- /dev/null +++ b/share/doc/psd/15.yacc/ss4 @@ -0,0 +1,367 @@ +.\" Copyright (C) Caldera International Inc. 2001-2002. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions are +.\" met: +.\" +.\" Redistributions of source code and documentation must retain the above +.\" copyright notice, this list of conditions and the following +.\" disclaimer. +.\" +.\" Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" +.\" All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" +.\" This product includes software developed or owned by Caldera +.\" International, Inc. Neither the name of Caldera International, Inc. +.\" nor the names of other contributors may be used to endorse or promote +.\" products derived from this software without specific prior written +.\" permission. +.\" +.\" USE OF THE SOFTWARE PROVIDED FOR UNDER THIS LICENSE BY CALDERA +.\" INTERNATIONAL, INC. AND CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR +.\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +.\" WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +.\" DISCLAIMED. IN NO EVENT SHALL CALDERA INTERNATIONAL, INC. BE LIABLE +.\" FOR ANY DIRECT, INDIRECT INCIDENTAL, SPECIAL, EXEMPLARY, OR +.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR +.\" BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, +.\" WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE +.\" OR OTHERWISE) RISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN +.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +.\" +.\" @(#)ss4 8.1 (Berkeley) 6/8/93 +.\" +.\" $FreeBSD$ +.SH +4: How the Parser Works +.PP +Yacc turns the specification file into a C program, which +parses the input according to the specification given. +The algorithm used to go from the +specification to the parser is complex, and will not be discussed +here (see +the references for more information). +The parser itself, however, is relatively simple, +and understanding how it works, while +not strictly necessary, will nevertheless make +treatment of error recovery and ambiguities much more +comprehensible. +.PP +The parser produced by Yacc consists +of a finite state machine with a stack. +The parser is also capable of reading and remembering the next +input token (called the +.I lookahead +token). +The +.I "current state" +is always the one on the top of the stack. +The states of the finite state machine are given +small integer labels; initially, the machine is in state 0, +the stack contains only state 0, and no lookahead token has been read. +.PP +The machine has only four actions available to it, called +.I shift , +.I reduce , +.I accept , +and +.I error . +A move of the parser is done as follows: +.IP 1. +Based on its current state, the parser decides +whether it needs a lookahead token to decide +what action should be done; if it needs one, and does +not have one, it calls +.I yylex +to obtain the next token. +.IP 2. +Using the current state, and the lookahead token +if needed, the parser decides on its next action, and +carries it out. +This may result in states being pushed onto the stack, or popped off of +the stack, and in the lookahead token being processed +or left alone. +.PP +The +.I shift +action is the most common action the parser takes. +Whenever a shift action is taken, there is always +a lookahead token. +For example, in state 56 there may be +an action: +.DS + IF shift 34 +.DE +which says, in state 56, if the lookahead token is IF, +the current state (56) is pushed down on the stack, +and state 34 becomes the current state (on the +top of the stack). +The lookahead token is cleared. +.PP +The +.I reduce +action keeps the stack from growing without +bounds. +Reduce actions are appropriate when the parser has seen +the right hand side of a grammar rule, +and is prepared to announce that it has seen +an instance of the rule, replacing the right hand side +by the left hand side. +It may be necessary to consult the lookahead token +to decide whether to reduce, but +usually it is not; in fact, the default +action (represented by a ``.'') is often a reduce action. +.PP +Reduce actions are associated with individual grammar rules. +Grammar rules are also given small integer +numbers, leading to some confusion. +The action +.DS + \fB.\fR reduce 18 +.DE +refers to +.I "grammar rule" +18, while the action +.DS + IF shift 34 +.DE +refers to +.I state +34. +.PP +Suppose the rule being reduced is +.DS +A \fB:\fR x y z ; +.DE +The reduce action depends on the +left hand symbol (A in this case), and the number of +symbols on the right hand side (three in this case). +To reduce, first pop off the top three states +from the stack +(In general, the number of states popped equals the number of symbols on the +right side of the rule). +In effect, these states were the ones +put on the stack while recognizing +.I x , +.I y , +and +.I z , +and no longer serve any useful purpose. +After popping these states, a state is uncovered +which was the state the parser was in before beginning to +process the rule. +Using this uncovered state, and the symbol +on the left side of the rule, perform what is in +effect a shift of A. +A new state is obtained, pushed onto the stack, and parsing continues. +There are significant differences between the processing of +the left hand symbol and an ordinary shift of a token, +however, so this action is called a +.I goto +action. +In particular, the lookahead token is cleared by a shift, and +is not affected by a goto. +In any case, the uncovered state contains an entry such as: +.DS + A goto 20 +.DE +causing state 20 to be pushed +onto the stack, and become the current state. +.PP +In effect, the reduce action ``turns back the clock'' in the parse, +popping the states off the stack to go back to the +state where the right hand side of the rule was first seen. +The parser then behaves as if it had seen the left side at that time. +If the right hand side of the rule is empty, +no states are popped off of the stack: the uncovered state +is in fact the current state. +.PP +The reduce action is also important in the treatment of user-supplied +actions and values. +When a rule is reduced, the code supplied with the rule is executed +before the stack is adjusted. +In addition to the stack holding the states, another stack, +running in parallel with it, holds the values returned +from the lexical analyzer and the actions. +When a shift takes place, the external variable +.I yylval +is copied onto the value stack. +After the return from the user code, the reduction is carried out. +When the +.I goto +action is done, the external variable +.I yyval +is copied onto the value stack. +The pseudo-variables $1, $2, etc., refer to the value stack. +.PP +The other two parser actions are conceptually much simpler. +The +.I accept +action indicates that the entire input has been seen and +that it matches the specification. +This action appears only when the lookahead token is +the endmarker, and indicates that the parser has successfully +done its job. +The +.I error +action, on the other hand, represents a place where the parser +can no longer continue parsing according to the specification. +The input tokens it has seen, together with the lookahead token, +cannot be followed by anything that would result +in a legal input. +The parser reports an error, and attempts to recover the situation and +resume parsing: the error recovery (as opposed to the detection of error) +will be covered in Section 7. +.PP +It is time for an example! +Consider the specification +.DS +%token DING DONG DELL +%% +rhyme : sound place + ; +sound : DING DONG + ; +place : DELL + ; +.DE +.PP +When Yacc is invoked with the +.B \-v +option, a file called +.I y.output +is produced, with a human-readable description of the parser. +The +.I y.output +file corresponding to the above grammar (with some statistics +stripped off the end) is: +.DS +state 0 + $accept : \_rhyme $end + + DING shift 3 + . error + + rhyme goto 1 + sound goto 2 + +state 1 + $accept : rhyme\_$end + + $end accept + . error + +state 2 + rhyme : sound\_place + + DELL shift 5 + . error + + place goto 4 + +state 3 + sound : DING\_DONG + + DONG shift 6 + . error + +state 4 + rhyme : sound place\_ (1) + + . reduce 1 + +state 5 + place : DELL\_ (3) + + . reduce 3 + +state 6 + sound : DING DONG\_ (2) + + . reduce 2 +.DE +Notice that, in addition to the actions for each state, there is a +description of the parsing rules being processed in each +state. The \_ character is used to indicate +what has been seen, and what is yet to come, in each rule. +Suppose the input is +.DS +DING DONG DELL +.DE +It is instructive to follow the steps of the parser while +processing this input. +.PP +Initially, the current state is state 0. +The parser needs to refer to the input in order to decide +between the actions available in state 0, so +the first token, +.I DING , +is read, becoming the lookahead token. +The action in state 0 on +.I DING +is +is ``shift 3'', so state 3 is pushed onto the stack, +and the lookahead token is cleared. +State 3 becomes the current state. +The next token, +.I DONG , +is read, becoming the lookahead token. +The action in state 3 on the token +.I DONG +is ``shift 6'', +so state 6 is pushed onto the stack, and the lookahead is cleared. +The stack now contains 0, 3, and 6. +In state 6, without even consulting the lookahead, +the parser reduces by rule 2. +.DS + sound : DING DONG +.DE +This rule has two symbols on the right hand side, so +two states, 6 and 3, are popped off of the stack, uncovering state 0. +Consulting the description of state 0, looking for a goto on +.I sound , +.DS + sound goto 2 +.DE +is obtained; thus state 2 is pushed onto the stack, +becoming the current state. +.PP +In state 2, the next token, +.I DELL , +must be read. +The action is ``shift 5'', so state 5 is pushed onto the stack, which now has +0, 2, and 5 on it, and the lookahead token is cleared. +In state 5, the only action is to reduce by rule 3. +This has one symbol on the right hand side, so one state, 5, +is popped off, and state 2 is uncovered. +The goto in state 2 on +.I place , +the left side of rule 3, +is state 4. +Now, the stack contains 0, 2, and 4. +In state 4, the only action is to reduce by rule 1. +There are two symbols on the right, so the top two states are popped off, +uncovering state 0 again. +In state 0, there is a goto on +.I rhyme +causing the parser to enter state 1. +In state 1, the input is read; the endmarker is obtained, +indicated by ``$end'' in the +.I y.output +file. +The action in state 1 when the endmarker is seen is to accept, +successfully ending the parse. +.PP +The reader is urged to consider how the parser works +when confronted with such incorrect strings as +.I "DING DONG DONG" , +.I "DING DONG" , +.I "DING DONG DELL DELL" , +etc. +A few minutes spend with this and other simple examples will +probably be repaid when problems arise in more complicated contexts. diff --git a/share/doc/psd/15.yacc/ss5 b/share/doc/psd/15.yacc/ss5 new file mode 100644 index 000000000000..e2c34626f333 --- /dev/null +++ b/share/doc/psd/15.yacc/ss5 @@ -0,0 +1,339 @@ +.\" Copyright (C) Caldera International Inc. 2001-2002. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions are +.\" met: +.\" +.\" Redistributions of source code and documentation must retain the above +.\" copyright notice, this list of conditions and the following +.\" disclaimer. +.\" +.\" Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" +.\" All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" +.\" This product includes software developed or owned by Caldera +.\" International, Inc. Neither the name of Caldera International, Inc. +.\" nor the names of other contributors may be used to endorse or promote +.\" products derived from this software without specific prior written +.\" permission. +.\" +.\" USE OF THE SOFTWARE PROVIDED FOR UNDER THIS LICENSE BY CALDERA +.\" INTERNATIONAL, INC. AND CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR +.\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +.\" WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +.\" DISCLAIMED. IN NO EVENT SHALL CALDERA INTERNATIONAL, INC. BE LIABLE +.\" FOR ANY DIRECT, INDIRECT INCIDENTAL, SPECIAL, EXEMPLARY, OR +.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR +.\" BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, +.\" WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE +.\" OR OTHERWISE) RISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN +.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +.\" +.\" @(#)ss5 8.1 (Berkeley) 6/8/93 +.\" +.\" $FreeBSD$ +.SH +5: Ambiguity and Conflicts +.PP +A set of grammar rules is +.I ambiguous +if there is some input string that can be structured in two or more different ways. +For example, the grammar rule +.DS +expr : expr \'\-\' expr +.DE +is a natural way of expressing the fact that one way of forming an arithmetic expression +is to put two other expressions together with a minus sign between them. +Unfortunately, this grammar rule does not +completely specify the way that all complex inputs +should be structured. +For example, if the input is +.DS +expr \- expr \- expr +.DE +the rule allows this input to be structured as either +.DS +( expr \- expr ) \- expr +.DE +or as +.DS +expr \- ( expr \- expr ) +.DE +(The first is called +.I "left association" , +the second +.I "right association" ). +.PP +Yacc detects such ambiguities when it is attempting to build the parser. +It is instructive to consider the problem that confronts the parser when it is +given an input such as +.DS +expr \- expr \- expr +.DE +When the parser has read the second expr, the input that it has seen: +.DS +expr \- expr +.DE +matches the right side of the grammar rule above. +The parser could +.I reduce +the input by applying this rule; +after applying the rule; +the input is reduced to +.I expr (the +left side of the rule). +The parser would then read the final part of the input: +.DS +\- expr +.DE +and again reduce. +The effect of this is to take the left associative interpretation. +.PP +Alternatively, when the parser has seen +.DS +expr \- expr +.DE +it could defer the immediate application of the rule, and continue reading +the input until it had seen +.DS +expr \- expr \- expr +.DE +It could then apply the rule to the rightmost three symbols, reducing them to +.I expr +and leaving +.DS +expr \- expr +.DE +Now the rule can be reduced once more; the effect is to +take the right associative interpretation. +Thus, having read +.DS +expr \- expr +.DE +the parser can do two legal things, a shift or a reduction, and has no way of +deciding between them. +This is called a +.I "shift / reduce conflict" . +It may also happen that the parser has a choice of two legal reductions; +this is called a +.I "reduce / reduce conflict" . +Note that there are never any ``Shift/shift'' conflicts. +.PP +When there are shift/reduce or reduce/reduce conflicts, Yacc still produces a parser. +It does this by selecting one of the valid steps wherever it has a choice. +A rule describing which choice to make in a given situation is called +a +.I "disambiguating rule" . +.PP +Yacc invokes two disambiguating rules by default: +.IP 1. +In a shift/reduce conflict, the default is to do the shift. +.IP 2. +In a reduce/reduce conflict, the default is to reduce by the +.I earlier +grammar rule (in the input sequence). +.PP +Rule 1 implies that reductions are deferred whenever there is a choice, +in favor of shifts. +Rule 2 gives the user rather crude control over the behavior of the parser +in this situation, but reduce/reduce conflicts should be avoided whenever possible. +.PP +Conflicts may arise because of mistakes in input or logic, or because the grammar rules, while consistent, +require a more complex parser than Yacc can construct. +The use of actions within rules can also cause conflicts, if the action must +be done before the parser can be sure which rule is being recognized. +In these cases, the application of disambiguating rules is inappropriate, +and leads to an incorrect parser. +For this reason, Yacc +always reports the number of shift/reduce and reduce/reduce conflicts resolved by Rule 1 and Rule 2. +.PP +In general, whenever it is possible to apply disambiguating rules to produce a correct parser, it is also +possible to rewrite the grammar rules so that the same inputs are read but there are no +conflicts. +For this reason, most previous parser generators +have considered conflicts to be fatal errors. +Our experience has suggested that this rewriting is somewhat unnatural, +and produces slower parsers; thus, Yacc will produce parsers even in the presence of conflicts. +.PP +As an example of the power of disambiguating rules, consider a fragment from a programming +language involving an ``if-then-else'' construction: +.DS +stat : IF \'(\' cond \')\' stat + | IF \'(\' cond \')\' stat ELSE stat + ; +.DE +In these rules, +.I IF +and +.I ELSE +are tokens, +.I cond +is a nonterminal symbol describing +conditional (logical) expressions, and +.I stat +is a nonterminal symbol describing statements. +The first rule will be called the +.ul +simple-if +rule, and the +second the +.ul +if-else +rule. +.PP +These two rules form an ambiguous construction, since input of the form +.DS +IF ( C1 ) IF ( C2 ) S1 ELSE S2 +.DE +can be structured according to these rules in two ways: +.DS +IF ( C1 ) { + IF ( C2 ) S1 + } +ELSE S2 +.DE +or +.DS +IF ( C1 ) { + IF ( C2 ) S1 + ELSE S2 + } +.DE +The second interpretation is the one given in most programming languages +having this construct. +Each +.I ELSE +is associated with the last preceding +``un-\fIELSE'\fRd'' +.I IF . +In this example, consider the situation where the parser has seen +.DS +IF ( C1 ) IF ( C2 ) S1 +.DE +and is looking at the +.I ELSE . +It can immediately +reduce +by the simple-if rule to get +.DS +IF ( C1 ) stat +.DE +and then read the remaining input, +.DS +ELSE S2 +.DE +and reduce +.DS +IF ( C1 ) stat ELSE S2 +.DE +by the if-else rule. +This leads to the first of the above groupings of the input. +.PP +On the other hand, the +.I ELSE +may be shifted, +.I S2 +read, and then the right hand portion of +.DS +IF ( C1 ) IF ( C2 ) S1 ELSE S2 +.DE +can be reduced by the if-else rule to get +.DS +IF ( C1 ) stat +.DE +which can be reduced by the simple-if rule. +This leads to the second of the above groupings of the input, which +is usually desired. +.PP +Once again the parser can do two valid things \- there is a shift/reduce conflict. +The application of disambiguating rule 1 tells the parser to shift in this case, +which leads to the desired grouping. +.PP +This shift/reduce conflict arises only when there is a particular current input symbol, +.I ELSE , +and particular inputs already seen, such as +.DS +IF ( C1 ) IF ( C2 ) S1 +.DE +In general, there may be many conflicts, and each one +will be associated with an input symbol and +a set of previously read inputs. +The previously read inputs are characterized by the +state +of the parser. +.PP +The conflict messages of Yacc are best understood +by examining the verbose (\fB\-v\fR) option output file. +For example, the output corresponding to the above +conflict state might be: +.DS L +23: shift/reduce conflict (shift 45, reduce 18) on ELSE + +state 23 + + stat : IF ( cond ) stat\_ (18) + stat : IF ( cond ) stat\_ELSE stat + + ELSE shift 45 + . reduce 18 + +.DE +The first line describes the conflict, giving the state and the input symbol. +The ordinary state description follows, giving +the grammar rules active in the state, and the parser actions. +Recall that the underline marks the +portion of the grammar rules which has been seen. +Thus in the example, in state 23 the parser has seen input corresponding +to +.DS +IF ( cond ) stat +.DE +and the two grammar rules shown are active at this time. +The parser can do two possible things. +If the input symbol is +.I ELSE , +it is possible to shift into state +45. +State 45 will have, as part of its description, the line +.DS +stat : IF ( cond ) stat ELSE\_stat +.DE +since the +.I ELSE +will have been shifted in this state. +Back in state 23, the alternative action, described by ``\fB.\fR'', +is to be done if the input symbol is not mentioned explicitly in the above actions; thus, +in this case, if the input symbol is not +.I ELSE , +the parser reduces by grammar rule 18: +.DS +stat : IF \'(\' cond \')\' stat +.DE +Once again, notice that the numbers following ``shift'' commands refer to other states, +while the numbers following ``reduce'' commands refer to grammar +rule numbers. +In the +.I y.output +file, the rule numbers are printed after those rules which can be reduced. +In most one states, there will be at most reduce action possible in the +state, and this will be the default command. +The user who encounters unexpected shift/reduce conflicts will probably want to +look at the verbose output to decide whether the default actions are appropriate. +In really tough cases, the user might need to know more about +the behavior and construction of the parser than can be covered here. +In this case, one of the theoretical references +.[ +Aho Johnson Surveys Parsing +.] +.[ +Aho Johnson Ullman Deterministic Ambiguous +.] +.[ +Aho Ullman Principles Design +.] +might be consulted; the services of a local guru might also be appropriate. diff --git a/share/doc/psd/15.yacc/ss6 b/share/doc/psd/15.yacc/ss6 new file mode 100644 index 000000000000..513d042a8203 --- /dev/null +++ b/share/doc/psd/15.yacc/ss6 @@ -0,0 +1,183 @@ +.\" Copyright (C) Caldera International Inc. 2001-2002. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions are +.\" met: +.\" +.\" Redistributions of source code and documentation must retain the above +.\" copyright notice, this list of conditions and the following +.\" disclaimer. +.\" +.\" Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" +.\" All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" +.\" This product includes software developed or owned by Caldera +.\" International, Inc. Neither the name of Caldera International, Inc. +.\" nor the names of other contributors may be used to endorse or promote +.\" products derived from this software without specific prior written +.\" permission. +.\" +.\" USE OF THE SOFTWARE PROVIDED FOR UNDER THIS LICENSE BY CALDERA +.\" INTERNATIONAL, INC. AND CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR +.\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +.\" WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +.\" DISCLAIMED. IN NO EVENT SHALL CALDERA INTERNATIONAL, INC. BE LIABLE +.\" FOR ANY DIRECT, INDIRECT INCIDENTAL, SPECIAL, EXEMPLARY, OR +.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR +.\" BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, +.\" WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE +.\" OR OTHERWISE) RISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN +.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +.\" +.\" @(#)ss6 8.1 (Berkeley) 6/8/93 +.\" +.\" $FreeBSD$ +.SH +6: Precedence +.PP +There is one common situation +where the rules given above for resolving conflicts are not sufficient; +this is in the parsing of arithmetic expressions. +Most of the commonly used constructions for arithmetic expressions can be naturally +described by the notion of +.I precedence +levels for operators, together with information about left +or right associativity. +It turns out that ambiguous grammars with appropriate disambiguating rules +can be used to create parsers that are faster and easier to +write than parsers constructed from unambiguous grammars. +The basic notion is to write grammar rules +of the form +.DS +expr : expr OP expr +.DE +and +.DS +expr : UNARY expr +.DE +for all binary and unary operators desired. +This creates a very ambiguous grammar, with many parsing conflicts. +As disambiguating rules, the user specifies the precedence, or binding +strength, of all the operators, and the associativity +of the binary operators. +This information is sufficient to allow Yacc to resolve the parsing conflicts +in accordance with these rules, and construct a parser that realizes the desired +precedences and associativities. +.PP +The precedences and associativities are attached to tokens in the declarations section. +This is done by a series of lines beginning with a Yacc keyword: %left, %right, +or %nonassoc, followed by a list of tokens. +All of the tokens on the same line are assumed to have the same precedence level +and associativity; the lines are listed in +order of increasing precedence or binding strength. +Thus, +.DS +%left \'+\' \'\-\' +%left \'*\' \'/\' +.DE +describes the precedence and associativity of the four arithmetic operators. +Plus and minus are left associative, and have lower precedence than +star and slash, which are also left associative. +The keyword %right is used to describe right associative operators, +and the keyword %nonassoc is used to describe operators, like +the operator .LT. in Fortran, that may not associate with themselves; thus, +.DS +A .LT. B .LT. C +.DE +is illegal in Fortran, and such an operator would be described with the keyword +%nonassoc in Yacc. +As an example of the behavior of these declarations, the description +.DS +%right \'=\' +%left \'+\' \'\-\' +%left \'*\' \'/\' + +%% + +expr : expr \'=\' expr + | expr \'+\' expr + | expr \'\-\' expr + | expr \'*\' expr + | expr \'/\' expr + | NAME + ; +.DE +might be used to structure the input +.DS +a = b = c*d \- e \- f*g +.DE +as follows: +.DS +a = ( b = ( ((c*d)\-e) \- (f*g) ) ) +.DE +When this mechanism is used, +unary operators must, in general, be given a precedence. +Sometimes a unary operator and a binary operator +have the same symbolic representation, but different precedences. +An example is unary and binary \'\-\'; unary minus may be given the same +strength as multiplication, or even higher, while binary minus has a lower strength than +multiplication. +The keyword, %prec, changes the precedence level associated with a particular grammar rule. +%prec appears immediately after the body of the grammar rule, before the action or closing semicolon, +and is followed by a token name or literal. +It +causes the precedence of the grammar rule to become that of the following token name or literal. +For example, to make unary minus have the same precedence as multiplication the rules might resemble: +.DS +%left \'+\' \'\-\' +%left \'*\' \'/\' + +%% + +expr : expr \'+\' expr + | expr \'\-\' expr + | expr \'*\' expr + | expr \'/\' expr + | \'\-\' expr %prec \'*\' + | NAME + ; +.DE +.PP +A token declared +by %left, %right, and %nonassoc need not be, but may be, declared by %token as well. +.PP +The precedences and associativities are used by Yacc to +resolve parsing conflicts; they give rise to disambiguating rules. +Formally, the rules work as follows: +.IP 1. +The precedences and associativities are recorded for those tokens and literals +that have them. +.IP 2. +A precedence and associativity is associated with each grammar rule; it is the precedence +and associativity of the last token or literal in the body of the rule. +If the %prec construction is used, it overrides this default. +Some grammar rules may have no precedence and associativity associated with them. +.IP 3. +When there is a reduce/reduce conflict, or there is a shift/reduce conflict +and either the input symbol or the grammar rule has no precedence and associativity, +then the two disambiguating rules given at the beginning of the section are used, +and the conflicts are reported. +.IP 4. +If there is a shift/reduce conflict, and both the grammar rule and the input character +have precedence and associativity associated with them, then the conflict is resolved +in favor of the action (shift or reduce) associated with the higher precedence. +If the precedences are the same, then the associativity is used; left +associative implies reduce, right associative implies shift, and nonassociating +implies error. +.PP +Conflicts resolved by precedence are not counted in the number of shift/reduce and reduce/reduce +conflicts reported by Yacc. +This means that mistakes in the specification of precedences may +disguise errors in the input grammar; it is a good idea to be sparing +with precedences, and use them in an essentially ``cookbook'' fashion, +until some experience has been gained. +The +.I y.output +file +is very useful in deciding whether the parser is actually doing +what was intended. diff --git a/share/doc/psd/15.yacc/ss7 b/share/doc/psd/15.yacc/ss7 new file mode 100644 index 000000000000..b6440c793857 --- /dev/null +++ b/share/doc/psd/15.yacc/ss7 @@ -0,0 +1,161 @@ +.\" Copyright (C) Caldera International Inc. 2001-2002. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions are +.\" met: +.\" +.\" Redistributions of source code and documentation must retain the above +.\" copyright notice, this list of conditions and the following +.\" disclaimer. +.\" +.\" Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" +.\" All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" +.\" This product includes software developed or owned by Caldera +.\" International, Inc. Neither the name of Caldera International, Inc. +.\" nor the names of other contributors may be used to endorse or promote +.\" products derived from this software without specific prior written +.\" permission. +.\" +.\" USE OF THE SOFTWARE PROVIDED FOR UNDER THIS LICENSE BY CALDERA +.\" INTERNATIONAL, INC. AND CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR +.\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +.\" WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +.\" DISCLAIMED. IN NO EVENT SHALL CALDERA INTERNATIONAL, INC. BE LIABLE +.\" FOR ANY DIRECT, INDIRECT INCIDENTAL, SPECIAL, EXEMPLARY, OR +.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR +.\" BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, +.\" WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE +.\" OR OTHERWISE) RISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN +.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +.\" +.\" @(#)ss7 8.1 (Berkeley) 6/8/93 +.\" +.\" $FreeBSD$ +.SH +7: Error Handling +.PP +Error handling is an extremely difficult area, and many of the problems are semantic ones. +When an error is found, for example, it may be necessary to reclaim parse tree storage, +delete or alter symbol table entries, and, typically, set switches to avoid generating any further output. +.PP +It is seldom acceptable to stop all processing when an error is found; it is more useful to continue +scanning the input to find further syntax errors. +This leads to the problem of getting the parser ``restarted'' after an error. +A general class of algorithms to do this involves discarding a number of tokens +from the input string, and attempting to adjust the parser so that input can continue. +.PP +To allow the user some control over this process, +Yacc provides a simple, but reasonably general, feature. +The token name ``error'' is reserved for error handling. +This name can be used in grammar rules; +in effect, it suggests places where errors are expected, and recovery might take place. +The parser pops its stack until it enters a state where the token ``error'' is legal. +It then behaves as if the token ``error'' were the current lookahead token, +and performs the action encountered. +The lookahead token is then reset to the token that caused the error. +If no special error rules have been specified, the processing halts when an error is detected. +.PP +In order to prevent a cascade of error messages, the parser, after +detecting an error, remains in error state until three tokens have been successfully +read and shifted. +If an error is detected when the parser is already in error state, +no message is given, and the input token is quietly deleted. +.PP +As an example, a rule of the form +.DS +stat : error +.DE +would, in effect, mean that on a syntax error the parser would attempt to skip over the statement +in which the error was seen. +More precisely, the parser will +scan ahead, looking for three tokens that might legally follow +a statement, and start processing at the first of these; if +the beginnings of statements are not sufficiently distinctive, it may make a +false start in the middle of a statement, and end up reporting a +second error where there is in fact no error. +.PP +Actions may be used with these special error rules. +These actions might attempt to reinitialize tables, reclaim symbol table space, etc. +.PP +Error rules such as the above are very general, but difficult to control. +Somewhat easier are rules such as +.DS +stat : error \';\' +.DE +Here, when there is an error, the parser attempts to skip over the statement, but +will do so by skipping to the next \';\'. +All tokens after the error and before the next \';\' cannot be shifted, and are discarded. +When the \';\' is seen, this rule will be reduced, and any ``cleanup'' +action associated with it performed. +.PP +Another form of error rule arises in interactive applications, where +it may be desirable to permit a line to be reentered after an error. +A possible error rule might be +.DS +input : error \'\en\' { printf( "Reenter last line: " ); } input + { $$ = $4; } +.DE +There is one potential difficulty with this approach; +the parser must correctly process three input tokens before it +admits that it has correctly resynchronized after the error. +If the reentered line contains an error +in the first two tokens, the parser deletes the offending tokens, +and gives no message; this is clearly unacceptable. +For this reason, there is a mechanism that +can be used to force the parser +to believe that an error has been fully recovered from. +The statement +.DS +yyerrok ; +.DE +in an action +resets the parser to its normal mode. +The last example is better written +.DS +input : error \'\en\' + { yyerrok; + printf( "Reenter last line: " ); } + input + { $$ = $4; } + ; +.DE +.PP +As mentioned above, the token seen immediately +after the ``error'' symbol is the input token at which the +error was discovered. +Sometimes, this is inappropriate; for example, an +error recovery action might +take upon itself the job of finding the correct place to resume input. +In this case, +the previous lookahead token must be cleared. +The statement +.DS +yyclearin ; +.DE +in an action will have this effect. +For example, suppose the action after error +were to call some sophisticated resynchronization routine, +supplied by the user, that attempted to advance the input to the +beginning of the next valid statement. +After this routine was called, the next token returned by yylex would presumably +be the first token in a legal statement; +the old, illegal token must be discarded, and the error state reset. +This could be done by a rule like +.DS +stat : error + { resynch(); + yyerrok ; + yyclearin ; } + ; +.DE +.PP +These mechanisms are admittedly crude, but do allow for a simple, fairly effective recovery of the parser +from many errors; +moreover, the user can get control to deal with +the error actions required by other portions of the program. diff --git a/share/doc/psd/15.yacc/ss8 b/share/doc/psd/15.yacc/ss8 new file mode 100644 index 000000000000..75c66ab383a5 --- /dev/null +++ b/share/doc/psd/15.yacc/ss8 @@ -0,0 +1,130 @@ +.\" Copyright (C) Caldera International Inc. 2001-2002. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions are +.\" met: +.\" +.\" Redistributions of source code and documentation must retain the above +.\" copyright notice, this list of conditions and the following +.\" disclaimer. +.\" +.\" Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" +.\" All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" +.\" This product includes software developed or owned by Caldera +.\" International, Inc. Neither the name of Caldera International, Inc. +.\" nor the names of other contributors may be used to endorse or promote +.\" products derived from this software without specific prior written +.\" permission. +.\" +.\" USE OF THE SOFTWARE PROVIDED FOR UNDER THIS LICENSE BY CALDERA +.\" INTERNATIONAL, INC. AND CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR +.\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +.\" WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +.\" DISCLAIMED. IN NO EVENT SHALL CALDERA INTERNATIONAL, INC. BE LIABLE +.\" FOR ANY DIRECT, INDIRECT INCIDENTAL, SPECIAL, EXEMPLARY, OR +.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR +.\" BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, +.\" WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE +.\" OR OTHERWISE) RISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN +.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +.\" +.\" @(#)ss8 8.1 (Berkeley) 6/8/93 +.\" +.\" $FreeBSD$ +.SH +8: The Yacc Environment +.PP +When the user inputs a specification +to Yacc, the output is a file of C programs, called +.I y.tab.c +on most +systems +(due to local file system conventions, the names may differ from +installation to installation). +The function produced by Yacc is called +.I yyparse \|; +it is an integer valued function. +When it is called, it in turn repeatedly calls +.I yylex , +the lexical analyzer +supplied by the user (see Section 3) +to obtain input tokens. +Eventually, either an error is detected, in which case +(if no error recovery is possible) +.I yyparse +returns the value 1, +or the lexical analyzer returns the endmarker token +and the parser accepts. +In this case, +.I yyparse +returns the value 0. +.PP +The user must provide a certain amount of environment for this +parser in order to obtain a working program. +For example, as with every C program, a program called +.I main +must be defined, that eventually calls +.I yyparse . +In addition, a routine called +.I yyerror +prints a message +when a syntax error is detected. +.PP +These two routines must be supplied in one form or another by the +user. +To ease the initial effort of using Yacc, a library has been +provided with default versions of +.I main +and +.I yyerror . +The name of this library is system dependent; +on many systems the library is accessed by a +.B \-ly +argument to the loader. +To show the triviality of these default programs, the source is +given below: +.DS +main(){ + return( yyparse() ); + } +.DE +and +.DS +# include <stdio.h> + +yyerror(s) char *s; { + fprintf( stderr, "%s\en", s ); + } +.DE +The argument to +.I yyerror +is a string containing an error message, usually +the string ``syntax error''. +The average application will want to do better than this. +Ordinarily, the program should keep track of the input line number, and print it +along with the message when a syntax error is detected. +The external integer variable +.I yychar +contains the lookahead token number at the time the error was detected; +this may be of some interest in giving better diagnostics. +Since the +.I main +program is probably supplied by the user (to read arguments, etc.) +the Yacc library is useful only in small +projects, or in the earliest stages of larger ones. +.PP +The external integer variable +.I yydebug +is normally set to 0. +If it is set to a nonzero value, the parser will output a +verbose description of its actions, including +a discussion of which input symbols have been read, and +what the parser actions are. +Depending on the operating environment, +it may be possible to set this variable by using a debugging system. diff --git a/share/doc/psd/15.yacc/ss9 b/share/doc/psd/15.yacc/ss9 new file mode 100644 index 000000000000..9d05fecd4abd --- /dev/null +++ b/share/doc/psd/15.yacc/ss9 @@ -0,0 +1,206 @@ +.\" Copyright (C) Caldera International Inc. 2001-2002. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions are +.\" met: +.\" +.\" Redistributions of source code and documentation must retain the above +.\" copyright notice, this list of conditions and the following +.\" disclaimer. +.\" +.\" Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" +.\" All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" +.\" This product includes software developed or owned by Caldera +.\" International, Inc. Neither the name of Caldera International, Inc. +.\" nor the names of other contributors may be used to endorse or promote +.\" products derived from this software without specific prior written +.\" permission. +.\" +.\" USE OF THE SOFTWARE PROVIDED FOR UNDER THIS LICENSE BY CALDERA +.\" INTERNATIONAL, INC. AND CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR +.\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +.\" WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +.\" DISCLAIMED. IN NO EVENT SHALL CALDERA INTERNATIONAL, INC. BE LIABLE +.\" FOR ANY DIRECT, INDIRECT INCIDENTAL, SPECIAL, EXEMPLARY, OR +.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR +.\" BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, +.\" WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE +.\" OR OTHERWISE) RISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN +.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +.\" +.\" @(#)ss9 8.1 (Berkeley) 6/8/93 +.\" +.\" $FreeBSD$ +.SH +9: Hints for Preparing Specifications +.PP +This section contains miscellaneous hints on preparing efficient, easy to change, +and clear specifications. +The individual subsections are more or less +independent. +.SH +Input Style +.PP +It is difficult to +provide rules with substantial actions +and still have a readable specification file. +The following style hints owe much to Brian Kernighan. +.IP a. +Use all capital letters for token names, all lower case letters for +nonterminal names. +This rule comes under the heading of ``knowing who to blame when +things go wrong.'' +.IP b. +Put grammar rules and actions on separate lines. +This allows either to be changed without +an automatic need to change the other. +.IP c. +Put all rules with the same left hand side together. +Put the left hand side in only once, and let all +following rules begin with a vertical bar. +.IP d. +Put a semicolon only after the last rule with a given left hand side, +and put the semicolon on a separate line. +This allows new rules to be easily added. +.IP e. +Indent rule bodies by two tab stops, and action bodies by three +tab stops. +.PP +The example in Appendix A is written following this style, as are +the examples in the text of this paper (where space permits). +The user must make up his own mind about these stylistic questions; +the central problem, however, is to make the rules visible through +the morass of action code. +.SH +Left Recursion +.PP +The algorithm used by the Yacc parser encourages so called ``left recursive'' +grammar rules: rules of the form +.DS +name : name rest_of_rule ; +.DE +These rules frequently arise when +writing specifications of sequences and lists: +.DS +list : item + | list \',\' item + ; +.DE +and +.DS +seq : item + | seq item + ; +.DE +In each of these cases, the first rule +will be reduced for the first item only, and the second rule +will be reduced for the second and all succeeding items. +.PP +With right recursive rules, such as +.DS +seq : item + | item seq + ; +.DE +the parser would be a bit bigger, and the items would be seen, and reduced, +from right to left. +More seriously, an internal stack in the parser +would be in danger of overflowing if a very long sequence were read. +Thus, the user should use left recursion wherever reasonable. +.PP +It is worth considering whether a sequence with zero +elements has any meaning, and if so, consider writing +the sequence specification with an empty rule: +.DS +seq : /* empty */ + | seq item + ; +.DE +Once again, the first rule would always be reduced exactly once, before the +first item was read, +and then the second rule would be reduced once for each item read. +Permitting empty sequences +often leads to increased generality. +However, conflicts might arise if Yacc is asked to decide +which empty sequence it has seen, when it hasn't seen enough to +know! +.SH +Lexical Tie-ins +.PP +Some lexical decisions depend on context. +For example, the lexical analyzer might want to +delete blanks normally, but not within quoted strings. +Or names might be entered into a symbol table in declarations, +but not in expressions. +.PP +One way of handling this situation is +to create a global flag that is +examined by the lexical analyzer, and set by actions. +For example, suppose a program +consists of 0 or more declarations, followed by 0 or more statements. +Consider: +.DS +%{ + int dflag; +%} + ... other declarations ... + +%% + +prog : decls stats + ; + +decls : /* empty */ + { dflag = 1; } + | decls declaration + ; + +stats : /* empty */ + { dflag = 0; } + | stats statement + ; + + ... other rules ... +.DE +The flag +.I dflag +is now 0 when reading statements, and 1 when reading declarations, +.ul +except for the first token in the first statement. +This token must be seen by the parser before it can tell that +the declaration section has ended and the statements have +begun. +In many cases, this single token exception does not +affect the lexical scan. +.PP +This kind of ``backdoor'' approach can be elaborated +to a noxious degree. +Nevertheless, it represents a way of doing some things +that are difficult, if not impossible, to +do otherwise. +.SH +Reserved Words +.PP +Some programming languages +permit the user to +use words like ``if'', which are normally reserved, +as label or variable names, provided that such use does not +conflict with the legal use of these names in the programming language. +This is extremely hard to do in the framework of Yacc; +it is difficult to pass information to the lexical analyzer +telling it ``this instance of `if' is a keyword, and that instance is a variable''. +The user can make a stab at it, using the +mechanism described in the last subsection, +but it is difficult. +.PP +A number of ways of making this easier are under advisement. +Until then, it is better that the keywords be +.I reserved \|; +that is, be forbidden for use as variable names. +There are powerful stylistic reasons for preferring this, anyway. diff --git a/share/doc/psd/15.yacc/ss_ b/share/doc/psd/15.yacc/ss_ new file mode 100644 index 000000000000..7dd9ea84cdf9 --- /dev/null +++ b/share/doc/psd/15.yacc/ss_ @@ -0,0 +1,94 @@ +.\" Copyright (C) Caldera International Inc. 2001-2002. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions are +.\" met: +.\" +.\" Redistributions of source code and documentation must retain the above +.\" copyright notice, this list of conditions and the following +.\" disclaimer. +.\" +.\" Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" +.\" All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" +.\" This product includes software developed or owned by Caldera +.\" International, Inc. Neither the name of Caldera International, Inc. +.\" nor the names of other contributors may be used to endorse or promote +.\" products derived from this software without specific prior written +.\" permission. +.\" +.\" USE OF THE SOFTWARE PROVIDED FOR UNDER THIS LICENSE BY CALDERA +.\" INTERNATIONAL, INC. AND CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR +.\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +.\" WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +.\" DISCLAIMED. IN NO EVENT SHALL CALDERA INTERNATIONAL, INC. BE LIABLE +.\" FOR ANY DIRECT, INDIRECT INCIDENTAL, SPECIAL, EXEMPLARY, OR +.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR +.\" BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, +.\" WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE +.\" OR OTHERWISE) RISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN +.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +.\" +.\" @(#)ss.. 8.1 (Berkeley) 6/8/93 +.\" +.\" $FreeBSD$ +.EH 'PSD:15-%''Yacc: Yet Another Compiler-Compiler' +.OH 'Yacc: Yet Another Compiler-Compiler''PSD:15-%' +.\".RP +.ND "July 31, 1978" +.TL +Yacc: +Yet Another Compiler-Compiler +.AU "MH 2C-559" 3968 +Stephen C. Johnson +AT&T Bell Laboratories +Murray Hill, New Jersey 07974 +.AI +.AB +.PP +Computer program input generally has some structure; +in fact, every computer program that does input can be thought of as defining +an ``input language'' which it accepts. +An input language may be as complex as a programming language, or as simple as +a sequence of numbers. +Unfortunately, usual input facilities +are limited, difficult to use, +and often are lax about checking their inputs for validity. +.PP +Yacc provides a general tool for describing +the input to a computer program. +The Yacc user specifies the structures +of his input, together with code to be invoked as +each such structure is recognized. +Yacc turns such a specification into a subroutine that +handles the input process; +frequently, it is convenient and appropriate to have most +of the flow of control in the user's application +handled by this subroutine. +.PP +The input subroutine produced by Yacc calls a user-supplied routine to +return the next basic input item. +Thus, the user can specify his input in terms of individual input characters, or +in terms of higher level constructs such as names and numbers. +The user-supplied routine may also handle idiomatic features such as +comment and continuation conventions, which typically defy easy grammatical specification. +.PP +Yacc is written in portable C. +The class of specifications accepted is a very general one: LALR(1) +grammars with disambiguating rules. +.PP +In addition to compilers for C, APL, Pascal, RATFOR, etc., Yacc +has also been used for less conventional languages, +including a phototypesetter language, several desk calculator languages, a document retrieval system, +and a Fortran debugging system. +.AE +.\" .OK +.\"Computer Languages +.\"Compilers +.\"Formal Language Theory +.\" .CS 23 11 34 0 0 8 diff --git a/share/doc/psd/15.yacc/ssa b/share/doc/psd/15.yacc/ssa new file mode 100644 index 000000000000..17e815edfcb1 --- /dev/null +++ b/share/doc/psd/15.yacc/ssa @@ -0,0 +1,150 @@ +.\" Copyright (C) Caldera International Inc. 2001-2002. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions are +.\" met: +.\" +.\" Redistributions of source code and documentation must retain the above +.\" copyright notice, this list of conditions and the following +.\" disclaimer. +.\" +.\" Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" +.\" All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" +.\" This product includes software developed or owned by Caldera +.\" International, Inc. Neither the name of Caldera International, Inc. +.\" nor the names of other contributors may be used to endorse or promote +.\" products derived from this software without specific prior written +.\" permission. +.\" +.\" USE OF THE SOFTWARE PROVIDED FOR UNDER THIS LICENSE BY CALDERA +.\" INTERNATIONAL, INC. AND CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR +.\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +.\" WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +.\" DISCLAIMED. IN NO EVENT SHALL CALDERA INTERNATIONAL, INC. BE LIABLE +.\" FOR ANY DIRECT, INDIRECT INCIDENTAL, SPECIAL, EXEMPLARY, OR +.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR +.\" BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, +.\" WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE +.\" OR OTHERWISE) RISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN +.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +.\" +.\" @(#)ssa 8.1 (Berkeley) 6/8/93 +.\" +.\" $FreeBSD$ +.SH +Appendix A: A Simple Example +.PP +This example gives the complete Yacc specification for a small desk calculator; +the desk calculator has 26 registers, labeled ``a'' through ``z'', and accepts +arithmetic expressions made up of the operators +, \-, *, /, +% (mod operator), & (bitwise and), | (bitwise or), and assignment. +If an expression at the top level is an assignment, the value is not +printed; otherwise it is. +As in C, an integer that begins with 0 (zero) is assumed to be octal; +otherwise, it is assumed to be decimal. +.PP +As an example of a Yacc specification, the desk calculator +does a reasonable job of showing how precedences and ambiguities +are used, and demonstrating simple error recovery. +The major oversimplifications are that the +lexical analysis phase is much simpler than for most applications, and the +output is produced immediately, line by line. +Note the way that decimal and octal integers are read in by the grammar rules; +This job is probably better done by the lexical analyzer. +.sp +.nf +.ta .5i 1i 1.5i 2i 2.5i + +%{ +# include <stdio.h> +# include <ctype.h> + +int regs[26]; +int base; + +%} + +%start list + +%token DIGIT LETTER + +%left \'|\' +%left \'&\' +%left \'+\' \'\-\' +%left \'*\' \'/\' \'%\' +%left UMINUS /* supplies precedence for unary minus */ + +%% /* beginning of rules section */ + +list : /* empty */ + | list stat \'\en\' + | list error \'\en\' + { yyerrok; } + ; + +stat : expr + { printf( "%d\en", $1 ); } + | LETTER \'=\' expr + { regs[$1] = $3; } + ; + +expr : \'(\' expr \')\' + { $$ = $2; } + | expr \'+\' expr + { $$ = $1 + $3; } + | expr \'\-\' expr + { $$ = $1 \- $3; } + | expr \'*\' expr + { $$ = $1 * $3; } + | expr \'/\' expr + { $$ = $1 / $3; } + | expr \'%\' expr + { $$ = $1 % $3; } + | expr \'&\' expr + { $$ = $1 & $3; } + | expr \'|\' expr + { $$ = $1 | $3; } + | \'\-\' expr %prec UMINUS + { $$ = \- $2; } + | LETTER + { $$ = regs[$1]; } + | number + ; + +number : DIGIT + { $$ = $1; base = ($1==0) ? 8 : 10; } + | number DIGIT + { $$ = base * $1 + $2; } + ; + +%% /* start of programs */ + +yylex() { /* lexical analysis routine */ + /* returns LETTER for a lower case letter, yylval = 0 through 25 */ + /* return DIGIT for a digit, yylval = 0 through 9 */ + /* all other characters are returned immediately */ + + int c; + + while( (c=getchar()) == \' \' ) { /* skip blanks */ } + + /* c is now nonblank */ + + if( islower( c ) ) { + yylval = c \- \'a\'; + return ( LETTER ); + } + if( isdigit( c ) ) { + yylval = c \- \'0\'; + return( DIGIT ); + } + return( c ); + } +.fi +.bp diff --git a/share/doc/psd/15.yacc/ssb b/share/doc/psd/15.yacc/ssb new file mode 100644 index 000000000000..2dba020cd015 --- /dev/null +++ b/share/doc/psd/15.yacc/ssb @@ -0,0 +1,147 @@ +.\" Copyright (C) Caldera International Inc. 2001-2002. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions are +.\" met: +.\" +.\" Redistributions of source code and documentation must retain the above +.\" copyright notice, this list of conditions and the following +.\" disclaimer. +.\" +.\" Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" +.\" All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" +.\" This product includes software developed or owned by Caldera +.\" International, Inc. Neither the name of Caldera International, Inc. +.\" nor the names of other contributors may be used to endorse or promote +.\" products derived from this software without specific prior written +.\" permission. +.\" +.\" USE OF THE SOFTWARE PROVIDED FOR UNDER THIS LICENSE BY CALDERA +.\" INTERNATIONAL, INC. AND CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR +.\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +.\" WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +.\" DISCLAIMED. IN NO EVENT SHALL CALDERA INTERNATIONAL, INC. BE LIABLE +.\" FOR ANY DIRECT, INDIRECT INCIDENTAL, SPECIAL, EXEMPLARY, OR +.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR +.\" BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, +.\" WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE +.\" OR OTHERWISE) RISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN +.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +.\" +.\" @(#)ssb 8.1 (Berkeley) 6/8/93 +.\" +.\" $FreeBSD$ +.SH +Appendix B: Yacc Input Syntax +.PP +This Appendix has a description of the Yacc input syntax, as a Yacc specification. +Context dependencies, etc., are not considered. +Ironically, the Yacc input specification language +is most naturally specified as an LR(2) grammar; the sticky +part comes when an identifier is seen in a rule, immediately +following an action. +If this identifier is followed by a colon, it is the start of the +next rule; otherwise +it is a continuation of the current rule, which just happens to have +an action embedded in it. +As implemented, the lexical analyzer looks +ahead after seeing an identifier, and +decide whether the next token (skipping blanks, newlines, comments, etc.) +is a colon. +If so, it returns the token C_IDENTIFIER. +Otherwise, it returns IDENTIFIER. +Literals (quoted strings) are also returned as IDENTIFIERS, +but never as part of C_IDENTIFIERs. +.sp +.nf +.ta .6i 1.2i 1.8i 2.4i 3i 3.6i + + /* grammar for the input to Yacc */ + + /* basic entities */ +%token IDENTIFIER /* includes identifiers and literals */ +%token C_IDENTIFIER /* identifier (but not literal) followed by colon */ +%token NUMBER /* [0-9]+ */ + + /* reserved words: %type => TYPE, %left => LEFT, etc. */ + +%token LEFT RIGHT NONASSOC TOKEN PREC TYPE START UNION + +%token MARK /* the %% mark */ +%token LCURL /* the %{ mark */ +%token RCURL /* the %} mark */ + + /* ascii character literals stand for themselves */ + +%start spec + +%% + +spec : defs MARK rules tail + ; + +tail : MARK { \fIIn this action, eat up the rest of the file\fR } + | /* empty: the second MARK is optional */ + ; + +defs : /* empty */ + | defs def + ; + +def : START IDENTIFIER + | UNION { \fICopy union definition to output\fR } + | LCURL { \fICopy C code to output file\fR } RCURL + | ndefs rword tag nlist + ; + +rword : TOKEN + | LEFT + | RIGHT + | NONASSOC + | TYPE + ; + +tag : /* empty: union tag is optional */ + | \'<\' IDENTIFIER \'>\' + ; + +nlist : nmno + | nlist nmno + | nlist \',\' nmno + ; + +nmno : IDENTIFIER /* NOTE: literal illegal with %type */ + | IDENTIFIER NUMBER /* NOTE: illegal with %type */ + ; + + /* rules section */ + +rules : C_IDENTIFIER rbody prec + | rules rule + ; + +rule : C_IDENTIFIER rbody prec + | '|' rbody prec + ; + +rbody : /* empty */ + | rbody IDENTIFIER + | rbody act + ; + +act : \'{\' { \fICopy action, translate $$, etc.\fR } \'}\' + ; + +prec : /* empty */ + | PREC IDENTIFIER + | PREC IDENTIFIER act + | prec \';\' + ; +.fi +.bp diff --git a/share/doc/psd/15.yacc/ssc b/share/doc/psd/15.yacc/ssc new file mode 100644 index 000000000000..95fca5c516b0 --- /dev/null +++ b/share/doc/psd/15.yacc/ssc @@ -0,0 +1,347 @@ +.\" Copyright (C) Caldera International Inc. 2001-2002. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions are +.\" met: +.\" +.\" Redistributions of source code and documentation must retain the above +.\" copyright notice, this list of conditions and the following +.\" disclaimer. +.\" +.\" Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" +.\" All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" +.\" This product includes software developed or owned by Caldera +.\" International, Inc. Neither the name of Caldera International, Inc. +.\" nor the names of other contributors may be used to endorse or promote +.\" products derived from this software without specific prior written +.\" permission. +.\" +.\" USE OF THE SOFTWARE PROVIDED FOR UNDER THIS LICENSE BY CALDERA +.\" INTERNATIONAL, INC. AND CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR +.\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +.\" WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +.\" DISCLAIMED. IN NO EVENT SHALL CALDERA INTERNATIONAL, INC. BE LIABLE +.\" FOR ANY DIRECT, INDIRECT INCIDENTAL, SPECIAL, EXEMPLARY, OR +.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR +.\" BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, +.\" WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE +.\" OR OTHERWISE) RISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN +.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +.\" @(#)ssc 8.1 (Berkeley) 8/14/93 +.\" +.\" $FreeBSD$ +.SH +Appendix C: An Advanced Example +.PP +This Appendix gives an example of a grammar using some +of the advanced features discussed in Section 10. +The desk calculator example in Appendix A is +modified to provide a desk calculator that +does floating point interval arithmetic. +The calculator understands floating point +constants, the arithmetic operations +, \-, *, /, +unary \-, and = (assignment), and has 26 floating +point variables, ``a'' through ``z''. +Moreover, it also understands +.I intervals , +written +.DS + ( x , y ) +.DE +where +.I x +is less than or equal to +.I y . +There are 26 interval valued variables ``A'' through ``Z'' +that may also be used. +The usage is similar to that in Appendix A; assignments +return no value, and print nothing, while expressions print +the (floating or interval) value. +.PP +This example explores a number of interesting features +of Yacc and C. +Intervals are represented by a structure, consisting of the +left and right endpoint values, stored as +.I double 's. +This structure is given a type name, INTERVAL, by using +.I typedef . +The Yacc value stack can also contain floating point scalars, and +integers (used to index into the arrays holding the variable values). +Notice that this entire strategy depends strongly on being able to +assign structures and unions in C. +In fact, many of the actions call functions that return structures +as well. +.PP +It is also worth noting the use of YYERROR to handle error conditions: +division by an interval containing 0, and an interval presented in +the wrong order. +In effect, the error recovery mechanism of Yacc is used to throw away the +rest of the offending line. +.PP +In addition to the mixing of types on the value stack, +this grammar also demonstrates an interesting use of syntax to +keep track of the type (e.g. scalar or interval) of intermediate +expressions. +Note that a scalar can be automatically promoted to an interval if +the context demands an interval value. +This causes a large number of conflicts when the grammar is run through +Yacc: 18 Shift/Reduce and 26 Reduce/Reduce. +The problem can be seen by looking at the two input lines: +.DS + 2.5 + ( 3.5 \- 4. ) +.DE +and +.DS + 2.5 + ( 3.5 , 4. ) +.DE +Notice that the 2.5 is to be used in an interval valued expression +in the second example, but this fact is not known until +the ``,'' is read; by this time, 2.5 is finished, and the parser cannot go back +and change its mind. +More generally, it might be necessary to look ahead an arbitrary number of +tokens to decide whether to convert a scalar to an interval. +This problem is evaded by having two rules for each binary interval +valued operator: one when the left operand is a scalar, and one when +the left operand is an interval. +In the second case, the right operand must be an interval, +so the conversion will be applied automatically. +Despite this evasion, there are still many cases where the +conversion may be applied or not, leading to the above +conflicts. +They are resolved by listing the rules that yield scalars first +in the specification file; in this way, the conflicts will +be resolved in the direction of keeping scalar +valued expressions scalar valued until they are forced to become +intervals. +.PP +This way of handling multiple types is very instructive, but not very general. +If there were many kinds of expression types, instead of just two, +the number of rules needed would increase dramatically, and the conflicts +even more dramatically. +Thus, while this example is instructive, it is better practice in a +more normal programming language environment to +keep the type information as part of the value, and not as part +of the grammar. +.PP +Finally, a word about the lexical analysis. +The only unusual feature is the treatment of floating point constants. +The C library routine +.I atof +is used to do the actual conversion from a character string +to a double precision value. +If the lexical analyzer detects an error, +it responds by returning a token that +is illegal in the grammar, provoking a syntax error +in the parser, and thence error recovery. +.LD + +%{ + +# include <stdio.h> +# include <ctype.h> + +typedef struct interval { + double lo, hi; + } INTERVAL; + +INTERVAL vmul(), vdiv(); + +double atof(); + +double dreg[ 26 ]; +INTERVAL vreg[ 26 ]; + +%} + +%start lines + +%union { + int ival; + double dval; + INTERVAL vval; + } + +%token <ival> DREG VREG /* indices into dreg, vreg arrays */ + +%token <dval> CONST /* floating point constant */ + +%type <dval> dexp /* expression */ + +%type <vval> vexp /* interval expression */ + + /* precedence information about the operators */ + +%left \'+\' \'\-\' +%left \'*\' \'/\' +%left UMINUS /* precedence for unary minus */ + +%% + +lines : /* empty */ + | lines line + ; + +line : dexp \'\en\' + { printf( "%15.8f\en", $1 ); } + | vexp \'\en\' + { printf( "(%15.8f , %15.8f )\en", $1.lo, $1.hi ); } + | DREG \'=\' dexp \'\en\' + { dreg[$1] = $3; } + | VREG \'=\' vexp \'\en\' + { vreg[$1] = $3; } + | error \'\en\' + { yyerrok; } + ; + +dexp : CONST + | DREG + { $$ = dreg[$1]; } + | dexp \'+\' dexp + { $$ = $1 + $3; } + | dexp \'\-\' dexp + { $$ = $1 \- $3; } + | dexp \'*\' dexp + { $$ = $1 * $3; } + | dexp \'/\' dexp + { $$ = $1 / $3; } + | \'\-\' dexp %prec UMINUS + { $$ = \- $2; } + | \'(\' dexp \')\' + { $$ = $2; } + ; + +vexp : dexp + { $$.hi = $$.lo = $1; } + | \'(\' dexp \',\' dexp \')\' + { + $$.lo = $2; + $$.hi = $4; + if( $$.lo > $$.hi ){ + printf( "interval out of order\en" ); + YYERROR; + } + } + | VREG + { $$ = vreg[$1]; } + | vexp \'+\' vexp + { $$.hi = $1.hi + $3.hi; + $$.lo = $1.lo + $3.lo; } + | dexp \'+\' vexp + { $$.hi = $1 + $3.hi; + $$.lo = $1 + $3.lo; } + | vexp \'\-\' vexp + { $$.hi = $1.hi \- $3.lo; + $$.lo = $1.lo \- $3.hi; } + | dexp \'\-\' vexp + { $$.hi = $1 \- $3.lo; + $$.lo = $1 \- $3.hi; } + | vexp \'*\' vexp + { $$ = vmul( $1.lo, $1.hi, $3 ); } + | dexp \'*\' vexp + { $$ = vmul( $1, $1, $3 ); } + | vexp \'/\' vexp + { if( dcheck( $3 ) ) YYERROR; + $$ = vdiv( $1.lo, $1.hi, $3 ); } + | dexp \'/\' vexp + { if( dcheck( $3 ) ) YYERROR; + $$ = vdiv( $1, $1, $3 ); } + | \'\-\' vexp %prec UMINUS + { $$.hi = \-$2.lo; $$.lo = \-$2.hi; } + | \'(\' vexp \')\' + { $$ = $2; } + ; + +%% + +# define BSZ 50 /* buffer size for floating point numbers */ + + /* lexical analysis */ + +yylex(){ + register c; + + while( (c=getchar()) == \' \' ){ /* skip over blanks */ } + + if( isupper( c ) ){ + yylval.ival = c \- \'A\'; + return( VREG ); + } + if( islower( c ) ){ + yylval.ival = c \- \'a\'; + return( DREG ); + } + + if( isdigit( c ) || c==\'.\' ){ + /* gobble up digits, points, exponents */ + + char buf[BSZ+1], *cp = buf; + int dot = 0, exp = 0; + + for( ; (cp\-buf)<BSZ ; ++cp,c=getchar() ){ + + *cp = c; + if( isdigit( c ) ) continue; + if( c == \'.\' ){ + if( dot++ || exp ) return( \'.\' ); /* will cause syntax error */ + continue; + } + + if( c == \'e\' ){ + if( exp++ ) return( \'e\' ); /* will cause syntax error */ + continue; + } + + /* end of number */ + break; + } + *cp = \'\e0\'; + if( (cp\-buf) >= BSZ ) printf( "constant too long: truncated\en" ); + else ungetc( c, stdin ); /* push back last char read */ + yylval.dval = atof( buf ); + return( CONST ); + } + return( c ); + } + +INTERVAL hilo( a, b, c, d ) double a, b, c, d; { + /* returns the smallest interval containing a, b, c, and d */ + /* used by *, / routines */ + INTERVAL v; + + if( a>b ) { v.hi = a; v.lo = b; } + else { v.hi = b; v.lo = a; } + + if( c>d ) { + if( c>v.hi ) v.hi = c; + if( d<v.lo ) v.lo = d; + } + else { + if( d>v.hi ) v.hi = d; + if( c<v.lo ) v.lo = c; + } + return( v ); + } + +INTERVAL vmul( a, b, v ) double a, b; INTERVAL v; { + return( hilo( a*v.hi, a*v.lo, b*v.hi, b*v.lo ) ); + } + +dcheck( v ) INTERVAL v; { + if( v.hi >= 0. && v.lo <= 0. ){ + printf( "divisor interval contains 0.\en" ); + return( 1 ); + } + return( 0 ); + } + +INTERVAL vdiv( a, b, v ) double a, b; INTERVAL v; { + return( hilo( a/v.hi, a/v.lo, b/v.hi, b/v.lo ) ); + } +.DE +.bp diff --git a/share/doc/psd/15.yacc/ssd b/share/doc/psd/15.yacc/ssd new file mode 100644 index 000000000000..988e0a0da09e --- /dev/null +++ b/share/doc/psd/15.yacc/ssd @@ -0,0 +1,76 @@ +.\" Copyright (C) Caldera International Inc. 2001-2002. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions are +.\" met: +.\" +.\" Redistributions of source code and documentation must retain the above +.\" copyright notice, this list of conditions and the following +.\" disclaimer. +.\" +.\" Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" +.\" All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" +.\" This product includes software developed or owned by Caldera +.\" International, Inc. Neither the name of Caldera International, Inc. +.\" nor the names of other contributors may be used to endorse or promote +.\" products derived from this software without specific prior written +.\" permission. +.\" +.\" USE OF THE SOFTWARE PROVIDED FOR UNDER THIS LICENSE BY CALDERA +.\" INTERNATIONAL, INC. AND CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR +.\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +.\" WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +.\" DISCLAIMED. IN NO EVENT SHALL CALDERA INTERNATIONAL, INC. BE LIABLE +.\" FOR ANY DIRECT, INDIRECT INCIDENTAL, SPECIAL, EXEMPLARY, OR +.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR +.\" BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, +.\" WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE +.\" OR OTHERWISE) RISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN +.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +.\" @(#)ssd 8.1 (Berkeley) 6/8/93 +.\" +.\" $FreeBSD$ +.SH +Appendix D: Old Features Supported but not Encouraged +.PP +This Appendix mentions synonyms and features which are supported for historical +continuity, but, for various reasons, are not encouraged. +.IP 1. +Literals may also be delimited by double quotes ``"''. +.IP 2. +Literals may be more than one character long. +If all the characters are alphabetic, numeric, or \_, the type number of the literal is defined, +just as if the literal did not have the quotes around it. +Otherwise, it is difficult to find the value for such literals. +.IP +The use of multi-character literals is likely to mislead those unfamiliar with +Yacc, since it suggests that Yacc is doing a job which must be actually done by the lexical analyzer. +.IP 3. +Most places where % is legal, backslash ``\e'' may be used. +In particular, \e\e is the same as %%, \eleft the same as %left, etc. +.IP 4. +There are a number of other synonyms: +.DS +%< is the same as %left +%> is the same as %right +%binary and %2 are the same as %nonassoc +%0 and %term are the same as %token +%= is the same as %prec +.DE +.IP 5. +Actions may also have the form +.DS +={ . . . } +.DE +and the curly braces can be dropped if the action is a +single C statement. +.IP 6. +C code between %{ and %} used to be permitted at the +head of the rules section, as well as in the +declaration section. diff --git a/share/doc/psd/16.lex/Makefile b/share/doc/psd/16.lex/Makefile new file mode 100644 index 000000000000..6dea7c028991 --- /dev/null +++ b/share/doc/psd/16.lex/Makefile @@ -0,0 +1,9 @@ +# @(#)Makefile 8.1 (Berkeley) 6/8/93 +# $FreeBSD$ + +VOLUME= psd/16.lex +SRCS= lex.ms +MACROS= -ms +USE_TBL= + +.include <bsd.doc.mk> diff --git a/share/doc/psd/16.lex/lex.ms b/share/doc/psd/16.lex/lex.ms new file mode 100644 index 000000000000..8b3c82ee23b3 --- /dev/null +++ b/share/doc/psd/16.lex/lex.ms @@ -0,0 +1,2345 @@ +.\" Copyright (C) Caldera International Inc. 2001-2002. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions are +.\" met: +.\" +.\" Redistributions of source code and documentation must retain the above +.\" copyright notice, this list of conditions and the following +.\" disclaimer. +.\" +.\" Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" +.\" All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" +.\" This product includes software developed or owned by Caldera +.\" International, Inc. Neither the name of Caldera International, Inc. +.\" nor the names of other contributors may be used to endorse or promote +.\" products derived from this software without specific prior written +.\" permission. +.\" +.\" USE OF THE SOFTWARE PROVIDED FOR UNDER THIS LICENSE BY CALDERA +.\" INTERNATIONAL, INC. AND CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR +.\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +.\" WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +.\" DISCLAIMED. IN NO EVENT SHALL CALDERA INTERNATIONAL, INC. BE LIABLE +.\" FOR ANY DIRECT, INDIRECT INCIDENTAL, SPECIAL, EXEMPLARY, OR +.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR +.\" BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, +.\" WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE +.\" OR OTHERWISE) RISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN +.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +.\" +.\" @(#)lex.ms 8.1 (Berkeley) 6/8/93 +.\" +.\" $FreeBSD$ +.nr tm 0 +.de MH +Bell Laboratories, Murray Hill, NJ 07974. +.. +.EH 'PSD:16-%''Lex \- A Lexical Analyzer Generator' +.OH 'Lex \- A Lexical Analyzer Generator''PSD:16-%' +.hc ~ +.bd I 2 +.de TS +.br +.nf +.sp 1v +.ul 0 +.. +.de TE +.sp 1v +.fi +.. +.\".de PT +.\".if \\n%>1 'tl ''\s7LEX\s0\s9\(mi%\s0'' +.\".if \\n%>1 'sp +.\".. +.ND July 21, 1975 +.\".RP +.\".TM 75-1274-15 39199 39199-11 +.TL +Lex \- A Lexical Analyzer ~Generator~ +.AU ``MH 2C-569'' 6377 +M. E. Lesk and E. Schmidt +.AI +.MH +.AB +.ps +1 +NOTE: This document describes the historical Unix version of \fIlex\fP. +FreeBSD is supplied with \fIflex\fP\| which is a compatible replacement. +See the extensive documentation in \fIflex(1)\fP\| for details. +.ps +.sp +.bd I 2 +.\".nr PS 8 +.\".nr VS 9 +.\".ps 8 +.\".vs 9p +Lex helps write programs whose control flow +is directed by instances of regular +expressions in the input stream. +It is well suited for editor-script type transformations and +for segmenting input in preparation for +a parsing routine. +.PP +Lex source is a table of regular expressions and corresponding program fragments. +The table is translated to a program +which reads an input stream, copying it to an output stream +and partitioning the input +into strings which match the given expressions. +As each such string is recognized the corresponding +program fragment is executed. +The recognition of the expressions +is performed by a deterministic finite automaton +generated by Lex. +The program fragments written by the user are executed in the order in which the +corresponding regular expressions occur in the input stream. +.PP +The lexical analysis +programs written with Lex accept ambiguous specifications +and choose the longest +match possible at each input point. +If necessary, substantial look~ahead +is performed on the input, but the +input stream will be backed up to the +end of the current partition, so that the user +has general freedom to manipulate it. +.PP +Lex can generate analyzers in either C or Ratfor, a language +which can be translated automatically to portable Fortran. +It is available on the PDP-11 UNIX, Honeywell GCOS, +and IBM OS systems. +This manual, however, will only discuss generating analyzers +in C on the UNIX system, which is the only supported +form of Lex under UNIX Version 7. +Lex is designed to simplify +interfacing with Yacc, for those +with access to this compiler-compiler system. +.\".nr PS 9 +.\".nr VS 11 +.AE +.2C +.NH +Introduction. +.PP +Lex is a program generator designed for +lexical processing of character input streams. +It accepts a high-level, problem oriented specification +for character string matching, +and +produces a program in a general purpose language which recognizes +regular expressions. +The regular expressions are specified by the user in the +source specifications given to Lex. +The Lex written code recognizes these expressions +in an input stream and partitions the input stream into +strings matching the expressions. At the bound~aries +between strings +program sections +provided by the user are executed. +The Lex source file associates the regular expressions and the +program fragments. +As each expression appears in the input to the program written by Lex, +the corresponding fragment is executed. +.PP +The user supplies the additional code +beyond expression matching +needed to complete his tasks, possibly +including code written by other generators. +The program that recognizes the expressions is generated in the +general purpose programming language employed for the +user's program fragments. +Thus, a high level expression +language is provided to write the string expressions to be +matched while the user's freedom to write actions +is unimpaired. +This avoids forcing the user who wishes to use a string manipulation +language for input analysis to write processing programs in the same +and often inappropriate string handling language. +.PP +Lex is not a complete language, but rather a generator representing +a new language feature which can be added to +different programming languages, called ``host languages.'' +Just as general purpose languages +can produce code to run on different computer hardware, +Lex can write code in different host languages. +The host language is used for the output code generated by Lex +and also for the program fragments added by the user. +Compatible run-time libraries for the different host languages +are also provided. +This makes Lex adaptable to different environments and +different users. +Each application +may be directed to the combination of hardware and host language appropriate +to the task, the user's background, and the properties of local +implementations. +At present, the only supported host language is C, +although Fortran (in the form of Ratfor [2] has been available +in the past. +Lex itself exists on UNIX, GCOS, and OS/370; but the +code generated by Lex may be taken anywhere the appropriate +compilers exist. +.PP +Lex turns the user's expressions and actions +(called +.ul +source +in this memo) into the host general-purpose language; +the generated program is named +.ul +yylex. +The +.ul +yylex +program +will recognize expressions +in a stream +(called +.ul +input +in this memo) +and perform the specified actions for each expression as it is detected. +See Figure 1. +.\" .GS +.TS +center; +l _ r +l|c|r +l _ r +l _ r +l|c|r +l _ r +c s s +c s s. + +Source \(-> Lex \(-> yylex + +.sp 2 + +Input \(-> yylex \(-> Output + +.sp +An overview of Lex +Figure 1 +.TE +.\" .GE +.PP +For a trivial example, consider a program to delete +from the input +all blanks or tabs at the ends of lines. +.TS +center; +l l. +%% +[ \et]+$ ; +.TE +is all that is required. +The program +contains a %% delimiter to mark the beginning of the rules, and +one rule. +This rule contains a regular expression +which matches one or more +instances of the characters blank or tab +(written \et for visibility, in accordance with the C language convention) +just prior to the end of a line. +The brackets indicate the character +class made of blank and tab; the + indicates ``one or more ...''; +and the $ indicates ``end of line,'' as in QED. +No action is specified, +so the program generated by Lex (yylex) will ignore these characters. +Everything else will be copied. +To change any remaining +string of blanks or tabs to a single blank, +add another rule: +.TS +center; +l l. +%% +[ \et]+$ ; +[ \et]+ printf(" "); +.TE +The finite automaton generated for this +source will scan for both rules at once, +observing at +the termination of the string of blanks or tabs +whether or not there is a newline character, and executing +the desired rule action. +The first rule matches all strings of blanks or tabs +at the end of lines, and the second +rule all remaining strings of blanks or tabs. +.PP +Lex can be used alone for simple transformations, or +for analysis and statistics gathering on a lexical level. +Lex can also be used with a parser generator +to perform the lexical analysis phase; it is particularly +easy to interface Lex and Yacc [3]. +Lex programs recognize only regular expressions; +Yacc writes parsers that accept a large class of context free grammars, +but require a lower level analyzer to recognize input tokens. +Thus, a combination of Lex and Yacc is often appropriate. +When used as a preprocessor for a later parser generator, +Lex is used to partition the input stream, +and the parser generator assigns structure to +the resulting pieces. +The flow of control +in such a case (which might be the first half of a compiler, +for example) is shown in Figure 2. +Additional programs, +written by other generators +or by hand, can +be added easily to programs written by Lex. +.\" .BS 2 +.ps 9 +.vs 11 +.TS +center; +l c c c l +l c c c l +l c c c l +l _ c _ l +l|c|c|c|l +l _ c _ l +l c c c l +l _ c _ l +l|c|c|c|l +l _ c _ l +l c s s l +l c s s l. + lexical grammar + rules rules + \(da \(da + + Lex Yacc + + \(da \(da + +Input \(-> yylex \(-> yyparse \(-> Parsed input + +.sp + Lex with Yacc + Figure 2 +.TE +.ps 10 +.vs 12 +.\" .BE +Yacc users +will realize that the name +.ul +yylex +is what Yacc expects its lexical analyzer to be named, +so that the use of this name by Lex simplifies +interfacing. +.PP +Lex generates a deterministic finite automaton from the regular expressions +in the source [4]. +The automaton is interpreted, rather than compiled, in order +to save space. +The result is still a fast analyzer. +In particular, the time taken by a Lex program +to recognize and partition an input stream is +proportional to the length of the input. +The number of Lex rules or +the complexity of the rules is +not important in determining speed, +unless rules which include +forward context require a significant amount of re~scanning. +What does increase with the number and complexity of rules +is the size of the finite +automaton, and therefore the size of the program +generated by Lex. +.PP +In the program written by Lex, the user's fragments +(representing the +.ul +actions +to be performed as each regular expression +is found) +are gathered +as cases of a switch. +The automaton interpreter directs the control flow. +Opportunity is provided for the user to insert either +declarations or additional statements in the routine containing +the actions, or to +add subroutines outside this action routine. +.PP +Lex is not limited to source which can +be interpreted on the basis of one character +look~ahead. +For example, +if there are two rules, one looking for +.I ab +and another for +.I abcdefg , +and the input stream is +.I abcdefh , +Lex will recognize +.I ab +and leave +the input pointer just before +.I "cd. . ." +Such backup is more costly +than the processing of simpler languages. +.2C +.NH +Lex Source. +.PP +The general format of Lex source is: +.TS +center; +l. +{definitions} +%% +{rules} +%% +{user subroutines} +.TE +where the definitions and the user subroutines +are often omitted. +The second +.I %% +is optional, but the first is required +to mark the beginning of the rules. +The absolute minimum Lex program is thus +.TS +center; +l. +%% +.TE +(no definitions, no rules) which translates into a program +which copies the input to the output unchanged. +.PP +In the outline of Lex programs shown above, the +.I +rules +.R +represent the user's control +decisions; they are a table, in which the left column +contains +.I +regular expressions +.R +(see section 3) +and the right column contains +.I +actions, +.R +program fragments to be executed when the expressions +are recognized. +Thus an individual rule might appear +.TS +center; +l l. +integer printf("found keyword INT"); +.TE +to look for the string +.I integer +in the input stream and +print the message ``found keyword INT'' whenever it appears. +In this example the host procedural language is C and +the C library function +.I +printf +.R +is used to print the string. +The end +of the expression is indicated by the first blank or tab character. +If the action is merely a single C expression, +it can just be given on the right side of the line; if it is +compound, or takes more than a line, it should be enclosed in +braces. +As a slightly more useful example, suppose it is desired to +change a number of words from British to American spelling. +Lex rules such as +.TS +center; +l l. +colour printf("color"); +mechanise printf("mechanize"); +petrol printf("gas"); +.TE +would be a start. These rules are not quite enough, +since +the word +.I petroleum +would become +.I gaseum ; +a way of dealing +with this will be described later. +.2C +.NH +Lex Regular Expressions. +.PP +The definitions of regular expressions are very similar to those +in QED [5]. +A regular +expression specifies a set of strings to be matched. +It contains text characters (which match the corresponding +characters in the strings being compared) +and operator characters (which specify +repetitions, choices, and other features). +The letters of the alphabet and the digits are +always text characters; thus the regular expression +.TS +center; +l l. +integer +.TE +matches the string +.ul +integer +wherever it appears +and the expression +.TS +center; +l. +a57D +.TE +looks for the string +.ul +a57D. +.PP +.I +Operators. +.R +The operator characters are +.TS +center; +l. +" \e [ ] ^ \- ? . \(** + | ( ) $ / { } % < > +.TE +and if they are to be used as text characters, an escape +should be used. +The quotation mark operator (") +indicates that whatever is contained between a pair of quotes +is to be taken as text characters. +Thus +.TS +center; +l. +xyz"++" +.TE +matches the string +.I xyz++ +when it appears. Note that a part of a string may be quoted. +It is harmless but unnecessary to quote an ordinary +text character; the expression +.TS +center; +l. +"xyz++" +.TE +is the same as the one above. +Thus by quoting every non-alphanumeric character +being used as a text character, the user can avoid remembering +the list above of current +operator characters, and is safe should further extensions to Lex +lengthen the list. +.PP +An operator character may also be turned into a text character +by preceding it with \e as in +.TS +center; +l. +xyz\e+\e+ +.TE +which +is another, less readable, equivalent of the above expressions. +Another use of the quoting mechanism is to get a blank into +an expression; normally, as explained above, blanks or tabs end +a rule. +Any blank character not contained within [\|] (see below) must +be quoted. +Several normal C escapes with \e +are recognized: \en is newline, \et is tab, and \eb is backspace. +To enter \e itself, use \e\e. +Since newline is illegal in an expression, \en must be used; +it is not +required to escape tab and backspace. +Every character but blank, tab, newline and the list above is always +a text character. +.PP +.I +Character classes. +.R +Classes of characters can be specified using the operator pair [\|]. +The construction +.I [abc] +matches a +single character, which may be +.I a , +.I b , +or +.I c . +Within square brackets, +most operator meanings are ignored. +Only three characters are special: +these are \e \(mi and ^. The \(mi character +indicates ranges. For example, +.TS +center; +l. +[a\(miz0\(mi9<>_] +.TE +indicates the character class containing all the lower case letters, +the digits, +the angle brackets, and underline. +Ranges may be given in either order. +Using \(mi between any pair of characters which are +not both upper case letters, both lower case letters, or both digits +is implementation dependent and will get a warning message. +(E.g., [0\-z] in ASCII is many more characters +than it is in EBCDIC). +If it is desired to include the +character \(mi in a character class, it should be first or +last; thus +.TS +center; +l. +[\(mi+0\(mi9] +.TE +matches all the digits and the two signs. +.PP +In character classes, +the ^ operator must appear as the first character +after the left bracket; it indicates that the resulting string +is to be complemented with respect to the computer character set. +Thus +.TS +center; +l. +[^abc] +.TE +matches all characters except a, b, or c, including +all special or control characters; or +.TS +center; +l. +[^a\-zA\-Z] +.TE +is any character which is not a letter. +The \e character provides the usual escapes within +character class brackets. +.PP +.I +Arbitrary character. +.R +To match almost any character, the operator character +.TS +center; +l. +\&. +.TE +is the class of all characters except newline. +Escaping into octal is possible although non-portable: +.TS +center; +l. +[\e40\-\e176] +.TE +matches all printable characters in the ASCII character set, from octal +40 (blank) to octal 176 (tilde). +.PP +.I +Optional expressions. +.R +The operator +.I ? +indicates +an optional element of an expression. +Thus +.TS +center; +l. +ab?c +.TE +matches either +.I ac +or +.I abc . +.PP +.I +Repeated expressions. +.R +Repetitions of classes are indicated by the operators +.I \(** +and +.I + . +.TS +center; +l. +\f2a\(**\f1 +.TE +is any number of consecutive +.I a +characters, including zero; while +.TS +center; +l. +a+ +.TE +is one or more instances of +.I a. +For example, +.TS +center; +l. +[a\-z]+ +.TE +is all strings of lower case letters. +And +.TS +center; +l. +[A\(miZa\(miz][A\(miZa\(miz0\(mi9]\(** +.TE +indicates all alphanumeric strings with a leading +alphabetic character. +This is a typical expression for recognizing identifiers in +computer languages. +.PP +.I +Alternation and Grouping. +.R +The operator | +indicates alternation: +.TS +center; +l. +(ab\||\|cd) +.TE +matches either +.ul +ab +or +.ul +cd. +Note that parentheses are used for grouping, although +they are +not necessary on the outside level; +.TS +center; +l. +ab\||\|cd +.TE +would have sufficed. +Parentheses +can be used for more complex expressions: +.TS +center; +l. +(ab\||\|cd+)?(ef)\(** +.TE +matches such strings as +.I abefef , +.I efefef , +.I cdef , +or +.I cddd\| ; +but not +.I abc , +.I abcd , +or +.I abcdef . +.PP +.I +Context sensitivity. +.R +Lex will recognize a small amount of surrounding +context. The two simplest operators for this are +.I ^ +and +.I $ . +If the first character of an expression is +.I ^ , +the expression will only be matched at the beginning +of a line (after a newline character, or at the beginning of +the input stream). +This can never conflict with the other meaning of +.I ^ , +complementation +of character classes, since that only applies within +the [\|] operators. +If the very last character is +.I $ , +the expression will only be matched at the end of a line (when +immediately followed by newline). +The latter operator is a special case of the +.I / +operator character, +which indicates trailing context. +The expression +.TS +center; +l. +ab/cd +.TE +matches the string +.I ab , +but only if followed by +.ul +cd. +Thus +.TS +center; +l. +ab$ +.TE +is the same as +.TS +center; +l. +ab/\en +.TE +Left context is handled in Lex by +.I +start conditions +.R +as explained in section 10. If a rule is only to be executed +when the Lex automaton interpreter is in start condition +.I +x, +.R +the rule should be prefixed by +.TS +center; +l. +<x> +.TE +using the angle bracket operator characters. +If we considered ``being at the beginning of a line'' to be +start condition +.I ONE , +then the ^ operator +would be equivalent to +.TS +center; +l. +<ONE> +.TE +Start conditions are explained more fully later. +.PP +.I +Repetitions and Definitions. +.R +The operators {} specify +either repetitions (if they enclose numbers) +or +definition expansion (if they enclose a name). For example +.TS +center; +l. +{digit} +.TE +looks for a predefined string named +.I digit +and inserts it +at that point in the expression. +The definitions are given in the first part of the Lex +input, before the rules. +In contrast, +.TS +center; +l. +a{1,5} +.TE +looks for 1 to 5 occurrences of +.I a . +.PP +Finally, initial +.I % +is special, being the separator +for Lex source segments. +.2C +.NH +Lex Actions. +.PP +When an expression written as above is matched, Lex +executes the corresponding action. This section describes +some features of Lex which aid in writing actions. Note +that there is a default action, which +consists of copying the input to the output. This +is performed on all strings not otherwise matched. Thus +the Lex user who wishes to absorb the entire input, without +producing any output, must provide rules to match everything. +When Lex is being used with Yacc, this is the normal +situation. +One may consider that actions are what is done instead of +copying the input to the output; thus, in general, +a rule which merely copies can be omitted. +Also, a character combination +which is omitted from the rules +and which appears as input +is likely to be printed on the output, thus calling +attention to the gap in the rules. +.PP +One of the simplest things that can be done is to ignore +the input. Specifying a C null statement, \fI;\fR as an action +causes this result. A frequent rule is +.TS +center; +l l. +[ \et\en] ; +.TE +which causes the three spacing characters (blank, tab, and newline) +to be ignored. +.PP +Another easy way to avoid writing actions is the action character +|, which indicates that the action for this rule is the action +for the next rule. +The previous example could also have been written +.TS +center, tab(#); +l l. +" "#| +"\et"#| +"\en"#; +.TE +with the same result, although in different style. +The quotes around \en and \et are not required. +.PP +In more complex actions, the user +will +often want to know the actual text that matched some expression +like +.I [a\(miz]+ . +Lex leaves this text in an external character +array named +.I +yytext. +.R +Thus, to print the name found, +a rule like +.TS +center; +l l. +[a\-z]+ printf("%s", yytext); +.TE +will print +the string in +.I +yytext. +.R +The C function +.I +printf +.R +accepts a format argument and data to be printed; +in this case, the format is ``print string'' (% indicating +data conversion, and +.I s +indicating string type), +and the data are the characters +in +.I +yytext. +.R +So this just places +the matched string +on the output. +This action +is so common that +it may be written as ECHO: +.TS +center; +l l. +[a\-z]+ ECHO; +.TE +is the same as the above. +Since the default action is just to +print the characters found, one might ask why +give a rule, like this one, which merely specifies +the default action? +Such rules are often required +to avoid matching some other rule +which is not desired. For example, if there is a rule +which matches +.I read +it will normally match the instances of +.I read +contained in +.I bread +or +.I readjust ; +to avoid +this, +a rule +of the form +.I [a\(miz]+ +is needed. +This is explained further below. +.PP +Sometimes it is more convenient to know the end of what +has been found; hence Lex also provides a count +.I +yyleng +.R +of the number of characters matched. +To count both the number +of words and the number of characters in words in the input, the user might write +.TS +center; +l l. +[a\-zA\-Z]+ {words++; chars += yyleng;} +.TE +which accumulates in +.ul +chars +the number +of characters in the words recognized. +The last character in the string matched can +be accessed by +.TS +center; +l. +yytext[yyleng\-1] +.TE +.PP +Occasionally, a Lex +action may decide that a rule has not recognized the correct +span of characters. +Two routines are provided to aid with this situation. +First, +.I +yymore() +.R +can be called to indicate that the next input expression recognized is to be +tacked on to the end of this input. Normally, +the next input string would overwrite the current +entry in +.I +yytext. +.R +Second, +.I +yyless (n) +.R +may be called to indicate that not all the characters matched +by the currently successful expression are wanted right now. +The argument +.I +n +.R +indicates the number of characters +in +.I +yytext +.R +to be retained. +Further characters previously matched +are +returned to the input. This provides the same sort of +look~ahead offered by the / operator, +but in a different form. +.PP +.I +Example: +.R +Consider a language which defines +a string as a set of characters between quotation (") marks, and provides that +to include a " in a string it must be preceded by a \e. The +regular expression which matches that is somewhat confusing, +so that it might be preferable to write +.TS +center; +l l. +\e"[^"]\(** { + if (yytext[yyleng\-1] == \(fm\e\e\(fm) + yymore(); + else + ... normal user processing + } +.TE +which will, when faced with a string such as +.I +"abc\e"def\|" +.R +first match +the five characters +\fI"abc\e\|\fR; +then +the call to +.I yymore() +will +cause the next part of the string, +\fI"def\|\fR, +to be tacked on the end. +Note that the final quote terminating the string should be picked +up in the code labeled ``normal processing''. +.PP +The function +.I +yyless() +.R +might be used to reprocess +text in various circumstances. Consider the C problem of distinguishing +the ambiguity of ``=\(mia''. +Suppose it is desired to treat this as ``=\(mi a'' +but print a message. A rule might be +.ps 9 +.vs 11 +.TS +center; +l l. +=\(mi[a\-zA\-Z] { + printf("Op (=\(mi) ambiguous\en"); + yyless(yyleng\-1); + ... action for =\(mi ... + } +.TE +.ps 10 +.vs 12 +which prints a message, returns the letter after the +operator to the input stream, and treats the operator as ``=\(mi''. +Alternatively it might be desired to treat this as ``= \(mia''. +To do this, just return the minus +sign as well as the letter to the input: +.ps 9 +.vs 11 +.TS +center; +l l. +=\(mi[a\-zA\-Z] { + printf("Op (=\(mi) ambiguous\en"); + yyless(yyleng\-2); + ... action for = ... + } +.TE +.ps 10 +.vs 12 +will perform the other interpretation. +Note that the expressions for the two cases might more easily +be written +.TS +center; +l l. +=\(mi/[A\-Za\-z] +.TE +in the first case and +.TS +center; +l. +=/\-[A\-Za\-z] +.TE +in the second; +no backup would be required in the rule action. +It is not necessary to recognize the whole identifier +to observe the ambiguity. +The +possibility of ``=\(mi3'', however, makes +.TS +center; +l. +=\(mi/[^ \et\en] +.TE +a still better rule. +.PP +In addition to these routines, Lex also permits +access to the I/O routines +it uses. +They are: +.IP 1) +.I +input() +.R +which returns the next input character; +.IP 2) +.I +output(c) +.R +which writes the character +.I +c +.R +on the output; and +.IP 3) +.I +unput(c) +.R +pushes the character +.I +c +.R +back onto the input stream to be read later by +.I +input(). +.R +.LP +By default these routines are provided as macro definitions, +but the user can override them and supply private versions. +These routines +define the relationship between external files and +internal characters, and must all be retained +or modified consistently. +They may be redefined, to +cause input or output to be transmitted to or from strange +places, including other programs or internal memory; +but the character set used must be consistent in all routines; +a value of zero returned by +.I +input +.R +must mean end of file; and +the relationship between +.I +unput +.R +and +.I +input +.R +must be retained +or the Lex look~ahead will not work. +Lex does not look ahead at all if it does not have to, +but every rule ending in +.ft I ++ \(** ? +.ft R +or +.ft I +$ +.ft R +or containing +.ft I +/ +.ft R +implies look~ahead. +Look~ahead is also necessary to match an expression that is a prefix +of another expression. +See below for a discussion of the character set used by Lex. +The standard Lex library imposes +a 100 character limit on backup. +.PP +Another Lex library routine that the user will sometimes want +to redefine is +.I +yywrap() +.R +which is called whenever Lex reaches an end-of-file. +If +.I +yywrap +.R +returns a 1, Lex continues with the normal wrapup on end of input. +Sometimes, however, it is convenient to arrange for more +input to arrive +from a new source. +In this case, the user should provide +a +.I +yywrap +.R +which +arranges for new input and +returns 0. This instructs Lex to continue processing. +The default +.I +yywrap +.R +always returns 1. +.PP +This routine is also a convenient place +to print tables, summaries, etc. at the end +of a program. Note that it is not +possible to write a normal rule which recognizes +end-of-file; the only access to this condition is +through +.I +yywrap. +.R +In fact, unless a private version of +.I +input() +.R +is supplied +a file containing nulls +cannot be handled, +since a value of 0 returned by +.I +input +.R +is taken to be end-of-file. +.PP +.2C +.NH +Ambiguous Source Rules. +.PP +Lex can handle ambiguous specifications. +When more than one expression can match the +current input, Lex chooses as follows: +.IP 1) +The longest match is preferred. +.IP 2) +Among rules which matched the same number of characters, +the rule given first is preferred. +.LP +Thus, suppose the rules +.TS +center; +l l. +integer keyword action ...; +[a\-z]+ identifier action ...; +.TE +to be given in that order. If the input is +.I integers , +it is taken as an identifier, because +.I [a\-z]+ +matches 8 characters while +.I integer +matches only 7. +If the input is +.I integer , +both rules match 7 characters, and +the keyword rule is selected because it was given first. +Anything shorter (e.g. \fIint\fR\|) will +not match the expression +.I integer +and so the identifier interpretation is used. +.PP +The principle of preferring the longest +match makes rules containing +expressions like +.I \&.\(** +dangerous. +For example, +.TS +center; +l. +\&\(fm.\(**\(fm +.TE +might seem a good way of recognizing +a string in single quotes. +But it is an invitation for the program to read far +ahead, looking for a distant +single quote. +Presented with the input +.TS +center; +l l. +\&\(fmfirst\(fm quoted string here, \(fmsecond\(fm here +.TE +the above expression will match +.TS +center; +l l. +\&\(fmfirst\(fm quoted string here, \(fmsecond\(fm +.TE +which is probably not what was wanted. +A better rule is of the form +.TS +center; +l. +\&\(fm[^\(fm\en]\(**\(fm +.TE +which, on the above input, will stop +after +.I \(fmfirst\(fm . +The consequences +of errors like this are mitigated by the fact +that the +.I \&. +operator will not match newline. +Thus expressions like +.I \&.\(** +stop on the +current line. +Don't try to defeat this with expressions like +.I (.|\en)+ +or +equivalents; +the Lex generated program will try to read +the entire input file, causing +internal buffer overflows. +.PP +Note that Lex is normally partitioning +the input stream, not searching for all possible matches +of each expression. +This means that each character is accounted for +once and only once. +For example, suppose it is desired to +count occurrences of both \fIshe\fR and \fIhe\fR in an input text. +Some Lex rules to do this might be +.TS +center; +l l. +she s++; +he h++; +\en | +\&. ; +.TE +where the last two rules ignore everything besides \fIhe\fR and \fIshe\fR. +Remember that . does not include newline. +Since \fIshe\fR includes \fIhe\fR, Lex will normally +.I +not +.R +recognize +the instances of \fIhe\fR included in \fIshe\fR, +since once it has passed a \fIshe\fR those characters are gone. +.PP +Sometimes the user would like to override this choice. The action +REJECT +means ``go do the next alternative.'' +It causes whatever rule was second choice after the current +rule to be executed. +The position of the input pointer is adjusted accordingly. +Suppose the user really wants to count the included instances of \fIhe\fR: +.TS +center; +l l. +she {s++; REJECT;} +he {h++; REJECT;} +\en | +\&. ; +.TE +these rules are one way of changing the previous example +to do just that. +After counting each expression, it is rejected; whenever appropriate, +the other expression will then be counted. In this example, of course, +the user could note that \fIshe\fR includes \fIhe\fR but not +vice versa, and omit the REJECT action on \fIhe\fR; +in other cases, however, it +would not be possible a priori to tell +which input characters +were in both classes. +.PP +Consider the two rules +.TS +center; +l l. +a[bc]+ { ... ; REJECT;} +a[cd]+ { ... ; REJECT;} +.TE +If the input is +.I ab , +only the first rule matches, +and on +.I ad +only the second matches. +The input string +.I accb +matches the first rule for four characters +and then the second rule for three characters. +In contrast, the input +.I accd +agrees with +the second rule for four characters and then the first +rule for three. +.PP +In general, REJECT is useful whenever +the purpose of Lex is not to partition the input +stream but to detect all examples of some items +in the input, and the instances of these items +may overlap or include each other. +Suppose a digram table of the input is desired; +normally the digrams overlap, that is the word +.I the +is considered to contain +both +.I th +and +.I he . +Assuming a two-dimensional array named +.ul +digram +to be incremented, the appropriate +source is +.TS +center; +l l. +%% +[a\-z][a\-z] { + digram[yytext[0]][yytext[1]]++; + REJECT; + } +\&. ; +\en ; +.TE +where the REJECT is necessary to pick up +a letter pair beginning at every character, rather than at every +other character. +.2C +.NH +Lex Source Definitions. +.PP +Remember the format of the Lex +source: +.TS +center; +l. +{definitions} +%% +{rules} +%% +{user routines} +.TE +So far only the rules have been described. The user needs +additional options, +though, to define variables for use in his program and for use +by Lex. +These can go either in the definitions section +or in the rules section. +.PP +Remember that Lex is turning the rules into a program. +Any source not intercepted by Lex is copied +into the generated program. There are three classes +of such things. +.IP 1) +Any line which is not part of a Lex rule or action +which begins with a blank or tab is copied into +the Lex generated program. +Such source input prior to the first %% delimiter will be external +to any function in the code; if it appears immediately after the first +%%, +it appears in an appropriate place for declarations +in the function written by Lex which contains the actions. +This material must look like program fragments, +and should precede the first Lex rule. +.IP +As a side effect of the above, lines which begin with a blank +or tab, and which contain a comment, +are passed through to the generated program. +This can be used to include comments in either the Lex source or +the generated code. The comments should follow the host +language convention. +.IP 2) +Anything included between lines containing +only +.I %{ +and +.I %} +is +copied out as above. The delimiters are discarded. +This format permits entering text like preprocessor statements that +must begin in column 1, +or copying lines that do not look like programs. +.IP 3) +Anything after the third %% delimiter, regardless of formats, etc., +is copied out after the Lex output. +.PP +Definitions intended for Lex are given +before the first %% delimiter. Any line in this section +not contained between %{ and %}, and begining +in column 1, is assumed to define Lex substitution strings. +The format of such lines is +.TS +center; +l l. +name translation +.TE +and it +causes the string given as a translation to +be associated with the name. +The name and translation +must be separated by at least one blank or tab, and the name must begin with a letter. +The translation can then be called out +by the {name} syntax in a rule. +Using {D} for the digits and {E} for an exponent field, +for example, might abbreviate rules to recognize numbers: +.TS +center, tab(#); +l l. +D#[0\-9] +E#[DEde][\-+]?{D}+ +%% +{D}+#printf("integer"); +{D}+"."{D}\(**({E})?#| +{D}\(**"."{D}+({E})?#| +{D}+{E}#printf("real"); +.TE +Note the first two rules for real numbers; +both require a decimal point and contain +an optional exponent field, +but the first requires at least one digit before the +decimal point and the second requires at least one +digit after the decimal point. +To correctly handle the problem +posed by a Fortran expression such as +.I 35.EQ.I , +which does not contain a real number, a context-sensitive +rule such as +.TS +center; +l l. +[0\-9]+/"."EQ printf("integer"); +.TE +could be used in addition to the normal rule for integers. +.PP +The definitions +section may also contain other commands, including the +selection of a host language, a character set table, +a list of start conditions, or adjustments to the default +size of arrays within Lex itself for larger source programs. +These possibilities +are discussed below under ``Summary of Source Format,'' +section 12. +.2C +.NH +Usage. +.PP +There are two steps in +compiling a Lex source program. +First, the Lex source must be turned into a generated program +in the host general purpose language. +Then this program must be compiled and loaded, usually with +a library of Lex subroutines. +The generated program +is on a file named +.I lex.yy.c . +The I/O library is defined in terms of the C standard +library [6]. +.PP +The C programs generated by Lex are slightly different +on OS/370, because the +OS compiler is less powerful than the UNIX or GCOS compilers, +and does less at compile time. +C programs generated on GCOS and UNIX are the same. +.PP +.I +UNIX. +.R +The library is accessed by the loader flag +.I \-ll . +So an appropriate +set of commands is +.KS +.in 5 +lex source +cc lex.yy.c \-ll +.in 0 +.KE +The resulting program is placed on the usual file +.I +a.out +.R +for later execution. +To use Lex with Yacc see below. +Although the default Lex I/O routines use the C standard library, +the Lex automata themselves do not do so; +if private versions of +.I +input, +output +.R +and +.I unput +are given, the library can be avoided. +.PP +.2C +.NH +Lex and Yacc. +.PP +If you want to use Lex with Yacc, note that what Lex writes is a program +named +.I +yylex(), +.R +the name required by Yacc for its analyzer. +Normally, the default main program on the Lex library +calls this routine, but if Yacc is loaded, and its main +program is used, Yacc will call +.I +yylex(). +.R +In this case each Lex rule should end with +.TS +center; +l. +return(token); +.TE +where the appropriate token value is returned. +An easy way to get access +to Yacc's names for tokens is to +compile the Lex output file as part of +the Yacc output file by placing the line +.TS +center; +l. +# include "lex.yy.c" +.TE +in the last section of Yacc input. +Supposing the grammar to be +named ``good'' and the lexical rules to be named ``better'' +the UNIX command sequence can just be: +.TS +center; +l. +yacc good +lex better +cc y.tab.c \-ly \-ll +.TE +The Yacc library (\-ly) should be loaded before the Lex library, +to obtain a main program which invokes the Yacc parser. +The generations of Lex and Yacc programs can be done in +either order. +.2C +.NH +Examples. +.PP +As a trivial problem, consider copying an input file while +adding 3 to every positive number divisible by 7. +Here is a suitable Lex source program +.TS +center; +l l. +%% + int k; +[0\-9]+ { + k = atoi(yytext); + if (k%7 == 0) + printf("%d", k+3); + else + printf("%d",k); + } +.TE +to do just that. +The rule [0\-9]+ recognizes strings of digits; +.I +atoi +.R +converts the digits to binary +and stores the result in +.ul +k. +The operator % (remainder) is used to check whether +.ul +k +is divisible by 7; if it is, +it is incremented by 3 as it is written out. +It may be objected that this program will alter such +input items as +.I 49.63 +or +.I X7 . +Furthermore, it increments the absolute value +of all negative numbers divisible by 7. +To avoid this, just add a few more rules after the active one, +as here: +.TS +center; +l l. +%% + int k; +\-?[0\-9]+ { + k = atoi(yytext); + printf("%d", + k%7 == 0 ? k+3 : k); + } +\-?[0\-9.]+ ECHO; +[A-Za-z][A-Za-z0-9]+ ECHO; +.TE +Numerical strings containing +a ``.'' or preceded by a letter will be picked up by +one of the last two rules, and not changed. +The +.I if\-else +has been replaced by +a C conditional expression to save space; +the form +.ul +a?b:c +means ``if +.I a +then +.I b +else +.I c ''. +.PP +For an example of statistics gathering, here +is a program which histograms the lengths +of words, where a word is defined as a string of letters. +.TS +center; +l l. + int lengs[100]; +%% +[a\-z]+ lengs[yyleng]++; +\&. | +\en ; +%% +.T& +l s. +yywrap() +{ +int i; +printf("Length No. words\en"); +for(i=0; i<100; i++) + if (lengs[i] > 0) + printf("%5d%10d\en",i,lengs[i]); +return(1); +} +.TE +This program +accumulates the histogram, while producing no output. At the end +of the input it prints the table. +The final statement +.I +return(1); +.R +indicates that Lex is to perform wrapup. If +.I +yywrap +.R +returns zero (false) +it implies that further input is available +and the program is +to continue reading and processing. +To provide a +.I +yywrap +.R +that never +returns true causes an infinite loop. +.PP +As a larger example, +here are some parts of a program written by N. L. Schryer +to convert double precision Fortran to single precision Fortran. +Because Fortran does not distinguish upper and lower case letters, +this routine begins by defining a set of classes including +both cases of each letter: +.TS +center; +l l. +a [aA] +b [bB] +c [cC] +\&... +z [zZ] +.TE +An additional class recognizes white space: +.TS +center; +l l. +W [ \et]\(** +.TE +The first rule changes +``double precision'' to ``real'', or ``DOUBLE PRECISION'' to ``REAL''. +.TS +center; +l. +{d}{o}{u}{b}{l}{e}{W}{p}{r}{e}{c}{i}{s}{i}{o}{n} { + printf(yytext[0]==\(fmd\(fm? "real" : "REAL"); + } +.TE +Care is taken throughout this program to preserve the case +(upper or lower) +of the original program. +The conditional operator is used to +select the proper form of the keyword. +The next rule copies continuation card indications to +avoid confusing them with constants: +.TS +center; +l l. +^" "[^ 0] ECHO; +.TE +In the regular expression, the quotes surround the +blanks. +It is interpreted as +``beginning of line, then five blanks, then +anything but blank or zero.'' +Note the two different meanings of +.I ^ . +There follow some rules to change double precision +constants to ordinary floating constants. +.TS +center; +l. +[0\-9]+{W}{d}{W}[+\-]?{W}[0\-9]+ | +[0\-9]+{W}"."{W}{d}{W}[+\-]?{W}[0\-9]+ | +"."{W}[0\-9]+{W}{d}{W}[+\-]?{W}[0\-9]+ { + /\(** convert constants \(**/ + for(p=yytext; \(**p != 0; p++) + { + if (\(**p == \(fmd\(fm || \(**p == \(fmD\(fm) + \(**p=+ \(fme\(fm\- \(fmd\(fm; + ECHO; + } +.TE +After the floating point constant is recognized, it is +scanned by the +.ul +for +loop +to find the letter +.I d +or +.I D . +The program then adds +.I \(fme\(fm\-\(fmd\(fm , +which converts +it to the next letter of the alphabet. +The modified constant, now single-precision, +is written out again. +There follow a series of names which must be respelled to remove +their initial \fId\fR. +By using the +array +.I +yytext +.R +the same action suffices for all the names (only a sample of +a rather long list is given here). +.TS +center; +l l. +{d}{s}{i}{n} | +{d}{c}{o}{s} | +{d}{s}{q}{r}{t} | +{d}{a}{t}{a}{n} | +\&... +{d}{f}{l}{o}{a}{t} printf("%s",yytext+1); +.TE +Another list of names must have initial \fId\fR changed to initial \fIa\fR: +.TS +center; +l l. +{d}{l}{o}{g} | +{d}{l}{o}{g}10 | +{d}{m}{i}{n}1 | +{d}{m}{a}{x}1 { + yytext[0] =+ \(fma\(fm \- \(fmd\(fm; + ECHO; + } +.TE +And one routine +must have initial \fId\fR changed to initial \fIr\fR: +.TS +center, tab(#); +l l. +{d}1{m}{a}{c}{h}#{yytext[0] =+ \(fmr\(fm \- \(fmd\(fm; +#ECHO; +#} +.TE +To avoid such names as \fIdsinx\fR being detected as instances +of \fIdsin\fR, some final rules pick up longer words as identifiers +and copy some surviving characters: +.TS +center; +l l. +[A\-Za\-z][A\-Za\-z0\-9]\(** | +[0\-9]+ | +\en | +\&. ECHO; +.TE +Note that this program is not complete; it +does not deal with the spacing problems in Fortran or +with the use of keywords as identifiers. +.br +.2C +.NH +Left Context Sensitivity. +.PP +Sometimes +it is desirable to have several sets of lexical rules +to be applied at different times in the input. +For example, a compiler preprocessor might distinguish +preprocessor statements and analyze them differently +from ordinary statements. +This requires +sensitivity +to prior context, and there are several ways of handling +such problems. +The \fI^\fR operator, for example, is a prior context operator, +recognizing immediately preceding left context just as \fI$\fR recognizes +immediately following right context. +Adjacent left context could be extended, to produce a facility similar to +that for adjacent right context, but it is unlikely +to be as useful, since often the relevant left context +appeared some time earlier, such as at the beginning of a line. +.PP +This section describes three means of dealing +with different environments: a simple use of flags, +when only a few rules change from one environment to another, +the use of +.I +start conditions +.R +on rules, +and the possibility of making multiple lexical analyzers all run +together. +In each case, there are rules which recognize the need to change the +environment in which the +following input text is analyzed, and set some parameter +to reflect the change. This may be a flag explicitly tested by +the user's action code; such a flag is the simplest way of dealing +with the problem, since Lex is not involved at all. +It may be more convenient, +however, +to have Lex remember the flags as initial conditions on the rules. +Any rule may be associated with a start condition. It will only +be recognized when Lex is in +that start condition. +The current start condition may be changed at any time. +Finally, if the sets of rules for the different environments +are very dissimilar, +clarity may be best achieved by writing several distinct lexical +analyzers, and switching from one to another as desired. +.PP +Consider the following problem: copy the input to the output, +changing the word \fImagic\fR to \fIfirst\fR on every line which began +with the letter \fIa\fR, changing \fImagic\fR to \fIsecond\fR on every line +which began with the letter \fIb\fR, and changing +\fImagic\fR to \fIthird\fR on every line which began +with the letter \fIc\fR. All other words and all other lines +are left unchanged. +.PP +These rules are so simple that the easiest way +to do this job is with a flag: +.TS +center; +l l. + int flag; +%% +^a {flag = \(fma\(fm; ECHO;} +^b {flag = \(fmb\(fm; ECHO;} +^c {flag = \(fmc\(fm; ECHO;} +\en {flag = 0 ; ECHO;} +magic { + switch (flag) + { + case \(fma\(fm: printf("first"); break; + case \(fmb\(fm: printf("second"); break; + case \(fmc\(fm: printf("third"); break; + default: ECHO; break; + } + } +.TE +should be adequate. +.PP +To handle the same problem with start conditions, each +start condition must be introduced to Lex in the definitions section +with a line reading +.TS +center; +l l. +%Start name1 name2 ... +.TE +where the conditions may be named in any order. +The word \fIStart\fR may be abbreviated to \fIs\fR or \fIS\fR. +The conditions may be referenced at the +head of a rule with the <> brackets: +.TS +center; +l. +<name1>expression +.TE +is a rule which is only recognized when Lex is in the +start condition \fIname1\fR. +To enter a start condition, +execute the action statement +.TS +center; +l. +BEGIN name1; +.TE +which changes the start condition to \fIname1\fR. +To resume the normal state, +.TS +center; +l. +BEGIN 0; +.TE +resets the initial condition +of the Lex automaton interpreter. +A rule may be active in several +start conditions: +.TS +center; +l. +<name1,name2,name3> +.TE +is a legal prefix. Any rule not beginning with the +<> prefix operator is always active. +.PP +The same example as before can be written: +.TS +center; +l l. +%START AA BB CC +%% +^a {ECHO; BEGIN AA;} +^b {ECHO; BEGIN BB;} +^c {ECHO; BEGIN CC;} +\en {ECHO; BEGIN 0;} +<AA>magic printf("first"); +<BB>magic printf("second"); +<CC>magic printf("third"); +.TE +where the logic is exactly the same as in the previous +method of handling the problem, but Lex does the work +rather than the user's code. +.2C +.NH +Character Set. +.PP +The programs generated by Lex handle +character I/O only through the routines +.I +input, +output, +.R +and +.I +unput. +.R +Thus the character representation +provided in these routines +is accepted by Lex and employed to return +values in +.I +yytext. +.R +For internal use +a character is represented as a small integer +which, if the standard library is used, +has a value equal to the integer value of the bit +pattern representing the character on the host computer. +Normally, the letter +.I a +is represented as the same form as the character constant +.I \(fma\(fm . +If this interpretation is changed, by providing I/O +routines which translate the characters, +Lex must be told about +it, by giving a translation table. +This table must be in the definitions section, +and must be bracketed by lines containing only +``%T''. +The table contains lines of the form +.TS +center; +l. +{integer} {character string} +.TE +which indicate the value associated with each character. +Thus the next example +.\" .GS 2 +.TS +center; +l l. +%T + 1 Aa + 2 Bb +\&... +26 Zz +27 \en +28 + +29 \- +30 0 +31 1 +\&... +39 9 +%T +.TE +.sp +.ce 1 +Sample character table. +.\" .GE +maps the lower and upper case letters together into the integers 1 through 26, +newline into 27, + and \- into 28 and 29, and the +digits into 30 through 39. +Note the escape for newline. +If a table is supplied, every character that is to appear either +in the rules or in any valid input must be included +in the table. +No character +may be assigned the number 0, and no character may be +assigned a bigger number than the size of the hardware character set. +.2C +.NH +Summary of Source Format. +.PP +The general form of a Lex source file is: +.TS +center; +l. +{definitions} +%% +{rules} +%% +{user subroutines} +.TE +The definitions section contains +a combination of +.IP 1) +Definitions, in the form ``name space translation''. +.IP 2) +Included code, in the form ``space code''. +.IP 3) +Included code, in the form +.TS +center; +l. +%{ +code +%} +.TE +.ns +.IP 4) +Start conditions, given in the form +.TS +center; +l. +%S name1 name2 ... +.TE +.ns +.IP 5) +Character set tables, in the form +.TS +center; +l. +%T +number space character-string +\&... +%T +.TE +.ns +.IP 6) +Changes to internal array sizes, in the form +.TS +center; +l. +%\fIx\fR\0\0\fInnn\fR +.TE +where \fInnn\fR is a decimal integer representing an array size +and \fIx\fR selects the parameter as follows: +.TS +center; +c c +c l. +Letter Parameter +p positions +n states +e tree nodes +a transitions +k packed character classes +o output array size +.TE +.LP +Lines in the rules section have the form ``expression action'' +where the action may be continued on succeeding +lines by using braces to delimit it. +.PP +Regular expressions in Lex use the following +operators: +.br +.TS +center; +l l. +x the character "x" +"x" an "x", even if x is an operator. +\ex an "x", even if x is an operator. +[xy] the character x or y. +[x\-z] the characters x, y or z. +[^x] any character but x. +\&. any character but newline. +^x an x at the beginning of a line. +<y>x an x when Lex is in start condition y. +x$ an x at the end of a line. +x? an optional x. +x\(** 0,1,2, ... instances of x. +x+ 1,2,3, ... instances of x. +x|y an x or a y. +(x) an x. +x/y an x but only if followed by y. +{xx} the translation of xx from the + definitions section. +x{m,n} \fIm\fR through \fIn\fR occurrences of x +.TE +.NH +Caveats and Bugs. +.PP +There are pathological expressions which +produce exponential growth of the tables when +converted to deterministic machines; +fortunately, they are rare. +.PP +REJECT does not rescan the input; instead it remembers the results of the previous +scan. This means that if a rule with trailing context is found, and +REJECT executed, the user +must not have used +.ul +unput +to change the characters forthcoming +from the input stream. +This is the only restriction on the user's ability to manipulate +the not-yet-processed input. +.PP +.2C +.NH +Acknowledgments. +.PP +As should +be obvious from the above, the outside of Lex +is patterned +on Yacc and the inside on Aho's string matching routines. +Therefore, both S. C. Johnson and A. V. Aho +are really originators +of much of Lex, +as well as debuggers of it. +Many thanks are due to both. +.PP +The code of the current version of Lex was designed, written, +and debugged by Eric Schmidt. +.if 0 .SG MH-1274-MEL-unix +.sp 1 +.2C +.NH +References. +.sp 1v +.IP 1. +B. W. Kernighan and D. M. Ritchie, +.I +The C Programming Language, +.R +Prentice-Hall, N. J. (1978). +.IP 2. +B. W. Kernighan, +.I +Ratfor: A Preprocessor for a Rational Fortran, +.R +Software \- Practice and Experience, +\fB5\fR, pp. 395-496 (1975). +.IP 3. +S. C. Johnson, +.I +Yacc: Yet Another Compiler Compiler, +.R +Computing Science Technical Report No. 32, +1975, +.MH +.if \n(tm (also TM 75-1273-6) +.IP 4. +A. V. Aho and M. J. Corasick, +.I +Efficient String Matching: An Aid to Bibliographic Search, +.R +Comm. ACM +.B +18, +.R +333-340 (1975). +.IP 5. +B. W. Kernighan, D. M. Ritchie and K. L. Thompson, +.I +QED Text Editor, +.R +Computing Science Technical Report No. 5, +1972, +.MH +.IP 6. +D. M. Ritchie, +private communication. +See also +M. E. Lesk, +.I +The Portable C Library, +.R +Computing Science Technical Report No. 31, +.MH +.if \n(tm (also TM 75-1274-11) diff --git a/share/doc/psd/17.m4/Makefile b/share/doc/psd/17.m4/Makefile new file mode 100644 index 000000000000..c48921f09a15 --- /dev/null +++ b/share/doc/psd/17.m4/Makefile @@ -0,0 +1,8 @@ +# @(#)Makefile 8.1 (Berkeley) 6/8/93 +# $FreeBSD$ + +VOLUME= psd/17.m4 +SRCS= m4.ms +MACROS= -ms + +.include <bsd.doc.mk> diff --git a/share/doc/psd/17.m4/m4.ms b/share/doc/psd/17.m4/m4.ms new file mode 100644 index 000000000000..c7a2fd941a86 --- /dev/null +++ b/share/doc/psd/17.m4/m4.ms @@ -0,0 +1,973 @@ +.\" Copyright (C) Caldera International Inc. 2001-2002. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions are +.\" met: +.\" +.\" Redistributions of source code and documentation must retain the above +.\" copyright notice, this list of conditions and the following +.\" disclaimer. +.\" +.\" Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" +.\" All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" +.\" This product includes software developed or owned by Caldera +.\" International, Inc. Neither the name of Caldera International, Inc. +.\" nor the names of other contributors may be used to endorse or promote +.\" products derived from this software without specific prior written +.\" permission. +.\" +.\" USE OF THE SOFTWARE PROVIDED FOR UNDER THIS LICENSE BY CALDERA +.\" INTERNATIONAL, INC. AND CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR +.\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +.\" WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +.\" DISCLAIMED. IN NO EVENT SHALL CALDERA INTERNATIONAL, INC. BE LIABLE +.\" FOR ANY DIRECT, INDIRECT INCIDENTAL, SPECIAL, EXEMPLARY, OR +.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR +.\" BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, +.\" WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE +.\" OR OTHERWISE) RISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN +.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +.\" +.\" @(#)m4.ms 6.3 (Berkeley) 6/5/93 +.\" +.\" $FreeBSD$ +.de MH +Bell Laboratories, Murray Hill, NJ 07974. +.. +.EH 'PSD:17-%''The M4 Macro Processor' +.OH 'The M4 Macro Processor''PSD:17-%' +.if n .ls 2 +.tr _\(em +.tr *\(** +.de UC +\&\\$3\s-1\\$1\\s0\&\\$2 +.. +.de IT +.if n .ul +\&\\$3\f2\\$1\fP\&\\$2 +.. +.de UL +.if n .ul +\&\\$3\f3\\$1\fP\&\\$2 +.. +.de P1 +.DS I 3n +.if n .ls 2 +.nf +.if n .ta 5 10 15 20 25 30 35 40 45 50 55 60 +.if t .ta .4i .8i 1.2i 1.6i 2i 2.4i 2.8i 3.2i 3.6i 4i 4.4i 4.8i 5.2i 5.6i +.if t .tr -\(mi|\(bv'\(fm^\(no*\(** +.tr `\(ga'\(aa +.if t .tr _\(ul +.ft 3 +.lg 0 +.. +.de P2 +.ps \\n(PS +.vs \\n(VSp +.ft R +.if n .ls 2 +.tr --||''^^!! +.if t .tr _\(em +.fi +.lg +.DE +.if t .tr _\(em +.. +.hw semi-colon +.hw estab-lished +.hy 14 +. \"2=not last lines; 4= no -xx; 8=no xx- +. \"special chars in programs +. \" start of text +.\".RP +.\" .....TR 59 +.\" .....TM 77-1273-6 39199 39199-11 +.ND "July 1, 1977" +.TL +The M4 Macro Processor +.AU "MH 2C-518" 6021 +Brian W. Kernighan +.AU "MH 2C-517" 3770 +Dennis M. Ritchie +.AI +.MH +.AB +.PP +M4 is a macro processor available on +.UX +and +.UC GCOS . +Its primary use has been as a +front end for Ratfor for those +cases where parameterless macros +are not adequately powerful. +It has also been used for languages as disparate as C and Cobol. +M4 is particularly suited for functional languages like Fortran, PL/I and C +since macros are specified in a functional notation. +.PP +M4 provides features seldom found even in much larger +macro processors, +including +.IP " \(bu" +arguments +.IP " \(bu" +condition testing +.IP " \(bu" +arithmetic capabilities +.IP " \(bu" +string and substring functions +.IP " \(bu" +file manipulation +.LP +.PP +This paper is a user's manual for M4. +.AE +.\" .CS 6 0 6 0 0 1 +.if t .2C +.SH +Introduction +.PP +A macro processor is a useful way to enhance a programming language, +to make it more palatable +or more readable, +or to tailor it to a particular application. +The +.UL #define +statement in C +and the analogous +.UL define +in Ratfor +are examples of the basic facility provided by +any macro processor _ +replacement of text by other text. +.PP +The M4 macro processor is an extension of a macro processor called M3 +which was written by D. M. Ritchie +for the AP-3 minicomputer; +M3 was in turn based on a macro processor implemented for [1]. +Readers unfamiliar with the basic ideas of macro processing +may wish to read some of the discussion there. +.PP +M4 is a suitable front end for Ratfor and C, +and has also been used successfully with Cobol. +Besides the straightforward replacement of one string of text by another, +it provides +macros with arguments, +conditional macro expansion, +arithmetic, +file manipulation, +and some specialized string processing functions. +.PP +The basic operation of M4 +is to copy its input to its output. +As the input is read, however, each alphanumeric ``token'' +(that is, string of letters and digits) is checked. +If it is the name of a macro, +then the name of the macro is replaced by its defining text, +and the resulting string is pushed back onto the +input to be rescanned. +Macros may be called with arguments, in which case the arguments are collected +and substituted into the right places in the defining text +before it is rescanned. +.PP +M4 provides a collection of about twenty built-in +macros +which perform various useful operations; +in addition, the user can define new macros. +Built-ins and user-defined macros work exactly the same way, except that +some of the built-in macros have side effects +on the state of the process. +.SH +Usage +.PP +On +.UC UNIX , +use +.P1 +m4 [files] +.P2 +Each argument file is processed in order; +if there are no arguments, or if an argument +is `\-', +the standard input is read at that point. +The processed text is written on the standard output, +which may be captured for subsequent processing with +.P1 +m4 [files] >outputfile +.P2 +On +.UC GCOS , +usage is identical, but the program is called +.UL \&./m4 . +.SH +Defining Macros +.PP +The primary built-in function of M4 +is +.UL define , +which is used to define new macros. +The input +.P1 +define(name, stuff) +.P2 +causes the string +.UL name +to be defined as +.UL stuff . +All subsequent occurrences of +.UL name +will be replaced by +.UL stuff . +.UL name +must be alphanumeric and must begin with a letter +(the underscore \(ul counts as a letter). +.UL stuff +is any text that contains balanced parentheses; +it may stretch over multiple lines. +.PP +Thus, as a typical example, +.P1 +define(N, 100) + ... +if (i > N) +.P2 +defines +.UL N +to be 100, and uses this ``symbolic constant'' in a later +.UL if +statement. +.PP +The left parenthesis must immediately follow the word +.UL define , +to signal that +.UL define +has arguments. +If a macro or built-in name is not followed immediately by `(', +it is assumed to have no arguments. +This is the situation for +.UL N +above; +it is actually a macro with no arguments, +and thus when it is used there need be no (...) following it. +.PP +You should also notice that a macro name is only recognized as such +if it appears surrounded by non-alphanumerics. +For example, in +.P1 +define(N, 100) + ... +if (NNN > 100) +.P2 +the variable +.UL NNN +is absolutely unrelated to the defined macro +.UL N , +even though it contains a lot of +.UL N 's. +.PP +Things may be defined in terms of other things. +For example, +.P1 +define(N, 100) +define(M, N) +.P2 +defines both M and N to be 100. +.PP +What happens if +.UL N +is redefined? +Or, to say it another way, is +.UL M +defined as +.UL N +or as 100? +In M4, +the latter is true _ +.UL M +is 100, so even if +.UL N +subsequently changes, +.UL M +does not. +.PP +This behavior arises because +M4 expands macro names into their defining text as soon as it possibly can. +Here, that means that when the string +.UL N +is seen as the arguments of +.UL define +are being collected, it is immediately replaced by 100; +it's just as if you had said +.P1 +define(M, 100) +.P2 +in the first place. +.PP +If this isn't what you really want, there are two ways out of it. +The first, which is specific to this situation, +is to interchange the order of the definitions: +.P1 +define(M, N) +define(N, 100) +.P2 +Now +.UL M +is defined to be the string +.UL N , +so when you ask for +.UL M +later, you'll always get the value of +.UL N +at that time +(because the +.UL M +will be replaced by +.UL N +which will be replaced by 100). +.SH +Quoting +.PP +The more general solution is to delay the expansion of +the arguments of +.UL define +by +.ul +quoting +them. +Any text surrounded by the single quotes \(ga and \(aa +is not expanded immediately, but has the quotes stripped off. +If you say +.P1 +define(N, 100) +define(M, `N') +.P2 +the quotes around the +.UL N +are stripped off as the argument is being collected, +but they have served their purpose, and +.UL M +is defined as +the string +.UL N , +not 100. +The general rule is that M4 always strips off +one level of single quotes whenever it evaluates +something. +This is true even outside of +macros. +If you want the word +.UL define +to appear in the output, +you have to quote it in the input, +as in +.P1 + `define' = 1; +.P2 +.PP +As another instance of the same thing, which is a bit more surprising, +consider redefining +.UL N : +.P1 +define(N, 100) + ... +define(N, 200) +.P2 +Perhaps regrettably, the +.UL N +in the second definition is +evaluated as soon as it's seen; +that is, it is +replaced by +100, so it's as if you had written +.P1 +define(100, 200) +.P2 +This statement is ignored by M4, since you can only define things that look +like names, but it obviously doesn't have the effect you wanted. +To really redefine +.UL N , +you must delay the evaluation by quoting: +.P1 +define(N, 100) + ... +define(`N', 200) +.P2 +In M4, +it is often wise to quote the first argument of a macro. +.PP +If \` and \' are not convenient for some reason, +the quote characters can be changed with the built-in +.UL changequote : +.P1 +changequote([, ]) +.P2 +makes the new quote characters the left and right brackets. +You can restore the original characters with just +.P1 +changequote +.P2 +.PP +There are two additional built-ins related to +.UL define . +.UL undefine +removes the definition of some macro or built-in: +.P1 +undefine(`N') +.P2 +removes the definition of +.UL N . +(Why are the quotes absolutely necessary?) +Built-ins can be removed with +.UL undefine , +as in +.P1 +undefine(`define') +.P2 +but once you remove one, you can never get it back. +.PP +The built-in +.UL ifdef +provides a way to determine if a macro is currently defined. +In particular, M4 has pre-defined the names +.UL unix +and +.UL gcos +on the corresponding systems, so you can +tell which one you're using: +.P1 +ifdef(`unix', `define(wordsize,16)' ) +ifdef(`gcos', `define(wordsize,36)' ) +.P2 +makes a definition appropriate for the particular machine. +Don't forget the quotes! +.PP +.UL ifdef +actually permits three arguments; +if the name is undefined, the value of +.UL ifdef +is then the third argument, as in +.P1 +ifdef(`unix', on UNIX, not on UNIX) +.P2 +.SH +Arguments +.PP +So far we have discussed the simplest form of macro processing _ +replacing one string by another (fixed) string. +User-defined macros may also have arguments, so different invocations +can have different results. +Within the replacement text for a macro +(the second argument of its +.UL define ) +any occurrence of +.UL $n +will be replaced by the +.UL n th +argument when the macro +is actually used. +Thus, the macro +.UL bump , +defined as +.P1 +define(bump, $1 = $1 + 1) +.P2 +generates code to increment its argument by 1: +.P1 +bump(x) +.P2 +is +.P1 +x = x + 1 +.P2 +.PP +A macro can have as many arguments as you want, +but only the first nine are accessible, +through +.UL $1 +to +.UL $9 . +(The macro name itself is +.UL $0 , +although that is less commonly used.) +Arguments that are not supplied are replaced by null strings, +so +we can define a macro +.UL cat +which simply concatenates its arguments, like this: +.P1 +define(cat, $1$2$3$4$5$6$7$8$9) +.P2 +Thus +.P1 +cat(x, y, z) +.P2 +is equivalent to +.P1 +xyz +.P2 +.UL $4 +through +.UL $9 +are null, since no corresponding arguments were provided. +.PP +.PP +Leading unquoted blanks, tabs, or newlines that occur during argument collection +are discarded. +All other white space is retained. +Thus +.P1 +define(a, b c) +.P2 +defines +.UL a +to be +.UL b\ \ \ c . +.PP +Arguments are separated by commas, but parentheses are counted properly, +so a comma ``protected'' by parentheses does not terminate an argument. +That is, in +.P1 +define(a, (b,c)) +.P2 +there are only two arguments; +the second is literally +.UL (b,c) . +And of course a bare comma or parenthesis can be inserted by quoting it. +.SH +Arithmetic Built-ins +.PP +M4 provides two built-in functions for doing arithmetic +on integers (only). +The simplest is +.UL incr , +which increments its numeric argument by 1. +Thus to handle the common programming situation +where you want a variable to be defined as ``one more than N'', +write +.P1 +define(N, 100) +define(N1, `incr(N)') +.P2 +Then +.UL N1 +is defined as one more than the current value of +.UL N . +.PP +The more general mechanism for arithmetic is a built-in +called +.UL eval , +which is capable of arbitrary arithmetic on integers. +It provides the operators +(in decreasing order of precedence) +.DS +unary + and \(mi +** or ^ (exponentiation) +* / % (modulus) ++ \(mi +== != < <= > >= +! (not) +& or && (logical and) +\(or or \(or\(or (logical or) +.DE +Parentheses may be used to group operations where needed. +All the operands of +an expression given to +.UL eval +must ultimately be numeric. +The numeric value of a true relation +(like 1>0) +is 1, and false is 0. +The precision in +.UL eval +is +32 bits on +.UC UNIX +and 36 bits on +.UC GCOS . +.PP +As a simple example, suppose we want +.UL M +to be +.UL 2**N+1 . +Then +.P1 +define(N, 3) +define(M, `eval(2**N+1)') +.P2 +As a matter of principle, it is advisable +to quote the defining text for a macro +unless it is very simple indeed +(say just a number); +it usually gives the result you want, +and is a good habit to get into. +.SH +File Manipulation +.PP +You can include a new file in the input at any time by +the built-in function +.UL include : +.P1 +include(filename) +.P2 +inserts the contents of +.UL filename +in place of the +.UL include +command. +The contents of the file is often a set of definitions. +The value +of +.UL include +(that is, its replacement text) +is the contents of the file; +this can be captured in definitions, etc. +.PP +It is a fatal error if the file named in +.UL include +cannot be accessed. +To get some control over this situation, the alternate form +.UL sinclude +can be used; +.UL sinclude +(``silent include'') +says nothing and continues if it can't access the file. +.PP +It is also possible to divert the output of M4 to temporary files during processing, +and output the collected material upon command. +M4 maintains nine of these diversions, numbered 1 through 9. +If you say +.P1 +divert(n) +.P2 +all subsequent output is put onto the end of a temporary file +referred to as +.UL n . +Diverting to this file is stopped by another +.UL divert +command; +in particular, +.UL divert +or +.UL divert(0) +resumes the normal output process. +.PP +Diverted text is normally output all at once +at the end of processing, +with the diversions output in numeric order. +It is possible, however, to bring back diversions +at any time, +that is, to append them to the current diversion. +.P1 +undivert +.P2 +brings back all diversions in numeric order, and +.UL undivert +with arguments brings back the selected diversions +in the order given. +The act of undiverting discards the diverted stuff, +as does diverting into a diversion +whose number is not between 0 and 9 inclusive. +.PP +The value of +.UL undivert +is +.ul +not +the diverted stuff. +Furthermore, the diverted material is +.ul +not +rescanned for macros. +.PP +The built-in +.UL divnum +returns the number of the currently active diversion. +This is zero during normal processing. +.SH +System Command +.PP +You can run any program in the local operating system +with the +.UL syscmd +built-in. +For example, +.P1 +syscmd(date) +.P2 +on +.UC UNIX +runs the +.UL date +command. +Normally +.UL syscmd +would be used to create a file +for a subsequent +.UL include . +.PP +To facilitate making unique file names, the built-in +.UL maketemp +is provided, with specifications identical to the system function +.ul +mktemp: +a string of XXXXX in the argument is replaced +by the process id of the current process. +.SH +Conditionals +.PP +There is a built-in called +.UL ifelse +which enables you to perform arbitrary conditional testing. +In the simplest form, +.P1 +ifelse(a, b, c, d) +.P2 +compares the two strings +.UL a +and +.UL b . +If these are identical, +.UL ifelse +returns +the string +.UL c ; +otherwise it returns +.UL d . +Thus we might define a macro called +.UL compare +which compares two strings and returns ``yes'' or ``no'' +if they are the same or different. +.P1 +define(compare, `ifelse($1, $2, yes, no)') +.P2 +Note the quotes, +which prevent too-early evaluation of +.UL ifelse . +.PP +If the fourth argument is missing, it is treated as empty. +.PP +.UL ifelse +can actually have any number of arguments, +and thus provides a limited form of multi-way decision capability. +In the input +.P1 +ifelse(a, b, c, d, e, f, g) +.P2 +if the string +.UL a +matches the string +.UL b , +the result is +.UL c . +Otherwise, if +.UL d +is the same as +.UL e , +the result is +.UL f . +Otherwise the result is +.UL g . +If the final argument +is omitted, the result is null, +so +.P1 +ifelse(a, b, c) +.P2 +is +.UL c +if +.UL a +matches +.UL b , +and null otherwise. +.SH +String Manipulation +.PP +The built-in +.UL len +returns the length of the string that makes up its argument. +Thus +.P1 +len(abcdef) +.P2 +is 6, and +.UL len((a,b)) +is 5. +.PP +The built-in +.UL substr +can be used to produce substrings of strings. +.UL substr(s,\ i,\ n) +returns the substring of +.UL s +that starts at the +.UL i th +position +(origin zero), +and is +.UL n +characters long. +If +.UL n +is omitted, the rest of the string is returned, +so +.P1 +substr(`now is the time', 1) +.P2 +is +.P1 +ow is the time +.P2 +If +.UL i +or +.UL n +are out of range, various sensible things happen. +.PP +.UL index(s1,\ s2) +returns the index (position) in +.UL s1 +where the string +.UL s2 +occurs, or \-1 +if it doesn't occur. +As with +.UL substr , +the origin for strings is 0. +.PP +The built-in +.UL translit +performs character transliteration. +.P1 +translit(s, f, t) +.P2 +modifies +.UL s +by replacing any character found in +.UL f +by the corresponding character of +.UL t . +That is, +.P1 +translit(s, aeiou, 12345) +.P2 +replaces the vowels by the corresponding digits. +If +.UL t +is shorter than +.UL f , +characters which don't have an entry in +.UL t +are deleted; as a limiting case, +if +.UL t +is not present at all, +characters from +.UL f +are deleted from +.UL s . +So +.P1 +translit(s, aeiou) +.P2 +deletes vowels from +.UL s . +.PP +There is also a built-in called +.UL dnl +which deletes all characters that follow it up to +and including the next newline; +it is useful mainly for throwing away +empty lines that otherwise tend to clutter up M4 output. +For example, if you say +.P1 +define(N, 100) +define(M, 200) +define(L, 300) +.P2 +the newline at the end of each line is not part of the definition, +so it is copied into the output, where it may not be wanted. +If you add +.UL dnl +to each of these lines, the newlines will disappear. +.PP +Another way to achieve this, due to J. E. Weythman, +is +.P1 +divert(-1) + define(...) + ... +divert +.P2 +.SH +Printing +.PP +The built-in +.UL errprint +writes its arguments out on the standard error file. +Thus you can say +.P1 +errprint(`fatal error') +.P2 +.PP +.UL dumpdef +is a debugging aid which +dumps the current definitions of defined terms. +If there are no arguments, you get everything; +otherwise you get the ones you name as arguments. +Don't forget to quote the names! +.SH +Summary of Built-ins +.PP +Each entry is preceded by the +page number where it is described. +.DS +.tr '\'`\` +.ta .25i +3 changequote(L, R) +1 define(name, replacement) +4 divert(number) +4 divnum +5 dnl +5 dumpdef(`name', `name', ...) +5 errprint(s, s, ...) +4 eval(numeric expression) +3 ifdef(`name', this if true, this if false) +5 ifelse(a, b, c, d) +4 include(file) +3 incr(number) +5 index(s1, s2) +5 len(string) +4 maketemp(...XXXXX...) +4 sinclude(file) +5 substr(string, position, number) +4 syscmd(s) +5 translit(str, from, to) +3 undefine(`name') +4 undivert(number,number,...) +.DE +.SH +Acknowledgements +.PP +We are indebted to Rick Becker, John Chambers, +Doug McIlroy, +and especially Jim Weythman, +whose pioneering use of M4 has led to several valuable improvements. +We are also deeply grateful to Weythman for several substantial contributions +to the code. +.\" .SG +.SH +References +.LP +.IP [1] +B. W. Kernighan and P. J. Plauger, +.ul +Software Tools, +Addison-Wesley, Inc., 1976. diff --git a/share/doc/psd/18.gprof/Makefile b/share/doc/psd/18.gprof/Makefile new file mode 100644 index 000000000000..2a5bed166b17 --- /dev/null +++ b/share/doc/psd/18.gprof/Makefile @@ -0,0 +1,14 @@ +# From: @(#)Makefile 8.1 (Berkeley) 8/14/93 +# $FreeBSD$ + +VOLUME= psd/18.gprof +SRCS= header.me abstract.me intro.me profiling.me gathering.me \ + postp.me present.me refs.me +EXTRA= postp1.pic postp2.pic postp3.pic pres1.pic pres2.pic +MACROS= -me +USE_SOELIM= +USE_PIC= +USE_TBL= +USE_EQN= + +.include <bsd.doc.mk> diff --git a/share/doc/psd/18.gprof/abstract.me b/share/doc/psd/18.gprof/abstract.me new file mode 100644 index 000000000000..28e8066fc331 --- /dev/null +++ b/share/doc/psd/18.gprof/abstract.me @@ -0,0 +1,66 @@ +.\" Copyright (c) 1982, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)abstract.me 8.1 (Berkeley) 6/8/93 +.\" +.sp 1 +\fB\s+2gprof: a Call Graph Execution Profiler\s-2\fP\** +.(f +\**This work was supported by grant MCS80-05144 +from the National Science Foundation. +.)f +.sp 1 +by +\fISusan L. Graham\fP +\fIPeter B. Kessler\fP +\fIMarshall K. McKusick\fP +.sp 1 +Computer Science Division +Electrical Engineering and Computer Science Department +University of California, Berkeley +Berkeley, California 94720 +.ce 0 +.sp 1 +.sp 0.5i +.sh 0 "Abstract" +.pp +Large complex programs are composed of many small routines +that implement abstractions for the routines that call them. +To be useful, an execution profiler must attribute +execution time in a way that is significant for the +logical structure of a program +as well as for its textual decomposition. +This data must then be displayed to the user +in a convenient and informative way. +The \fBgprof\fP profiler +accounts for the running time of called routines +in the running time of the routines that call them. +The design and use of this profiler is described. diff --git a/share/doc/psd/18.gprof/gathering.me b/share/doc/psd/18.gprof/gathering.me new file mode 100644 index 000000000000..17130c330e68 --- /dev/null +++ b/share/doc/psd/18.gprof/gathering.me @@ -0,0 +1,231 @@ +.\" Copyright (c) 1982, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)gathering.me 8.1 (Berkeley) 6/8/93 +.\" +.sh 1 "Gathering Profile Data" +.pp +Routine calls or statement executions can be measured by having a +compiler augment the code at strategic points. +The additions can be inline increments to counters [Knuth71] +[Satterthwaite72] [Joy79] or calls to +monitoring routines [Unix]. +The counter increment overhead is low, and is suitable for +profiling statements. +A call of the monitoring routine has an overhead comparable with a +call of a regular routine, and is therefore only suited +to profiling on a routine by routine basis. +However, the monitoring routine solution has certain advantages. +Whatever counters are needed by the monitoring routine can be +managed by the monitoring routine itself, rather than being +distributed around the code. +In particular, a monitoring routine can easily be called from separately +compiled programs. +In addition, different monitoring routines can be linked into the +program +being measured +to assemble different profiling data without having to +change the compiler or recompile the program. +We have exploited this approach; +our compilers for C, Fortran77, and Pascal can insert calls to a +monitoring routine in the prologue for each routine. +Use of the monitoring routine requires no planning on part of a +programmer other than to request that augmented routine +prologues be produced during compilation. +.pp +We are interested in gathering three pieces of information during +program execution: call counts and execution times for +each profiled routine, and the arcs of the dynamic call graph +traversed by this execution of the program. +By post-processing of this data we can build the dynamic call +graph for this execution of the program and propagate times along +the edges of this graph to attribute times for routines to the +routines that invoke them. +.pp +Gathering of the profiling information should not greatly +interfere with the running of the program. +Thus, the monitoring routine must not produce trace output each +time it is invoked. +The volume of data thus produced would be unmanageably large, +and the time required to record it would overwhelm the running +time of most programs. +Similarly, the monitoring routine can not do the analysis of +the profiling data (e.g. assembling the call graph, propagating +times around it, discovering cycles, etc.) during program +execution. +Our solution is to gather profiling data in memory during program +execution and to condense it to a file as the profiled +program exits. +This file is then processed by a separate program to produce the +listing of the profile data. +An advantage of this approach is that the profile data for +several executions of a program can be combined by the +post-processing to provide a profile of many +executions. +.pp +The execution time monitoring consists of three parts. +The first part allocates and initializes the runtime monitoring data +structures before the program begins execution. +The second part is the monitoring routine invoked from the +prologue of each profiled routine. +The third part condenses the data structures and writes them +to a file as the program terminates. +The monitoring routine is discussed in detail in the following sections. +.sh 2 "Execution Counts" +.pp +The \fBgprof\fP monitoring routine counts the number of times +each profiled routine is called. +The monitoring routine also records the arc in the call graph +that activated the profiled routine. +The count is associated with the arc in the call graph +rather than with the routine. +Call counts for routines can then be determined by summing the counts +on arcs directed into that routine. +In a machine-dependent fashion, the monitoring routine notes its +own return address. +This address is in the prologue of some profiled routine that is +the destination of an arc in the dynamic call graph. +The monitoring routine also discovers the return address for that +routine, thus identifying the call site, or source of the arc. +The source of the arc is in the \fIcaller\fP, and the destination is in +the \fIcallee\fP. +For example, if a routine A calls a routine B, A is the caller, +and B is the callee. +The prologue of B will include a call to the monitoring routine +that will note the arc from A to B and either initialize or +increment a counter for that arc. +.pp +One can not afford to have the monitoring routine output tracing +information as each arc is identified. +Therefore, the monitoring routine maintains a table of all the +arcs discovered, with counts of the numbers of times each is +traversed during execution. +This table is accessed once per routine call. +Access to it +must be as fast as possible so as not to overwhelm the time +required to execute the program. +.pp +Our solution is to access the table through a hash table. +We use the call site as the primary key with the callee +address being the secondary key. +Since each call site typically calls only one callee, we can +reduce (usually to one) the number of minor lookups based on the callee. +Another alternative would use the callee as the primary key and the +call site as the secondary key. +Such an organization has the advantage of associating callers with +callees, at the expense of longer lookups in the monitoring +routine. +We are fortunate to be running in a virtual memory environment, +and (for the sake of speed) were able to allocate enough space +for the primary hash table to allow a one-to-one mapping from +call site addresses to the primary hash table. +Thus our hash function is trivial to calculate and collisions +occur only for call sites that call multiple +destinations (e.g. functional parameters and functional variables). +A one level hash function using both call site and callee would +result in an unreasonably large hash table. +Further, the number of dynamic call sites and callees is not known during +execution of the profiled program. +.pp +Not all callers and callees can be identified by the monitoring +routine. +Routines that were compiled without the profiling augmentations +will not call the monitoring routine as part of their prologue, +and thus no arcs will be recorded whose destinations are in these +routines. +One need not profile all the routines in a program. +Routines that are not profiled run at full speed. +Certain routines, notably exception handlers, are invoked by +non-standard calling sequences. +Thus the monitoring routine may know the destination of an arc +(the callee), +but find it difficult or +impossible to determine the source of the arc (the caller). +Often in these cases the apparent source of the arc is not a call +site at all. +Such anomalous invocations are declared ``spontaneous''. +.sh 2 "Execution Times" +.pp +The execution times for routines can be gathered in at least two +ways. +One method measures the execution time of a routine by measuring +the elapsed time from routine entry to routine exit. +Unfortunately, time measurement is complicated on time-sharing +systems by the time-slicing of the program. +A second method samples the value of the program counter at some +interval, and infers execution time from the distribution of the +samples within the program. +This technique is particularly suited to time-sharing systems, +where the time-slicing can serve as the basis for sampling +the program counter. +Notice that, whereas the first method could provide exact timings, +the second is inherently a statistical approximation. +.pp +The sampling method need not require support from the operating +system: all that is needed is the ability to set and respond to +``alarm clock'' interrupts that run relative to program time. +It is imperative that the intervals be uniform since the +sampling of the program counter rather than the duration of the +interval is the basis of the distribution. +If sampling is done too often, the interruptions to sample the +program counter will overwhelm the running of the profiled program. +On the other hand, the program must run for enough sampled +intervals that the distribution of the samples accurately +represents the distribution of time for the execution of the +program. +As with routine call tracing, the monitoring routine can not +afford to output information for each program counter +sample. +In our computing environment, the operating system can provide a +histogram of the location of the program counter at the end of +each clock tick (1/60th of a second) in which a program runs. +The histogram is assembled in memory as the program runs. +This facility is enabled by our monitoring routine. +We have adjusted the granularity of the histogram so that +program counter values map one-to-one onto the histogram. +We make the simplifying assumption that all calls to a specific +routine require the same amount of time to execute. +This assumption may disguise that some calls +(or worse, some call sites) always invoke a routine +such that its execution is faster (or slower) +than the average time for that routine. +.pp +When the profiled program terminates, +the arc table and the histogram of +program counter samples is written to a file. +The arc table is condensed to consist of the source and destination +addresses of the arc and the count of the number of times the arc +was traversed by this execution of the program. +The recorded histogram consists of counters of the number of +times the program counter was found to be in each of the ranges covered +by the histogram. +The ranges themselves are summarized as a +lower and upper bound and a step size. diff --git a/share/doc/psd/18.gprof/header.me b/share/doc/psd/18.gprof/header.me new file mode 100644 index 000000000000..aef606d1fedd --- /dev/null +++ b/share/doc/psd/18.gprof/header.me @@ -0,0 +1,38 @@ +.\" Copyright (c) 1982, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)header.me 8.1 (Berkeley) 8/14/93 +.\" +.\"he 'gprof''Graham, Kessler, McKusick' +.\"fo 'Draft of \*(td''%' +.\"ls 2 +.eh 'PSD:18-%''gprof \*- a Call Graph Execution Profiler' +.oh 'gprof \*- A Call Graph Execution Profiler''PSD:18-%' diff --git a/share/doc/psd/18.gprof/intro.me b/share/doc/psd/18.gprof/intro.me new file mode 100644 index 000000000000..3a872b2e2f0c --- /dev/null +++ b/share/doc/psd/18.gprof/intro.me @@ -0,0 +1,81 @@ +.\" Copyright (c) 1982, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)intro.me 8.1 (Berkeley) 6/8/93 +.\" +.sh 1 "Programs to be Profiled" +.pp +Software research environments +normally include many large programs +both for production use and for experimental investigation. +These programs are typically modular, +in accordance with generally accepted principles +of good program design. +Often they consist of numerous small routines +that implement various abstractions. +Sometimes such large programs are written +by one programmer +who has understood the requirements for +these abstractions, and has programmed them +appropriately. +More frequently the program has +had multiple authors and has +evolved over time, changing the demands placed +on the implementation of the abstractions without +changing the implementation itself. +Finally, the program may be assembled from a library +of abstraction implementations +unexamined by the programmer. +.pp +Once a large program is executable, +it is often desirable to increase its speed, +especially if small portions of the program +are found to dominate its execution time. +The purpose of the \fBgprof\fP profiling tool is to +help the user evaluate alternative implementations +of abstractions. +We developed this tool in response to our efforts +to improve a code generator we were writing [Graham82]. +.pp +The \fBgprof\fP design takes advantage of the fact that the programs +to be measured are large, structured and hierarchical. +We provide a profile in which the execution time +for a set of routines that implement an +abstraction is collected and charged +to that abstraction. +The profile can be used to compare and assess the costs of +various implementations. +.pp +The profiler can be linked into a program without +special planning by the programmer. +The overhead for using \fBgprof\fP is low; +both in terms of added execution time and in the +volume of profiling information recorded. diff --git a/share/doc/psd/18.gprof/postp.me b/share/doc/psd/18.gprof/postp.me new file mode 100644 index 000000000000..d71fefb3e321 --- /dev/null +++ b/share/doc/psd/18.gprof/postp.me @@ -0,0 +1,190 @@ +.\" Copyright (c) 1982, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)postp.me 8.1 (Berkeley) 6/8/93 +.\" +.EQ +delim $$ +gsize 11 +.EN +.sh 1 "Post Processing" +.pp +Having gathered the arcs of the call graph and timing information +for an execution of the program, +we are interested in attributing the time for each routine to the +routines that call it. +We build a dynamic call graph with arcs from caller to callee, +and propagate time from descendants to ancestors +by topologically sorting the call graph. +Time propagation is performed from the leaves of the +call graph toward the roots, according to the order +assigned by a topological numbering algorithm. +The topological numbering ensures that +all edges in the graph go from higher numbered nodes to lower +numbered nodes. +An example is given in Figure 1. +If we propagate time from nodes in the +order assigned by the algorithm, +execution time can be propagated from descendants to ancestors +after a single traversal of each arc in the call graph. +Each parent receives some fraction of a child's time. +Thus time is charged to the +caller in addition to being charged to the callee. +.(z +.so postp1.pic +.ce 2 +Topological ordering +Figure 1. +.ce 0 +.)z +.pp +Let $C sub e$ be the number of calls to some routine, +$e$, and $C sub e sup r$ be the number of +calls from a caller $r$ to a callee $e$. +Since we are assuming each call to a routine takes the +average amount of time for all calls to that routine, +the caller is accountable for +$C sub e sup r / C sub e$ +of the time spent by the callee. +Let the $S sub e$ be the $selftime$ of a routine, $e$. +The selftime of a routine can be determined from the +timing information gathered during profiled program execution. +The total time, $T sub r$, we wish to account to a routine +$r$, is then given by the recurrence equation: +.EQ +T sub r ~ = ~ {S sub r} ~ + ~ + sum from {r ~ roman CALLS ~ e} + {T sub e times {{C sub e sup r} over {C sub e}}} +.EN +where $r ~ roman CALLS ~ e$ is a relation showing all routines +$e$ called by a routine $r$. +This relation is easily available from the call graph. +.pp +However, if the execution contains recursive calls, +the call graph has cycles that +cannot be topologically sorted. +In these cases, we discover strongly-connected +components in the call graph, +treat each such component as a single node, +and then sort the resulting graph. +We use a variation of Tarjan's strongly-connected +components algorithm +that discovers strongly-connected components as it is assigning +topological order numbers [Tarjan72]. +.pp +Time propagation within strongly connected +components is a problem. +For example, a self-recursive routine +(a trivial cycle in the call graph) +is accountable for all the time it +uses in all its recursive instantiations. +In our scheme, this time should be +shared among its call graph parents. +The arcs from a routine to itself are of interest, +but do not participate in time propagation. +Thus the simple equation for time propagation +does not work within strongly connected components. +Time is not propagated from one member of a cycle to another, +since, by definition, this involves propagating time from a routine +to itself. +In addition, children of one member of a cycle +must be considered children of all members of the cycle. +Similarly, parents of one member of the cycle must inherit +all members of the cycle as descendants. +It is for these reasons that we collapse connected components. +Our solution collects all members of a cycle together, +summing the time and call counts for all members. +All calls into the cycle are made to share the total +time of the cycle, and all descendants of the cycle +propagate time into the cycle as a whole. +Calls among the members of the cycle +do not propagate any time, +though they are listed in the call graph profile. +.pp +Figure 2 shows a modified version of the call graph of Figure 1, +in which the nodes labelled 3 and 7 in Figure 1 are mutually +recursive. +The topologically sorted graph after the cycle is collapsed is +given in Figure 3. +.(z +.so postp2.pic +.ce 2 +Cycle to be collapsed. +Figure 2. +.ce 0 +.)z +.(z +.so postp3.pic +.ce 2 +Topological numbering after cycle collapsing. +Figure 3. +.ce 0 +.)z +.pp +Since the technique described above only collects the +dynamic call graph, +and the program typically does not call every routine +on each execution, +different executions can introduce different cycles in the +dynamic call graph. +Since cycles often have a significant effect on time propagation, +it is desirable to incorporate the static call graph so that cycles +will have the same members regardless of how the program runs. +.pp +The static call graph can be constructed from the source text +of the program. +However, discovering the static call graph from the source text +would require two moderately difficult steps: +finding the source text for the program +(which may not be available), +and scanning and parsing that text, +which may be in any one of several languages. +.pp +In our programming system, +the static calling information is also contained in the +executable version of the program, +which we already have available, +and which is in language-independent form. +One can examine the instructions +in the object program, +looking for calls to routines, and note which +routines can be called. +This technique allows us to add arcs to those already in the +dynamic call graph. +If a statically discovered arc already exists in the dynamic call +graph, no action is required. +Statically discovered arcs that do not exist in the dynamic call +graph are added to the graph with a traversal count of zero. +Thus they are never responsible for any time propagation. +However, they may affect the structure of the graph. +Since they may complete strongly connected components, +the static call graph construction is +done before topological ordering. diff --git a/share/doc/psd/18.gprof/postp1.pic b/share/doc/psd/18.gprof/postp1.pic new file mode 100644 index 000000000000..1446092e877c --- /dev/null +++ b/share/doc/psd/18.gprof/postp1.pic @@ -0,0 +1,54 @@ +.\" Copyright (c) 1986, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)postp1.pic 8.1 (Berkeley) 6/8/93 +.\" +.PS +circle diam .3i "8" +circle diam .3i "9" at 1st circle + (2i,0i) +circle diam .3i "3" at 1st circle + (0.5i,-0.5i) +circle diam .3i "7" at 2nd circle - (0.5i, 0.5i) +circle diam .3i "2" at 1st circle - (0i,1i) +circle diam .3i "5" at 5th circle + (1i,0i) +circle diam .3i "6" at 2nd circle - (0i,1i) +circle diam .3i "1" at 3rd circle - (0i,1i) +circle diam .3i "4" at 4th circle - (0i,1i) +arrow from 1st circle to 3rd circle chop .15i chop .15i +arrow from 1st circle to 4th circle chop .15i chop .15i +arrow from 2nd circle to 4th circle chop .15i chop .15i +arrow from 3rd circle to 5th circle chop .15i chop .15i +arrow from 4th circle to 5th circle chop .15i chop .15i +arrow from 4th circle to 6th circle chop .15i chop .15i +arrow from 4th circle to 7th circle chop .15i chop .15i +arrow from 5th circle to 8th circle chop .15i chop .15i +arrow from 6th circle to 8th circle chop .15i chop .15i +arrow from 6th circle to 9th circle chop .15i chop .15i +.PE diff --git a/share/doc/psd/18.gprof/postp2.pic b/share/doc/psd/18.gprof/postp2.pic new file mode 100644 index 000000000000..3b31736e91a8 --- /dev/null +++ b/share/doc/psd/18.gprof/postp2.pic @@ -0,0 +1,56 @@ +.\" Copyright (c) 1986, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)postp2.pic 8.1 (Berkeley) 6/8/93 +.\" +.PS +circle diam .3i "\(ci" +circle diam .3i "\(ci" at 1st circle + (2i,0i) +circle diam .3i "\(bu" at 1st circle + (0.5i,-0.5i) +circle diam .3i "\(bu" at 2nd circle - (0.5i, 0.5i) +circle diam .3i "\(ci" at 1st circle - (0i,1i) +circle diam .3i "\(ci" at 5th circle + (1i,0i) +circle diam .3i "\(ci" at 2nd circle - (0i,1i) +circle diam .3i "\(ci" at 3rd circle - (0i,1i) +circle diam .3i "\(ci" at 4th circle - (0i,1i) +arrow from 1st circle to 3rd circle chop .15i chop .15i +arrow from 1st circle to 4th circle chop .15i chop .15i +arrow from 2nd circle to 4th circle chop .15i chop .15i +spline -> from 3rd circle right .5i up .075i then right .5i down .075i chop .15i chop .15i +spline -> from 4th circle left .5i down .075i then left .5i up .075i chop .15i chop .15i +arrow from 3rd circle to 5th circle chop .15i chop .15i +arrow from 4th circle to 5th circle chop .15i chop .15i +arrow from 4th circle to 6th circle chop .15i chop .15i +arrow from 4th circle to 7th circle chop .15i chop .15i +arrow from 5th circle to 8th circle chop .15i chop .15i +arrow from 6th circle to 8th circle chop .15i chop .15i +arrow from 6th circle to 9th circle chop .15i chop .15i +.PE diff --git a/share/doc/psd/18.gprof/postp3.pic b/share/doc/psd/18.gprof/postp3.pic new file mode 100644 index 000000000000..65eb2a78f51b --- /dev/null +++ b/share/doc/psd/18.gprof/postp3.pic @@ -0,0 +1,51 @@ +.\" Copyright (c) 1986, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)postp3.pic 8.1 (Berkeley) 6/8/93 +.\" +.PS +circle diam .3i "7" +circle diam .3i "8" at 1st circle + (2i,0i) +EL: ellipse wid 1i ht .3i "\fB6\fR\h'.7i'\fB6\fR" at 1st circle + (1i,-0.5i) +circle diam .3i "2" at 1st circle - (0i,1i) +circle diam .3i "4" at 3th circle + (1i,0i) +circle diam .3i "5" at 2nd circle - (0i,1i) +circle diam .3i "1" at 3rd circle + (0.5i,-0.5i) +circle diam .3i "3" at 5th circle - (0.5i,0.5i) +arrow from 1st circle to EL.nw chop .15i chop 0i +arrow from 2nd circle to EL.ne chop .15i chop 0i +arrow from EL.sw to 3rd circle chop 0i chop .15i +arrow from EL.s to 4th circle chop 0i chop .15i +arrow from EL.se to 5th circle chop 0i chop .15i +arrow from 3rd circle to 6th circle chop .15i chop .15i +arrow from 4th circle to 6th circle chop .15i chop .15i +arrow from 4th circle to 7th circle chop .15i chop .15i +.PE diff --git a/share/doc/psd/18.gprof/pres1.pic b/share/doc/psd/18.gprof/pres1.pic new file mode 100644 index 000000000000..0c311a19e0ee --- /dev/null +++ b/share/doc/psd/18.gprof/pres1.pic @@ -0,0 +1,56 @@ +.\" Copyright (c) 1986, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)pres1.pic 8.1 (Berkeley) 6/8/93 +.\" +.PS +ellipse ht .3i wid .75i "\s-1CALLER1\s+1" +ellipse ht .3i wid .75i "\s-1CALLER2\s+1" at 1st ellipse + (2i,0i) +ellipse ht .3i wid .8i "\s-1EXAMPLE\s+1" at 1st ellipse + (1i,-.5i) +ellipse ht .3i wid .5i "\s-1SUB1\s+1" at 1st ellipse - (0i,1i) +ellipse ht .3i wid .5i "\s-1SUB2\s+1" at 3rd ellipse - (0i,.5i) +ellipse ht .3i wid .5i "\s-1SUB3\s+1" at 2nd ellipse - (0i,1i) +line <- from 1st ellipse up .5i left .5i chop .1875i +line <- from 1st ellipse up .5i right .5i chop .1875i +line <- from 2nd ellipse up .5i left .5i chop .1875i +line <- from 2nd ellipse up .5i right .5i chop .1875i +arrow from 1st ellipse to 3rd ellipse chop +arrow from 2nd ellipse to 3rd ellipse chop +arrow from 3rd ellipse to 4th ellipse chop +arrow from 3rd ellipse to 5th ellipse chop .15i chop .15i +arrow from 3rd ellipse to 6th ellipse chop +arrow from 4th ellipse down .5i left .5i chop .1875i +arrow from 4th ellipse down .5i right .5i chop .1875i +arrow from 5th ellipse down .5i left .5i chop .1875i +arrow from 5th ellipse down .5i right .5i chop .1875i +arrow from 6th ellipse down .5i left .5i chop .1875i +arrow from 6th ellipse down .5i right .5i chop .1875i +.PE diff --git a/share/doc/psd/18.gprof/pres2.pic b/share/doc/psd/18.gprof/pres2.pic new file mode 100644 index 000000000000..c3a4ea069594 --- /dev/null +++ b/share/doc/psd/18.gprof/pres2.pic @@ -0,0 +1,52 @@ +.\" Copyright (c) 1986, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)pres2.pic 8.1 (Berkeley) 6/8/93 +.\" +.PS +ellipse ht .3i wid .6i "\s-1CALC1\s+1" +ellipse ht .3i wid .6i "\s-1CALC2\s+1" at 1st ellipse + (.75i,0i) +ellipse ht .3i wid .6i "\s-1CALC3\s+1" at 1st ellipse + (1.5i,0i) +ellipse ht .3i wid .8i "\s-1FORMAT1\s+1" at 1st ellipse - (0i,.5i) +ellipse ht .3i wid .8i "\s-1FORMAT2\s+1" at 3rd ellipse - (0i,.5i) +ellipse ht .3i wid .75i "\s-1\"WRITE\"\s+1" at 5th ellipse - (.75i,.5i) +line <- from 1st ellipse up .5i left .4i chop .1825i +line <- from 1st ellipse up .5i right .4i chop .1825i +line <- from 2nd ellipse up .5i left .4i chop .1825i +line <- from 2nd ellipse up .5i right .4i chop .1825i +line <- from 3rd ellipse up .5i left .4i chop .1825i +line <- from 3rd ellipse up .5i right .4i chop .1825i +arrow from 1st ellipse to 4th ellipse chop .15i +arrow from 2nd ellipse to 5th ellipse chop +arrow from 3rd ellipse to 5th ellipse chop .15i +arrow from 4th ellipse to 6th ellipse chop +arrow from 5th ellipse to 6th ellipse chop +.PE diff --git a/share/doc/psd/18.gprof/present.me b/share/doc/psd/18.gprof/present.me new file mode 100644 index 000000000000..1dd7f62ad812 --- /dev/null +++ b/share/doc/psd/18.gprof/present.me @@ -0,0 +1,306 @@ +.\" Copyright (c) 1982, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)present.me 8.1 (Berkeley) 6/8/93 +.\" +.sh 1 "Data Presentation" +.pp +The data is presented to the user in two different formats. +The first presentation simply lists the routines +without regard to the amount of time their descendants use. +The second presentation incorporates the call graph of the +program. +.sh 2 "The Flat Profile +.pp +The flat profile consists of a list of all the routines +that are called during execution of the program, +with the count of the number of times they are called +and the number of seconds of execution time for which they +are themselves accountable. +The routines are listed in decreasing order of execution time. +A list of the routines that are never called during execution of +the program is also available +to verify that nothing important is omitted by +this execution. +The flat profile gives a quick overview of the routines that are used, +and shows the routines that are themselves responsible +for large fractions of the execution time. +In practice, +this profile usually shows that no single function +is overwhelmingly responsible for +the total time of the program. +Notice that for this profile, +the individual times sum to the total execution time. +.sh 2 "The Call Graph Profile" +.sz 10 +.(z +.TS +box center; +c c c c c l l +c c c c c l l +c c c c c l l +l n n n c l l. + called/total \ \ parents +index %time self descendants called+self name index + called/total \ \ children +_ + 0.20 1.20 4/10 \ \ \s-1CALLER1\s+1 [7] + 0.30 1.80 6/10 \ \ \s-1CALLER2\s+1 [1] +[2] 41.5 0.50 3.00 10+4 \s-1EXAMPLE\s+1 [2] + 1.50 1.00 20/40 \ \ \s-1SUB1\s+1 <cycle1> [4] + 0.00 0.50 1/5 \ \ \s-1SUB2\s+1 [9] + 0.00 0.00 0/5 \ \ \s-1SUB3\s+1 [11] +.TE +.ce 2 +Profile entry for \s-1EXAMPLE\s+1. +Figure 4. +.)z +.pp +Ideally, we would like to print the call graph of the program, +but we are limited by the two-dimensional nature of our output +devices. +We cannot assume that a call graph is planar, +and even if it is, that we can print a planar version of it. +Instead, we choose to list each routine, +together with information about +the routines that are its direct parents and children. +This listing presents a window into the call graph. +Based on our experience, +both parent information and child information +is important, +and should be available without searching +through the output. +.pp +The major entries of the call graph profile are the entries from the +flat profile, augmented by the time propagated to each +routine from its descendants. +This profile is sorted by the sum of the time for the routine +itself plus the time inherited from its descendants. +The profile shows which of the higher level routines +spend large portions of the total execution time +in the routines that they call. +For each routine, we show the amount of time passed by each child +to the routine, which includes time for the child itself +and for the descendants of the child +(and thus the descendants of the routine). +We also show the percentage these times represent of the total time +accounted to the child. +Similarly, the parents of each routine are listed, +along with time, +and percentage of total routine time, +propagated to each one. +.pp +Cycles are handled as single entities. +The cycle as a whole is shown as though it were a single routine, +except that members of the cycle are listed in place of the children. +Although the number of calls of each member +from within the cycle are shown, +they do not affect time propagation. +When a child is a member of a cycle, +the time shown is the appropriate fraction of the time +for the whole cycle. +Self-recursive routines have their calls broken +down into calls from the outside and self-recursive calls. +Only the outside calls affect the propagation of time. +.pp +The following example is a typical fragment of a call graph. +.(b +.so pres1.pic +.)b +The entry in the call graph profile listing for this example is +shown in Figure 4. +.pp +The entry is for routine \s-1EXAMPLE\s+1, which has +the Caller routines as its parents, +and the Sub routines as its children. +The reader should keep in mind that all information +is given \fIwith respect to \s-1EXAMPLE\s+1\fP. +The index in the first column shows that \s-1EXAMPLE\s+1 +is the second entry in the profile listing. +The \s-1EXAMPLE\s+1 routine is called ten times, four times by \s-1CALLER1\s+1, +and six times by \s-1CALLER2\s+1. +Consequently 40% of \s-1EXAMPLE\s+1's time is propagated to \s-1CALLER1\s+1, +and 60% of \s-1EXAMPLE\s+1's time is propagated to \s-1CALLER2\s+1. +The self and descendant fields of the parents +show the amount of self and descendant time \s-1EXAMPLE\s+1 +propagates to them (but not the time used by +the parents directly). +Note that \s-1EXAMPLE\s+1 calls itself recursively four times. +The routine \s-1EXAMPLE\s+1 calls routine \s-1SUB1\s+1 twenty times, \s-1SUB2\s+1 once, +and never calls \s-1SUB3\s+1. +Since \s-1SUB2\s+1 is called a total of five times, +20% of its self and descendant time is propagated to \s-1EXAMPLE\s+1's +descendant time field. +Because \s-1SUB1\s+1 is a member of \fIcycle 1\fR, +the self and descendant times +and call count fraction +are those for the cycle as a whole. +Since cycle 1 is called a total of forty times +(not counting calls among members of the cycle), +it propagates 50% of the cycle's self and descendant +time to \s-1EXAMPLE\s+1's descendant time field. +Finally each name is followed by an index that shows +where on the listing to find the entry for that routine. +.sh 1 "Using the Profiles" +.pp +The profiler is a useful tool for improving +a set of routines that implement an abstraction. +It can be helpful in identifying poorly coded routines, +and in evaluating the new algorithms and code that replace them. +Taking full advantage of the profiler +requires a careful examination of the call graph profile, +and a thorough knowledge of the abstractions underlying +the program. +.pp +The easiest optimization that can be performed +is a small change +to a control construct or data structure that improves the +running time of the program. +An obvious starting point +is a routine that is called many times. +For example, suppose an output +routine is the only parent +of a routine that formats the data. +If this format routine is expanded inline in the +output routine, the overhead of a function call and +return can be saved for each datum that needs to be formatted. +.pp +The drawback to inline expansion is that the data abstractions +in the program may become less parameterized, +hence less clearly defined. +The profiling will also become less useful since the loss of +routines will make its output more granular. +For example, +if the symbol table functions ``lookup'', ``insert'', and ``delete'' +are all merged into a single parameterized routine, +it will be impossible to determine the costs +of any one of these individual functions from the profile. +.pp +Further potential for optimization lies in routines that +implement data abstractions whose total execution +time is long. +For example, a lookup routine might be called only a few +times, but use an inefficient linear search algorithm, +that might be replaced with a binary search. +Alternately, the discovery that a rehashing function is being +called excessively, can lead to a different +hash function or a larger hash table. +If the data abstraction function cannot easily be speeded up, +it may be advantageous to cache its results, +and eliminate the need to rerun +it for identical inputs. +These and other ideas for program improvement are discussed in +[Bentley81]. +.pp +This tool is best used in an iterative approach: +profiling the program, +eliminating one bottleneck, +then finding some other part of the program +that begins to dominate execution time. +For instance, we have used \fBgprof\fR on itself; +eliminating, rewriting, and inline expanding routines, +until reading +data files (hardly a target for optimization!) +represents the dominating factor in its execution time. +.pp +Certain types of programs are not easily analyzed by \fBgprof\fR. +They are typified by programs that exhibit a large degree of +recursion, such as recursive descent compilers. +The problem is that most of the major routines are grouped +into a single monolithic cycle. +As in the symbol table abstraction that is placed +in one routine, +it is impossible to distinguish which members of the cycle are +responsible for the execution time. +Unfortunately there are no easy modifications to these programs that +make them amenable to analysis. +.pp +A completely different use of the profiler is to analyze the control +flow of an unfamiliar program. +If you receive a program from another user that you need to modify +in some small way, +it is often unclear where the changes need to be made. +By running the program on an example and then using \fBgprof\fR, +you can get a view of the structure of the program. +.pp +Consider an example in which you need to change the output format +of the program. +For purposes of this example suppose that the call graph +of the output portion of the program has the following structure: +.(b +.so pres2.pic +.)b +Initially you look through the \fBgprof\fR +output for the system call ``\s-1WRITE\s+1''. +The format routine you will need to change is probably +among the parents of the ``\s-1WRITE\s+1'' procedure. +The next step is to look at the profile entry for each +of parents of ``\s-1WRITE\s+1'', +in this example either ``\s-1FORMAT1\s+1'' or ``\s-1FORMAT2\s+1'', +to determine which one to change. +Each format routine will have one or more parents, +in this example ``\s-1CALC1\s+1'', ``\s-1CALC2\s+1'', and ``\s-1CALC3\s+1''. +By inspecting the source code for each of these routines +you can determine which format routine generates the output that +you wish to modify. +Since the \fBgprof\fR entry shows all the +potential calls to the format routine you intend to change, +you can determine if your modifications will affect output that +should be left alone. +If you desire to change the output of ``\s-1CALC2\s+1'', but not ``\s-1CALC3\s+1'', +then formatting routine ``\s-1FORMAT2\s+1'' needs to be split +into two separate routines, +one of which implements the new format. +You can then retarget just the call by ``\s-1CALC2\s+1'' +that needs the new format. +It should be noted that the static call information is particularly +useful here since the test case you run probably will not +exercise the entire program. +.sh 1 "Conclusions" +.pp +We have created a profiler that aids in the evaluation +of modular programs. +For each routine in the program, +the profile shows the extent to which that routine +helps support various abstractions, +and how that routine uses other abstractions. +The profile accurately assesses the cost of routines +at all levels of the program decomposition. +The profiler is easily used, +and can be compiled into the program without any prior planning by +the programmer. +It adds only five to thirty percent execution overhead to the program +being profiled, +produces no additional output until after the program finishes, +and allows the program to be measured in its actual environment. +Finally, the profiler runs on a time-sharing system +using only the normal services provided by the operating system +and compilers. diff --git a/share/doc/psd/18.gprof/profiling.me b/share/doc/psd/18.gprof/profiling.me new file mode 100644 index 000000000000..227aedf5cf78 --- /dev/null +++ b/share/doc/psd/18.gprof/profiling.me @@ -0,0 +1,115 @@ +.\" Copyright (c) 1982, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)profiling.me 8.1 (Berkeley) 6/8/93 +.\" +.sh 1 "Types of Profiling" +.pp +There are several different uses for program profiles, +and each may require different information from the profiles, +or different presentation of the information. +We distinguish two broad categories of profiles: +those that present counts of statement or routine invocations, +and those that display timing information about statements +or routines. +Counts are typically presented in tabular form, +often in parallel with a listing of the source code. +Timing information could be similarly presented; +but more than one measure of time might be associated with each +statement or routine. +For example, +in the framework used by \fBgprof\fP +each profiled segment would display two times: +one for the time used by the segment itself, and another for the +time inherited from code segments it invokes. +.pp +Execution counts are used in many different contexts. +The exact number of times a routine or statement is activated +can be used to determine if an algorithm is performing as +expected. +Cursory inspection of such counters may show algorithms whose +complexity is unsuited to the task at hand. +Careful interpretation of counters can often suggest +improvements to acceptable algorithms. +Precise examination can uncover subtle errors in an +algorithm. +At this level, profiling counters are similar to +debugging statements whose purpose is to show the number of times +a piece of code is executed. +Another view of such counters is as boolean values. +One may be interested that a portion of code has executed at +all, for exhaustive testing, or to check that one implementation +of an abstraction completely replaces a previous one. +.pp +Execution counts are not necessarily proportional to the amount +of time required to execute the routine or statement. +Further, the execution time of a routine will not be the same for +all calls on the routine. +The criteria for establishing execution time +must be decided. +If a routine implements an abstraction by invoking other abstractions, +the time spent in the routine will not accurately reflect the +time required by the abstraction it implements. +Similarly, if an abstraction is implemented by several +routines the time required by the abstraction will be distributed +across those routines. +.pp +Given the execution time of individual routines, +\fBgprof\fP accounts to each routine the time spent +for it by the routines it invokes. +This accounting is done by assembling a \fIcall graph\fP with nodes that +are the routines of the program and directed arcs that represent +calls from call sites to routines. +We distinguish among three different call graphs for a program. +The \fIcomplete call graph\fP incorporates all routines and all +potential arcs, +including arcs that represent calls to functional parameters +or functional variables. +This graph contains the other two graphs as subgraphs. +The \fIstatic call graph\fP includes all routines and all possible arcs +that are not calls to functional parameters or variables. +The \fIdynamic call graph\fP includes only those routines and +arcs traversed by the profiled execution of the program. +This graph need not include all routines, nor need it include all +potential arcs between the routines it covers. +It may, however, include arcs to functional parameters or +variables that the static call graph may omit. +The static call graph can be determined from the (static) program text. +The dynamic call graph is determined only by profiling an +execution of the program. +The complete call graph for a monolithic program could be determined +by data flow analysis techniques. +The complete call graph for programs that change +during execution, by modifying themselves or dynamically loading +or overlaying code, may never be determinable. +Both the static call graph and the dynamic call graph are used +by \fBgprof\fP, but it does not search for the complete call +graph. diff --git a/share/doc/psd/18.gprof/refs.me b/share/doc/psd/18.gprof/refs.me new file mode 100644 index 000000000000..580d08030f51 --- /dev/null +++ b/share/doc/psd/18.gprof/refs.me @@ -0,0 +1,63 @@ +.\" Copyright (c) 1982, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)refs.me 8.1 (Berkeley) 6/8/93 +.\" +.sh 1 "References" +.ls 1 +.ip [Bentley81] +Bentley, J. L., +``Writing Efficient Code'', +Department of Computer Science, +Carnegie-Mellon University, +Pittsburgh, Pennsylvania, +CMU-CS-81-116, 1981. +.ip [Graham82] +Graham, S. L., Henry, R. R., Schulman, R. A., +``An Experiment in Table Driven Code Generation'', +SIGPLAN '82 Symposium on Compiler Construction, +June, 1982. +.ip [Joy79] +Joy, W. N., Graham, S. L., Haley, C. B. ``Berkeley Pascal User's Manual'', +Version 1.1, Computer Science Division +University of California, Berkeley, CA. April 1979. +.ip [Knuth71] +Knuth, D. E. ``An empirical study of FORTRAN programs'', +Software - Practice and Experience, 1, 105-133. 1971 +.ip [Satterthwaite72] +Satterthwaite, E. ``Debugging Tools for High Level Languages'', +Software - Practice and Experience, 2, 197-217, 1972 +.ip [Tarjan72] +Tarjan, R. E., ``Depth first search and linear graph algorithm,'' +\fISIAM J. Computing\fP \fB1\fP:2, 146-160, 1972. +.ip [Unix] +Unix Programmer's Manual, ``\fBprof\fR command'', section 1, +Bell Laboratories, Murray Hill, NJ. January 1979. diff --git a/share/doc/psd/20.ipctut/Makefile b/share/doc/psd/20.ipctut/Makefile new file mode 100644 index 000000000000..934cdeaf5eb6 --- /dev/null +++ b/share/doc/psd/20.ipctut/Makefile @@ -0,0 +1,14 @@ +# From: @(#)Makefile 8.1 (Berkeley) 8/14/93 +# $FreeBSD$ + +VOLUME= psd/20.ipctut +SRCS= tutor.me +MACROS= -me +EXTRA= dgramread.c dgramsend.c fig2.pic fig3.pic fig8.pic pipe.c \ + socketpair.c strchkread.c streamread.c streamwrite.c \ + udgramread.c udgramsend.c ustreamread.c ustreamwrite.c +USE_SOELIM= +USE_PIC= +USE_TBL= + +.include <bsd.doc.mk> diff --git a/share/doc/psd/20.ipctut/dgramread.c b/share/doc/psd/20.ipctut/dgramread.c new file mode 100644 index 000000000000..193fca90067a --- /dev/null +++ b/share/doc/psd/20.ipctut/dgramread.c @@ -0,0 +1,83 @@ +.\" Copyright (c) 1986, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)dgramread.c 8.1 (Berkeley) 6/8/93 +.\" +#include <sys/types.h> +#include <sys/socket.h> +#include <netinet/in.h> +#include <stdio.h> + +/* + * In the included file <netinet/in.h> a sockaddr_in is defined as follows: + * struct sockaddr_in { + * short sin_family; + * u_short sin_port; + * struct in_addr sin_addr; + * char sin_zero[8]; + * }; + * + * This program creates a datagram socket, binds a name to it, then reads + * from the socket. + */ +main() +{ + int sock, length; + struct sockaddr_in name; + char buf[1024]; + + /* Create socket from which to read. */ + sock = socket(AF_INET, SOCK_DGRAM, 0); + if (sock < 0) { + perror("opening datagram socket"); + exit(1); + } + /* Create name with wildcards. */ + name.sin_family = AF_INET; + name.sin_addr.s_addr = INADDR_ANY; + name.sin_port = 0; + if (bind(sock, &name, sizeof(name))) { + perror("binding datagram socket"); + exit(1); + } + /* Find assigned port value and print it out. */ + length = sizeof(name); + if (getsockname(sock, &name, &length)) { + perror("getting socket name"); + exit(1); + } + printf("Socket has port #%d\en", ntohs(name.sin_port)); + /* Read from the socket */ + if (read(sock, buf, 1024) < 0) + perror("receiving datagram packet"); + printf("-->%s\en", buf); + close(sock); +} diff --git a/share/doc/psd/20.ipctut/dgramsend.c b/share/doc/psd/20.ipctut/dgramsend.c new file mode 100644 index 000000000000..4bd1e5a13015 --- /dev/null +++ b/share/doc/psd/20.ipctut/dgramsend.c @@ -0,0 +1,80 @@ +.\" Copyright (c) 1986, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)dgramsend.c 8.1 (Berkeley) 6/8/93 +.\" +#include <sys/types.h> +#include <sys/socket.h> +#include <netinet/in.h> +#include <netdb.h> +#include <stdio.h> + +#define DATA "The sea is calm tonight, the tide is full . . ." + +/* + * Here I send a datagram to a receiver whose name I get from the command + * line arguments. The form of the command line is dgramsend hostname + * portnumber + */ + +main(argc, argv) + int argc; + char *argv[]; +{ + int sock; + struct sockaddr_in name; + struct hostent *hp, *gethostbyname(); + + /* Create socket on which to send. */ + sock = socket(AF_INET, SOCK_DGRAM, 0); + if (sock < 0) { + perror("opening datagram socket"); + exit(1); + } + /* + * Construct name, with no wildcards, of the socket to send to. + * Gethostbyname() returns a structure including the network address + * of the specified host. The port number is taken from the command + * line. + */ + hp = gethostbyname(argv[1]); + if (hp == 0) { + fprintf(stderr, "%s: unknown host\en", argv[1]); + exit(2); + } + bcopy(hp->h_addr, &name.sin_addr, hp->h_length); + name.sin_family = AF_INET; + name.sin_port = htons(atoi(argv[2])); + /* Send message. */ + if (sendto(sock, DATA, sizeof(DATA), 0, &name, sizeof(name)) < 0) + perror("sending datagram message"); + close(sock); +} diff --git a/share/doc/psd/20.ipctut/fig2.pic b/share/doc/psd/20.ipctut/fig2.pic new file mode 100644 index 000000000000..ffbc193f733e --- /dev/null +++ b/share/doc/psd/20.ipctut/fig2.pic @@ -0,0 +1,77 @@ +.\" Copyright (c) 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" @(#)fig2.pic 8.1 (Berkeley) 8/14/93 +.PS +.ps +.ps 10 +arc at 5.407,4.723 from 5.375,4.838 to 5.362,4.612 cw +arc at 5.907,7.973 from 5.875,8.088 to 5.862,7.862 cw +line from 5.963,5.513 to 6.925,5.513 +line from 5.963,5.650 to 6.925,5.650 +line from 5.963,5.787 to 6.925,5.787 +line from 5.963,5.912 to 6.925,5.912 +line from 5.963,6.050 to 6.925,6.050 +line from 5.963,6.200 to 6.925,6.200 to 6.925,5.375 to 5.963,5.375 to 5.963,6.200 +ellipse at 6.388,6.713 wid 0.475 ht 0.475 +line from 6.388,6.463 to 6.388,6.200 +line from 3.150,6.200 to 4.112,6.200 to 4.112,5.375 to 3.150,5.375 to 3.150,6.200 +line from 3.150,6.050 to 4.112,6.050 +line from 3.150,5.912 to 4.112,5.912 +line from 3.150,5.787 to 4.112,5.787 +line from 3.150,5.650 to 4.112,5.650 +line from 3.150,5.513 to 4.112,5.513 +ellipse at 3.575,6.713 wid 0.475 ht 0.475 +line from 3.575,6.463 to 3.575,6.200 +line from 3.650,8.762 to 4.612,8.762 +line from 3.650,8.900 to 4.612,8.900 +line from 3.650,9.037 to 4.612,9.037 +line from 3.650,9.162 to 4.612,9.162 +line from 3.650,9.300 to 4.612,9.300 +line from 3.650,9.450 to 4.612,9.450 to 4.612,8.625 to 3.650,8.625 to 3.650,9.450 +ellipse at 4.075,9.963 wid 0.475 ht 0.475 +ellipse at 3.950,4.725 wid 0.225 ht 0.225 +ellipse at 4.450,7.975 wid 0.225 ht 0.225 +dashwid = 0.037i +line dotted from 1.925,7.513 to 8.238,7.513 +line from 6.050,6.138 to 5.737,6.138 to 5.737,4.700 to 5.550,4.700 +line from 5.650,4.725 to 5.550,4.700 to 5.650,4.675 +line from 6.050,6.013 to 4.050,4.888 +line from 4.125,4.958 to 4.050,4.888 to 4.149,4.915 +line from 3.975,6.000 to 4.525,5.987 to 3.925,4.875 +line from 3.950,4.975 to 3.925,4.875 to 3.994,4.951 +line from 3.975,6.112 to 5.650,6.112 to 5.650,4.750 to 5.550,4.763 +line from 5.652,4.775 to 5.550,4.763 to 5.646,4.725 +line from 4.075,9.713 to 4.075,9.450 +line from 4.475,9.363 to 6.150,9.363 to 6.150,8.000 to 6.050,8.012 +line from 6.152,8.025 to 6.050,8.012 to 6.146,7.975 +line from 4.475,9.250 to 5.025,9.238 to 4.425,8.125 +line from 4.450,8.225 to 4.425,8.125 to 4.494,8.201 +.ps +.ps 20 +line from 4.362,4.775 to 4.162,4.725 to 4.362,4.675 +line from 4.162,4.725 to 4.838,4.725 +.ps +.ps 10 +line from 3.962,4.600 to 5.375,4.600 +line from 3.950,4.838 to 5.375,4.838 +line from 4.450,8.088 to 5.875,8.088 +line from 4.463,7.850 to 5.875,7.850 +.ps +.ps 20 +line from 4.862,8.025 to 4.662,7.975 to 4.862,7.925 +line from 4.662,7.975 to 5.338,7.975 +.ps +.ps 11 +.ft +.ft R +"Child" at 6.362,7.106 +.ps +.ps 12 +"Parent" at 3.362,7.096 ljust +"Parent" at 3.862,10.346 ljust +"PIPE" at 4.987,4.671 ljust +"PIPE" at 5.425,7.921 ljust +.ps +.ft +.PE diff --git a/share/doc/psd/20.ipctut/fig2.xfig b/share/doc/psd/20.ipctut/fig2.xfig new file mode 100644 index 000000000000..59b46be37b8d --- /dev/null +++ b/share/doc/psd/20.ipctut/fig2.xfig @@ -0,0 +1,100 @@ +#FIG 2.0 +80 2 +5 1 0 1 0 0 0 0 0.000 0 0 0 432.554 462.170 430 453 442 461 429 471 +5 1 0 1 0 0 0 0 0.000 0 0 0 472.554 202.170 470 193 482 201 469 211 +6 414 279 589 424 +6 473 340 557 414 +2 1 0 1 0 0 0 0 0.000 0 0 + 477 399 554 399 9999 9999 +2 1 0 1 0 0 0 0 0.000 0 0 + 477 388 554 388 9999 9999 +2 1 0 1 0 0 0 0 0.000 0 0 + 477 377 554 377 9999 9999 +2 1 0 1 0 0 0 0 0.000 0 0 + 477 367 554 367 9999 9999 +2 1 0 1 0 0 0 0 0.000 0 0 + 477 356 554 356 9999 9999 +2 2 0 1 0 0 0 0 0.000 0 0 + 477 344 554 344 554 410 477 410 477 344 9999 9999 +-6 +1 3 0 1 0 0 0 0 0.000 1 0.000 511 303 19 19 511 303 525 317 +2 1 0 1 0 0 0 0 0.000 0 0 + 511 323 511 344 9999 9999 +-6 +6 189 279 364 424 +6 248 340 332 414 +2 2 0 1 0 0 0 0 0.000 0 0 + 252 344 329 344 329 410 252 410 252 344 9999 9999 +2 1 0 1 0 0 0 0 0.000 0 0 + 252 356 329 356 9999 9999 +2 1 0 1 0 0 0 0 0.000 0 0 + 252 367 329 367 9999 9999 +2 1 0 1 0 0 0 0 0.000 0 0 + 252 377 329 377 9999 9999 +2 1 0 1 0 0 0 0 0.000 0 0 + 252 388 329 388 9999 9999 +2 1 0 1 0 0 0 0 0.000 0 0 + 252 399 329 399 9999 9999 +-6 +1 3 0 1 0 0 0 0 0.000 1 0.000 286 303 19 19 286 303 300 317 +2 1 0 1 0 0 0 0 0.000 0 0 + 286 323 286 344 9999 9999 +-6 +6 288 80 372 154 +2 1 0 1 0 0 0 0 0.000 0 0 + 292 139 369 139 9999 9999 +2 1 0 1 0 0 0 0 0.000 0 0 + 292 128 369 128 9999 9999 +2 1 0 1 0 0 0 0 0.000 0 0 + 292 117 369 117 9999 9999 +2 1 0 1 0 0 0 0 0.000 0 0 + 292 107 369 107 9999 9999 +2 1 0 1 0 0 0 0 0.000 0 0 + 292 96 369 96 9999 9999 +2 2 0 1 0 0 0 0 0.000 0 0 + 292 84 369 84 369 150 292 150 292 84 9999 9999 +-6 +1 3 0 1 0 0 0 0 0.000 1 0.000 326 43 19 19 326 43 340 57 +1 3 0 1 0 0 0 0 0.000 1 0.000 316 462 9 9 316 462 322 469 +1 3 0 1 0 0 0 0 0.000 1 0.000 356 202 9 9 356 202 362 209 +2 1 2 1 0 0 0 0 3.000 0 0 + 154 239 659 239 9999 9999 +2 1 0 1 0 0 0 0 0.000 1 0 + 0 0 1.000 4.000 8.000 + 484 349 459 349 459 464 444 464 9999 9999 +2 1 0 1 0 0 0 0 0.000 1 0 + 0 0 1.000 4.000 8.000 + 484 359 324 449 9999 9999 +2 1 0 1 0 0 0 0 0.000 1 0 + 0 0 1.000 4.000 8.000 + 318 360 362 361 314 450 9999 9999 +2 1 0 1 0 0 0 0 0.000 1 0 + 0 0 1.000 4.000 8.000 + 318 351 452 351 452 460 444 459 9999 9999 +2 1 0 1 0 0 0 0 0.000 0 0 + 326 63 326 84 9999 9999 +2 1 0 1 0 0 0 0 0.000 1 0 + 0 0 1.000 4.000 8.000 + 358 91 492 91 492 200 484 199 9999 9999 +2 1 0 1 0 0 0 0 0.000 1 0 + 0 0 1.000 4.000 8.000 + 358 100 402 101 354 190 9999 9999 +2 1 0 2 0 0 0 0 0.000 0 1 + 0 0 2.000 8.000 16.000 + 333 462 387 462 9999 9999 +2 1 0 1 0 0 0 0 0.000 0 0 + 317 472 430 472 9999 9999 +2 1 0 1 0 0 0 0 0.000 0 0 + 316 453 430 453 9999 9999 +2 1 0 1 0 0 0 0 0.000 0 0 + 356 193 470 193 9999 9999 +2 1 0 1 0 0 0 0 0.000 0 0 + 357 212 470 212 9999 9999 +2 1 0 2 0 0 0 0 0.000 0 1 + 0 0 2.000 8.000 16.000 + 373 202 427 202 9999 9999 +4 1 0 11 0 0 0 0.000 1 7 24 509 274 Child +4 0 0 12 0 0 0 0.000 1 9 33 269 275 Parent +4 0 0 12 0 0 0 0.000 1 9 33 309 15 Parent +4 0 0 12 0 0 0 0.000 1 9 26 399 469 PIPE +4 0 0 12 0 0 0 0.000 1 9 26 434 209 PIPE diff --git a/share/doc/psd/20.ipctut/fig3.pic b/share/doc/psd/20.ipctut/fig3.pic new file mode 100644 index 000000000000..15a4a73bf7d5 --- /dev/null +++ b/share/doc/psd/20.ipctut/fig3.pic @@ -0,0 +1,69 @@ +.\" Copyright (c) 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" @(#)fig3.pic 8.1 (Berkeley) 8/14/93 +.PS +.ps +.ps 10 +ellipse at 5.787,8.012 wid 0.275 ht 0.275 +ellipse at 4.175,8.012 wid 0.275 ht 0.275 +dashwid = 0.037i +line dotted from 5.550,8.012 to 4.362,8.012 +line from 4.462,8.037 to 4.362,8.012 to 4.462,7.987 +line dotted from 4.362,7.950 to 5.550,7.950 +line from 5.450,7.925 to 5.550,7.950 to 5.450,7.975 +ellipse at 3.737,4.763 wid 0.275 ht 0.275 +ellipse at 5.350,4.763 wid 0.275 ht 0.275 +line dotted from 3.925,4.700 to 5.112,4.700 +line from 5.013,4.675 to 5.112,4.700 to 5.013,4.725 +line dotted from 5.112,4.763 to 3.925,4.763 +line from 4.025,4.788 to 3.925,4.763 to 4.025,4.737 +line from 5.963,5.513 to 6.925,5.513 +line from 5.963,5.650 to 6.925,5.650 +line from 5.963,5.787 to 6.925,5.787 +line from 5.963,5.912 to 6.925,5.912 +line from 5.963,6.050 to 6.925,6.050 +line from 5.963,6.200 to 6.925,6.200 to 6.925,5.375 to 5.963,5.375 to 5.963,6.200 +ellipse at 6.388,6.713 wid 0.475 ht 0.475 +line from 6.388,6.463 to 6.388,6.200 +line from 3.150,6.200 to 4.112,6.200 to 4.112,5.375 to 3.150,5.375 to 3.150,6.200 +line from 3.150,6.050 to 4.112,6.050 +line from 3.150,5.912 to 4.112,5.912 +line from 3.150,5.787 to 4.112,5.787 +line from 3.150,5.650 to 4.112,5.650 +line from 3.150,5.513 to 4.112,5.513 +ellipse at 3.575,6.713 wid 0.475 ht 0.475 +line from 3.575,6.463 to 3.575,6.200 +line from 3.650,8.762 to 4.612,8.762 +line from 3.650,8.900 to 4.612,8.900 +line from 3.650,9.037 to 4.612,9.037 +line from 3.650,9.162 to 4.612,9.162 +line from 3.650,9.300 to 4.612,9.300 +line from 3.650,9.450 to 4.612,9.450 to 4.612,8.625 to 3.650,8.625 to 3.650,9.450 +ellipse at 4.075,9.963 wid 0.475 ht 0.475 +line from 3.975,6.112 to 5.650,6.112 to 5.650,4.750 to 5.550,4.763 +line from 5.652,4.775 to 5.550,4.763 to 5.646,4.725 +line from 6.050,6.138 to 5.737,6.138 to 5.737,4.700 to 5.550,4.700 +line from 5.650,4.725 to 5.550,4.700 to 5.650,4.675 +line dotted from 1.925,7.513 to 8.238,7.513 +line from 6.050,6.013 to 4.050,4.888 +line from 4.125,4.958 to 4.050,4.888 to 4.149,4.915 +line from 3.975,6.000 to 4.525,5.987 to 3.925,4.875 +line from 3.950,4.975 to 3.925,4.875 to 3.994,4.951 +line from 4.075,9.713 to 4.075,9.450 +line from 4.475,9.363 to 6.150,9.363 to 6.150,8.000 to 6.050,8.012 +line from 6.152,8.025 to 6.050,8.012 to 6.146,7.975 +line from 4.475,9.250 to 5.025,9.238 to 4.425,8.125 +line from 4.450,8.225 to 4.425,8.125 to 4.494,8.201 +.ps +.ps 11 +.ft +.ft R +"Child" at 6.362,7.106 +.ps +.ps 12 +"Parent" at 3.362,7.096 ljust +"Parent" at 3.862,10.346 ljust +.ps +.ft +.PE diff --git a/share/doc/psd/20.ipctut/fig3.xfig b/share/doc/psd/20.ipctut/fig3.xfig new file mode 100644 index 000000000000..ed65b70bd184 --- /dev/null +++ b/share/doc/psd/20.ipctut/fig3.xfig @@ -0,0 +1,100 @@ +#FIG 2.0 +80 2 +6 309 184 479 214 +1 3 0 1 0 0 0 0 0.000 1 0.000 463 199 11 11 463 199 468 209 +1 3 0 1 0 0 0 0 0.000 1 0.000 334 199 11 11 334 199 339 209 +2 1 2 1 0 0 0 0 3.000 1 0 + 0 0 1.000 4.000 8.000 + 444 199 349 199 9999 9999 +2 1 2 1 0 0 0 0 3.000 1 0 + 0 0 1.000 4.000 8.000 + 349 204 444 204 9999 9999 +-6 +6 274 444 444 474 +1 3 0 1 0 0 0 0 0.000 1 0.000 299 459 11 11 299 459 304 469 +1 3 0 1 0 0 0 0 0.000 1 0.000 428 459 11 11 428 459 433 469 +2 1 2 1 0 0 0 0 3.000 1 0 + 0 0 1.000 4.000 8.000 + 314 464 409 464 9999 9999 +2 1 2 1 0 0 0 0 3.000 1 0 + 0 0 1.000 4.000 8.000 + 409 459 314 459 9999 9999 +-6 +6 414 279 589 424 +6 473 340 557 414 +2 1 0 1 0 0 0 0 0.000 0 0 + 477 399 554 399 9999 9999 +2 1 0 1 0 0 0 0 0.000 0 0 + 477 388 554 388 9999 9999 +2 1 0 1 0 0 0 0 0.000 0 0 + 477 377 554 377 9999 9999 +2 1 0 1 0 0 0 0 0.000 0 0 + 477 367 554 367 9999 9999 +2 1 0 1 0 0 0 0 0.000 0 0 + 477 356 554 356 9999 9999 +2 2 0 1 0 0 0 0 0.000 0 0 + 477 344 554 344 554 410 477 410 477 344 9999 9999 +-6 +1 3 0 1 0 0 0 0 0.000 1 0.000 511 303 19 19 511 303 525 317 +2 1 0 1 0 0 0 0 0.000 0 0 + 511 323 511 344 9999 9999 +-6 +6 189 279 364 424 +6 248 340 332 414 +2 2 0 1 0 0 0 0 0.000 0 0 + 252 344 329 344 329 410 252 410 252 344 9999 9999 +2 1 0 1 0 0 0 0 0.000 0 0 + 252 356 329 356 9999 9999 +2 1 0 1 0 0 0 0 0.000 0 0 + 252 367 329 367 9999 9999 +2 1 0 1 0 0 0 0 0.000 0 0 + 252 377 329 377 9999 9999 +2 1 0 1 0 0 0 0 0.000 0 0 + 252 388 329 388 9999 9999 +2 1 0 1 0 0 0 0 0.000 0 0 + 252 399 329 399 9999 9999 +-6 +1 3 0 1 0 0 0 0 0.000 1 0.000 286 303 19 19 286 303 300 317 +2 1 0 1 0 0 0 0 0.000 0 0 + 286 323 286 344 9999 9999 +-6 +6 288 80 372 154 +2 1 0 1 0 0 0 0 0.000 0 0 + 292 139 369 139 9999 9999 +2 1 0 1 0 0 0 0 0.000 0 0 + 292 128 369 128 9999 9999 +2 1 0 1 0 0 0 0 0.000 0 0 + 292 117 369 117 9999 9999 +2 1 0 1 0 0 0 0 0.000 0 0 + 292 107 369 107 9999 9999 +2 1 0 1 0 0 0 0 0.000 0 0 + 292 96 369 96 9999 9999 +2 2 0 1 0 0 0 0 0.000 0 0 + 292 84 369 84 369 150 292 150 292 84 9999 9999 +-6 +1 3 0 1 0 0 0 0 0.000 1 0.000 326 43 19 19 326 43 340 57 +2 1 0 1 0 0 0 0 0.000 1 0 + 0 0 1.000 4.000 8.000 + 318 351 452 351 452 460 444 459 9999 9999 +2 1 0 1 0 0 0 0 0.000 1 0 + 0 0 1.000 4.000 8.000 + 484 349 459 349 459 464 444 464 9999 9999 +2 1 2 1 0 0 0 0 3.000 0 0 + 154 239 659 239 9999 9999 +2 1 0 1 0 0 0 0 0.000 1 0 + 0 0 1.000 4.000 8.000 + 484 359 324 449 9999 9999 +2 1 0 1 0 0 0 0 0.000 1 0 + 0 0 1.000 4.000 8.000 + 318 360 362 361 314 450 9999 9999 +2 1 0 1 0 0 0 0 0.000 0 0 + 326 63 326 84 9999 9999 +2 1 0 1 0 0 0 0 0.000 1 0 + 0 0 1.000 4.000 8.000 + 358 91 492 91 492 200 484 199 9999 9999 +2 1 0 1 0 0 0 0 0.000 1 0 + 0 0 1.000 4.000 8.000 + 358 100 402 101 354 190 9999 9999 +4 1 0 11 0 0 0 0.000 1 7 24 509 274 Child +4 0 0 12 0 0 0 0.000 1 9 33 269 275 Parent +4 0 0 12 0 0 0 0.000 1 9 33 309 15 Parent diff --git a/share/doc/psd/20.ipctut/fig8.pic b/share/doc/psd/20.ipctut/fig8.pic new file mode 100644 index 000000000000..92b8833760e3 --- /dev/null +++ b/share/doc/psd/20.ipctut/fig8.pic @@ -0,0 +1,79 @@ +.\" Copyright (c) 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" @(#)fig8.pic 8.1 (Berkeley) 8/14/93 +.PS +.ps +.ps 11 +.ft +.ft R +"Process 1" at 3.800,7.106 rjust +"Process 2" at 6.612,7.106 rjust +.ps +.ps 10 +line from 3.150,6.200 to 4.112,6.200 to 4.112,5.375 to 3.150,5.375 to 3.150,6.200 +line from 3.150,6.050 to 4.112,6.050 +line from 3.150,5.912 to 4.112,5.912 +line from 3.150,5.787 to 4.112,5.787 +line from 3.150,5.650 to 4.112,5.650 +line from 3.150,5.513 to 4.112,5.513 +ellipse at 3.575,6.713 wid 0.475 ht 0.475 +line from 3.575,6.463 to 3.575,6.200 +line from 5.963,5.513 to 6.925,5.513 +line from 5.963,5.650 to 6.925,5.650 +line from 5.963,5.787 to 6.925,5.787 +line from 5.963,5.912 to 6.925,5.912 +line from 5.963,6.050 to 6.925,6.050 +line from 5.963,6.200 to 6.925,6.200 to 6.925,5.375 to 5.963,5.375 to 5.963,6.200 +ellipse at 6.388,6.713 wid 0.475 ht 0.475 +line from 6.388,6.463 to 6.388,6.200 +line from 3.087,8.637 to 4.050,8.637 +line from 3.087,8.775 to 4.050,8.775 +line from 3.087,8.912 to 4.050,8.912 +line from 3.087,9.037 to 4.050,9.037 +line from 3.087,9.175 to 4.050,9.175 +line from 3.087,9.325 to 4.050,9.325 to 4.050,8.500 to 3.087,8.500 to 3.087,9.325 +ellipse at 3.513,9.838 wid 0.475 ht 0.475 +line from 3.513,9.588 to 3.513,9.325 +line from 5.900,9.325 to 6.862,9.325 to 6.862,8.500 to 5.900,8.500 to 5.900,9.325 +line from 5.900,9.175 to 6.862,9.175 +line from 5.900,9.037 to 6.862,9.037 +line from 5.900,8.912 to 6.862,8.912 +line from 5.900,8.775 to 6.862,8.775 +line from 5.900,8.637 to 6.862,8.637 +ellipse at 6.325,9.838 wid 0.475 ht 0.475 +line from 6.325,9.588 to 6.325,9.325 +.ps +.ps 11 +"Process 2" at 6.550,10.231 rjust +"Process 1" at 3.737,10.231 rjust +.ps +.ps 10 +ellipse at 6.112,4.888 wid 0.275 ht 0.275 +ellipse at 5.350,4.763 wid 0.275 ht 0.275 +ellipse at 3.737,4.763 wid 0.275 ht 0.275 +ellipse at 4.550,7.950 wid 0.275 ht 0.275 +ellipse at 5.487,7.950 wid 0.275 ht 0.275 +line from 6.050,6.013 to 5.175,6.013 to 5.987,5.013 +line from 5.905,5.074 to 5.987,5.013 to 5.944,5.106 +line from 6.050,6.138 to 5.737,6.138 to 5.737,4.700 to 5.550,4.700 +line from 5.650,4.725 to 5.550,4.700 to 5.650,4.675 +dashwid = 0.037i +line dotted from 1.925,7.513 to 8.238,7.513 +line from 3.975,6.000 to 4.525,5.987 to 3.925,4.875 +line from 3.950,4.975 to 3.925,4.875 to 3.994,4.951 +line dotted from 5.112,4.763 to 3.925,4.763 +line from 4.025,4.788 to 3.925,4.763 to 4.025,4.737 +line dotted from 3.925,4.700 to 5.112,4.700 +line from 5.013,4.675 to 5.112,4.700 to 5.013,4.725 +line from 6.050,9.012 to 5.487,9.012 to 5.487,8.137 +line from 5.462,8.237 to 5.487,8.137 to 5.513,8.237 +line from 3.737,9.137 to 4.550,9.137 to 4.550,8.137 +line from 4.525,8.237 to 4.550,8.137 to 4.575,8.237 +.ps +.ps 11 +"NAME" at 6.737,4.918 rjust +"NAME" at 6.112,8.043 rjust +.ps +.ft +.PE diff --git a/share/doc/psd/20.ipctut/fig8.xfig b/share/doc/psd/20.ipctut/fig8.xfig new file mode 100644 index 000000000000..f1a525768572 --- /dev/null +++ b/share/doc/psd/20.ipctut/fig8.xfig @@ -0,0 +1,116 @@ +#FIG 2.0 +80 2 +6 224 254 589 279 +4 2 0 11 0 0 0 0.000 1 7 38 304 274 Process 1 +4 2 0 11 0 0 0 0.000 1 7 38 529 274 Process 2 +-6 +6 189 279 364 424 +6 248 340 332 414 +2 2 0 1 0 0 0 0 0.000 0 0 + 252 344 329 344 329 410 252 410 252 344 9999 9999 +2 1 0 1 0 0 0 0 0.000 0 0 + 252 356 329 356 9999 9999 +2 1 0 1 0 0 0 0 0.000 0 0 + 252 367 329 367 9999 9999 +2 1 0 1 0 0 0 0 0.000 0 0 + 252 377 329 377 9999 9999 +2 1 0 1 0 0 0 0 0.000 0 0 + 252 388 329 388 9999 9999 +2 1 0 1 0 0 0 0 0.000 0 0 + 252 399 329 399 9999 9999 +-6 +1 3 0 1 0 0 0 0 0.000 1 0.000 286 303 19 19 286 303 300 317 +2 1 0 1 0 0 0 0 0.000 0 0 + 286 323 286 344 9999 9999 +-6 +6 414 279 589 424 +6 473 340 557 414 +2 1 0 1 0 0 0 0 0.000 0 0 + 477 399 554 399 9999 9999 +2 1 0 1 0 0 0 0 0.000 0 0 + 477 388 554 388 9999 9999 +2 1 0 1 0 0 0 0 0.000 0 0 + 477 377 554 377 9999 9999 +2 1 0 1 0 0 0 0 0.000 0 0 + 477 367 554 367 9999 9999 +2 1 0 1 0 0 0 0 0.000 0 0 + 477 356 554 356 9999 9999 +2 2 0 1 0 0 0 0 0.000 0 0 + 477 344 554 344 554 410 477 410 477 344 9999 9999 +-6 +1 3 0 1 0 0 0 0 0.000 1 0.000 511 303 19 19 511 303 525 317 +2 1 0 1 0 0 0 0 0.000 0 0 + 511 323 511 344 9999 9999 +-6 +6 184 29 359 174 +6 243 90 327 164 +2 1 0 1 0 0 0 0 0.000 0 0 + 247 149 324 149 9999 9999 +2 1 0 1 0 0 0 0 0.000 0 0 + 247 138 324 138 9999 9999 +2 1 0 1 0 0 0 0 0.000 0 0 + 247 127 324 127 9999 9999 +2 1 0 1 0 0 0 0 0.000 0 0 + 247 117 324 117 9999 9999 +2 1 0 1 0 0 0 0 0.000 0 0 + 247 106 324 106 9999 9999 +2 2 0 1 0 0 0 0 0.000 0 0 + 247 94 324 94 324 160 247 160 247 94 9999 9999 +-6 +1 3 0 1 0 0 0 0 0.000 1 0.000 281 53 19 19 281 53 295 67 +2 1 0 1 0 0 0 0 0.000 0 0 + 281 73 281 94 9999 9999 +-6 +6 409 29 584 174 +6 468 90 552 164 +2 2 0 1 0 0 0 0 0.000 0 0 + 472 94 549 94 549 160 472 160 472 94 9999 9999 +2 1 0 1 0 0 0 0 0.000 0 0 + 472 106 549 106 9999 9999 +2 1 0 1 0 0 0 0 0.000 0 0 + 472 117 549 117 9999 9999 +2 1 0 1 0 0 0 0 0.000 0 0 + 472 127 549 127 9999 9999 +2 1 0 1 0 0 0 0 0.000 0 0 + 472 138 549 138 9999 9999 +2 1 0 1 0 0 0 0 0.000 0 0 + 472 149 549 149 9999 9999 +-6 +1 3 0 1 0 0 0 0 0.000 1 0.000 506 53 19 19 506 53 520 67 +2 1 0 1 0 0 0 0 0.000 0 0 + 506 73 506 94 9999 9999 +-6 +6 219 4 584 29 +4 2 0 11 0 0 0 0.000 1 7 38 524 24 Process 2 +4 2 0 11 0 0 0 0.000 1 7 38 299 24 Process 1 +-6 +1 3 0 1 0 0 0 0 0.000 1 0.000 489 449 11 11 489 449 494 459 +1 3 0 1 0 0 0 0 0.000 1 0.000 428 459 11 11 428 459 433 469 +1 3 0 1 0 0 0 0 0.000 1 0.000 299 459 11 11 299 459 304 469 +1 3 0 1 0 0 0 0 0.000 1 0.000 364 204 11 11 364 204 369 214 +1 3 0 1 0 0 0 0 0.000 1 0.000 439 204 11 11 439 204 444 214 +2 1 0 1 0 0 0 0 0.000 1 0 + 0 0 1.000 4.000 8.000 + 484 359 414 359 479 439 9999 9999 +2 1 0 1 0 0 0 0 0.000 1 0 + 0 0 1.000 4.000 8.000 + 484 349 459 349 459 464 444 464 9999 9999 +2 1 2 1 0 0 0 0 3.000 0 0 + 154 239 659 239 9999 9999 +2 1 0 1 0 0 0 0 0.000 1 0 + 0 0 1.000 4.000 8.000 + 318 360 362 361 314 450 9999 9999 +2 1 2 1 0 0 0 0 3.000 1 0 + 0 0 1.000 4.000 8.000 + 409 459 314 459 9999 9999 +2 1 2 1 0 0 0 0 3.000 1 0 + 0 0 1.000 4.000 8.000 + 314 464 409 464 9999 9999 +2 1 0 1 0 0 0 0 0.000 1 0 + 0 0 1.000 4.000 8.000 + 484 119 439 119 439 189 9999 9999 +2 1 0 1 0 0 0 0 0.000 1 0 + 0 0 1.000 4.000 8.000 + 299 109 364 109 364 189 9999 9999 +4 2 0 11 0 0 0 0.000 1 7 32 539 449 NAME +4 2 0 11 0 0 0 0.000 1 7 32 489 199 NAME diff --git a/share/doc/psd/20.ipctut/pipe.c b/share/doc/psd/20.ipctut/pipe.c new file mode 100644 index 000000000000..86cb66354250 --- /dev/null +++ b/share/doc/psd/20.ipctut/pipe.c @@ -0,0 +1,74 @@ +.\" Copyright (c) 1986, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)pipe.c 8.1 (Berkeley) 6/8/93 +.\" +#include <stdio.h> + +#define DATA "Bright star, would I were steadfast as thou art . . ." + +/* + * This program creates a pipe, then forks. The child communicates to the + * parent over the pipe. Notice that a pipe is a one-way communications + * device. I can write to the output socket (sockets[1], the second socket + * of the array returned by pipe()) and read from the input socket + * (sockets[0]), but not vice versa. + */ + +main() +{ + int sockets[2], child; + + /* Create a pipe */ + if (pipe(sockets) < 0) { + perror("opening stream socket pair"); + exit(10); + } + + if ((child = fork()) == -1) + perror("fork"); + else if (child) { + char buf[1024]; + + /* This is still the parent. It reads the child's message. */ + close(sockets[1]); + if (read(sockets[0], buf, 1024) < 0) + perror("reading message"); + printf("-->%s\en", buf); + close(sockets[0]); + } else { + /* This is the child. It writes a message to its parent. */ + close(sockets[0]); + if (write(sockets[1], DATA, sizeof(DATA)) < 0) + perror("writing message"); + close(sockets[1]); + } +} diff --git a/share/doc/psd/20.ipctut/socketpair.c b/share/doc/psd/20.ipctut/socketpair.c new file mode 100644 index 000000000000..f525c76fc792 --- /dev/null +++ b/share/doc/psd/20.ipctut/socketpair.c @@ -0,0 +1,77 @@ +.\" Copyright (c) 1986, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)socketpair.c 8.1 (Berkeley) 6/8/93 +.\" +#include <sys/types.h> +#include <sys/socket.h> +#include <stdio.h> + +#define DATA1 "In Xanadu, did Kublai Khan . . ." +#define DATA2 "A stately pleasure dome decree . . ." + +/* + * This program creates a pair of connected sockets then forks and + * communicates over them. This is very similar to communication with pipes, + * however, socketpairs are two-way communications objects. Therefore I can + * send messages in both directions. + */ + +main() +{ + int sockets[2], child; + char buf[1024]; + + if (socketpair(AF_UNIX, SOCK_STREAM, 0, sockets) < 0) { + perror("opening stream socket pair"); + exit(1); + } + + if ((child = fork()) == -1) + perror("fork"); + else if (child) { /* This is the parent. */ + close(sockets[0]); + if (read(sockets[1], buf, 1024, 0) < 0) + perror("reading stream message"); + printf("-->%s\en", buf); + if (write(sockets[1], DATA2, sizeof(DATA2)) < 0) + perror("writing stream message"); + close(sockets[1]); + } else { /* This is the child. */ + close(sockets[1]); + if (write(sockets[0], DATA1, sizeof(DATA1)) < 0) + perror("writing stream message"); + if (read(sockets[0], buf, 1024, 0) < 0) + perror("reading stream message"); + printf("-->%s\en", buf); + close(sockets[0]); + } +} diff --git a/share/doc/psd/20.ipctut/strchkread.c b/share/doc/psd/20.ipctut/strchkread.c new file mode 100644 index 000000000000..a1e148bb438d --- /dev/null +++ b/share/doc/psd/20.ipctut/strchkread.c @@ -0,0 +1,106 @@ +.\" Copyright (c) 1986, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)strchkread.c 8.1 (Berkeley) 6/8/93 +.\" +#include <sys/types.h> +#include <sys/socket.h> +#include <sys/time.h> +#include <netinet/in.h> +#include <netdb.h> +#include <stdio.h> +#define TRUE 1 + +/* + * This program uses select() to check that someone is trying to connect + * before calling accept(). + */ + +main() +{ + int sock, length; + struct sockaddr_in server; + int msgsock; + char buf[1024]; + int rval; + fd_set ready; + struct timeval to; + + /* Create socket */ + sock = socket(AF_INET, SOCK_STREAM, 0); + if (sock < 0) { + perror("opening stream socket"); + exit(1); + } + /* Name socket using wildcards */ + server.sin_family = AF_INET; + server.sin_addr.s_addr = INADDR_ANY; + server.sin_port = 0; + if (bind(sock, &server, sizeof(server))) { + perror("binding stream socket"); + exit(1); + } + /* Find out assigned port number and print it out */ + length = sizeof(server); + if (getsockname(sock, &server, &length)) { + perror("getting socket name"); + exit(1); + } + printf("Socket has port #%d\en", ntohs(server.sin_port)); + + /* Start accepting connections */ + listen(sock, 5); + do { + FD_ZERO(&ready); + FD_SET(sock, &ready); + to.tv_sec = 5; + if (select(sock + 1, &ready, 0, 0, &to) < 0) { + perror("select"); + continue; + } + if (FD_ISSET(sock, &ready)) { + msgsock = accept(sock, (struct sockaddr *)0, (int *)0); + if (msgsock == -1) + perror("accept"); + else do { + bzero(buf, sizeof(buf)); + if ((rval = read(msgsock, buf, 1024)) < 0) + perror("reading stream message"); + else if (rval == 0) + printf("Ending connection\en"); + else + printf("-->%s\en", buf); + } while (rval > 0); + close(msgsock); + } else + printf("Do something else\en"); + } while (TRUE); +} diff --git a/share/doc/psd/20.ipctut/streamread.c b/share/doc/psd/20.ipctut/streamread.c new file mode 100644 index 000000000000..ffad802f63e0 --- /dev/null +++ b/share/doc/psd/20.ipctut/streamread.c @@ -0,0 +1,102 @@ +.\" Copyright (c) 1986, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)streamread.c 8.1 (Berkeley) 6/8/93 +.\" +#include <sys/types.h> +#include <sys/socket.h> +#include <netinet/in.h> +#include <netdb.h> +#include <stdio.h> +#define TRUE 1 + +/* + * This program creates a socket and then begins an infinite loop. Each time + * through the loop it accepts a connection and prints out messages from it. + * When the connection breaks, or a termination message comes through, the + * program accepts a new connection. + */ + +main() +{ + int sock, length; + struct sockaddr_in server; + int msgsock; + char buf[1024]; + int rval; + int i; + + /* Create socket */ + sock = socket(AF_INET, SOCK_STREAM, 0); + if (sock < 0) { + perror("opening stream socket"); + exit(1); + } + /* Name socket using wildcards */ + server.sin_family = AF_INET; + server.sin_addr.s_addr = INADDR_ANY; + server.sin_port = 0; + if (bind(sock, &server, sizeof(server))) { + perror("binding stream socket"); + exit(1); + } + /* Find out assigned port number and print it out */ + length = sizeof(server); + if (getsockname(sock, &server, &length)) { + perror("getting socket name"); + exit(1); + } + printf("Socket has port #%d\en", ntohs(server.sin_port)); + + /* Start accepting connections */ + listen(sock, 5); + do { + msgsock = accept(sock, 0, 0); + if (msgsock == -1) + perror("accept"); + else do { + bzero(buf, sizeof(buf)); + if ((rval = read(msgsock, buf, 1024)) < 0) + perror("reading stream message"); + i = 0; + if (rval == 0) + printf("Ending connection\en"); + else + printf("-->%s\en", buf); + } while (rval != 0); + close(msgsock); + } while (TRUE); + /* + * Since this program has an infinite loop, the socket "sock" is + * never explicitly closed. However, all sockets will be closed + * automatically when a process is killed or terminates normally. + */ +} diff --git a/share/doc/psd/20.ipctut/streamwrite.c b/share/doc/psd/20.ipctut/streamwrite.c new file mode 100644 index 000000000000..6205f135cf51 --- /dev/null +++ b/share/doc/psd/20.ipctut/streamwrite.c @@ -0,0 +1,81 @@ +.\" Copyright (c) 1986, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)streamwrite.c 8.1 (Berkeley) 6/8/93 +.\" +#include <sys/types.h> +#include <sys/socket.h> +#include <netinet/in.h> +#include <netdb.h> +#include <stdio.h> + +#define DATA "Half a league, half a league . . ." + +/* + * This program creates a socket and initiates a connection with the socket + * given in the command line. One message is sent over the connection and + * then the socket is closed, ending the connection. The form of the command + * line is streamwrite hostname portnumber + */ + +main(argc, argv) + int argc; + char *argv[]; +{ + int sock; + struct sockaddr_in server; + struct hostent *hp, *gethostbyname(); + char buf[1024]; + + /* Create socket */ + sock = socket(AF_INET, SOCK_STREAM, 0); + if (sock < 0) { + perror("opening stream socket"); + exit(1); + } + /* Connect socket using name specified by command line. */ + server.sin_family = AF_INET; + hp = gethostbyname(argv[1]); + if (hp == 0) { + fprintf(stderr, "%s: unknown host\en", argv[1]); + exit(2); + } + bcopy(hp->h_addr, &server.sin_addr, hp->h_length); + server.sin_port = htons(atoi(argv[2])); + + if (connect(sock, &server, sizeof(server)) < 0) { + perror("connecting stream socket"); + exit(1); + } + if (write(sock, DATA, sizeof(DATA)) < 0) + perror("writing on stream socket"); + close(sock); +} diff --git a/share/doc/psd/20.ipctut/tutor.me b/share/doc/psd/20.ipctut/tutor.me new file mode 100644 index 000000000000..fba4583f1392 --- /dev/null +++ b/share/doc/psd/20.ipctut/tutor.me @@ -0,0 +1,939 @@ +.\" Copyright (c) 1986, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)tutor.me 8.1 (Berkeley) 8/14/93 +.\" +.oh 'Introductory 4.4BSD IPC''PSD:20-%' +.eh 'PSD:20-%''Introductory 4.4BSD IPC' +.rs +.sp 2 +.sz 14 +.ft B +.ce 2 +An Introductory 4.4BSD +Interprocess Communication Tutorial +.sz 10 +.sp 2 +.ce +.i "Stuart Sechrest" +.ft +.sp +.ce 4 +Computer Science Research Group +Computer Science Division +Department of Electrical Engineering and Computer Science +University of California, Berkeley +.sp 2 +.ce +.i ABSTRACT +.sp +.(c +.pp +Berkeley UNIX\(dg 4.4BSD offers several choices for interprocess communication. +To aid the programmer in developing programs which are comprised of +cooperating +processes, the different choices are discussed and a series of example +programs are presented. These programs +demonstrate in a simple way the use of pipes, socketpairs, sockets +and the use of datagram and stream communication. The intent of this +document is to present a few simple example programs, not to describe the +networking system in full. +.)c +.sp 2 +.(f +\(dg\|UNIX is a trademark of AT&T Bell Laboratories. +.)f +.b +.sh 1 "Goals" +.r +.pp +Facilities for interprocess communication (IPC) and networking +were a major addition to UNIX in the Berkeley UNIX 4.2BSD release. +These facilities required major additions and some changes +to the system interface. +The basic idea of this interface is to make IPC similar to file I/O. +In UNIX a process has a set of I/O descriptors, from which one reads +and to which one writes. +Descriptors may refer to normal files, to devices (including terminals), +or to communication channels. +The use of a descriptor has three phases: its creation, +its use for reading and writing, and its destruction. By using descriptors +to write files, rather than simply naming the target file in the write +call, one gains a surprising amount of flexibility. Often, the program that +creates a descriptor will be different from the program that uses the +descriptor. For example the shell can create a descriptor for the output +of the `ls' +command that will cause the listing to appear in a file rather than +on a terminal. +Pipes are another form of descriptor that have been used in UNIX +for some time. +Pipes allow one-way data transmission from one process +to another; the two processes and the pipe must be set up by a common +ancestor. +.pp +The use of descriptors is not the only communication interface +provided by UNIX. +The signal mechanism sends a tiny amount of information from one +process to another. +The signaled process receives only the signal type, +not the identity of the sender, +and the number of possible signals is small. +The signal semantics limit the flexibility of the signaling mechanism +as a means of interprocess communication. +.pp +The identification of IPC with I/O is quite longstanding in UNIX and +has proved quite successful. At first, however, IPC was limited to +processes communicating within a single machine. With Berkeley UNIX +4.2BSD this expanded to include IPC between machines. This expansion +has necessitated some change in the way that descriptors are created. +Additionally, new possibilities for the meaning of read and write have +been admitted. Originally the meanings, or semantics, of these terms +were fairly simple. When you wrote something it was delivered. When +you read something, you were blocked until the data arrived. +Other possibilities exist, +however. One can write without full assurance of delivery if one can +check later to catch occasional failures. Messages can be kept as +discrete units or merged into a stream. +One can ask to read, but insist on not waiting if nothing is immediately +available. These new possibilities are allowed in the Berkeley UNIX IPC +interface. +.pp +Thus Berkeley UNIX 4.4BSD offers several choices for IPC. +This paper presents simple examples that illustrate some of +the choices. +The reader is presumed to be familiar with the C programming language +[Kernighan & Ritchie 1978], +but not necessarily with the system calls of the UNIX system or with +processes and interprocess communication. +The paper reviews the notion of a process and the types of +communication that are supported by Berkeley UNIX 4.4BSD. +A series of examples are presented that create processes that communicate +with one another. The programs show different ways of establishing +channels of communication. +Finally, the calls that actually transfer data are reviewed. +To clearly present how communication can take place, +the example programs have been cleared of anything that +might be construed as useful work. +They can, therefore, serve as models +for the programmer trying to construct programs which are comprised of +cooperating processes. +.b +.sh 1 "Processes" +.pp +A \fIprogram\fP is both a sequence of statements and a rough way of referring +to the computation that occurs when the compiled statements are run. +A \fIprocess\fP can be thought of as a single line of control in a program. +Most programs execute some statements, go through a few loops, branch in +various directions and then end. These are single process programs. +Programs can also have a point where control splits into two independent lines, +an action called \fIforking.\fP +In UNIX these lines can never join again. A call to the system routine +\fIfork()\fP, causes a process to split in this way. +The result of this call is that two independent processes will be +running, executing exactly the same code. +Memory values will be the same for all values set before the fork, but, +subsequently, each version will be able to change only the +value of its own copy of each variable. +Initially, the only difference between the two will be the value returned by +\fIfork().\fP The parent will receive a process id for the child, +the child will receive a zero. +Calls to \fIfork(),\fP +therefore, typically precede, or are included in, an if-statement. +.pp +A process views the rest of the system through a private table of descriptors. +The descriptors can represent open files or sockets (sockets are communication +objects that will be discussed below). Descriptors are referred to +by their index numbers in the table. The first three descriptors are often +known by special names, \fI stdin, stdout\fP and \fIstderr\fP. +These are the standard input, output and error. +When a process forks, its descriptor table is copied to the child. +Thus, if the parent's standard input is being taken from a terminal +(devices are also treated as files in UNIX), the child's input will +be taken from the +same terminal. Whoever reads first will get the input. If, before forking, +the parent changes its standard input so that it is reading from a +new file, the child will take its input from the new file. It is +also possible to take input from a socket, rather than from a file. +.b +.sh 1 "Pipes" +.r +.pp +Most users of UNIX know that they can pipe the output of a +program ``prog1'' to the input of another, ``prog2,'' by typing the command +\fI``prog1 | prog2.''\fP +This is called ``piping'' the output of one program +to another because the mechanism used to transfer the output is called a +pipe. +When the user types a command, the command is read by the shell, which +decides how to execute it. If the command is simple, for example, +.i "``prog1,''" +the shell forks a process, which executes the program, prog1, and then dies. +The shell waits for this termination and then prompts for the next +command. +If the command is a compound command, +.i "``prog1 | prog2,''" +the shell creates two processes connected by a pipe. One process +runs the program, prog1, the other runs prog2. The pipe is an I/O +mechanism with two ends, or sockets. Data that is written into one socket +can be read from the other. +.(z +.ft CW +.so pipe.c +.ft +.ce 1 +Figure 1\ \ Use of a pipe +.)z +.pp +Since a program specifies its input and output only by the descriptor table +indices, which appear as variables or constants, +the input source and output destination can be changed without +changing the text of the program. +It is in this way that the shell is able to set up pipes. Before executing +prog1, the process can close whatever is at \fIstdout\fP +and replace it with one +end of a pipe. Similarly, the process that will execute prog2 can substitute +the opposite end of the pipe for +\fIstdin.\fP +.pp +Let us now examine a program that creates a pipe for communication between +its child and itself (Figure 1). +A pipe is created by a parent process, which then forks. +When a process forks, the parent's descriptor table is copied into +the child's. +.pp +In Figure 1, the parent process makes a call to the system routine +\fIpipe().\fP +This routine creates a pipe and places descriptors for the sockets +for the two ends of the pipe in the process's descriptor table. +\fIPipe()\fP +is passed an array into which it places the index numbers of the +sockets it created. +The two ends are not equivalent. The socket whose index is +returned in the low word of the array is opened for reading only, +while the socket in the high end is opened only for writing. +This corresponds to the fact that the standard input is the first +descriptor of a process's descriptor table and the standard output +is the second. After creating the pipe, the parent creates the child +with which it will share the pipe by calling \fIfork().\fP +Figure 2 illustrates the effect of a fork. +The parent process's descriptor table points to both ends of the pipe. +After the fork, both parent's and child's descriptor tables point to +the pipe. +The child can then use the pipe to send a message to the parent. +.(z +.so fig2.pic +.ce 2 +Figure 2\ \ Sharing a pipe between parent and child +.ce 0 +.)z +.pp +Just what is a pipe? +It is a one-way communication mechanism, with one end opened +for reading and the other end for writing. +Therefore, parent and child need to agree on which way to turn +the pipe, from parent to child or the other way around. +Using the same pipe for communication both from parent to child and +from child to parent would be possible (since both processes have +references to both ends), but very complicated. +If the parent and child are to have a two-way conversation, +the parent creates two pipes, one for use in each direction. +(In accordance with their plans, both parent and child in the example above +close the socket that they will not use. It is not required that unused +descriptors be closed, but it is good practice.) +A pipe is also a \fIstream\fP communication mechanism; that +is, all messages sent through the pipe are placed in order +and reliably delivered. When the reader asks for a certain +number of bytes from this +stream, he is given as many bytes as are available, up +to the amount of the request. Note that these bytes may have come from +the same call to \fIwrite()\fR or from several calls to \fIwrite()\fR +which were concatenated. +.b +.sh 1 "Socketpairs" +.r +.pp +Berkeley UNIX 4.4BSD provides a slight generalization of pipes. A pipe is a +pair of connected sockets for one-way stream communication. One may +obtain a pair of connected sockets for two-way stream communication +by calling the routine \fIsocketpair().\fP +The program in Figure 3 calls \fIsocketpair()\fP +to create such a connection. The program uses the link for +communication in both directions. Since socketpairs are +an extension of pipes, their use resembles that of pipes. +Figure 4 illustrates the result of a fork following a call to +\fIsocketpair().\fP +.pp +\fISocketpair()\fP +takes as +arguments a specification of a domain, a style of communication, and a +protocol. +These are the parameters shown in the example. +Domains and protocols will be discussed in the next section. +Briefly, +a domain is a space of names that may be bound +to sockets and implies certain other conventions. +Currently, socketpairs have only been implemented for one +domain, called the UNIX domain. +The UNIX domain uses UNIX path names for naming sockets. +It only allows communication +between sockets on the same machine. +.pp +Note that the header files +.i "<sys/socket.h>" +and +.i "<sys/types.h>." +are required in this program. +The constants AF_UNIX and SOCK_STREAM are defined in +.i "<sys/socket.h>," +which in turn requires the file +.i "<sys/types.h>" +for some of its definitions. +.(z +.ft CW +.so socketpair.c +.ft +.ce 1 +Figure 3\ \ Use of a socketpair +.)z +.(z +.so fig3.pic +.ce 1 +Figure 4\ \ Sharing a socketpair between parent and child +.)z +.b +.sh 1 "Domains and Protocols" +.r +.pp +Pipes and socketpairs are a simple solution for communicating between +a parent and child or between child processes. +What if we wanted to have processes that have no common ancestor +with whom to set up communication? +Neither standard UNIX pipes nor socketpairs are +the answer here, since both mechanisms require a common ancestor to +set up the communication. +We would like to have two processes separately create sockets +and then have messages sent between them. This is often the +case when providing or using a service in the system. This is +also the case when the communicating processes are on separate machines. +In Berkeley UNIX 4.4BSD one can create individual sockets, give them names and +send messages between them. +.pp +Sockets created by different programs use names to refer to one another; +names generally must be translated into addresses for use. +The space from which an address is drawn is referred to as a +.i domain. +There are several domains for sockets. +Two that will be used in the examples here are the UNIX domain (or AF_UNIX, +for Address Format UNIX) and the Internet domain (or AF_INET). +UNIX domain IPC is an experimental facility in 4.2BSD and 4.3BSD. +In the UNIX domain, a socket is given a path name within the file system +name space. +A file system node is created for the socket and other processes may +then refer to the socket by giving the proper pathname. +UNIX domain names, therefore, allow communication between any two processes +that work in the same file system. +The Internet domain is the UNIX implementation of the DARPA Internet +standard protocols IP/TCP/UDP. +Addresses in the Internet domain consist of a machine network address +and an identifying number, called a port. +Internet domain names allow communication between machines. +.pp +Communication follows some particular ``style.'' +Currently, communication is either through a \fIstream\fP +or by \fIdatagram.\fP +Stream communication implies several things. Communication takes +place across a connection between two sockets. The communication +is reliable, error-free, and, as in pipes, no message boundaries are +kept. Reading from a stream may result in reading the data sent from +one or several calls to \fIwrite()\fP +or only part of the data from a single call, if there is not enough room +for the entire message, or if not all the data from a large message +has been transferred. +The protocol implementing such a style will retransmit messages +received with errors. It will also return error messages if one tries to +send a message after the connection has been broken. +Datagram communication does not use connections. Each message is +addressed individually. If the address is correct, it will generally +be received, although this is not guaranteed. Often datagrams are +used for requests that require a response from the +recipient. If no response +arrives in a reasonable amount of time, the request is repeated. +The individual datagrams will be kept separate when they are read, that +is, message boundaries are preserved. +.pp +The difference in performance between the two styles of communication is +generally less important than the difference in semantics. The +performance gain that one might find in using datagrams must be weighed +against the increased complexity of the program, which must now concern +itself with lost or out of order messages. If lost messages may simply be +ignored, the quantity of traffic may be a consideration. The expense +of setting up a connection is best justified by frequent use of the connection. +Since the performance of a protocol changes as it is tuned for different +situations, it is best to seek the most up-to-date information when +making choices for a program in which performance is crucial. +.pp +A protocol is a set of rules, data formats and conventions that regulate the +transfer of data between participants in the communication. +In general, there is one protocol for each socket type (stream, +datagram, etc.) within each domain. +The code that implements a protocol +keeps track of the names that are bound to sockets, +sets up connections and transfers data between sockets, +perhaps sending the data across a network. +This code also keeps track of the names that are bound to sockets. +It is possible for several protocols, differing only in low level +details, to implement the same style of communication within +a particular domain. Although it is possible to select +which protocol should be used, for nearly all uses it is sufficient to +request the default protocol. This has been done in all of the example +programs. +.pp +One specifies the domain, style and protocol of a socket when +it is created. For example, in Figure 5a the call to \fIsocket()\fP +causes the creation of a datagram socket with the default protocol +in the UNIX domain. +.b +.sh 1 "Datagrams in the UNIX Domain" +.r +.(z +.ft CW +.so udgramread.c +.ft +.ce 1 +Figure 5a\ \ Reading UNIX domain datagrams +.)z +.pp +Let us now look at two programs that create sockets separately. +The programs in Figures 5a and 5b use datagram communication +rather than a stream. +The structure used to name UNIX domain sockets is defined +in the file \fI<sys/un.h>.\fP +The definition has also been included in the example for clarity. +.pp +Each program creates a socket with a call to \fIsocket().\fP +These sockets are in the UNIX domain. +Once a name has been decided upon it is attached to a socket by the +system call \fIbind().\fP +The program in Figure 5a uses the name ``socket'', +which it binds to its socket. +This name will appear in the working directory of the program. +The routines in Figure 5b use its +socket only for sending messages. It does not create a name for +the socket because no other process has to refer to it. +.(z +.ft CW +.so udgramsend.c +.ft +.ce 1 +Figure 5b\ \ Sending a UNIX domain datagrams +.)z +.pp +Names in the UNIX domain are path names. Like file path names they may +be either absolute (e.g. ``/dev/imaginary'') or relative (e.g. ``socket''). +Because these names are used to allow processes to rendezvous, relative +path names can pose difficulties and should be used with care. +When a name is bound into the name space, a file (inode) is allocated in the +file system. If +the inode is not deallocated, the name will continue to exist even after +the bound socket is closed. This can cause subsequent runs of a program +to find that a name is unavailable, and can cause +directories to fill up with these +objects. The names are removed by calling \fIunlink()\fP or using +the \fIrm\fP\|(1) command. +Names in the UNIX domain are only used for rendezvous. They are not used +for message delivery once a connection is established. Therefore, in +contrast with the Internet domain, unbound sockets need not be (and are +not) automatically given addresses when they are connected. +.pp +There is no established means of communicating names to interested +parties. In the example, the program in Figure 5b gets the +name of the socket to which it will send its message through its +command line arguments. Once a line of communication has been created, +one can send the names of additional, perhaps new, sockets over the link. +Facilities will have to be built that will make the distribution of +names less of a problem than it now is. +.b +.sh 1 "Datagrams in the Internet Domain" +.r +.(z +.ft CW +.so dgramread.c +.ft +.ce 1 +Figure 6a\ \ Reading Internet domain datagrams +.)z +.pp +The examples in Figure 6a and 6b are very close to the previous example +except that the socket is in the Internet domain. +The structure of Internet domain addresses is defined in the file +\fI<netinet/in.h>\fP. +Internet addresses specify a host address (a 32-bit number) +and a delivery slot, or port, on that +machine. These ports are managed by the system routines that implement +a particular protocol. +Unlike UNIX domain names, Internet socket names are not entered into +the file system and, therefore, +they do not have to be unlinked after the socket has been closed. +When a message must be sent between machines it is sent to +the protocol routine on the destination machine, which interprets the +address to determine to which socket the message should be delivered. +Several different protocols may be active on +the same machine, but, in general, they will not communicate with one another. +As a result, different protocols are allowed to use the same port numbers. +Thus, implicitly, an Internet address is a triple including a protocol as +well as the port and machine address. +An \fIassociation\fP is a temporary or permanent specification +of a pair of communicating sockets. +An association is thus identified by the tuple +<\fIprotocol, local machine address, local port, +remote machine address, remote port\fP>. +An association may be transient when using datagram sockets; +the association actually exists during a \fIsend\fP operation. +.(z +.ft CW +.so dgramsend.c +.ft +.ce 1 +Figure 6b\ \ Sending an Internet domain datagram +.)z +.pp +The protocol for a socket is chosen when the socket is created. The +local machine address for a socket can be any valid network address of the +machine, if it has more than one, or it can be the wildcard value +INADDR_ANY. +The wildcard value is used in the program in Figure 6a. +If a machine has several network addresses, it is likely +that messages sent to any of the addresses should be deliverable to +a socket. This will be the case if the wildcard value has been chosen. +Note that even if the wildcard value is chosen, a program sending messages +to the named socket must specify a valid network address. One can be willing +to receive from ``anywhere,'' but one cannot send a message ``anywhere.'' +The program in Figure 6b is given the destination host name as a command +line argument. +To determine a network address to which it can send the message, it looks +up +the host address by the call to \fIgethostbyname()\fP. +The returned structure includes the host's network address, +which is copied into the structure specifying the +destination of the message. +.pp +The port number can be thought of as the number of a mailbox, into +which the protocol places one's messages. Certain daemons, offering +certain advertised services, have reserved +or ``well-known'' port numbers. These fall in the range +from 1 to 1023. Higher numbers are available to general users. +Only servers need to ask for a particular number. +The system will assign an unused port number when an address +is bound to a socket. +This may happen when an explicit \fIbind\fP +call is made with a port number of 0, or +when a \fIconnect\fP or \fIsend\fP +is performed on an unbound socket. +Note that port numbers are not automatically reported back to the user. +After calling \fIbind(),\fP asking for port 0, one may call +\fIgetsockname()\fP to discover what port was actually assigned. +The routine \fIgetsockname()\fP +will not work for names in the UNIX domain. +.pp +The format of the socket address is specified in part by standards within the +Internet domain. The specification includes the order of the bytes in +the address. Because machines differ in the internal representation +they ordinarily use +to represent integers, printing out the port number as returned by +\fIgetsockname()\fP may result in a misinterpretation. To +print out the number, it is necessary to use the routine \fIntohs()\fP +(for \fInetwork to host: short\fP) to convert the number from the +network representation to the host's representation. On some machines, +such as 68000-based machines, this is a null operation. On others, +such as VAXes, this results in a swapping of bytes. Another routine +exists to convert a short integer from the host format to the network format, +called \fIhtons()\fP; similar routines exist for long integers. +For further information, refer to the +entry for \fIbyteorder\fP in section 3 of the manual. +.b +.sh 1 "Connections" +.r +.pp +To send data between stream sockets (having communication style SOCK_STREAM), +the sockets must be connected. +Figures 7a and 7b show two programs that create such a connection. +The program in 7a is relatively simple. +To initiate a connection, this program simply creates +a stream socket, then calls \fIconnect()\fP, +specifying the address of the socket to which +it wishes its socket connected. Provided that the target socket exists and +is prepared to handle a connection, connection will be complete, +and the program can begin to send +messages. Messages will be delivered in order without message +boundaries, as with pipes. The connection is destroyed when either +socket is closed (or soon thereafter). If a process persists +in sending messages after the connection is closed, a SIGPIPE signal +is sent to the process by the operating system. Unless explicit action +is taken to handle the signal (see the manual page for \fIsignal\fP +or \fIsigvec\fP), +the process will terminate and the shell +will print the message ``broken pipe.'' +.(z +.ft CW +.so streamwrite.c +.ft +.ce 1 +Figure 7a\ \ Initiating an Internet domain stream connection +.)z +.(z +.ft CW +.so streamread.c +.ft +.ce 1 +Figure 7b\ \ Accepting an Internet domain stream connection +.sp 2 +.ft CW +.so strchkread.c +.ft +.ce 1 +Figure 7c\ \ Using select() to check for pending connections +.)z +.(z +.so fig8.pic +.sp +.ce 1 +Figure 8\ \ Establishing a stream connection +.)z +.pp +Forming a connection is asymmetrical; one process, such as the +program in Figure 7a, requests a connection with a particular socket, +the other process accepts connection requests. +Before a connection can be accepted a socket must be created and an address +bound to it. This +situation is illustrated in the top half of Figure 8. Process 2 +has created a socket and bound a port number to it. Process 1 has created an +unnamed socket. +The address bound to process 2's socket is then made known to process 1 and, +perhaps to several other potential communicants as well. +If there are several possible communicants, +this one socket might receive several requests for connections. +As a result, a new socket is created for each connection. This new socket +is the endpoint for communication within this process for this connection. +A connection may be destroyed by closing the corresponding socket. +.pp +The program in Figure 7b is a rather trivial example of a server. It +creates a socket to which it binds a name, which it then advertises. +(In this case it prints out the socket number.) The program then calls +\fIlisten()\fP for this socket. +Since several clients may attempt to connect more or less +simultaneously, a queue of pending connections is maintained in the system +address space. \fIListen()\fP +marks the socket as willing to accept connections and initializes the queue. +When a connection is requested, it is listed in the queue. If the +queue is full, an error status may be returned to the requester. +The maximum length of this queue is specified by the second argument of +\fIlisten()\fP; the maximum length is limited by the system. +Once the listen call has been completed, the program enters +an infinite loop. On each pass through the loop, a new connection is +accepted and removed from the queue, and, hence, a new socket for the +connection is created. The bottom half of Figure 8 shows the result of +Process 1 connecting with the named socket of Process 2, and Process 2 +accepting the connection. After the connection is created, the +service, in this case printing out the messages, is performed and the +connection socket closed. The \fIaccept()\fP +call will take a pending connection +request from the queue if one is available, or block waiting for a request. +Messages are read from the connection socket. +Reads from an active connection will normally block until data is available. +The number of bytes read is returned. When a connection is destroyed, +the read call returns immediately. The number of bytes returned will +be zero. +.pp +The program in Figure 7c is a slight variation on the server in Figure 7b. +It avoids blocking when there are no pending connection requests by +calling \fIselect()\fP +to check for pending requests before calling \fIaccept().\fP +This strategy is useful when connections may be received +on more than one socket, or when data may arrive on other connected +sockets before another connection request. +.pp +The programs in Figures 9a and 9b show a program using stream communication +in the UNIX domain. Streams in the UNIX domain can be used for this sort +of program in exactly the same way as Internet domain streams, except for +the form of the names and the restriction of the connections to a single +file system. There are some differences, however, in the functionality of +streams in the two domains, notably in the handling of +\fIout-of-band\fP data (discussed briefly below). These differences +are beyond the scope of this paper. +.(z +.ft CW +.so ustreamwrite.c +.ft +.ce 1 +Figure 9a\ \ Initiating a UNIX domain stream connection +.sp 2 +.ft CW +.so ustreamread.c +.ft +.ce 1 +Figure 9b\ \ Accepting a UNIX domain stream connection +.)z +.b +.sh 1 "Reads, Writes, Recvs, etc." +.r +.pp +UNIX 4.4BSD has several system calls for reading and writing information. +The simplest calls are \fIread() \fP and \fIwrite().\fP \fIWrite()\fP +takes as arguments the index of a descriptor, a pointer to a buffer +containing the data and the size of the data. +The descriptor may indicate either a file or a connected socket. +``Connected'' can mean either a connected stream socket (as described +in Section 8) or a datagram socket for which a \fIconnect()\fP +call has provided a default destination (see the \fIconnect()\fP manual page). +\fIRead()\fP also takes a descriptor that indicates either a file or a socket. +\fIWrite()\fP requires a connected socket since no destination is +specified in the parameters of the system call. +\fIRead()\fP can be used for either a connected or an unconnected socket. +These calls are, therefore, quite flexible and may be used to +write applications that require no assumptions about the source of +their input or the destination of their output. +There are variations on \fIread() \fP and \fIwrite()\fP +that allow the source and destination of the input and output to use +several separate buffers, while retaining the flexibility to handle +both files and sockets. These are \fIreadv()\fP and \fI writev(),\fP +for read and write \fIvector.\fP +.pp +It is sometimes necessary to send high priority data over a +connection that may have unread low priority data at the +other end. For example, a user interface process may be interpreting +commands and sending them on to another process through a stream connection. +The user interface may have filled the stream with as yet unprocessed +requests when the user types +a command to cancel all outstanding requests. +Rather than have the high priority data wait +to be processed after the low priority data, it is possible to +send it as \fIout-of-band\fP +(OOB) data. The notification of pending OOB data results in the generation of +a SIGURG signal, if this signal has been enabled (see the manual +page for \fIsignal\fP or \fIsigvec\fP). +See [Leffler 1986] for a more complete description of the OOB mechanism. +There are a pair of calls similar to \fIread\fP and \fIwrite\fP +that allow options, including sending +and receiving OOB information; these are \fI send()\fP +and \fIrecv().\fP +These calls are used only with sockets; specifying a descriptor for a file will +result in the return of an error status. These calls also allow +\fIpeeking\fP at data in a stream. +That is, they allow a process to read data without removing the data from +the stream. One use of this facility is to read ahead in a stream +to determine the size of the next item to be read. +When not using these options, these calls have the same functions as +\fIread()\fP and \fIwrite().\fP +.pp +To send datagrams, one must be allowed to specify the destination. +The call \fIsendto()\fP +takes a destination address as an argument and is therefore used for +sending datagrams. The call \fIrecvfrom()\fP +is often used to read datagrams, since this call returns the address +of the sender, if it is available, along with the data. +If the identity of the sender does not matter, one may use \fIread()\fP +or \fIrecv().\fP +.pp +Finally, there are a pair of calls that allow the sending and +receiving of messages from multiple buffers, when the address of the +recipient must be specified. These are \fIsendmsg()\fP and +\fIrecvmsg().\fP +These calls are actually quite general and have other uses, +including, in the UNIX domain, the transmission of a file descriptor from one +process to another. +.pp +The various options for reading and writing are shown in Figure 10, +together with their parameters. The parameters for each system call +reflect the differences in function of the different calls. +In the examples given in this paper, the calls \fIread()\fP and +\fIwrite()\fP have been used whenever possible. +.(z +.ft CW + /* + * The variable descriptor may be the descriptor of either a file + * or of a socket. + */ + cc = read(descriptor, buf, nbytes) + int cc, descriptor; char *buf; int nbytes; + + /* + * An iovec can include several source buffers. + */ + cc = readv(descriptor, iov, iovcnt) + int cc, descriptor; struct iovec *iov; int iovcnt; + + cc = write(descriptor, buf, nbytes) + int cc, descriptor; char *buf; int nbytes; + + cc = writev(descriptor, iovec, ioveclen) + int cc, descriptor; struct iovec *iovec; int ioveclen; + + /* + * The variable ``sock'' must be the descriptor of a socket. + * Flags may include MSG_OOB and MSG_PEEK. + */ + cc = send(sock, msg, len, flags) + int cc, sock; char *msg; int len, flags; + + cc = sendto(sock, msg, len, flags, to, tolen) + int cc, sock; char *msg; int len, flags; + struct sockaddr *to; int tolen; + + cc = sendmsg(sock, msg, flags) + int cc, sock; struct msghdr msg[]; int flags; + + cc = recv(sock, buf, len, flags) + int cc, sock; char *buf; int len, flags; + + cc = recvfrom(sock, buf, len, flags, from, fromlen) + int cc, sock; char *buf; int len, flags; + struct sockaddr *from; int *fromlen; + + cc = recvmsg(sock, msg, flags) + int cc, socket; struct msghdr msg[]; int flags; +.ft +.sp 1 +.ce 1 +Figure 10\ \ Varieties of read and write commands +.)z +.b +.sh 1 "Choices" +.r +.pp +This paper has presented examples of some of the forms +of communication supported by +Berkeley UNIX 4.4BSD. These have been presented in an order chosen for +ease of presentation. It is useful to review these options emphasizing the +factors that make each attractive. +.pp +Pipes have the advantage of portability, in that they are supported in all +UNIX systems. They also are relatively +simple to use. Socketpairs share this simplicity and have the additional +advantage of allowing bidirectional communication. The major shortcoming +of these mechanisms is that they require communicating processes to be +descendants of a common process. They do not allow intermachine communication. +.pp +The two communication domains, UNIX and Internet, allow processes with no common +ancestor to communicate. +Of the two, only the Internet domain allows +communication between machines. +This makes the Internet domain a necessary +choice for processes running on separate machines. +.pp +The choice between datagrams and stream communication is best made by +carefully considering the semantic and performance +requirements of the application. +Streams can be both advantageous and disadvantageous. One disadvantage +is that a process is only allowed a limited number of open streams, +as there are usually only 64 entries available in the open descriptor +table. This can cause problems if a single server must talk with a large +number of clients. +Another is that for delivering a short message the stream setup and +teardown time can be unnecessarily long. Weighed against this are +the reliability built into the streams. This will often be the +deciding factor in favor of streams. +.b +.sh 1 "What to do Next" +.r +.pp +Many of the examples presented here can serve as models for multiprocess +programs and for programs distributed across several machines. +In developing a new multiprocess program, it is often easiest to +first write the code to create the processes and communication paths. +After this code is debugged, the code specific to the application can +be added. +.pp +An introduction to the UNIX system and programming using UNIX system calls +can be found in [Kernighan and Pike 1984]. +Further documentation of the Berkeley UNIX 4.4BSD IPC mechanisms can be +found in [Leffler et al. 1986]. +More detailed information about particular calls and protocols +is provided in sections +2, 3 and 4 of the +UNIX Programmer's Manual [CSRG 1986]. +In particular the following manual pages are relevant: +.(b +.TS +l l. +creating and naming sockets socket(2), bind(2) +establishing connections listen(2), accept(2), connect(2) +transferring data read(2), write(2), send(2), recv(2) +addresses inet(4F) +protocols tcp(4P), udp(4P). +.TE +.)b +.(b +.sp +.b +Acknowledgements +.pp +I would like to thank Sam Leffler and Mike Karels for their help in +understanding the IPC mechanisms and all the people whose comments +have helped in writing and improving this report. +.pp +This work was sponsored by the Defense Advanced Research Projects Agency +(DoD), ARPA Order No. 4031, monitored by the Naval Electronics Systems +Command under contract No. N00039-C-0235. +The views and conclusions contained in this document are those of the +author and should not be interpreted as representing official policies, +either expressed or implied, of the Defense Research Projects Agency +or of the US Government. +.)b +.(b +.sp +.b +References +.r +.sp +.ls 1 +B.W. Kernighan & R. Pike, 1984, +.i "The UNIX Programming Environment." +Englewood Cliffs, N.J.: Prentice-Hall. +.sp +.ls 1 +B.W. Kernighan & D.M. Ritchie, 1978, +.i "The C Programming Language," +Englewood Cliffs, N.J.: Prentice-Hall. +.sp +.ls 1 +S.J. Leffler, R.S. Fabry, W.N. Joy, P. Lapsley, S. Miller & C. Torek, 1986, +.i "An Advanced 4.4BSD Interprocess Communication Tutorial." +Computer Systems Research Group, +Department of Electrical Engineering and Computer Science, +University of California, Berkeley. +.sp +.ls 1 +Computer Systems Research Group, 1986, +.i "UNIX Programmer's Manual, 4.4 Berkeley Software Distribution." +Computer Systems Research Group, +Department of Electrical Engineering and Computer Science, +University of California, Berkeley. +.)b diff --git a/share/doc/psd/20.ipctut/udgramread.c b/share/doc/psd/20.ipctut/udgramread.c new file mode 100644 index 000000000000..2cb605d76a1d --- /dev/null +++ b/share/doc/psd/20.ipctut/udgramread.c @@ -0,0 +1,80 @@ +.\" Copyright (c) 1986, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)udgramread.c 8.1 (Berkeley) 6/8/93 +.\" +#include <sys/types.h> +#include <sys/socket.h> +#include <sys/un.h> + +/* + * In the included file <sys/un.h> a sockaddr_un is defined as follows + * struct sockaddr_un { + * short sun_family; + * char sun_path[108]; + * }; + */ + +#include <stdio.h> + +#define NAME "socket" + +/* + * This program creates a UNIX domain datagram socket, binds a name to it, + * then reads from the socket. + */ +main() +{ + int sock, length; + struct sockaddr_un name; + char buf[1024]; + + /* Create socket from which to read. */ + sock = socket(AF_UNIX, SOCK_DGRAM, 0); + if (sock < 0) { + perror("opening datagram socket"); + exit(1); + } + /* Create name. */ + name.sun_family = AF_UNIX; + strcpy(name.sun_path, NAME); + if (bind(sock, &name, sizeof(struct sockaddr_un))) { + perror("binding name to datagram socket"); + exit(1); + } + printf("socket -->%s\en", NAME); + /* Read from the socket */ + if (read(sock, buf, 1024) < 0) + perror("receiving datagram packet"); + printf("-->%s\en", buf); + close(sock); + unlink(NAME); +} diff --git a/share/doc/psd/20.ipctut/udgramsend.c b/share/doc/psd/20.ipctut/udgramsend.c new file mode 100644 index 000000000000..3e3ba932403e --- /dev/null +++ b/share/doc/psd/20.ipctut/udgramsend.c @@ -0,0 +1,68 @@ +.\" Copyright (c) 1986, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)udgramsend.c 8.1 (Berkeley) 6/8/93 +.\" +#include <sys/types.h> +#include <sys/socket.h> +#include <sys/un.h> +#include <stdio.h> + +#define DATA "The sea is calm tonight, the tide is full . . ." + +/* + * Here I send a datagram to a receiver whose name I get from the command + * line arguments. The form of the command line is udgramsend pathname + */ + +main(argc, argv) + int argc; + char *argv[]; +{ + int sock; + struct sockaddr_un name; + + /* Create socket on which to send. */ + sock = socket(AF_UNIX, SOCK_DGRAM, 0); + if (sock < 0) { + perror("opening datagram socket"); + exit(1); + } + /* Construct name of socket to send to. */ + name.sun_family = AF_UNIX; + strcpy(name.sun_path, argv[1]); + /* Send message. */ + if (sendto(sock, DATA, sizeof(DATA), 0, + &name, sizeof(struct sockaddr_un)) < 0) { + perror("sending datagram message"); + } + close(sock); +} diff --git a/share/doc/psd/20.ipctut/ustreamread.c b/share/doc/psd/20.ipctut/ustreamread.c new file mode 100644 index 000000000000..97fadb95f873 --- /dev/null +++ b/share/doc/psd/20.ipctut/ustreamread.c @@ -0,0 +1,96 @@ +.\" Copyright (c) 1986, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)ustreamread.c 8.1 (Berkeley) 6/8/93 +.\" +#include <sys/types.h> +#include <sys/socket.h> +#include <sys/un.h> +#include <stdio.h> + +#define NAME "socket" + +/* + * This program creates a socket in the UNIX domain and binds a name to it. + * After printing the socket's name it begins a loop. Each time through the + * loop it accepts a connection and prints out messages from it. When the + * connection breaks, or a termination message comes through, the program + * accepts a new connection. + */ +main() +{ + int sock, msgsock, rval; + struct sockaddr_un server; + char buf[1024]; + + /* Create socket */ + sock = socket(AF_UNIX, SOCK_STREAM, 0); + if (sock < 0) { + perror("opening stream socket"); + exit(1); + } + /* Name socket using file system name */ + server.sun_family = AF_UNIX; + strcpy(server.sun_path, NAME); + if (bind(sock, &server, sizeof(struct sockaddr_un))) { + perror("binding stream socket"); + exit(1); + } + printf("Socket has name %s\en", server.sun_path); + /* Start accepting connections */ + listen(sock, 5); + for (;;) { + msgsock = accept(sock, 0, 0); + if (msgsock == -1) + perror("accept"); + else do { + bzero(buf, sizeof(buf)); + if ((rval = read(msgsock, buf, 1024)) < 0) + perror("reading stream message"); + else if (rval == 0) + printf("Ending connection\en"); + else + printf("-->%s\en", buf); + } while (rval > 0); + close(msgsock); + } + /* + * The following statements are not executed, because they follow an + * infinite loop. However, most ordinary programs will not run + * forever. In the UNIX domain it is necessary to tell the file + * system that one is through using NAME. In most programs one uses + * the call unlink() as below. Since the user will have to kill this + * program, it will be necessary to remove the name by a command from + * the shell. + */ + close(sock); + unlink(NAME); +} diff --git a/share/doc/psd/20.ipctut/ustreamwrite.c b/share/doc/psd/20.ipctut/ustreamwrite.c new file mode 100644 index 000000000000..bdc0b95d2d45 --- /dev/null +++ b/share/doc/psd/20.ipctut/ustreamwrite.c @@ -0,0 +1,71 @@ +.\" Copyright (c) 1986, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)ustreamwrite.c 8.1 (Berkeley) 6/8/93 +.\" +#include <sys/types.h> +#include <sys/socket.h> +#include <sys/un.h> +#include <stdio.h> + +#define DATA "Half a league, half a league . . ." + +/* + * This program connects to the socket named in the command line and sends a + * one line message to that socket. The form of the command line is + * ustreamwrite pathname + */ +main(argc, argv) + int argc; + char *argv[]; +{ + int sock; + struct sockaddr_un server; + char buf[1024]; + + /* Create socket */ + sock = socket(AF_UNIX, SOCK_STREAM, 0); + if (sock < 0) { + perror("opening stream socket"); + exit(1); + } + /* Connect socket using name specified by command line. */ + server.sun_family = AF_UNIX; + strcpy(server.sun_path, argv[1]); + + if (connect(sock, &server, sizeof(struct sockaddr_un)) < 0) { + close(sock); + perror("connecting stream socket"); + exit(1); + } + if (write(sock, DATA, sizeof(DATA)) < 0) + perror("writing on stream socket"); +} diff --git a/share/doc/psd/21.ipc/0.t b/share/doc/psd/21.ipc/0.t new file mode 100644 index 000000000000..d28199a1bd3c --- /dev/null +++ b/share/doc/psd/21.ipc/0.t @@ -0,0 +1,93 @@ +.\" Copyright (c) 1986, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)0.t 8.1 (Berkeley) 6/8/93 +.\" +.EH 'PSD:21-%''Advanced 4.4BSD IPC Tutorial' +.OH 'Advanced 4.4BSD IPC Tutorial''PSD:21-%' +.ds lq `` +.ds rq '' +.de DT +.if t .ta .5i 1.25i 2.5i 3.75i +.\" 3.5i went to 3.8i +.if n .ta .7i 1.75i 3.8i +.. +.bd S B 3 +.TL +An Advanced 4.4BSD Interprocess Communication Tutorial +.AU +Samuel J. Leffler +.AU +Robert S. Fabry +.AU +William N. Joy +.AU +Phil Lapsley +.AI +Computer Systems Research Group +Department of Electrical Engineering and Computer Science +University of California, Berkeley +Berkeley, California 94720 +.sp 2 +.AU +Steve Miller +.AU +Chris Torek +.AI +Heterogeneous Systems Laboratory +Department of Computer Science +University of Maryland, College Park +College Park, Maryland 20742 +.de IR +\fI\\$1\fP\\$2 +.. +.de UX +UNIX\\$1 +.. +.AB +.PP +.FS +* \s-2UNIX\s0 is a trademark of UNIX System Laboratories, Inc. +in the US and some other countries. +.FE +This document provides an introduction to the interprocess +communication facilities included in the +4.4BSD release of the +.UX * +system. +.PP +It discusses the overall model for interprocess communication +and introduces the interprocess communication primitives +which have been added to the system. The majority of the +document considers the use of these primitives in developing +applications. The reader is expected to be familiar with +the C programming language as all examples are written in C. +.AE diff --git a/share/doc/psd/21.ipc/1.t b/share/doc/psd/21.ipc/1.t new file mode 100644 index 000000000000..f4e48ffc23dd --- /dev/null +++ b/share/doc/psd/21.ipc/1.t @@ -0,0 +1,106 @@ +.\" Copyright (c) 1986, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)1.t 8.1 (Berkeley) 8/14/93 +.\" +.\".ds LH "4.4BSD IPC Primer +.\".ds RH Introduction +.\".ds RF "Leffler/Fabry/Joy +.\".ds LF "\*(DY +.\".ds CF " +.nr H1 1 +.LP +.bp +.LG +.B +.ce +1. INTRODUCTION +.sp 2 +.R +.NL +One of the most important additions to UNIX in 4.2BSD was interprocess +communication. +These facilities were the result of +more than two years of discussion and research. The facilities +provided in 4.2BSD incorporated many of the ideas from current +research, while trying to maintain the UNIX philosophy of +simplicity and conciseness. +The 4.3BSD release of Berkeley UNIX +improved upon some of the IPC facilities +while providing an upward-compatible interface. +4.4BSD adds support for ISO protocols and IP multicasting. +The BSD interprocess communication +facilities have become a defacto standard for UNIX. +.PP +UNIX has previously been very weak in the area of interprocess +communication. Prior to the 4BSD facilities, the only +standard mechanism which allowed two processes to communicate were +pipes (the mpx files which were part of Version 7 were +experimental). Unfortunately, pipes are very restrictive +in that +the two communicating processes must be related through a +common ancestor. +Further, the semantics of pipes makes them almost impossible +to maintain in a distributed environment. +.PP +Earlier attempts at extending the IPC facilities of UNIX have +met with mixed reaction. The majority of the problems have +been related to the fact that these facilities have been tied to +the UNIX file system, either through naming or implementation. +Consequently, the IPC facilities provided in 4.2BSD were +designed as a totally independent subsystem. The BSD IPC +allows processes to rendezvous in many ways. +Processes may rendezvous through a UNIX file system-like +name space (a space where all names are path names) +as well as through a +network name space. In fact, new name spaces may +be added at a future time with only minor changes visible +to users. Further, the communication facilities +have been extended to include more than the simple byte stream +provided by a pipe. These extensions have resulted +in a completely new part of the system which users will need +time to familiarize themselves with. It is likely that as +more use is made of these facilities they will be refined; +only time will tell. +.PP +This document provides a high-level description +of the IPC facilities in 4.4BSD and their use. +It is designed to complement the manual pages for the IPC primitives +by examples of their use. +The remainder of this document is organized in four sections. +Section 2 introduces the IPC-related system calls and the basic model +of communication. Section 3 describes some of the supporting +library routines users may find useful in constructing distributed +applications. Section 4 is concerned with the client/server model +used in developing applications and includes examples of the +two major types of servers. Section 5 delves into advanced topics +which sophisticated users are likely to encounter when using +the IPC facilities. diff --git a/share/doc/psd/21.ipc/2.t b/share/doc/psd/21.ipc/2.t new file mode 100644 index 000000000000..6f0845436756 --- /dev/null +++ b/share/doc/psd/21.ipc/2.t @@ -0,0 +1,714 @@ +.\" Copyright (c) 1986, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)2.t 8.1 (Berkeley) 8/14/93 +.\" +.\".ds RH "Basics +.bp +.nr H1 2 +.nr H2 0 +.\" The next line is a major hack to get around internal changes in the groff +.\" implementation of .NH. +.nr nh*hl 1 +.bp +.LG +.B +.ce +2. BASICS +.sp 2 +.R +.NL +.PP +The basic building block for communication is the \fIsocket\fP. +A socket is an endpoint of communication to which a name may +be \fIbound\fP. Each socket in use has a \fItype\fP +and one or more associated processes. Sockets exist within +\fIcommunication domains\fP. +A communication domain is an +abstraction introduced to bundle common properties of +processes communicating through sockets. +One such property is the scheme used to name sockets. For +example, in the UNIX communication domain sockets are +named with UNIX path names; e.g. a +socket may be named \*(lq/dev/foo\*(rq. Sockets normally +exchange data only with +sockets in the same domain (it may be possible to cross domain +boundaries, but only if some translation process is +performed). The +4.4BSD IPC facilities support four separate communication domains: +the UNIX domain, for on-system communication; +the Internet domain, which is used by +processes which communicate +using the Internet standard communication protocols; +the NS domain, which is used by processes which +communicate using the Xerox standard communication +protocols*; +.FS +* See \fIInternet Transport Protocols\fP, Xerox System Integration +Standard (XSIS)028112 for more information. This document is +almost a necessity for one trying to write NS applications. +.FE +and the ISO OSI protocols, which are not documented in this tutorial. +The underlying communication +facilities provided by these domains have a significant influence +on the internal system implementation as well as the interface to +socket facilities available to a user. An example of the +latter is that a socket \*(lqoperating\*(rq in the UNIX domain +sees a subset of the error conditions which are possible +when operating in the Internet (or NS) domain. +.NH 2 +Socket types +.PP +Sockets are +typed according to the communication properties visible to a +user. +Processes are presumed to communicate only between sockets of +the same type, although there is +nothing that prevents communication between sockets of different +types should the underlying communication +protocols support this. +.PP +Four types of sockets currently are available to a user. +A \fIstream\fP socket provides for the bidirectional, reliable, +sequenced, and unduplicated flow of data without record boundaries. +Aside from the bidirectionality of data flow, a pair of connected +stream sockets provides an interface nearly identical to that of pipes\(dg. +.FS +\(dg In the UNIX domain, in fact, the semantics are identical and, +as one might expect, pipes have been implemented internally +as simply a pair of connected stream sockets. +.FE +.PP +A \fIdatagram\fP socket supports bidirectional flow of data which +is not promised to be sequenced, reliable, or unduplicated. +That is, a process +receiving messages on a datagram socket may find messages duplicated, +and, possibly, +in an order different from the order in which it was sent. +An important characteristic of a datagram +socket is that record boundaries in data are preserved. Datagram +sockets closely model the facilities found in many contemporary +packet switched networks such as the Ethernet. +.PP +A \fIraw\fP socket provides users access to +the underlying communication +protocols which support socket abstractions. +These sockets are normally datagram oriented, though their +exact characteristics are dependent on the interface provided by +the protocol. Raw sockets are not intended for the general user; they +have been provided mainly for those interested in developing new +communication protocols, or for gaining access to some of the more +esoteric facilities of an existing protocol. The use of raw sockets +is considered in section 5. +.PP +A \fIsequenced packet\fP socket is similar to a stream socket, +with the exception that record boundaries are preserved. This +interface is provided only as part of the NS socket abstraction, +and is very important in most serious NS applications. +Sequenced-packet sockets allow the user to manipulate the +SPP or IDP headers on a packet or a group of packets either +by writing a prototype header along with whatever data is +to be sent, or by specifying a default header to be used with +all outgoing data, and allows the user to receive the headers +on incoming packets. The use of these options is considered in +section 5. +.PP +Another potential socket type which has interesting properties is +the \fIreliably delivered +message\fP socket. +The reliably delivered message socket has +similar properties to a datagram socket, but with +reliable delivery. There is currently no support for this +type of socket, but a reliably delivered message protocol +similar to Xerox's Packet Exchange Protocol (PEX) may be +simulated at the user level. More information on this topic +can be found in section 5. +.NH 2 +Socket creation +.PP +To create a socket the \fIsocket\fP system call is used: +.DS +s = socket(domain, type, protocol); +.DE +This call requests that the system create a socket in the specified +\fIdomain\fP and of the specified \fItype\fP. A particular protocol may +also be requested. If the protocol is left unspecified (a value +of 0), the system will select an appropriate protocol from those +protocols which comprise the communication domain and which +may be used to support the requested socket type. The user is +returned a descriptor (a small integer number) which may be used +in later system calls which operate on sockets. The domain is specified as +one of the manifest constants defined in the file <\fIsys/socket.h\fP>. +For the UNIX domain the constant is AF_UNIX*; for the Internet +.FS +* The manifest constants are named AF_whatever as they indicate +the ``address format'' to use in interpreting names. +.FE +domain AF_INET; and for the NS domain, AF_NS. +The socket types are also defined in this file +and one of SOCK_STREAM, SOCK_DGRAM, SOCK_RAW, or SOCK_SEQPACKET +must be specified. +To create a stream socket in the Internet domain the following +call might be used: +.DS +s = socket(AF_INET, SOCK_STREAM, 0); +.DE +This call would result in a stream socket being created with the TCP +protocol providing the underlying communication support. To +create a datagram socket for on-machine use the call might +be: +.DS +s = socket(AF_UNIX, SOCK_DGRAM, 0); +.DE +.PP +The default protocol (used when the \fIprotocol\fP argument to the +\fIsocket\fP call is 0) should be correct for most every +situation. However, it is possible to specify a protocol +other than the default; this will be covered in +section 5. +.PP +There are several reasons a socket call may fail. Aside from +the rare occurrence of lack of memory (ENOBUFS), a socket +request may fail due to a request for an unknown protocol +(EPROTONOSUPPORT), or a request for a type of socket for +which there is no supporting protocol (EPROTOTYPE). +.NH 2 +Binding local names +.PP +A socket is created without a name. Until a name is bound +to a socket, processes have no way to reference it and, consequently, +no messages may be received on it. +Communicating processes are bound +by an \fIassociation\fP. In the Internet and NS domains, +an association +is composed of local and foreign +addresses, and local and foreign ports, +while in the UNIX domain, an association is composed of +local and foreign path names (the phrase ``foreign pathname'' +means a pathname created by a foreign process, not a pathname +on a foreign system). +In most domains, associations must be unique. +In the Internet domain there +may never be duplicate <protocol, local address, local port, foreign +address, foreign port> tuples. UNIX domain sockets need not always +be bound to a name, but when bound +there may never be duplicate <protocol, local pathname, foreign +pathname> tuples. +The pathnames may not refer to files +already existing on the system +in 4.3; the situation may change in future releases. +.PP +The \fIbind\fP system call allows a process to specify half of +an association, <local address, local port> +(or <local pathname>), while the \fIconnect\fP +and \fIaccept\fP primitives are used to complete a socket's association. +.PP +In the Internet domain, +binding names to sockets can be fairly complex. +Fortunately, it is usually not necessary to specifically bind an +address and port number to a socket, because the +\fIconnect\fP and \fIsend\fP calls will automatically +bind an appropriate address if they are used with an +unbound socket. The process of binding names to NS +sockets is similar in most ways to that of +binding names to Internet sockets. +.PP +The \fIbind\fP system call is used as follows: +.DS +bind(s, name, namelen); +.DE +The bound name is a variable length byte string which is interpreted +by the supporting protocol(s). Its interpretation may vary from +communication domain to communication domain (this is one of +the properties which comprise the \*(lqdomain\*(rq). +As mentioned, in the +Internet domain names contain an Internet address and port +number. NS domain names contain an NS address and +port number. In the UNIX domain, names contain a path name and +a family, which is always AF_UNIX. If one wanted to bind +the name \*(lq/tmp/foo\*(rq to a UNIX domain socket, the +following code would be used*: +.FS +* Note that, although the tendency here is to call the \*(lqaddr\*(rq +structure \*(lqsun\*(rq, doing so would cause problems if the code +were ever ported to a Sun workstation. +.FE +.DS +#include <sys/un.h> + ... +struct sockaddr_un addr; + ... +strcpy(addr.sun_path, "/tmp/foo"); +addr.sun_family = AF_UNIX; +bind(s, (struct sockaddr *) &addr, strlen(addr.sun_path) + + sizeof (addr.sun_len) + sizeof (addr.sun_family)); +.DE +Note that in determining the size of a UNIX domain address null +bytes are not counted, which is why \fIstrlen\fP is used. In +the current implementation of UNIX domain IPC, +the file name +referred to in \fIaddr.sun_path\fP is created as a socket +in the system file space. +The caller must, therefore, have +write permission in the directory where +\fIaddr.sun_path\fP is to reside, and this file should be deleted by the +caller when it is no longer needed. Future versions of 4BSD +may not create this file. +.PP +In binding an Internet address things become more +complicated. The actual call is similar, +.DS +#include <sys/types.h> +#include <netinet/in.h> + ... +struct sockaddr_in sin; + ... +bind(s, (struct sockaddr *) &sin, sizeof (sin)); +.DE +but the selection of what to place in the address \fIsin\fP +requires some discussion. We will come back to the problem +of formulating Internet addresses in section 3 when +the library routines used in name resolution are discussed. +.PP +Binding an NS address to a socket is even more +difficult, +especially since the Internet library routines do not +work with NS hostnames. The actual call is again similar: +.DS +#include <sys/types.h> +#include <netns/ns.h> + ... +struct sockaddr_ns sns; + ... +bind(s, (struct sockaddr *) &sns, sizeof (sns)); +.DE +Again, discussion of what to place in a \*(lqstruct sockaddr_ns\*(rq +will be deferred to section 3. +.NH 2 +Connection establishment +.PP +Connection establishment is usually asymmetric, +with one process a \*(lqclient\*(rq and the other a \*(lqserver\*(rq. +The server, when willing to offer its advertised services, +binds a socket to a well-known address associated with the service +and then passively \*(lqlistens\*(rq on its socket. +It is then possible for an unrelated process to rendezvous +with the server. +The client requests services from the server by initiating a +\*(lqconnection\*(rq to the server's socket. +On the client side the \fIconnect\fP call is +used to initiate a connection. Using the UNIX domain, this +might appear as, +.DS +struct sockaddr_un server; + ... +connect(s, (struct sockaddr *)&server, strlen(server.sun_path) + + sizeof (server.sun_family)); +.DE +while in the Internet domain, +.DS +struct sockaddr_in server; + ... +connect(s, (struct sockaddr *)&server, sizeof (server)); +.DE +and in the NS domain, +.DS +struct sockaddr_ns server; + ... +connect(s, (struct sockaddr *)&server, sizeof (server)); +.DE +where \fIserver\fP in the example above would contain either the UNIX +pathname, Internet address and port number, or NS address and +port number of the server to which the +client process wishes to speak. +If the client process's socket is unbound at the time of +the connect call, +the system will automatically select and bind a name to +the socket if necessary; c.f. section 5.4. +This is the usual way that local addresses are bound +to a socket. +.PP +An error is returned if the connection was unsuccessful +(any name automatically bound by the system, however, remains). +Otherwise, the socket is associated with the server and +data transfer may begin. Some of the more common errors returned +when a connection attempt fails are: +.IP ETIMEDOUT +.br +After failing to establish a connection for a period of time, +the system decided there was no point in retrying the +connection attempt any more. This usually occurs because +the destination host is down, or because problems in +the network resulted in transmissions being lost. +.IP ECONNREFUSED +.br +The host refused service for some reason. +This is usually +due to a server process +not being present at the requested name. +.IP "ENETDOWN or EHOSTDOWN" +.br +These operational errors are +returned based on status information delivered to +the client host by the underlying communication services. +.IP "ENETUNREACH or EHOSTUNREACH" +.br +These operational errors can occur either because the network +or host is unknown (no route to the network or host is present), +or because of status information returned by intermediate +gateways or switching nodes. Many times the status returned +is not sufficient to distinguish a network being down from a +host being down, in which case the system +indicates the entire network is unreachable. +.PP +For the server to receive a client's connection it must perform +two steps after binding its socket. +The first is to indicate a willingness to listen for +incoming connection requests: +.DS +listen(s, 5); +.DE +The second parameter to the \fIlisten\fP call specifies the maximum +number of outstanding connections which may be queued awaiting +acceptance by the server process; this number +may be limited by the system. Should a connection be +requested while the queue is full, the connection will not be +refused, but rather the individual messages which comprise the +request will be ignored. This gives a harried server time to +make room in its pending connection queue while the client +retries the connection request. Had the connection been returned +with the ECONNREFUSED error, the client would be unable to tell +if the server was up or not. As it is now it is still possible +to get the ETIMEDOUT error back, though this is unlikely. The +backlog figure supplied with the listen call is currently limited +by the system to a maximum of 5 pending connections on any +one queue. This avoids the problem of processes hogging system +resources by setting an infinite backlog, then ignoring +all connection requests. +.PP +With a socket marked as listening, a server may \fIaccept\fP +a connection: +.DS +struct sockaddr_in from; + ... +fromlen = sizeof (from); +newsock = accept(s, (struct sockaddr *)&from, &fromlen); +.DE +(For the UNIX domain, \fIfrom\fP would be declared as a +\fIstruct sockaddr_un\fP, and for the NS domain, \fIfrom\fP +would be declared as a \fIstruct sockaddr_ns\fP, +but nothing different would need +to be done as far as \fIfromlen\fP is concerned. In the examples +which follow, only Internet routines will be discussed.) A new +descriptor is returned on receipt of a connection (along with +a new socket). If the server wishes to find out who its client is, +it may supply a buffer for the client socket's name. The value-result +parameter \fIfromlen\fP is initialized by the server to indicate how +much space is associated with \fIfrom\fP, then modified on return +to reflect the true size of the name. If the client's name is not +of interest, the second parameter may be a null pointer. +.PP +\fIAccept\fP normally blocks. That is, \fIaccept\fP +will not return until a connection is available or the system call +is interrupted by a signal to the process. Further, there is no +way for a process to indicate it will accept connections from only +a specific individual, or individuals. It is up to the user process +to consider who the connection is from and close down the connection +if it does not wish to speak to the process. If the server process +wants to accept connections on more than one socket, or wants to avoid blocking +on the accept call, there are alternatives; they will be considered +in section 5. +.NH 2 +Data transfer +.PP +With a connection established, data may begin to flow. To send +and receive data there are a number of possible calls. +With the peer entity at each end of a connection +anchored, a user can send or receive a message without specifying +the peer. As one might expect, in this case, then +the normal \fIread\fP and \fIwrite\fP system calls are usable, +.DS +write(s, buf, sizeof (buf)); +read(s, buf, sizeof (buf)); +.DE +In addition to \fIread\fP and \fIwrite\fP, +the new calls \fIsend\fP and \fIrecv\fP +may be used: +.DS +send(s, buf, sizeof (buf), flags); +recv(s, buf, sizeof (buf), flags); +.DE +While \fIsend\fP and \fIrecv\fP are virtually identical to +\fIread\fP and \fIwrite\fP, +the extra \fIflags\fP argument is important. The flags, +defined in \fI<sys/socket.h>\fP, may be +specified as a non-zero value if one or more +of the following is required: +.DS +.TS +l l. +MSG_OOB send/receive out of band data +MSG_PEEK look at data without reading +MSG_DONTROUTE send data without routing packets +.TE +.DE +Out of band data is a notion specific to stream sockets, and one +which we will not immediately consider. The option to have data +sent without routing applied to the outgoing packets is currently +used only by the routing table management process, and is +unlikely to be of interest to the casual user. The ability +to preview data is, however, of interest. When MSG_PEEK +is specified with a \fIrecv\fP call, any data present is returned +to the user, but treated as still \*(lqunread\*(rq. That +is, the next \fIread\fP or \fIrecv\fP call applied to the socket will +return the data previously previewed. +.NH 2 +Discarding sockets +.PP +Once a socket is no longer of interest, it may be discarded +by applying a \fIclose\fP to the descriptor, +.DS +close(s); +.DE +If data is associated with a socket which promises reliable delivery +(e.g. a stream socket) when a close takes place, the system will +continue to attempt to transfer the data. +However, after a fairly long period of +time, if the data is still undelivered, it will be discarded. +Should a user have no use for any pending data, it may +perform a \fIshutdown\fP on the socket prior to closing it. +This call is of the form: +.DS +shutdown(s, how); +.DE +where \fIhow\fP is 0 if the user is no longer interested in reading +data, 1 if no more data will be sent, or 2 if no data is to +be sent or received. +.NH 2 +Connectionless sockets +.PP +To this point we have been concerned mostly with sockets which +follow a connection oriented model. However, there is also +support for connectionless interactions typical of the datagram +facilities found in contemporary packet switched networks. +A datagram socket provides a symmetric interface to data +exchange. While processes are still likely to be client +and server, there is no requirement for connection establishment. +Instead, each message includes the destination address. +.PP +Datagram sockets are created as before. +If a particular local address is needed, +the \fIbind\fP operation must precede the first data transmission. +Otherwise, the system will set the local address and/or port +when data is first sent. +To send data, the \fIsendto\fP primitive is used, +.DS +sendto(s, buf, buflen, flags, (struct sockaddr *)&to, tolen); +.DE +The \fIs\fP, \fIbuf\fP, \fIbuflen\fP, and \fIflags\fP +parameters are used as before. +The \fIto\fP and \fItolen\fP +values are used to indicate the address of the intended recipient of the +message. When +using an unreliable datagram interface, it is +unlikely that any errors will be reported to the sender. When +information is present locally to recognize a message that can +not be delivered (for instance when a network is unreachable), +the call will return \-1 and the global value \fIerrno\fP will +contain an error number. +.PP +To receive messages on an unconnected datagram socket, the +\fIrecvfrom\fP primitive is provided: +.DS +recvfrom(s, buf, buflen, flags, (struct sockaddr *)&from, &fromlen); +.DE +Once again, the \fIfromlen\fP parameter is handled in +a value-result fashion, initially containing the size of +the \fIfrom\fP buffer, and modified on return to indicate +the actual size of the address from which the datagram was received. +.PP +In addition to the two calls mentioned above, datagram +sockets may also use the \fIconnect\fP call to associate +a socket with a specific destination address. In this case, any +data sent on the socket will automatically be addressed +to the connected peer, and only data received from that +peer will be delivered to the user. Only one connected +address is permitted for each socket at one time; +a second connect will change the destination address, +and a connect to a null address (family AF_UNSPEC) +will disconnect. +Connect requests on datagram sockets return immediately, +as this simply results in the system recording +the peer's address (as compared to a stream socket, where a +connect request initiates establishment of an end to end +connection). \fIAccept\fP and \fIlisten\fP are not +used with datagram sockets. +.PP +While a datagram socket socket is connected, +errors from recent \fIsend\fP calls may be returned +asynchronously. +These errors may be reported on subsequent operations +on the socket, +or a special socket option used with \fIgetsockopt\fP, SO_ERROR, +may be used to interrogate the error status. +A \fIselect\fP for reading or writing will return true +when an error indication has been received. +The next operation will return the error, and the error status is cleared. +Other of the less +important details of datagram sockets are described +in section 5. +.NH 2 +Input/Output multiplexing +.PP +One last facility often used in developing applications +is the ability to multiplex i/o requests among multiple +sockets and/or files. This is done using the \fIselect\fP +call: +.DS +#include <sys/time.h> +#include <sys/types.h> + ... + +fd_set readmask, writemask, exceptmask; +struct timeval timeout; + ... +select(nfds, &readmask, &writemask, &exceptmask, &timeout); +.DE +\fISelect\fP takes as arguments pointers to three sets, one for +the set of file descriptors for which the caller wishes to +be able to read data on, one for those descriptors to which +data is to be written, and one for which exceptional conditions +are pending; out-of-band data is the only +exceptional condition currently implemented by the socket +If the user is not interested +in certain conditions (i.e., read, write, or exceptions), +the corresponding argument to the \fIselect\fP should +be a null pointer. +.PP +Each set is actually a structure containing an array of +long integer bit masks; the size of the array is set +by the definition FD_SETSIZE. +The array is be +long enough to hold one bit for each of FD_SETSIZE file descriptors. +.PP +The macros FD_SET(\fIfd, &mask\fP) and +FD_CLR(\fIfd, &mask\fP) +have been provided for adding and removing file descriptor +\fIfd\fP in the set \fImask\fP. The +set should be zeroed before use, and +the macro FD_ZERO(\fI&mask\fP) has been provided +to clear the set \fImask\fP. +The parameter \fInfds\fP in the \fIselect\fP call specifies the range +of file descriptors (i.e. one plus the value of the largest +descriptor) to be examined in a set. +.PP +A timeout value may be specified if the selection +is not to last more than a predetermined period of time. If +the fields in \fItimeout\fP are set to 0, the selection takes +the form of a +\fIpoll\fP, returning immediately. If the last parameter is +a null pointer, the selection will block indefinitely*. +.FS +* To be more specific, a return takes place only when a +descriptor is selectable, or when a signal is received by +the caller, interrupting the system call. +.FE +\fISelect\fP normally returns the number of file descriptors selected; +if the \fIselect\fP call returns due to the timeout expiring, then +the value 0 is returned. +If the \fIselect\fP terminates because of an error or interruption, +a \-1 is returned with the error number in \fIerrno\fP, +and with the file descriptor masks unchanged. +.PP +Assuming a successful return, the three sets will +indicate which +file descriptors are ready to be read from, written to, or +have exceptional conditions pending. +The status of a file descriptor in a select mask may be +tested with the \fIFD_ISSET(fd, &mask)\fP macro, which +returns a non-zero value if \fIfd\fP is a member of the set +\fImask\fP, and 0 if it is not. +.PP +To determine if there are connections waiting +on a socket to be used with an \fIaccept\fP call, +\fIselect\fP can be used, followed by +a \fIFD_ISSET(fd, &mask)\fP macro to check for read +readiness on the appropriate socket. If \fIFD_ISSET\fP +returns a non-zero value, indicating permission to read, then a +connection is pending on the socket. +.PP +As an example, to read data from two sockets, \fIs1\fP and +\fIs2\fP as it is available from each and with a one-second +timeout, the following code +might be used: +.DS +#include <sys/time.h> +#include <sys/types.h> + ... +fd_set read_template; +struct timeval wait; + ... +for (;;) { + wait.tv_sec = 1; /* one second */ + wait.tv_usec = 0; + + FD_ZERO(&read_template); + + FD_SET(s1, &read_template); + FD_SET(s2, &read_template); + + nb = select(FD_SETSIZE, &read_template, (fd_set *) 0, (fd_set *) 0, &wait); + if (nb <= 0) { + \fIAn error occurred during the \fPselect\fI, or + the \fPselect\fI timed out.\fP + } + + if (FD_ISSET(s1, &read_template)) { + \fISocket #1 is ready to be read from.\fP + } + + if (FD_ISSET(s2, &read_template)) { + \fISocket #2 is ready to be read from.\fP + } +} +.DE +.PP +In 4.2, the arguments to \fIselect\fP were pointers to integers +instead of pointers to \fIfd_set\fPs. This type of call +will still work as long as the number of file descriptors +being examined is less than the number of bits in an +integer; however, the methods illustrated above should +be used in all current programs. +.PP +\fISelect\fP provides a synchronous multiplexing scheme. +Asynchronous notification of output completion, input availability, +and exceptional conditions is possible through use of the +SIGIO and SIGURG signals described in section 5. diff --git a/share/doc/psd/21.ipc/3.t b/share/doc/psd/21.ipc/3.t new file mode 100644 index 000000000000..6e7eb0622364 --- /dev/null +++ b/share/doc/psd/21.ipc/3.t @@ -0,0 +1,411 @@ +.\" Copyright (c) 1986, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)3.t 8.1 (Berkeley) 6/8/93 +.\" +.\" $FreeBSD$ +.\" +.\".ds RH "Network Library Routines +.bp +.nr H1 3 +.nr H2 0 +.bp +.LG +.B +.ce +3. NETWORK LIBRARY ROUTINES +.sp 2 +.R +.NL +.PP +The discussion in section 2 indicated the possible need to +locate and construct network addresses when using the +interprocess communication facilities in a distributed +environment. To aid in this task a number of routines +have been added to the standard C run-time library. +In this section we will consider the new routines provided +to manipulate network addresses. While the 4.4BSD networking +facilities support the Internet protocols +and the Xerox NS protocols, +most of the routines presented +in this section do not apply to the NS domain. Unless otherwise +stated, it should be assumed that the routines presented in this +section do not apply to the NS domain. +.PP +Locating a service on a remote host requires many levels of +mapping before client and server may +communicate. A service is assigned a name which is intended +for human consumption; e.g. \*(lqthe \fIlogin server\fP on host +monet\*(rq. +This name, and the name of the peer host, must then be translated +into network \fIaddresses\fP which are not necessarily suitable +for human consumption. Finally, the address must then used in locating +a physical \fIlocation\fP and \fIroute\fP to the service. The +specifics of these three mappings are likely to vary between +network architectures. For instance, it is desirable for a network +to not require hosts to +be named in such a way that their physical location is known by +the client host. Instead, underlying services in the network +may discover the actual location of the host at the time a client +host wishes to communicate. This ability to have hosts named in +a location independent manner may induce overhead in connection +establishment, as a discovery process must take place, +but allows a host to be physically mobile without requiring it to +notify its clientele of its current location. +.PP +Standard routines are provided for: mapping host names +to network addresses, network names to network numbers, +protocol names to protocol numbers, and service names +to port numbers and the appropriate protocol to +use in communicating with the server process. The +file <\fInetdb.h\fP> must be included when using any of these +routines. +.NH 2 +Host names +.PP +An Internet host name to address mapping is represented by +the \fIhostent\fP structure: +.DS +.if t .ta 0.6i 1.1i 2.6i +struct hostent { + char *h_name; /* official name of host */ + char **h_aliases; /* alias list */ + int h_addrtype; /* host address type (e.g., AF_INET) */ + int h_length; /* length of address */ + char **h_addr_list; /* list of addresses, null terminated */ +}; + +#define h_addr h_addr_list[0] /* first address, network byte order */ +.DE +The routine \fIgethostbyname\fP(3N) takes an Internet host name +and returns a \fIhostent\fP structure, +while the routine \fIgethostbyaddr\fP(3N) +maps Internet host addresses into a \fIhostent\fP structure. +.PP +The official name of the host and its public aliases are +returned by these routines, +along with the address type (family) and a null terminated list of +variable length address. This list of addresses is +required because it is possible +for a host to have many addresses, all having the same name. +The \fIh_addr\fP definition is provided for backward compatibility, +and is defined to be the first address in the list of addresses +in the \fIhostent\fP structure. +.PP +The database for these calls is provided either by the +file \fI/etc/hosts\fP (\fIhosts\fP\|(5)), +or by use of a nameserver, \fInamed\fP\|(8). +Because of the differences in these databases and their access protocols, +the information returned may differ. +When using the host table version of \fIgethostbyname\fP, +only one address will be returned, but all listed aliases will be included. +The nameserver version may return alternate addresses, +but will not provide any aliases other than one given as argument. +.PP +Unlike Internet names, NS names are always mapped into host +addresses by the use of a standard NS \fIClearinghouse service\fP, +a distributed name and authentication server. The algorithms +for mapping NS names to addresses via a Clearinghouse are +rather complicated, and the routines are not part of the +standard libraries. The user-contributed Courier (Xerox +remote procedure call protocol) compiler contains routines +to accomplish this mapping; see the documentation and +examples provided therein for more information. It is +expected that almost all software that has to communicate +using NS will need to use the facilities of +the Courier compiler. +.PP +An NS host address is represented by the following: +.DS +union ns_host { + u_char c_host[6]; + u_short s_host[3]; +}; + +union ns_net { + u_char c_net[4]; + u_short s_net[2]; +}; + +struct ns_addr { + union ns_net x_net; + union ns_host x_host; + u_short x_port; +}; +.DE +The following code fragment inserts a known NS address into +a \fIns_addr\fP: +.DS +#include <sys/types.h> +#include <sys/socket.h> +#include <netns/ns.h> + ... +u_long netnum; +struct sockaddr_ns dst; + ... +bzero((char *)&dst, sizeof(dst)); + +/* + * There is no convenient way to assign a long + * integer to a ``union ns_net'' at present; in + * the future, something will hopefully be provided, + * but this is the portable way to go for now. + * The network number below is the one for the NS net + * that the desired host (gyre) is on. + */ +netnum = htonl(2266); +dst.sns_addr.x_net = *(union ns_net *) &netnum; +dst.sns_family = AF_NS; + +/* + * host 2.7.1.0.2a.18 == "gyre:Computer Science:UofMaryland" + */ +dst.sns_addr.x_host.c_host[0] = 0x02; +dst.sns_addr.x_host.c_host[1] = 0x07; +dst.sns_addr.x_host.c_host[2] = 0x01; +dst.sns_addr.x_host.c_host[3] = 0x00; +dst.sns_addr.x_host.c_host[4] = 0x2a; +dst.sns_addr.x_host.c_host[5] = 0x18; +dst.sns_addr.x_port = htons(75); +.DE +.NH 2 +Network names +.PP +As for host names, routines for mapping network names to numbers, +and back, are provided. These routines return a \fInetent\fP +structure: +.DS +.DT +/* + * Assumption here is that a network number + * fits in 32 bits -- probably a poor one. + */ +struct netent { + char *n_name; /* official name of net */ + char **n_aliases; /* alias list */ + int n_addrtype; /* net address type */ + int n_net; /* network number, host byte order */ +}; +.DE +The routines \fIgetnetbyname\fP(3N), \fIgetnetbynumber\fP(3N), +and \fIgetnetent\fP(3N) are the network counterparts to the +host routines described above. The routines extract their +information from \fI/etc/networks\fP. +.PP +NS network numbers are determined either by asking your local +Xerox Network Administrator (and hardcoding the information +into your code), or by querying the Clearinghouse for addresses. +The internetwork router is the only process +that needs to manipulate network numbers on a regular basis; if +a process wishes to communicate with a machine, it should ask the +Clearinghouse for that machine's address (which will include +the net number). +.NH 2 +Protocol names +.PP +For protocols, which are defined in \fI/etc/protocols\fP, +the \fIprotoent\fP structure defines the +protocol-name mapping +used with the routines \fIgetprotobyname\fP(3N), +\fIgetprotobynumber\fP(3N), +and \fIgetprotoent\fP(3N): +.DS +.DT +struct protoent { + char *p_name; /* official protocol name */ + char **p_aliases; /* alias list */ + int p_proto; /* protocol number */ +}; +.DE +.PP +In the NS domain, protocols are indicated by the "client type" +field of an IDP header. No protocol database exists; see section +5 for more information. +.NH 2 +Service names +.PP +Information regarding services is a bit more complicated. A service +is expected to reside at a specific \*(lqport\*(rq and employ +a particular communication protocol. This view is consistent with +the Internet domain, but inconsistent with other network architectures. +Further, a service may reside on multiple ports. +If this occurs, the higher level library routines +will have to be bypassed or extended. +Services available are contained in the file \fI/etc/services\fP. +A service mapping is described by the \fIservent\fP structure, +.DS +.DT +struct servent { + char *s_name; /* official service name */ + char **s_aliases; /* alias list */ + int s_port; /* port number, network byte order */ + char *s_proto; /* protocol to use */ +}; +.DE +The routine \fIgetservbyname\fP(3N) maps service +names to a servent structure by specifying a service name and, +optionally, a qualifying protocol. Thus the call +.DS +sp = getservbyname("telnet", (char *) 0); +.DE +returns the service specification for a telnet server using +any protocol, while the call +.DS +sp = getservbyname("telnet", "tcp"); +.DE +returns only that telnet server which uses the TCP protocol. +The routines \fIgetservbyport\fP(3N) and \fIgetservent\fP(3N) are +also provided. The \fIgetservbyport\fP routine has an interface similar +to that provided by \fIgetservbyname\fP; an optional protocol name may +be specified to qualify lookups. +.PP +In the NS domain, services are handled by a central dispatcher +provided as part of the Courier remote procedure call facilities. +Again, the reader is referred to the Courier compiler documentation +and to the Xerox standard* +.FS +* \fICourier: The Remote Procedure Call Protocol\fP, XSIS 038112. +.FE +for further details. +.NH 2 +Miscellaneous +.PP +With the support routines described above, an Internet application program +should rarely have to deal directly +with addresses. This allows +services to be developed as much as possible in a network independent +fashion. It is clear, however, that purging all network dependencies +is very difficult. So long as the user is required to supply network +addresses when naming services and sockets there will always some +network dependency in a program. For example, the normal +code included in client programs, such as the remote login program, +is of the form shown in Figure 1. +(This example will be considered in more detail in section 4.) +.PP +If we wanted to make the remote login program independent of the +Internet protocols and addressing scheme we would be forced to add +a layer of routines which masked the network dependent aspects from +the mainstream login code. For the current facilities available in +the system this does not appear to be worthwhile. +.PP +Aside from the address-related data base routines, there are several +other routines available in the run-time library which are of interest +to users. These are intended mostly to simplify manipulation of +names and addresses. Table 1 summarizes the routines +for manipulating variable length byte strings and handling byte +swapping of network addresses and values. +.KF +.DS B +.TS +box; +l | l +l | l. +Call Synopsis +_ +bcmp(s1, s2, n) compare byte-strings; 0 if same, not 0 otherwise +bcopy(s1, s2, n) copy n bytes from s1 to s2 +bzero(base, n) zero-fill n bytes starting at base +htonl(val) convert 32-bit quantity from host to network byte order +htons(val) convert 16-bit quantity from host to network byte order +ntohl(val) convert 32-bit quantity from network to host byte order +ntohs(val) convert 16-bit quantity from network to host byte order +.TE +.DE +.ce +Table 1. C run-time routines. +.KE +.PP +The byte swapping routines are provided because the operating +system expects addresses to be supplied in network order (aka ``big-endian'' order). On +``little-endian'' architectures, such as Intel x86 and VAX, +host byte ordering is different than +network byte ordering. Consequently, +programs are sometimes required to byte swap quantities. The +library routines which return network addresses provide them +in network order so that they may simply be copied into the structures +provided to the system. This implies users should encounter the +byte swapping problem only when \fIinterpreting\fP network addresses. +For example, if an Internet port is to be printed out the following +code would be required: +.DS +printf("port number %d\en", ntohs(sp->s_port)); +.DE +On machines where unneeded these routines are defined as null +macros. +.DS +.if t .ta .5i 1.0i 1.5i 2.0i +.if n .ta .7i 1.4i 2.1i 2.8i +#include <sys/types.h> +#include <sys/socket.h> +#include <netinet/in.h> +#include <stdio.h> +#include <netdb.h> + ... +main(argc, argv) + int argc; + char *argv[]; +{ + struct sockaddr_in server; + struct servent *sp; + struct hostent *hp; + int s; + ... + sp = getservbyname("login", "tcp"); + if (sp == NULL) { + fprintf(stderr, "rlogin: login/tcp: unknown service\en"); + exit(1); + } + hp = gethostbyname(argv[1]); + if (hp == NULL) { + fprintf(stderr, "rlogin: %s: unknown host\en", argv[1]); + exit(2); + } + bzero((char *)&server, sizeof (server)); + bcopy(hp->h_addr, (char *)&server.sin_addr, hp->h_length); + server.sin_family = hp->h_addrtype; + server.sin_port = sp->s_port; + s = socket(AF_INET, SOCK_STREAM, 0); + if (s < 0) { + perror("rlogin: socket"); + exit(3); + } + ... + /* Connect does the bind() for us */ + + if (connect(s, (char *)&server, sizeof (server)) < 0) { + perror("rlogin: connect"); + exit(5); + } + ... +} +.DE +.ce +Figure 1. Remote login client code. diff --git a/share/doc/psd/21.ipc/4.t b/share/doc/psd/21.ipc/4.t new file mode 100644 index 000000000000..22e6836d28a4 --- /dev/null +++ b/share/doc/psd/21.ipc/4.t @@ -0,0 +1,515 @@ +.\" Copyright (c) 1986, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)4.t 8.1 (Berkeley) 6/8/93 +.\" $FreeBSD$ +.\" +.\".ds RH "Client/Server Model +.bp +.nr H1 4 +.nr H2 0 +.sp 8i +.bp +.LG +.B +.ce +4. CLIENT/SERVER MODEL +.sp 2 +.R +.NL +.PP +The most commonly used paradigm in constructing distributed applications +is the client/server model. In this scheme client applications request +services from a server process. This implies an asymmetry in establishing +communication between the client and server which has been examined +in section 2. In this section we will look more closely at the interactions +between client and server, and consider some of the problems in developing +client and server applications. +.PP +The client and server require a well known set of conventions before +service may be rendered (and accepted). This set of conventions +comprises a protocol which must be implemented at both ends of a +connection. Depending on the situation, the protocol may be symmetric +or asymmetric. In a symmetric protocol, either side may play the +master or slave roles. In an asymmetric protocol, one side is +immutably recognized as the master, with the other as the slave. +An example of a symmetric protocol is the TELNET protocol used in +the Internet for remote terminal emulation. An example +of an asymmetric protocol is the Internet file transfer protocol, +FTP. No matter whether the specific protocol used in obtaining +a service is symmetric or asymmetric, when accessing a service there +is a \*(lqclient process\*(rq and a \*(lqserver process\*(rq. We +will first consider the properties of server processes, then +client processes. +.PP +A server process normally listens at a well known address for +service requests. That is, the server process remains dormant +until a connection is requested by a client's connection +to the server's address. At such a time +the server process ``wakes up'' and services the client, +performing whatever appropriate actions the client requests of it. +.PP +Alternative schemes which use a service server +may be used to eliminate a flock of server processes clogging the +system while remaining dormant most of the time. For Internet +servers in 4.4BSD, +this scheme has been implemented via \fIinetd\fP, the so called +``internet super-server.'' \fIInetd\fP listens at a variety +of ports, determined at start-up by reading a configuration file. +When a connection is requested to a port on which \fIinetd\fP is +listening, \fIinetd\fP executes the appropriate server program to handle the +client. With this method, clients are unaware that an +intermediary such as \fIinetd\fP has played any part in the +connection. \fIInetd\fP will be described in more detail in +section 5. +.PP +A similar alternative scheme is used by most Xerox services. In general, +the Courier dispatch process (if used) accepts connections from +processes requesting services of some sort or another. The client +processes request a particular <program number, version number, procedure +number> triple. If the dispatcher knows of such a program, it is +started to handle the request; if not, an error is reported to the +client. In this way, only one port is required to service a large +variety of different requests. Again, the Courier facilities are +not available without the use and installation of the Courier +compiler. The information presented in this section applies only +to NS clients and services that do not use Courier. +.NH 2 +Servers +.PP +In 4.4BSD most servers are accessed at well known Internet addresses +or UNIX domain names. For +example, the remote login server's main loop is of the form shown +in Figure 2. +.KF +.if t .ta .5i 1.0i 1.5i 2.0i 2.5i 3.0i 3.5i +.if n .ta .7i 1.4i 2.1i 2.8i 3.5i 4.2i 4.9i +.sp 0.5i +.DS +main(argc, argv) + int argc; + char *argv[]; +{ + int f; + struct sockaddr_in from; + struct servent *sp; + + sp = getservbyname("login", "tcp"); + if (sp == NULL) { + fprintf(stderr, "rlogind: login/tcp: unknown service\en"); + exit(1); + } + ... +#ifndef DEBUG + /* Disassociate server from controlling terminal */ + ... +#endif + + sin.sin_port = sp->s_port; /* Restricted port -- see section 5 */ + ... + f = socket(AF_INET, SOCK_STREAM, 0); + ... + if (bind(f, (struct sockaddr *) &sin, sizeof (sin)) < 0) { + ... + } + ... + listen(f, 5); + for (;;) { + int g, len = sizeof (from); + + g = accept(f, (struct sockaddr *) &from, &len); + if (g < 0) { + if (errno != EINTR) + syslog(LOG_ERR, "rlogind: accept: %m"); + continue; + } + if (fork() == 0) { + close(f); + doit(g, &from); + } + close(g); + } +} +.DE +.ce +Figure 2. Remote login server. +.sp 0.5i +.KE +.PP +The first step taken by the server is look up its service +definition: +.sp 1 +.nf +.in +5 +.if t .ta .5i 1.0i 1.5i 2.0i +.if n .ta .7i 1.4i 2.1i 2.8i +sp = getservbyname("login", "tcp"); +if (sp == NULL) { + fprintf(stderr, "rlogind: login/tcp: unknown service\en"); + exit(1); +} +.sp 1 +.in -5 +.fi +The result of the \fIgetservbyname\fP call +is used in later portions of the code to +define the Internet port at which it listens for service +requests (indicated by a connection). +.KS +.PP +Step two is to disassociate the server from the controlling +terminal of its invoker: +.DS + for (i = 0; i < 3; ++i) + close(i); + + open("/", O_RDONLY); + dup2(0, 1); + dup2(0, 2); + + i = open("/dev/tty", O_RDWR); + if (i >= 0) { + ioctl(i, TIOCNOTTY, 0); + close(i); + } +.DE +.KE +This step is important as the server will +likely not want to receive signals delivered to the process +group of the controlling terminal. Note, however, that +once a server has disassociated itself it can no longer +send reports of errors to a terminal, and must log errors +via \fIsyslog\fP. +.PP +Once a server has established a pristine environment, it +creates a socket and begins accepting service requests. +The \fIbind\fP call is required to insure the server listens +at its expected location. It should be noted that the +remote login server listens at a restricted port number, and must +therefore be run +with a user-id of root. +This concept of a ``restricted port number'' is 4BSD +specific, and is covered in section 5. +.PP +The main body of the loop is fairly simple: +.DS +.if t .ta .5i 1.0i 1.5i 2.0i +.if n .ta .7i 1.4i 2.1i 2.8i +for (;;) { + int g, len = sizeof (from); + + g = accept(f, (struct sockaddr *)&from, &len); + if (g < 0) { + if (errno != EINTR) + syslog(LOG_ERR, "rlogind: accept: %m"); + continue; + } + if (fork() == 0) { /* Child */ + close(f); + doit(g, &from); + } + close(g); /* Parent */ +} +.DE +An \fIaccept\fP call blocks the server until +a client requests service. This call could return a +failure status if the call is interrupted by a signal +such as SIGCHLD (to be discussed in section 5). Therefore, +the return value from \fIaccept\fP is checked to insure +a connection has actually been established, and +an error report is logged via \fIsyslog\fP if an error +has occurred. +.PP +With a connection +in hand, the server then forks a child process and invokes +the main body of the remote login protocol processing. Note +how the socket used by the parent for queuing connection +requests is closed in the child, while the socket created as +a result of the \fIaccept\fP is closed in the parent. The +address of the client is also handed the \fIdoit\fP routine +because it requires it in authenticating clients. +.NH 2 +Clients +.PP +The client side of the remote login service was shown +earlier in Figure 1. +One can see the separate, asymmetric roles of the client +and server clearly in the code. The server is a passive entity, +listening for client connections, while the client process is +an active entity, initiating a connection when invoked. +.PP +Let us consider more closely the steps taken +by the client remote login process. As in the server process, +the first step is to locate the service definition for a remote +login: +.DS +sp = getservbyname("login", "tcp"); +if (sp == NULL) { + fprintf(stderr, "rlogin: login/tcp: unknown service\en"); + exit(1); +} +.DE +Next the destination host is looked up with a +\fIgethostbyname\fP call: +.DS +hp = gethostbyname(argv[1]); +if (hp == NULL) { + fprintf(stderr, "rlogin: %s: unknown host\en", argv[1]); + exit(2); +} +.DE +With this accomplished, all that is required is to establish a +connection to the server at the requested host and start up the +remote login protocol. The address buffer is cleared, then filled +in with the Internet address of the foreign host and the port +number at which the login process resides on the foreign host: +.DS +bzero((char *)&server, sizeof (server)); +bcopy(hp->h_addr, (char *) &server.sin_addr, hp->h_length); +server.sin_family = hp->h_addrtype; +server.sin_port = sp->s_port; +.DE +A socket is created, and a connection initiated. Note +that \fIconnect\fP implicitly performs a \fIbind\fP +call, since \fIs\fP is unbound. +.DS +s = socket(hp->h_addrtype, SOCK_STREAM, 0); +if (s < 0) { + perror("rlogin: socket"); + exit(3); +} + ... +if (connect(s, (struct sockaddr *) &server, sizeof (server)) < 0) { + perror("rlogin: connect"); + exit(4); +} +.DE +The details of the remote login protocol will not be considered here. +.NH 2 +Connectionless servers +.PP +While connection-based services are the norm, some services +are based on the use of datagram sockets. One, in particular, +is the \*(lqrwho\*(rq service which provides users with status +information for hosts connected to a local area +network. This service, while predicated on the ability to +\fIbroadcast\fP information to all hosts connected to a particular +network, is of interest as an example usage of datagram sockets. +.PP +A user on any machine running the rwho server may find out +the current status of a machine with the \fIruptime\fP(1) program. +The output generated is illustrated in Figure 3. +.KF +.DS B +.TS +l r l l l l l. +arpa up 9:45, 5 users, load 1.15, 1.39, 1.31 +cad up 2+12:04, 8 users, load 4.67, 5.13, 4.59 +calder up 10:10, 0 users, load 0.27, 0.15, 0.14 +dali up 2+06:28, 9 users, load 1.04, 1.20, 1.65 +degas up 25+09:48, 0 users, load 1.49, 1.43, 1.41 +ear up 5+00:05, 0 users, load 1.51, 1.54, 1.56 +ernie down 0:24 +esvax down 17:04 +ingres down 0:26 +kim up 3+09:16, 8 users, load 2.03, 2.46, 3.11 +matisse up 3+06:18, 0 users, load 0.03, 0.03, 0.05 +medea up 3+09:39, 2 users, load 0.35, 0.37, 0.50 +merlin down 19+15:37 +miro up 1+07:20, 7 users, load 4.59, 3.28, 2.12 +monet up 1+00:43, 2 users, load 0.22, 0.09, 0.07 +oz down 16:09 +statvax up 2+15:57, 3 users, load 1.52, 1.81, 1.86 +ucbvax up 9:34, 2 users, load 6.08, 5.16, 3.28 +.TE +.DE +.ce +Figure 3. ruptime output. +.sp +.KE +.PP +Status information for each host is periodically broadcast +by rwho server processes on each machine. The same server +process also receives the status information and uses it +to update a database. This database is then interpreted +to generate the status information for each host. Servers +operate autonomously, coupled only by the local network and +its broadcast capabilities. +.PP +Note that the use of broadcast for such a task is fairly inefficient, +as all hosts must process each message, whether or not using an rwho server. +Unless such a service is sufficiently universal and is frequently used, +the expense of periodic broadcasts outweighs the simplicity. +.PP +Multicasting is an alternative to broadcasting. +Setting up multicast sockets is described in Section 5.10. +.PP +The rwho server, in a simplified form, is pictured in Figure +4. There are two separate tasks performed by the server. The +first task is to act as a receiver of status information broadcast +by other hosts on the network. This job is carried out in the +main loop of the program. Packets received at the rwho port +are interrogated to insure they've been sent by another rwho +server process, then are time stamped with their arrival time +and used to update a file indicating the status of the host. +When a host has not been heard from for an extended period of +time, the database interpretation routines assume the host is +down and indicate such on the status reports. This algorithm +is prone to error as a server may be down while a host is actually +up, but serves our current needs. +.KF +.DS +.if t .ta .5i 1.0i 1.5i 2.0i +.if n .ta .7i 1.4i 2.1i 2.8i +main() +{ + ... + sp = getservbyname("who", "udp"); + net = getnetbyname("localnet"); + sin.sin_addr = inet_makeaddr(INADDR_ANY, net); + sin.sin_port = sp->s_port; + ... + s = socket(AF_INET, SOCK_DGRAM, 0); + ... + on = 1; + if (setsockopt(s, SOL_SOCKET, SO_BROADCAST, &on, sizeof(on)) < 0) { + syslog(LOG_ERR, "setsockopt SO_BROADCAST: %m"); + exit(1); + } + bind(s, (struct sockaddr *) &sin, sizeof (sin)); + ... + signal(SIGALRM, onalrm); + onalrm(); + for (;;) { + struct whod wd; + int cc, whod, len = sizeof (from); + + cc = recvfrom(s, (char *)&wd, sizeof (struct whod), 0, + (struct sockaddr *)&from, &len); + if (cc <= 0) { + if (cc < 0 && errno != EINTR) + syslog(LOG_ERR, "rwhod: recv: %m"); + continue; + } + if (from.sin_port != sp->s_port) { + syslog(LOG_ERR, "rwhod: %d: bad from port", + ntohs(from.sin_port)); + continue; + } + ... + if (!verify(wd.wd_hostname)) { + syslog(LOG_ERR, "rwhod: malformed host name from %x", + ntohl(from.sin_addr.s_addr)); + continue; + } + (void) sprintf(path, "%s/whod.%s", RWHODIR, wd.wd_hostname); + whod = open(path, O_WRONLY | O_CREAT | O_TRUNC, 0666); + ... + (void) time(&wd.wd_recvtime); + (void) write(whod, (char *)&wd, cc); + (void) close(whod); + } +} +.DE +.ce +Figure 4. rwho server. +.sp +.KE +.PP +The second task performed by the server is to supply information +regarding the status of its host. This involves periodically +acquiring system status information, packaging it up in a message +and broadcasting it on the local network for other rwho servers +to hear. The supply function is triggered by a timer and +runs off a signal. Locating the system status +information is somewhat involved, but uninteresting. Deciding +where to transmit the resultant packet +is somewhat problematical, however. +.PP +Status information must be broadcast on the local network. +For networks which do not support the notion of broadcast another +scheme must be used to simulate or +replace broadcasting. One possibility is to enumerate the +known neighbors (based on the status messages received +from other rwho servers). This, unfortunately, +requires some bootstrapping information, +for a server will have no idea what machines are its +neighbors until it receives status messages from them. +Therefore, if all machines on a net are freshly booted, +no machine will have any +known neighbors and thus never receive, or send, any status information. +This is the identical problem faced by the routing table management +process in propagating routing status information. The standard +solution, unsatisfactory as it may be, is to inform one or more servers +of known neighbors and request that they always communicate with +these neighbors. If each server has at least one neighbor supplied +to it, status information may then propagate through +a neighbor to hosts which +are not (possibly) directly neighbors. If the server is able to +support networks which provide a broadcast capability, as well as +those which do not, then networks with an +arbitrary topology may share status information*. +.FS +* One must, however, be concerned about \*(lqloops\*(rq. +That is, if a host is connected to multiple networks, it +will receive status information from itself. This can lead +to an endless, wasteful, exchange of information. +.FE +.PP +It is important that software operating in a distributed +environment not have any site-dependent information compiled into it. +This would require a separate copy of the server at each host and +make maintenance a severe headache. 4.4BSD attempts to isolate +host-specific information from applications by providing system +calls which return the necessary information*. +.FS +* An example of such a system call is the \fIgethostname\fP(2) +call which returns the host's \*(lqofficial\*(rq name. +.FE +A mechanism exists, in the form of an \fIioctl\fP call, +for finding the collection +of networks to which a host is directly connected. +Further, a local network broadcasting mechanism +has been implemented at the socket level. +Combining these two features allows a process +to broadcast on any directly connected local +network which supports the notion of broadcasting +in a site independent manner. This allows 4.4BSD +to solve the problem of deciding how to propagate +status information in the case of \fIrwho\fP, or +more generally in broadcasting: +Such status information is broadcast to connected +networks at the socket level, where the connected networks +have been obtained via the appropriate \fIioctl\fP +calls. +The specifics of +such broadcastings are complex, however, and will +be covered in section 5. diff --git a/share/doc/psd/21.ipc/5.t b/share/doc/psd/21.ipc/5.t new file mode 100644 index 000000000000..8ce44b2018c1 --- /dev/null +++ b/share/doc/psd/21.ipc/5.t @@ -0,0 +1,1668 @@ +.\" Copyright (c) 1986, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)5.t 8.1 (Berkeley) 8/14/93 +.\" $FreeBSD$ +.\" +.\".ds RH "Advanced Topics +.bp +.nr H1 5 +.nr H2 0 +.LG +.B +.ce +5. ADVANCED TOPICS +.sp 2 +.R +.NL +.PP +A number of facilities have yet to be discussed. For most users +of the IPC the mechanisms already +described will suffice in constructing distributed +applications. However, others will find the need to utilize some +of the features which we consider in this section. +.NH 2 +Out of band data +.PP +The stream socket abstraction includes the notion of \*(lqout +of band\*(rq data. Out of band data is a logically independent +transmission channel associated with each pair of connected +stream sockets. Out of band data is delivered to the user +independently of normal data. +The abstraction defines that the out of band data facilities +must support the reliable delivery of at least one +out of band message at a time. This message may contain at least one +byte of data, and at least one message may be pending delivery +to the user at any one time. For communications protocols which +support only in-band signaling (i.e. the urgent data is +delivered in sequence with the normal data), the system normally extracts +the data from the normal data stream and stores it separately. +This allows users to choose between receiving the urgent data +in order and receiving it out of sequence without having to +buffer all the intervening data. It is possible +to ``peek'' (via MSG_PEEK) at out of band data. +If the socket has a process group, a SIGURG signal is generated +when the protocol is notified of its existence. +A process can set the process group +or process id to be informed by the SIGURG signal via the +appropriate \fIfcntl\fP call, as described below for +SIGIO. +If multiple sockets may have out of band data awaiting +delivery, a \fIselect\fP call for exceptional conditions +may be used to determine those sockets with such data pending. +Neither the signal nor the select indicate the actual arrival +of the out-of-band data, but only notification that it is pending. +.PP +In addition to the information passed, a logical mark is placed in +the data stream to indicate the point at which the out +of band data was sent. The remote login and remote shell +applications use this facility to propagate signals between +client and server processes. When a signal +flushs any pending output from the remote process(es), all +data up to the mark in the data stream is discarded. +.PP +To send an out of band message the MSG_OOB flag is supplied to +a \fIsend\fP or \fIsendto\fP calls, +while to receive out of band data MSG_OOB should be indicated +when performing a \fIrecvfrom\fP or \fIrecv\fP call. +To find out if the read pointer is currently pointing at +the mark in the data stream, the SIOCATMARK ioctl is provided: +.DS +ioctl(s, SIOCATMARK, &yes); +.DE +If \fIyes\fP is a 1 on return, the next read will return data +after the mark. Otherwise (assuming out of band data has arrived), +the next read will provide data sent by the client prior +to transmission of the out of band signal. The routine used +in the remote login process to flush output on receipt of an +interrupt or quit signal is shown in Figure 5. +It reads the normal data up to the mark (to discard it), +then reads the out-of-band byte. +.KF +.DS +#include <sys/ioctl.h> +#include <sys/file.h> + ... +oob() +{ + int out = FWRITE, mark; + char waste[BUFSIZ]; + + /* flush local terminal output */ + ioctl(1, TIOCFLUSH, (char *)&out); + for (;;) { + if (ioctl(rem, SIOCATMARK, &mark) < 0) { + perror("ioctl"); + break; + } + if (mark) + break; + (void) read(rem, waste, sizeof (waste)); + } + if (recv(rem, &mark, 1, MSG_OOB) < 0) { + perror("recv"); + ... + } + ... +} +.DE +.ce +Figure 5. Flushing terminal I/O on receipt of out of band data. +.sp +.KE +.PP +A process may also read or peek at the out-of-band data +without first reading up to the mark. +This is more difficult when the underlying protocol delivers +the urgent data in-band with the normal data, and only sends +notification of its presence ahead of time (e.g., the TCP protocol +used to implement streams in the Internet domain). +With such protocols, the out-of-band byte may not yet have arrived +when a \fIrecv\fP is done with the MSG_OOB flag. +In that case, the call will return an error of EWOULDBLOCK. +Worse, there may be enough in-band data in the input buffer +that normal flow control prevents the peer from sending the urgent data +until the buffer is cleared. +The process must then read enough of the queued data +that the urgent data may be delivered. +.PP +Certain programs that use multiple bytes of urgent data and must +handle multiple urgent signals (e.g., \fItelnet\fP\|(1C)) +need to retain the position of urgent data within the stream. +This treatment is available as a socket-level option, SO_OOBINLINE; +see \fIsetsockopt\fP\|(2) for usage. +With this option, the position of urgent data (the \*(lqmark\*(rq) +is retained, but the urgent data immediately follows the mark +within the normal data stream returned without the MSG_OOB flag. +Reception of multiple urgent indications causes the mark to move, +but no out-of-band data are lost. +.NH 2 +Non-Blocking Sockets +.PP +It is occasionally convenient to make use of sockets +which do not block; that is, I/O requests which +cannot complete immediately and +would therefore cause the process to be suspended awaiting completion are +not executed, and an error code is returned. +Once a socket has been created via +the \fIsocket\fP call, it may be marked as non-blocking +by \fIfcntl\fP as follows: +.DS +#include <fcntl.h> + ... +int s; + ... +s = socket(AF_INET, SOCK_STREAM, 0); + ... +if (fcntl(s, F_SETFL, FNDELAY) < 0) + perror("fcntl F_SETFL, FNDELAY"); + exit(1); +} + ... +.DE +.PP +When performing non-blocking I/O on sockets, one must be +careful to check for the error EWOULDBLOCK (stored in the +global variable \fIerrno\fP), which occurs when +an operation would normally block, but the socket it +was performed on is marked as non-blocking. +In particular, \fIaccept\fP, \fIconnect\fP, \fIsend\fP, \fIrecv\fP, +\fIread\fP, and \fIwrite\fP can +all return EWOULDBLOCK, and processes should be prepared +to deal with such return codes. +If an operation such as a \fIsend\fP cannot be done in its entirety, +but partial writes are sensible (for example, when using a stream socket), +the data that can be sent immediately will be processed, +and the return value will indicate the amount actually sent. +.NH 2 +Interrupt driven socket I/O +.PP +The SIGIO signal allows a process to be notified +via a signal when a socket (or more generally, a file +descriptor) has data waiting to be read. Use of +the SIGIO facility requires three steps: First, +the process must set up a SIGIO signal handler +by use of the \fIsignal\fP or \fIsigvec\fP calls. Second, +it must set the process id or process group id which is to receive +notification of pending input to its own process id, +or the process group id of its process group (note that +the default process group of a socket is group zero). +This is accomplished by use of an \fIfcntl\fP call. +Third, it must enable asynchronous notification of pending I/O requests +with another \fIfcntl\fP call. Sample code to +allow a given process to receive information on +pending I/O requests as they occur for a socket \fIs\fP +is given in Figure 6. With the addition of a handler for SIGURG, +this code can also be used to prepare for receipt of SIGURG signals. +.KF +.DS +#include <fcntl.h> + ... +int io_handler(); + ... +signal(SIGIO, io_handler); + +/* Set the process receiving SIGIO/SIGURG signals to us */ + +if (fcntl(s, F_SETOWN, getpid()) < 0) { + perror("fcntl F_SETOWN"); + exit(1); +} + +/* Allow receipt of asynchronous I/O signals */ + +if (fcntl(s, F_SETFL, FASYNC) < 0) { + perror("fcntl F_SETFL, FASYNC"); + exit(1); +} +.DE +.ce +Figure 6. Use of asynchronous notification of I/O requests. +.sp +.KE +.NH 2 +Signals and process groups +.PP +Due to the existence of the SIGURG and SIGIO signals each socket has an +associated process number, just as is done for terminals. +This value is initialized to zero, +but may be redefined at a later time with the F_SETOWN +\fIfcntl\fP, such as was done in the code above for SIGIO. +To set the socket's process id for signals, positive arguments +should be given to the \fIfcntl\fP call. To set the socket's +process group for signals, negative arguments should be +passed to \fIfcntl\fP. Note that the process number indicates +either the associated process id or the associated process +group; it is impossible to specify both at the same time. +A similar \fIfcntl\fP, F_GETOWN, is available for determining the +current process number of a socket. +.PP +Another signal which is useful when constructing server processes +is SIGCHLD. This signal is delivered to a process when any +child processes have changed state. Normally servers use +the signal to \*(lqreap\*(rq child processes that have exited +without explicitly awaiting their termination +or periodic polling for exit status. +For example, the remote login server loop shown in Figure 2 +may be augmented as shown in Figure 7. +.KF +.DS +int reaper(); + ... +signal(SIGCHLD, reaper); +listen(f, 5); +for (;;) { + int g, len = sizeof (from); + + g = accept(f, (struct sockaddr *)&from, &len,); + if (g < 0) { + if (errno != EINTR) + syslog(LOG_ERR, "rlogind: accept: %m"); + continue; + } + ... +} + ... +#include <wait.h> +reaper() +{ + union wait status; + + while (wait3(&status, WNOHANG, 0) > 0) + ; +} +.DE +.sp +.ce +Figure 7. Use of the SIGCHLD signal. +.sp +.KE +.PP +If the parent server process fails to reap its children, +a large number of \*(lqzombie\*(rq processes may be created. +.NH 2 +Pseudo terminals +.PP +Many programs will not function properly without a terminal +for standard input and output. Since sockets do not provide +the semantics of terminals, +it is often necessary to have a process communicating over +the network do so through a \fIpseudo-terminal\fP. A pseudo- +terminal is actually a pair of devices, master and slave, +which allow a process to serve as an active agent in communication +between processes and users. Data written on the slave side +of a pseudo-terminal is supplied as input to a process reading +from the master side, while data written on the master side are +processed as terminal input for the slave. +In this way, the process manipulating +the master side of the pseudo-terminal has control over the +information read and written on the slave side +as if it were manipulating the keyboard and reading the screen +on a real terminal. +The purpose of this abstraction is to +preserve terminal semantics over a network connection\(em +that is, the slave side appears as a normal terminal to +any process reading from or writing to it. +.PP +For example, the remote +login server uses pseudo-terminals for remote login sessions. +A user logging in to a machine across the network is provided +a shell with a slave pseudo-terminal as standard input, output, +and error. The server process then handles the communication +between the programs invoked by the remote shell and the user's +local client process. +When a user sends a character that generates an interrupt +on the remote machine that flushes terminal output, +the pseudo-terminal generates a control message for the server process. +The server then sends an out of band message +to the client process to signal a flush of data at the real terminal +and on the intervening data buffered in the network. +.PP +Under 4.4BSD, the name of the slave side of a pseudo-terminal is of the form +\fI/dev/ttyxy\fP, where \fIx\fP is a single letter +starting at `p' and continuing to `t'. +\fIy\fP is a hexadecimal digit (i.e., a single +character in the range 0 through 9 or `a' through `f'). +The master side of a pseudo-terminal is \fI/dev/ptyxy\fP, +where \fIx\fP and \fIy\fP correspond to the +slave side of the pseudo-terminal. +.PP +In general, the method of obtaining a pair of master and +slave pseudo-terminals is to +find a pseudo-terminal which +is not currently in use. +The master half of a pseudo-terminal is a single-open device; +thus, each master may be opened in turn until an open succeeds. +The slave side of the pseudo-terminal is then opened, +and is set to the proper terminal modes if necessary. +The process then \fIfork\fPs; the child closes +the master side of the pseudo-terminal, and \fIexec\fPs the +appropriate program. Meanwhile, the parent closes the +slave side of the pseudo-terminal and begins reading and +writing from the master side. Sample code making use of +pseudo-terminals is given in Figure 8; this code assumes +that a connection on a socket \fIs\fP exists, connected +to a peer who wants a service of some kind, and that the +process has disassociated itself from any previous controlling terminal. +.KF +.DS +gotpty = 0; +for (c = 'p'; !gotpty && c <= 's'; c++) { + line = "/dev/ptyXX"; + line[sizeof("/dev/pty")-1] = c; + line[sizeof("/dev/ptyp")-1] = '0'; + if (stat(line, &statbuf) < 0) + break; + for (i = 0; i < 16; i++) { + line[sizeof("/dev/ptyp")-1] = "0123456789abcdef"[i]; + master = open(line, O_RDWR); + if (master > 0) { + gotpty = 1; + break; + } + } +} +if (!gotpty) { + syslog(LOG_ERR, "All network ports in use"); + exit(1); +} + +line[sizeof("/dev/")-1] = 't'; +slave = open(line, O_RDWR); /* \fIslave\fP is now slave side */ +if (slave < 0) { + syslog(LOG_ERR, "Cannot open slave pty %s", line); + exit(1); +} + +ioctl(slave, TIOCGETP, &b); /* Set slave tty modes */ +b.sg_flags = CRMOD|XTABS|ANYP; +ioctl(slave, TIOCSETP, &b); + +i = fork(); +if (i < 0) { + syslog(LOG_ERR, "fork: %m"); + exit(1); +} else if (i) { /* Parent */ + close(slave); + ... +} else { /* Child */ + (void) close(s); + (void) close(master); + dup2(slave, 0); + dup2(slave, 1); + dup2(slave, 2); + if (slave > 2) + (void) close(slave); + ... +} +.DE +.ce +Figure 8. Creation and use of a pseudo terminal +.sp +.KE +.NH 2 +Selecting specific protocols +.PP +If the third argument to the \fIsocket\fP call is 0, +\fIsocket\fP will select a default protocol to use with +the returned socket of the type requested. +The default protocol is usually correct, and alternate choices are not +usually available. +However, when using ``raw'' sockets to communicate directly with +lower-level protocols or hardware interfaces, +the protocol argument may be important for setting up demultiplexing. +For example, raw sockets in the Internet family may be used to implement +a new protocol above IP, and the socket will receive packets +only for the protocol specified. +To obtain a particular protocol one determines the protocol number +as defined within the communication domain. For the Internet +domain one may use one of the library routines +discussed in section 3, such as \fIgetprotobyname\fP: +.DS +#include <sys/types.h> +#include <sys/socket.h> +#include <netinet/in.h> +#include <netdb.h> + ... +pp = getprotobyname("newtcp"); +s = socket(AF_INET, SOCK_STREAM, pp->p_proto); +.DE +This would result in a socket \fIs\fP using a stream +based connection, but with protocol type of ``newtcp'' +instead of the default ``tcp.'' +.PP +In the NS domain, the available socket protocols are defined in +<\fInetns/ns.h\fP>. To create a raw socket for Xerox Error Protocol +messages, one might use: +.DS +#include <sys/types.h> +#include <sys/socket.h> +#include <netns/ns.h> + ... +s = socket(AF_NS, SOCK_RAW, NSPROTO_ERROR); +.DE +.NH 2 +Address binding +.PP +As was mentioned in section 2, +binding addresses to sockets in the Internet and NS domains can be +fairly complex. As a brief reminder, these associations +are composed of local and foreign +addresses, and local and foreign ports. Port numbers are +allocated out of separate spaces, one for each system and one +for each domain on that system. +Through the \fIbind\fP system call, a +process may specify half of an association, the +<local address, local port> part, while the +\fIconnect\fP +and \fIaccept\fP +primitives are used to complete a socket's association by +specifying the <foreign address, foreign port> part. +Since the association is created in two steps the association +uniqueness requirement indicated previously could be violated unless +care is taken. Further, it is unrealistic to expect user +programs to always know proper values to use for the local address +and local port since a host may reside on multiple networks and +the set of allocated port numbers is not directly accessible +to a user. +.PP +To simplify local address binding in the Internet domain the notion of a +\*(lqwildcard\*(rq address has been provided. When an address +is specified as INADDR_ANY (a manifest constant defined in +<netinet/in.h>), the system interprets the address as +\*(lqany valid address\*(rq. For example, to bind a specific +port number to a socket, but leave the local address unspecified, +the following code might be used: +.DS +#include <sys/types.h> +#include <netinet/in.h> + ... +struct sockaddr_in sin; + ... +s = socket(AF_INET, SOCK_STREAM, 0); +sin.sin_family = AF_INET; +sin.sin_addr.s_addr = htonl(INADDR_ANY); +sin.sin_port = htons(MYPORT); +bind(s, (struct sockaddr *) &sin, sizeof (sin)); +.DE +Sockets with wildcarded local addresses may receive messages +directed to the specified port number, and sent to any +of the possible addresses assigned to a host. For example, +if a host has addresses 128.32.0.4 and 10.0.0.78, and a socket is bound as +above, the process will be +able to accept connection requests which are addressed to +128.32.0.4 or 10.0.0.78. +If a server process wished to only allow hosts on a +given network connect to it, it would bind +the address of the host on the appropriate network. +.PP +In a similar fashion, a local port may be left unspecified +(specified as zero), in which case the system will select an +appropriate port number for it. This shortcut will work +both in the Internet and NS domains. For example, to +bind a specific local address to a socket, but to leave the +local port number unspecified: +.DS +hp = gethostbyname(hostname); +if (hp == NULL) { + ... +} +bcopy(hp->h_addr, (char *) sin.sin_addr, hp->h_length); +sin.sin_port = htons(0); +bind(s, (struct sockaddr *) &sin, sizeof (sin)); +.DE +The system selects the local port number based on two criteria. +The first is that on 4BSD systems, +Internet ports below IPPORT_RESERVED (1024) (for the Xerox domain, +0 through 3000) are reserved +for privileged users (i.e., the super user); +Internet ports above IPPORT_USERRESERVED (50000) are reserved +for non-privileged servers. The second is +that the port number is not currently bound to some other +socket. In order to find a free Internet port number in the privileged +range the \fIrresvport\fP library routine may be used as follows +to return a stream socket in with a privileged port number: +.DS +int lport = IPPORT_RESERVED \- 1; +int s; +\&... +s = rresvport(&lport); +if (s < 0) { + if (errno == EAGAIN) + fprintf(stderr, "socket: all ports in use\en"); + else + perror("rresvport: socket"); + ... +} +.DE +The restriction on allocating ports was done to allow processes +executing in a \*(lqsecure\*(rq environment to perform authentication +based on the originating address and port number. For example, +the \fIrlogin\fP(1) command allows users to log in across a network +without being asked for a password, if two conditions hold: +First, the name of the system the user +is logging in from is in the file +\fI/etc/hosts.equiv\fP on the system he is logging +in to (or the system name and the user name are in +the user's \fI.rhosts\fP file in the user's home +directory), and second, that the user's rlogin +process is coming from a privileged port on the machine from which he is +logging. The port number and network address of the +machine from which the user is logging in can be determined either +by the \fIfrom\fP result of the \fIaccept\fP call, or +from the \fIgetpeername\fP call. +.PP +In certain cases the algorithm used by the system in selecting +port numbers is unsuitable for an application. This is because +associations are created in a two step process. For example, +the Internet file transfer protocol, FTP, specifies that data +connections must always originate from the same local port. However, +duplicate associations are avoided by connecting to different foreign +ports. In this situation the system would disallow binding the +same local address and port number to a socket if a previous data +connection's socket still existed. To override the default port +selection algorithm, an option call must be performed prior +to address binding: +.DS + ... +int on = 1; + ... +setsockopt(s, SOL_SOCKET, SO_REUSEADDR, &on, sizeof(on)); +bind(s, (struct sockaddr *) &sin, sizeof (sin)); +.DE +With the above call, local addresses may be bound which +are already in use. This does not violate the uniqueness +requirement as the system still checks at connect time to +be sure any other sockets with the same local address and +port do not have the same foreign address and port. +If the association already exists, the error EADDRINUSE is returned. +A related socket option, SO_REUSEPORT, which allows completely +duplicate bindings, is described in the IP multicasting section. +.NH 2 +Socket Options +.PP +It is possible to set and get a number of options on sockets +via the \fIsetsockopt\fP and \fIgetsockopt\fP system calls. +These options include such things as marking a socket for +broadcasting, not to route, to linger on close, etc. +In addition, there are protocol-specific options for IP and TCP, +as described in +.IR ip (4), +.IR tcp (4), +and in the section on multicasting below. +.PP +The general forms of the calls are: +.DS +setsockopt(s, level, optname, optval, optlen); +.DE +and +.DS +getsockopt(s, level, optname, optval, optlen); +.DE +.PP +The parameters to the calls are as follows: \fIs\fP +is the socket on which the option is to be applied. +\fILevel\fP specifies the protocol layer on which the +option is to be applied; in most cases this is +the ``socket level'', indicated by the symbolic constant +SOL_SOCKET, defined in \fI<sys/socket.h>.\fP +The actual option is specified in \fIoptname\fP, and is +a symbolic constant also defined in \fI<sys/socket.h>\fP. +\fIOptval\fP and \fIOptlen\fP point to the value of the +option (in most cases, whether the option is to be turned +on or off), and the length of the value of the option, +respectively. +For \fIgetsockopt\fP, \fIoptlen\fP is +a value-result parameter, initially set to the size of +the storage area pointed to by \fIoptval\fP, and modified +upon return to indicate the actual amount of storage used. +.PP +An example should help clarify things. It is sometimes +useful to determine the type (e.g., stream, datagram, etc.) +of an existing socket; programs +under \fIinetd\fP (described below) may need to perform this +task. This can be accomplished as follows via the +SO_TYPE socket option and the \fIgetsockopt\fP call: +.DS +#include <sys/types.h> +#include <sys/socket.h> + +int type, size; + +size = sizeof (int); + +if (getsockopt(s, SOL_SOCKET, SO_TYPE, (char *) &type, &size) < 0) { + ... +} +.DE +After the \fIgetsockopt\fP call, \fItype\fP will be set +to the value of the socket type, as defined in +\fI<sys/socket.h>\fP. If, for example, the socket were +a datagram socket, \fItype\fP would have the value +corresponding to SOCK_DGRAM. +.NH 2 +Broadcasting and determining network configuration +.PP +By using a datagram socket, it is possible to send broadcast +packets on many networks supported by the system. +The network itself must support broadcast; the system +provides no simulation of broadcast in software. +Broadcast messages can place a high load on a network since they force +every host on the network to service them. Consequently, +the ability to send broadcast packets has been limited +to sockets which are explicitly marked as allowing broadcasting. +Broadcast is typically used for one of two reasons: +it is desired to find a resource on a local network without prior +knowledge of its address, +or important functions such as routing require that information +be sent to all accessible neighbors. +.PP +Multicasting is an alternative to broadcasting. +Setting up IP multicast sockets is described in the next section. +.PP +To send a broadcast message, a datagram socket +should be created: +.DS +s = socket(AF_INET, SOCK_DGRAM, 0); +.DE +or +.DS +s = socket(AF_NS, SOCK_DGRAM, 0); +.DE +The socket is marked as allowing broadcasting, +.DS +int on = 1; + +setsockopt(s, SOL_SOCKET, SO_BROADCAST, &on, sizeof (on)); +.DE +and at least a port number should be bound to the socket: +.DS +sin.sin_family = AF_INET; +sin.sin_addr.s_addr = htonl(INADDR_ANY); +sin.sin_port = htons(MYPORT); +bind(s, (struct sockaddr *) &sin, sizeof (sin)); +.DE +or, for the NS domain, +.DS +sns.sns_family = AF_NS; +netnum = htonl(net); +sns.sns_addr.x_net = *(union ns_net *) &netnum; /* insert net number */ +sns.sns_addr.x_port = htons(MYPORT); +bind(s, (struct sockaddr *) &sns, sizeof (sns)); +.DE +The destination address of the message to be broadcast +depends on the network(s) on which the message is to be broadcast. +The Internet domain supports a shorthand notation for broadcast +on the local network, the address INADDR_BROADCAST (defined in +<\fInetinet/in.h\fP>. +To determine the list of addresses for all reachable neighbors +requires knowledge of the networks to which the host is connected. +Since this information should +be obtained in a host-independent fashion and may be impossible +to derive, 4.4BSD provides a method of +retrieving this information from the system data structures. +The SIOCGIFCONF \fIioctl\fP call returns the interface +configuration of a host in the form of a +single \fIifconf\fP structure; this structure contains +a ``data area'' which is made up of an array of +of \fIifreq\fP structures, one for each network interface +to which the host is connected. +These structures are defined in +\fI<net/if.h>\fP as follows: +.DS +.if t .ta .5i 1.0i 1.5i 3.5i +.if n .ta .7i 1.4i 2.1i 3.4i +struct ifconf { + int ifc_len; /* size of associated buffer */ + union { + caddr_t ifcu_buf; + struct ifreq *ifcu_req; + } ifc_ifcu; +}; + +#define ifc_buf ifc_ifcu.ifcu_buf /* buffer address */ +#define ifc_req ifc_ifcu.ifcu_req /* array of structures returned */ + +#define IFNAMSIZ 16 + +struct ifreq { + char ifr_name[IFNAMSIZ]; /* if name, e.g. "en0" */ + union { + struct sockaddr ifru_addr; + struct sockaddr ifru_dstaddr; + struct sockaddr ifru_broadaddr; + short ifru_flags; + caddr_t ifru_data; + } ifr_ifru; +}; + +.if t .ta \w' #define'u +\w' ifr_broadaddr'u +\w' ifr_ifru.ifru_broadaddr'u +#define ifr_addr ifr_ifru.ifru_addr /* address */ +#define ifr_dstaddr ifr_ifru.ifru_dstaddr /* other end of p-to-p link */ +#define ifr_broadaddr ifr_ifru.ifru_broadaddr /* broadcast address */ +#define ifr_flags ifr_ifru.ifru_flags /* flags */ +#define ifr_data ifr_ifru.ifru_data /* for use by interface */ +.DE +The actual call which obtains the +interface configuration is +.DS +struct ifconf ifc; +char buf[BUFSIZ]; + +ifc.ifc_len = sizeof (buf); +ifc.ifc_buf = buf; +if (ioctl(s, SIOCGIFCONF, (char *) &ifc) < 0) { + ... +} +.DE +After this call \fIbuf\fP will contain one \fIifreq\fP structure for +each network to which the host is connected, and +\fIifc.ifc_len\fP will have been modified to reflect the number +of bytes used by the \fIifreq\fP structures. +.PP +For each structure +there exists a set of ``interface flags'' which tell +whether the network corresponding to that interface is +up or down, point to point or broadcast, etc. The +SIOCGIFFLAGS \fIioctl\fP retrieves these +flags for an interface specified by an \fIifreq\fP +structure as follows: +.DS +struct ifreq *ifr; + +ifr = ifc.ifc_req; + +for (n = ifc.ifc_len / sizeof (struct ifreq); --n >= 0; ifr++) { + /* + * We must be careful that we don't use an interface + * devoted to an address family other than those intended; + * if we were interested in NS interfaces, the + * AF_INET would be AF_NS. + */ + if (ifr->ifr_addr.sa_family != AF_INET) + continue; + if (ioctl(s, SIOCGIFFLAGS, (char *) ifr) < 0) { + ... + } + /* + * Skip boring cases. + */ + if ((ifr->ifr_flags & IFF_UP) == 0 || + (ifr->ifr_flags & IFF_LOOPBACK) || + (ifr->ifr_flags & (IFF_BROADCAST | IFF_POINTTOPOINT)) == 0) + continue; +.DE +.PP +Once the flags have been obtained, the broadcast address +must be obtained. In the case of broadcast networks this is +done via the SIOCGIFBRDADDR \fIioctl\fP, while for point-to-point networks +the address of the destination host is obtained with SIOCGIFDSTADDR. +.DS +struct sockaddr dst; + +if (ifr->ifr_flags & IFF_POINTTOPOINT) { + if (ioctl(s, SIOCGIFDSTADDR, (char *) ifr) < 0) { + ... + } + bcopy((char *) ifr->ifr_dstaddr, (char *) &dst, sizeof (ifr->ifr_dstaddr)); +} else if (ifr->ifr_flags & IFF_BROADCAST) { + if (ioctl(s, SIOCGIFBRDADDR, (char *) ifr) < 0) { + ... + } + bcopy((char *) ifr->ifr_broadaddr, (char *) &dst, sizeof (ifr->ifr_broadaddr)); +} +.DE +.PP +After the appropriate \fIioctl\fP's have obtained the broadcast +or destination address (now in \fIdst\fP), the \fIsendto\fP call may be +used: +.DS + sendto(s, buf, buflen, 0, (struct sockaddr *)&dst, sizeof (dst)); +} +.DE +In the above loop one \fIsendto\fP occurs for every +interface to which the host is connected that supports the notion of +broadcast or point-to-point addressing. +If a process only wished to send broadcast +messages on a given network, code similar to that outlined above +would be used, but the loop would need to find the +correct destination address. +.PP +Received broadcast messages contain the senders address +and port, as datagram sockets are bound before +a message is allowed to go out. +.NH 2 +IP Multicasting +.PP +IP multicasting is the transmission of an IP datagram to a "host +group", a set of zero or more hosts identified by a single IP +destination address. A multicast datagram is delivered to all +members of its destination host group with the same "best-efforts" +reliability as regular unicast IP datagrams, i.e., the datagram is +not guaranteed to arrive intact at all members of the destination +group or in the same order relative to other datagrams. +.PP +The membership of a host group is dynamic; that is, hosts may join +and leave groups at any time. There is no restriction on the +location or number of members in a host group. A host may be a +member of more than one group at a time. A host need not be a member +of a group to send datagrams to it. +.PP +A host group may be permanent or transient. A permanent group has a +well-known, administratively assigned IP address. It is the address, +not the membership of the group, that is permanent; at any time a +permanent group may have any number of members, even zero. Those IP +multicast addresses that are not reserved for permanent groups are +available for dynamic assignment to transient groups which exist only +as long as they have members. +.PP +In general, a host cannot assume that datagrams sent to any host +group address will reach only the intended hosts, or that datagrams +received as a member of a transient host group are intended for the +recipient. Misdelivery must be detected at a level above IP, using +higher-level identifiers or authentication tokens. Information +transmitted to a host group address should be encrypted or governed +by administrative routing controls if the sender is concerned about +unwanted listeners. +.PP +IP multicasting is currently supported only on AF_INET sockets of type +SOCK_DGRAM and SOCK_RAW, and only on subnetworks for which the interface +driver has been modified to support multicasting. +.PP +The next subsections describe how to send and receive multicast datagrams. +.NH 3 +Sending IP Multicast Datagrams +.PP +To send a multicast datagram, specify an IP multicast address in the range +224.0.0.0 to 239.255.255.255 as the destination address +in a +.IR sendto (2) +call. +.PP +The definitions required for the multicast-related socket options are +found in \fI<netinet/in.h>\fP. +All IP addresses are passed in network byte-order. +.PP +By default, IP multicast datagrams are sent with a time-to-live (TTL) of 1, +which prevents them from being forwarded beyond a single subnetwork. A new +socket option allows the TTL for subsequent multicast datagrams to be set to +any value from 0 to 255, in order to control the scope of the multicasts: +.DS +u_char ttl; +setsockopt(sock, IPPROTO_IP, IP_MULTICAST_TTL, &ttl, sizeof(ttl)); +.DE +Multicast datagrams with a TTL of 0 will not be transmitted on any subnet, +but may be delivered locally if the sending host belongs to the destination +group and if multicast loopback has not been disabled on the sending socket +(see below). Multicast datagrams with TTL greater than one may be delivered +to more than one subnet if there are one or more multicast routers attached +to the first-hop subnet. To provide meaningful scope control, the multicast +routers support the notion of TTL "thresholds", which prevent datagrams with +less than a certain TTL from traversing certain subnets. The thresholds +enforce the following convention: +.TS +center; +l | l +l | n. +_ +Scope Initial TTL += +restricted to the same host 0 +restricted to the same subnet 1 +restricted to the same site 32 +restricted to the same region 64 +restricted to the same continent 128 +unrestricted 255 +_ +.TE +"Sites" and "regions" are not strictly defined, and sites may be further +subdivided into smaller administrative units, as a local matter. +.PP +An application may choose an initial TTL other than the ones listed above. +For example, an application might perform an "expanding-ring search" for a +network resource by sending a multicast query, first with a TTL of 0, and +then with larger and larger TTLs, until a reply is received, perhaps using +the TTL sequence 0, 1, 2, 4, 8, 16, 32. +.PP +The multicast router +.IR mrouted (8), +refuses to forward any +multicast datagram with a destination address between 224.0.0.0 and +224.0.0.255, inclusive, regardless of its TTL. This range of addresses is +reserved for the use of routing protocols and other low-level topology +discovery or maintenance protocols, such as gateway discovery and group +membership reporting. +.PP +The address 224.0.0.0 is +guaranteed not to be assigned to any group, and 224.0.0.1 is assigned +to the permanent group of all IP hosts (including gateways). This is +used to address all multicast hosts on the directly connected +network. There is no multicast address (or any other IP address) for +all hosts on the total Internet. The addresses of other well-known, +permanent groups are published in the "Assigned Numbers" RFC, +which is available from the InterNIC. +.PP +Each multicast transmission is sent from a single network interface, even if +the host has more than one multicast-capable interface. (If the host is +also serving as a multicast router, +a multicast may be \fIforwarded\fP to interfaces +other than originating interface, provided that the TTL is greater than 1.) +The default interface to be used for multicasting is the primary network +interface on the system. +A socket option +is available to override the default for subsequent transmissions from a +given socket: +.DS +struct in_addr addr; +setsockopt(sock, IPPROTO_IP, IP_MULTICAST_IF, &addr, sizeof(addr)); +.DE +where "addr" is the local IP address of the desired outgoing interface. +An address of INADDR_ANY may be used to revert to the default interface. +The local IP address of an interface can be obtained via the SIOCGIFCONF +ioctl. To determine if an interface supports multicasting, fetch the +interface flags via the SIOCGIFFLAGS ioctl and see if the IFF_MULTICAST +flag is set. (Normal applications should not need to use this option; it +is intended primarily for multicast routers and other system services +specifically concerned with internet topology.) +The SIOCGIFCONF and SIOCGIFFLAGS ioctls are described in the previous section. +.PP +If a multicast datagram is sent to a group to which the sending host itself +belongs (on the outgoing interface), a copy of the datagram is, by default, +looped back by the IP layer for local delivery. Another socket option gives +the sender explicit control over whether or not subsequent datagrams are +looped back: +.DS +u_char loop; +setsockopt(sock, IPPROTO_IP, IP_MULTICAST_LOOP, &loop, sizeof(loop)); +.DE +where \f2loop\f1 is set to 0 to disable loopback, +and set to 1 to enable loopback. +This option +improves performance for applications that may have no more than one +instance on a single host (such as a router demon), by eliminating +the overhead of receiving their own transmissions. It should generally not +be used by applications for which there may be more than one instance on a +single host (such as a conferencing program) or for which the sender does +not belong to the destination group (such as a time querying program). +.PP +A multicast datagram sent with an initial TTL greater than 1 may be delivered +to the sending host on a different interface from that on which it was sent, +if the host belongs to the destination group on that other interface. The +loopback control option has no effect on such delivery. +.NH 3 +Receiving IP Multicast Datagrams +.PP +Before a host can receive IP multicast datagrams, it must become a member +of one or more IP multicast groups. A process can ask the host to join +a multicast group by using the following socket option: +.DS +struct ip_mreq mreq; +setsockopt(sock, IPPROTO_IP, IP_ADD_MEMBERSHIP, &mreq, sizeof(mreq)) +.DE +where "mreq" is the following structure: +.DS +struct ip_mreq { + struct in_addr imr_multiaddr; /* \fImulticast group to join\fP */ + struct in_addr imr_interface; /* \fIinterface to join on\fP */ +} +.DE +Every membership is associated with a single interface, and it is possible +to join the same group on more than one interface. "imr_interface" should +be INADDR_ANY to choose the default multicast interface, or one of the +host's local addresses to choose a particular (multicast-capable) interface. +Up to IP_MAX_MEMBERSHIPS (currently 20) memberships may be added on a +single socket. +.PP +To drop a membership, use: +.DS +struct ip_mreq mreq; +setsockopt(sock, IPPROTO_IP, IP_DROP_MEMBERSHIP, &mreq, sizeof(mreq)); +.DE +where "mreq" contains the same values as used to add the membership. The +memberships associated with a socket are also dropped when the socket is +closed or the process holding the socket is killed. However, more than +one socket may claim a membership in a particular group, and the host +will remain a member of that group until the last claim is dropped. +.PP +The memberships associated with a socket do not necessarily determine which +datagrams are received on that socket. Incoming multicast packets are +accepted by the kernel IP layer if any socket has claimed a membership in the +destination group of the datagram; however, delivery of a multicast datagram +to a particular socket is based on the destination port (or protocol type, for +raw sockets), just as with unicast datagrams. +To receive multicast datagrams +sent to a particular port, it is necessary to bind to that local port, +leaving the local address unspecified (i.e., INADDR_ANY). +To receive multicast datagrams +sent to a particular group and port, bind to the local port, with +the local address set to the multicast group address. +Once bound to a multicast address, the socket cannot be used for sending data. +.PP +More than one process may bind to the same SOCK_DGRAM UDP port +or the same multicast group and port if the +.I bind +call is preceded by: +.DS +int on = 1; +setsockopt(sock, SOL_SOCKET, SO_REUSEPORT, &on, sizeof(on)); +.DE +All processes sharing the port must enable this option. +Every incoming multicast or broadcast UDP datagram destined to +the shared port is delivered to all sockets bound to the port. +For backwards compatibility reasons, this does not apply to incoming +unicast datagrams. Unicast +datagrams are never delivered to more than one socket, regardless of +how many sockets are bound to the datagram's destination port. +.PP +A final multicast-related extension is independent of IP: two new ioctls, +SIOCADDMULTI and SIOCDELMULTI, are available to add or delete link-level +(e.g., Ethernet) multicast addresses accepted by a particular interface. +The address to be added or deleted is passed as a sockaddr structure of +family AF_UNSPEC, within the standard ifreq structure. +.PP +These ioctls are +for the use of protocols other than IP, and require superuser privileges. +A link-level multicast address added via SIOCADDMULTI is not automatically +deleted when the socket used to add it goes away; it must be explicitly +deleted. It is inadvisable to delete a link-level address that may be +in use by IP. +.NH 3 +Sample Multicast Program +.PP +The following program sends or receives multicast packets. +If invoked with one argument, it sends a packet containing the current +time to an arbitrarily-chosen multicast group and UDP port. +If invoked with no arguments, it receives and prints these packets. +Start it as a sender on just one host and as a receiver on all the other hosts. +.DS +#include <sys/types.h> +#include <sys/socket.h> +#include <netinet/in.h> +#include <arpa/inet.h> +#include <time.h> +#include <stdio.h> + +#define EXAMPLE_PORT 60123 +#define EXAMPLE_GROUP "224.0.0.250" + +main(argc) + int argc; +{ + struct sockaddr_in addr; + int addrlen, fd, cnt; + struct ip_mreq mreq; + char message[50]; + + fd = socket(AF_INET, SOCK_DGRAM, 0); + if (fd < 0) { + perror("socket"); + exit(1); + } + + bzero(&addr, sizeof(addr)); + addr.sin_family = AF_INET; + addr.sin_addr.s_addr = htonl(INADDR_ANY); + addr.sin_port = htons(EXAMPLE_PORT); + addrlen = sizeof(addr); + + if (argc > 1) { /* Send */ + addr.sin_addr.s_addr = inet_addr(EXAMPLE_GROUP); + while (1) { + time_t t = time(0); + sprintf(message, "time is %-24.24s", ctime(&t)); + cnt = sendto(fd, message, sizeof(message), 0, + (struct sockaddr *)&addr, addrlen); + if (cnt < 0) { + perror("sendto"); + exit(1); + } + sleep(5); + } + } else { /* Receive */ + if (bind(fd, (struct sockaddr *)&addr, sizeof(addr)) < 0) { + perror("bind"); + exit(1); + } + + mreq.imr_multiaddr.s_addr = inet_addr(EXAMPLE_GROUP); + mreq.imr_interface.s_addr = htonl(INADDR_ANY); + if (setsockopt(fd, IPPROTO_IP, IP_ADD_MEMBERSHIP, + &mreq, sizeof(mreq)) < 0) { + perror("setsockopt mreq"); + exit(1); + } + + while (1) { + cnt = recvfrom(fd, message, sizeof(message), 0, + (struct sockaddr *)&addr, &addrlen); + if (cnt <= 0) { + if (cnt == 0) { + break; + } + perror("recvfrom"); + exit(1); + } + printf("%s: message = \e"%s\e"\en", + inet_ntoa(addr.sin_addr), message); + } + } +} +.DE +.\"---------------------------------------------------------------------- +.NH 2 +NS Packet Sequences +.PP +The semantics of NS connections demand that +the user both be able to look inside the network header associated +with any incoming packet and be able to specify what should go +in certain fields of an outgoing packet. +Using different calls to \fIsetsockopt\fP, it is possible +to indicate whether prototype headers will be associated by +the user with each outgoing packet (SO_HEADERS_ON_OUTPUT), +to indicate whether the headers received by the system should be +delivered to the user (SO_HEADERS_ON_INPUT), or to indicate +default information that should be associated with all +outgoing packets on a given socket (SO_DEFAULT_HEADERS). +.PP +The contents of a SPP header (minus the IDP header) are: +.DS +.if t .ta \w" #define"u +\w" u_short"u +2.0i +struct sphdr { + u_char sp_cc; /* connection control */ +#define SP_SP 0x80 /* system packet */ +#define SP_SA 0x40 /* send acknowledgement */ +#define SP_OB 0x20 /* attention (out of band data) */ +#define SP_EM 0x10 /* end of message */ + u_char sp_dt; /* datastream type */ + u_short sp_sid; /* source connection identifier */ + u_short sp_did; /* destination connection identifier */ + u_short sp_seq; /* sequence number */ + u_short sp_ack; /* acknowledge number */ + u_short sp_alo; /* allocation number */ +}; +.DE +Here, the items of interest are the \fIdatastream type\fP and +the \fIconnection control\fP fields. The semantics of the +datastream type are defined by the application(s) in question; +the value of this field is, by default, zero, but it can be +used to indicate things such as Xerox's Bulk Data Transfer +Protocol (in which case it is set to one). The connection control +field is a mask of the flags defined just below it. The user may +set or clear the end-of-message bit to indicate +that a given message is the last of a given substream type, +or may set/clear the attention bit as an alternate way to +indicate that a packet should be sent out-of-band. +As an example, to associate prototype headers with outgoing +SPP packets, consider: +.DS +#include <sys/types.h> +#include <sys/socket.h> +#include <netns/ns.h> +#include <netns/sp.h> + ... +struct sockaddr_ns sns, to; +int s, on = 1; +struct databuf { + struct sphdr proto_spp; /* prototype header */ + char buf[534]; /* max. possible data by Xerox std. */ +} buf; + ... +s = socket(AF_NS, SOCK_SEQPACKET, 0); + ... +bind(s, (struct sockaddr *) &sns, sizeof (sns)); +setsockopt(s, NSPROTO_SPP, SO_HEADERS_ON_OUTPUT, &on, sizeof(on)); + ... +buf.proto_spp.sp_dt = 1; /* bulk data */ +buf.proto_spp.sp_cc = SP_EM; /* end-of-message */ +strcpy(buf.buf, "hello world\en"); +sendto(s, (char *) &buf, sizeof(struct sphdr) + strlen("hello world\en"), + (struct sockaddr *) &to, sizeof(to)); + ... +.DE +Note that one must be careful when writing headers; if the prototype +header is not written with the data with which it is to be associated, +the kernel will treat the first few bytes of the data as the +header, with unpredictable results. +To turn off the above association, and to indicate that packet +headers received by the system should be passed up to the user, +one might use: +.DS +#include <sys/types.h> +#include <sys/socket.h> +#include <netns/ns.h> +#include <netns/sp.h> + ... +struct sockaddr sns; +int s, on = 1, off = 0; + ... +s = socket(AF_NS, SOCK_SEQPACKET, 0); + ... +bind(s, (struct sockaddr *) &sns, sizeof (sns)); +setsockopt(s, NSPROTO_SPP, SO_HEADERS_ON_OUTPUT, &off, sizeof(off)); +setsockopt(s, NSPROTO_SPP, SO_HEADERS_ON_INPUT, &on, sizeof(on)); + ... +.DE +.PP +Output is handled somewhat differently in the IDP world. +The header of an IDP-level packet looks like: +.DS +.if t .ta \w'struct 'u +\w" struct ns_addr"u +2.0i +struct idp { + u_short idp_sum; /* Checksum */ + u_short idp_len; /* Length, in bytes, including header */ + u_char idp_tc; /* Transport Control (i.e., hop count) */ + u_char idp_pt; /* Packet Type (i.e., level 2 protocol) */ + struct ns_addr idp_dna; /* Destination Network Address */ + struct ns_addr idp_sna; /* Source Network Address */ +}; +.DE +The primary field of interest in an IDP header is the \fIpacket type\fP +field. The standard values for this field are (as defined +in <\fInetns/ns.h\fP>): +.DS +.if t .ta \w" #define"u +\w" NSPROTO_ERROR"u +1.0i +#define NSPROTO_RI 1 /* Routing Information */ +#define NSPROTO_ECHO 2 /* Echo Protocol */ +#define NSPROTO_ERROR 3 /* Error Protocol */ +#define NSPROTO_PE 4 /* Packet Exchange */ +#define NSPROTO_SPP 5 /* Sequenced Packet */ +.DE +For SPP connections, the contents of this field are +automatically set to NSPROTO_SPP; for IDP packets, +this value defaults to zero, which means ``unknown''. +.PP +Setting the value of that field with SO_DEFAULT_HEADERS is +easy: +.DS +#include <sys/types.h> +#include <sys/socket.h> +#include <netns/ns.h> +#include <netns/idp.h> + ... +struct sockaddr sns; +struct idp proto_idp; /* prototype header */ +int s, on = 1; + ... +s = socket(AF_NS, SOCK_DGRAM, 0); + ... +bind(s, (struct sockaddr *) &sns, sizeof (sns)); +proto_idp.idp_pt = NSPROTO_PE; /* packet exchange */ +setsockopt(s, NSPROTO_IDP, SO_DEFAULT_HEADERS, (char *) &proto_idp, + sizeof(proto_idp)); + ... +.DE +.PP +Using SO_HEADERS_ON_OUTPUT is somewhat more difficult. When +SO_HEADERS_ON_OUTPUT is turned on for an IDP socket, the socket +becomes (for all intents and purposes) a raw socket. In this +case, all the fields of the prototype header (except the +length and checksum fields, which are computed by the kernel) +must be filled in correctly in order for the socket to send and +receive data in a sensible manner. To be more specific, the +source address must be set to that of the host sending the +data; the destination address must be set to that of the +host for whom the data is intended; the packet type must be +set to whatever value is desired; and the hopcount must be +set to some reasonable value (almost always zero). It should +also be noted that simply sending data using \fIwrite\fP +will not work unless a \fIconnect\fP or \fIsendto\fP call +is used, in spite of the fact that it is the destination +address in the prototype header that is used, not the one +given in either of those calls. For almost +all IDP applications , using SO_DEFAULT_HEADERS is easier and +more desirable than writing headers. +.NH 2 +Three-way Handshake +.PP +The semantics of SPP connections indicates that a three-way +handshake, involving changes in the datastream type, should \(em +but is not absolutely required to \(em take place before a SPP +connection is closed. Almost all SPP connections are +``well-behaved'' in this manner; when communicating with +any process, it is best to assume that the three-way handshake +is required unless it is known for certain that it is not +required. In a three-way close, the closing process +indicates that it wishes to close the connection by sending +a zero-length packet with end-of-message set and with +datastream type 254. The other side of the connection +indicates that it is OK to close by sending a zero-length +packet with end-of-message set and datastream type 255. Finally, +the closing process replies with a zero-length packet with +substream type 255; at this point, the connection is considered +closed. The following code fragments are simplified examples +of how one might handle this three-way handshake at the user +level; in the future, support for this type of close will +probably be provided as part of the C library or as part of +the kernel. The first code fragment below illustrates how a process +might handle three-way handshake if it sees that the process it +is communicating with wants to close the connection: +.DS +#include <sys/types.h> +#include <sys/socket.h> +#include <netns/ns.h> +#include <netns/sp.h> + ... +#ifndef SPPSST_END +#define SPPSST_END 254 +#define SPPSST_ENDREPLY 255 +#endif +struct sphdr proto_sp; +int s; + ... +read(s, buf, BUFSIZE); +if (((struct sphdr *)buf)->sp_dt == SPPSST_END) { + /* + * SPPSST_END indicates that the other side wants to + * close. + */ + proto_sp.sp_dt = SPPSST_ENDREPLY; + proto_sp.sp_cc = SP_EM; + setsockopt(s, NSPROTO_SPP, SO_DEFAULT_HEADERS, (char *)&proto_sp, + sizeof(proto_sp)); + write(s, buf, 0); + /* + * Write a zero-length packet with datastream type = SPPSST_ENDREPLY + * to indicate that the close is OK with us. The packet that we + * don't see (because we don't look for it) is another packet + * from the other side of the connection, with SPPSST_ENDREPLY + * on it it, too. Once that packet is sent, the connection is + * considered closed; note that we really ought to retransmit + * the close for some time if we do not get a reply. + */ + close(s); +} + ... +.DE +To indicate to another process that we would like to close the +connection, the following code would suffice: +.DS +#include <sys/types.h> +#include <sys/socket.h> +#include <netns/ns.h> +#include <netns/sp.h> + ... +#ifndef SPPSST_END +#define SPPSST_END 254 +#define SPPSST_ENDREPLY 255 +#endif +struct sphdr proto_sp; +int s; + ... +proto_sp.sp_dt = SPPSST_END; +proto_sp.sp_cc = SP_EM; +setsockopt(s, NSPROTO_SPP, SO_DEFAULT_HEADERS, (char *)&proto_sp, + sizeof(proto_sp)); +write(s, buf, 0); /* send the end request */ +proto_sp.sp_dt = SPPSST_ENDREPLY; +setsockopt(s, NSPROTO_SPP, SO_DEFAULT_HEADERS, (char *)&proto_sp, + sizeof(proto_sp)); +/* + * We assume (perhaps unwisely) + * that the other side will send the + * ENDREPLY, so we'll just send our final ENDREPLY + * as if we'd seen theirs already. + */ +write(s, buf, 0); +close(s); + ... +.DE +.NH 2 +Packet Exchange +.PP +The Xerox standard protocols include a protocol that is both +reliable and datagram-oriented. This protocol is known as +Packet Exchange (PEX or PE) and, like SPP, is layered on top +of IDP. PEX is important for a number of things: Courier +remote procedure calls may be expedited through the use +of PEX, and many Xerox servers are located by doing a PEX +``BroadcastForServers'' operation. Although there is no +implementation of PEX in the kernel, +it may be simulated at the user level with some clever coding +and the use of one peculiar \fIgetsockopt\fP. A PEX packet +looks like: +.DS +.if t .ta \w'struct 'u +\w" struct idp"u +2.0i +/* + * The packet-exchange header shown here is not defined + * as part of any of the system include files. + */ +struct pex { + struct idp p_idp; /* idp header */ + u_short ph_id[2]; /* unique transaction ID for pex */ + u_short ph_client; /* client type field for pex */ +}; +.DE +The \fIph_id\fP field is used to hold a ``unique id'' that +is used in duplicate suppression; the \fIph_client\fP +field indicates the PEX client type (similar to the packet +type field in the IDP header). PEX reliability stems from the +fact that it is an idempotent (``I send a packet to you, you +send a packet to me'') protocol. Processes on each side of +the connection may use the unique id to determine if they have +seen a given packet before (the unique id field differs on each +packet sent) so that duplicates may be detected, and to indicate +which message a given packet is in response to. If a packet with +a given unique id is sent and no response is received in a given +amount of time, the packet is retransmitted until it is decided +that no response will ever be received. To simulate PEX, one +must be able to generate unique ids -- something that is hard to +do at the user level with any real guarantee that the id is really +unique. Therefore, a means (via \fIgetsockopt\fP) has been provided +for getting unique ids from the kernel. The following code fragment +indicates how to get a unique id: +.DS +long uniqueid; +int s, idsize = sizeof(uniqueid); + ... +s = socket(AF_NS, SOCK_DGRAM, 0); + ... +/* get id from the kernel -- only on IDP sockets */ +getsockopt(s, NSPROTO_PE, SO_SEQNO, (char *)&uniqueid, &idsize); + ... +.DE +The retransmission and duplicate suppression code required to +simulate PEX fully is left as an exercise for the reader. +.NH 2 +Inetd +.PP +One of the daemons provided with 4.4BSD is \fIinetd\fP, the +so called ``internet super-server.'' +Having one daemon listen for requests for many daemons +instead of having each daemon listen for its own requests +reduces the number of idle daemons and simplies their implementation. +.I Inetd +handles +two types of services: standard and TCPMUX. +A standard service has a well-known port assigned to it and +is listed in +.I /etc/services +(see \f2services\f1(5)); +it may be a service that implements an official Internet standard or is a +BSD-specific service. +TCPMUX services are nonstandard and do not have a +well-known port assigned to them. +They are invoked from +.I inetd +when a program connects to the "tcpmux" well-known port and specifies +the service name. +This is useful for adding locally-developed servers. +.PP +\fIInetd\fP is invoked at boot +time, and determines from the file \fI/etc/inetd.conf\fP the +servers for which it is to listen. Once this information has been +read and a pristine environment created, \fIinetd\fP proceeds +to create one socket for each service it is to listen for, +binding the appropriate port number to each socket. +.PP +\fIInetd\fP then performs a \fIselect\fP on all these +sockets for read availability, waiting for somebody wishing +a connection to the service corresponding to +that socket. \fIInetd\fP then performs an \fIaccept\fP on +the socket in question, \fIfork\fPs, \fIdup\fPs the new +socket to file descriptors 0 and 1 (stdin and +stdout), closes other open file +descriptors, and \fIexec\fPs the appropriate server. +.PP +Servers making use of \fIinetd\fP are considerably simplified, +as \fIinetd\fP takes care of the majority of the IPC work +required in establishing a connection. The server invoked +by \fIinetd\fP expects the socket connected to its client +on file descriptors 0 and 1, and may immediately perform +any operations such as \fIread\fP, \fIwrite\fP, \fIsend\fP, +or \fIrecv\fP. Indeed, servers may use +buffered I/O as provided by the ``stdio'' conventions, as +long as they remember to use \fIfflush\fP when appropriate. +.PP +One call which may be of interest to individuals writing +servers under \fIinetd\fP is the \fIgetpeername\fP call, +which returns the address of the peer (process) connected +on the other end of the socket. For example, to log the +Internet address in ``dot notation'' (e.g., ``128.32.0.4'') +of a client connected to a server under +\fIinetd\fP, the following code might be used: +.DS +struct sockaddr_in name; +int namelen = sizeof (name); + ... +if (getpeername(0, (struct sockaddr *)&name, &namelen) < 0) { + syslog(LOG_ERR, "getpeername: %m"); + exit(1); +} else + syslog(LOG_INFO, "Connection from %s", inet_ntoa(name.sin_addr)); + ... +.DE +While the \fIgetpeername\fP call is especially useful when +writing programs to run with \fIinetd\fP, it can be used +under other circumstances. Be warned, however, that \fIgetpeername\fP will +fail on UNIX domain sockets. +.PP +Standard TCP +services are assigned unique well-known port numbers in the range of +0 to 1023 by the +Internet Assigned Numbers Authority (IANA@ISI.EDU). +The limited number of ports in this range are +assigned to official Internet protocols. +The TCPMUX service allows you to add +locally-developed protocols without needing an official TCP port assignment. +The TCPMUX protocol described in RFC-1078 is simple: +.QP +``A TCP client connects to a foreign host on TCP port 1. It sends the +service name followed by a carriage-return line-feed <CRLF>. +The service name is never case sensitive. +The server replies with a +single character indicating positive ("+") or negative ("\-") +acknowledgment, immediately followed by an optional message of +explanation, terminated with a <CRLF>. If the reply was positive, +the selected protocol begins; otherwise the connection is closed.'' +.LP +In 4.4BSD, the TCPMUX service is built into +.IR inetd , +that is, +.IR inetd +listens on TCP port 1 for requests for TCPMUX services listed +in \f2inetd.conf\f1. +.IR inetd (8) +describes the format of TCPMUX entries for \f2inetd.conf\f1. +.PP +The following is an example TCPMUX server and its \f2inetd.conf\f1 entry. +More sophisticated servers may want to do additional processing +before returning the positive or negative acknowledgement. +.DS +#include <sys/types.h> +#include <stdio.h> + +main() +{ + time_t t; + + printf("+Go\er\en"); + fflush(stdout); + time(&t); + printf("%d = %s", t, ctime(&t)); + fflush(stdout); +} +.DE +The \f2inetd.conf\f1 entry is: +.DS +tcpmux/current_time stream tcp nowait nobody /d/curtime curtime +.DE +Here's the portion of the client code that handles the TCPMUX handshake: +.DS +char line[BUFSIZ]; +FILE *fp; + ... + +/* Use stdio for reading data from the server */ +fp = fdopen(sock, "r"); +if (fp == NULL) { + fprintf(stderr, "Can't create file pointer\en"); + exit(1); +} + +/* Send service request */ +sprintf(line, "%s\er\en", "current_time"); +if (write(sock, line, strlen(line)) < 0) { + perror("write"); + exit(1); +} + +/* Get ACK/NAK response from the server */ +if (fgets(line, sizeof(line), fp) == NULL) { + if (feof(fp)) { + die(); + } else { + fprintf(stderr, "Error reading response\en"); + exit(1); + } +} + +/* Delete <CR> */ +if ((lp = index(line, '\r')) != NULL) { + *lp = '\0'; +} + +switch (line[0]) { + case '+': + printf("Got ACK: %s\en", &line[1]); + break; + case '-': + printf("Got NAK: %s\en", &line[1]); + exit(0); + default: + printf("Got unknown response: %s\en", line); + exit(1); +} + +/* Get rest of data from the server */ +while ((fgets(line, sizeof(line), fp)) != NULL) { + fputs(line, stdout); +} +.DE diff --git a/share/doc/psd/21.ipc/Makefile b/share/doc/psd/21.ipc/Makefile new file mode 100644 index 000000000000..67c3d6c40a17 --- /dev/null +++ b/share/doc/psd/21.ipc/Makefile @@ -0,0 +1,9 @@ +# From: @(#)Makefile 8.1 (Berkeley) 6/8/93 +# $FreeBSD$ + +VOLUME= psd/21.ipc +SRCS= 0.t 1.t 2.t 3.t 4.t 5.t +MACROS= -ms +USE_TBL= + +.include <bsd.doc.mk> diff --git a/share/doc/psd/21.ipc/spell.ok b/share/doc/psd/21.ipc/spell.ok new file mode 100644 index 000000000000..02b45d4d26dd --- /dev/null +++ b/share/doc/psd/21.ipc/spell.ok @@ -0,0 +1,347 @@ +4.2bsd +AF +ANYP +BUFSIZ +BUFSIZE +BroadcastForServers +CF +CLR +CRMOD +Clearinghouse +DARPA +DESTPORT +DGRAM +DONTROUTE +Datagram +EADDRINUSE +EADDRNOTAVAIL +EAGAIN +ECONNREFUSED +EHOSTDOWN +EHOSTUNREACH +EINTR +ENDREPLY +ENETDOWN +ENETUNREACH +ENOBUFS +EPROTONOSUPPORT +EPROTOTYPE +ETIMEDOUT +EWOULDBLOCK +Ethernet +FASYNC +FCREATE +FD +FNDELAY +FTP +FTRUNCATE +FWRITE +FWRONLY +Fabry +GETOWN +Gethostybyname +IDP +IFF +IFNAMSIZ +INADDR +INET +INFO +IP +IPC +IPPORT +ISSET +Inetd +LF +LH +LOOPBACK +Lapsley +Leffler +MSG +MYADDRESS +MYPORT +NS +NSPROTO +OB +OOB +OOBINLINE +Optlen +Optval +PE +PEX +POINTTOPOINT +PS1:8 +RDONLY +RDWR +REUSEADDR +RF +RH +RWHODIR +SEQNO +SEQPACKET +SETFL +SETOWN +SETSIZE +SIGALRM +SIGCHLD +SIGIO +SIGURG +SIOCATMARK +SIOCGIFBRDADDR +SIOCGIFCONF +SIOCGIFDSTADDR +SIOCGIFFLAGS +SIOCGPGRP +SIOCSPGRP +SOF +SP +SPP +SPPSST +Science:UofMaryland +TCP +TELNET +TIOCFLUSH +TIOCGETP +TIOCNOTTY +TIOCSETP +TRUNC +Torek +Tutorial''PS1:8 +USERRESERVED +VAX +WNOHANG +WRONLY +XSIS +XTABS +ack +addr +addr.s +addr.sa +addr.sun +addr.x +addrtype +alo +argc +argv +arpa +b.sg +bcmp +bcopy +broadaddr +buf +buf.buf +buf.proto +buflen +bzero +c.f +cad +caddr +calder +daemons +dali +databuf +datagram +datastream +dev +dna +doit +dst +dst.sin +dst.sns +dstaddr +dt +dup2 +en0 +endhostent +endif +ernie +errno +es +esvax +exceptmask +execptfds +fcntl +fcntl.h +fd +fflush +file.h +foo +fprintf +from.sin +fromlen +gethostbyaddr +gethostbyname +gethostbynameandnet +gethostent +gethostname +getnetbyname +getnetbynumber +getnetent +getpeername +getprotobyname +getprotobynumber +getprotoent +getservbyname +getservbyport +getservent +getsockopt +goto +gotpty +gyre +gyre:Computer +hardcoding +hopcount +host.c +hostent +hostname +hostnames +hosts.equiv +htonl +htons +idp +idp.h +idp.idp +idsize +if.h +ifc +ifc.ifc +ifconf +ifcu +ifcu.ifcu +ifndef +ifr +ifreq +ifru +ifru.ifru +in.h +inet +inetd +inetd.conf +ing +ingres +io +ioctl.h +ipc +kim +len +localnet +lport +lq +makeaddr +matisse +medea +miro +monet +name.sin +namelen +nameserver +nb +netdb.h +netent +netinet +netns +netnum +netof +newsock +newtcp +nfds +ns +ns.h +ntoa +ntohl +ntohs +onalrm +oob +optlen +optname +optval +oz +pathname +pathnames +pex +pgrp +ph +pp +proto +protoent +pt +pty +ptyXX +ptyp +ptyxy +queueing +readfds +readmask +recv +recvfrom +recvtime +rem +req +rhosts +rlogin +rlogind +rq +rresvport +ruptime +rwho +rwhod +sendto +servent +server.sin +server.sun +sethostent +setsockopt +sid +sigvec +sin.sin +sizeof +sna +snew +sns +sns.sns +sockaddr +socket.h +sp +sp.h +sp.sp +sphdr +spp +spp.sp +sprintf +statbuf +statvax +std +stderr +stdin +stdio.h +stdout +strcmp +strcpy +strlen +syslog +ta +tcp +telnet +time.h +timeval +tmp +tolen +ttyxy +tuples +types.h +ucbvax +udp +un +un.h +uniqueid +useable +usec +val +wait.h +wait.tv +wd +wd.wd +whod +wildcard +wildcarded +writefds +writemask diff --git a/share/doc/psd/22.rpcgen/Makefile b/share/doc/psd/22.rpcgen/Makefile new file mode 100644 index 000000000000..04d4e9ee471a --- /dev/null +++ b/share/doc/psd/22.rpcgen/Makefile @@ -0,0 +1,8 @@ +# $FreeBSD$ + +VOLUME= psd/22.rpcgen +SRCS= stubs rpcgen.ms +MACROS= -ms +USE_TBL= + +.include <bsd.doc.mk> diff --git a/share/doc/psd/22.rpcgen/rpcgen.ms b/share/doc/psd/22.rpcgen/rpcgen.ms new file mode 100644 index 000000000000..e663e7fd4f94 --- /dev/null +++ b/share/doc/psd/22.rpcgen/rpcgen.ms @@ -0,0 +1,1301 @@ +.\" +.\" Must use -- tbl -- for this one +.\" +.\" @(#)rpcgen.ms 2.2 88/08/04 4.0 RPCSRC +.\" $FreeBSD$ +.\" +.de BT +.if \\n%=1 .tl ''- % -'' +.. +.ND +.\" prevent excess underlining in nroff +.if n .fp 2 R +.OH '\fBrpcgen\fP Programming Guide''Page %' +.EH 'Page %''\fBrpcgen\fP Programming Guide' +.if \n%=1 .bp +.SH +\&\fBrpcgen\fP Programming Guide +.NH 0 +\&The \fBrpcgen\fP Protocol Compiler +.IX rpcgen "" \fIrpcgen\fP "" PAGE MAJOR +.LP +.IX RPC "" "" \fIrpcgen\fP +The details of programming applications to use Remote Procedure Calls +can be overwhelming. Perhaps most daunting is the writing of the XDR +routines necessary to convert procedure arguments and results into +their network format and vice-versa. +.LP +Fortunately, +.I rpcgen(1) +exists to help programmers write RPC applications simply and directly. +.I rpcgen +does most of the dirty work, allowing programmers to debug +the main features of their application, instead of requiring them to +spend most of their time debugging their network interface code. +.LP +.I rpcgen +is a compiler. It accepts a remote program interface definition written +in a language, called RPC Language, which is similar to C. It produces a C +language output which includes stub versions of the client routines, a +server skeleton, XDR filter routines for both parameters and results, and a +header file that contains common definitions. The client stubs interface +with the RPC library and effectively hide the network from their callers. +The server stub similarly hides the network from the server procedures that +are to be invoked by remote clients. +.I rpcgen 's +output files can be compiled and linked in the usual way. The developer +writes server procedures\(emin any language that observes Sun calling +conventions\(emand links them with the server skeleton produced by +.I rpcgen +to get an executable server program. To use a remote program, a programmer +writes an ordinary main program that makes local procedure calls to the +client stubs produced by +.I rpcgen . +Linking this program with +.I rpcgen 's +stubs creates an executable program. (At present the main program must be +written in C). +.I rpcgen +options can be used to suppress stub generation and to specify the transport +to be used by the server stub. +.LP +Like all compilers, +.I rpcgen +reduces development time +that would otherwise be spent coding and debugging low-level routines. +All compilers, including +.I rpcgen , +do this at a small cost in efficiency +and flexibility. However, many compilers allow escape hatches for +programmers to mix low-level code with high-level code. +.I rpcgen +is no exception. In speed-critical applications, hand-written routines +can be linked with the +.I rpcgen +output without any difficulty. Also, one may proceed by using +.I rpcgen +output as a starting point, and then rewriting it as necessary. +(If you need a discussion of RPC programming without +.I rpcgen , +see the +.I "Remote Procedure Call Programming Guide)\. +.NH 1 +\&Converting Local Procedures into Remote Procedures +.IX rpcgen "local procedures" \fIrpcgen\fP +.IX rpcgen "remote procedures" \fIrpcgen\fP +.LP +Assume an application that runs on a single machine, one which we want +to convert to run over the network. Here we will demonstrate such a +conversion by way of a simple example\(ema program that prints a +message to the console: +.ie t .DS +.el .DS L +.ft I +/* + * printmsg.c: print a message on the console + */ +.ft CW +#include <stdio.h> + +main(argc, argv) + int argc; + char *argv[]; +{ + char *message; + + if (argc < 2) { + fprintf(stderr, "usage: %s <message>\en", argv[0]); + exit(1); + } + message = argv[1]; + + if (!printmessage(message)) { + fprintf(stderr, "%s: couldn't print your message\en", + argv[0]); + exit(1); + } + printf("Message Delivered!\en"); + exit(0); +} +.ft I +/* + * Print a message to the console. + * Return a boolean indicating whether the message was actually printed. + */ +.ft CW +printmessage(msg) + char *msg; +{ + FILE *f; + + f = fopen("/dev/console", "w"); + if (f == NULL) { + return (0); + } + fprintf(f, "%s\en", msg); + fclose(f); + return(1); +} +.DE +.LP +And then, of course: +.ie t .DS +.el .DS L +.ft CW +example% \fBcc printmsg.c -o printmsg\fP +example% \fBprintmsg "Hello, there."\fP +Message delivered! +example% +.DE +.LP +If +.I printmessage() +was turned into a remote procedure, +then it could be called from anywhere in the network. +Ideally, one would just like to stick a keyword like +.I remote +in front of a +procedure to turn it into a remote procedure. Unfortunately, +we have to live within the constraints of the C language, since +it existed long before RPC did. But even without language +support, it's not very difficult to make a procedure remote. +.LP +In general, it's necessary to figure out what the types are for +all procedure inputs and outputs. In this case, we have a +procedure +.I printmessage() +which takes a string as input, and returns an integer +as output. Knowing this, we can write a protocol specification in RPC +language that describes the remote version of +.I printmessage (). +Here it is: +.ie t .DS +.el .DS L +.ft I +/* + * msg.x: Remote message printing protocol + */ +.ft CW + +program MESSAGEPROG { + version MESSAGEVERS { + int PRINTMESSAGE(string) = 1; + } = 1; +} = 99; +.DE +.LP +Remote procedures are part of remote programs, so we actually declared +an entire remote program here which contains the single procedure +.I PRINTMESSAGE . +This procedure was declared to be in version 1 of the +remote program. No null procedure (procedure 0) is necessary because +.I rpcgen +generates it automatically. +.LP +Notice that everything is declared with all capital letters. This is +not required, but is a good convention to follow. +.LP +Notice also that the argument type is \*Qstring\*U and not \*Qchar *\*U. This +is because a \*Qchar *\*U in C is ambiguous. Programmers usually intend it +to mean a null-terminated string of characters, but it could also +represent a pointer to a single character or a pointer to an array of +characters. In RPC language, a null-terminated string is +unambiguously called a \*Qstring\*U. +.LP +There are just two more things to write. First, there is the remote +procedure itself. Here's the definition of a remote procedure +to implement the +.I PRINTMESSAGE +procedure we declared above: +.ie t .DS +.el .DS L +.vs 11 +.ft I +/* + * msg_proc.c: implementation of the remote procedure "printmessage" + */ +.ft CW + +#include <stdio.h> +#include <rpc/rpc.h> /* \fIalways needed\fP */ +#include "msg.h" /* \fIneed this too: msg.h will be generated by rpcgen\fP */ + +.ft I +/* + * Remote verson of "printmessage" + */ +.ft CW +int * +printmessage_1(msg) + char **msg; +{ + static int result; /* \fImust be static!\fP */ + FILE *f; + + f = fopen("/dev/console", "w"); + if (f == NULL) { + result = 0; + return (&result); + } + fprintf(f, "%s\en", *msg); + fclose(f); + result = 1; + return (&result); +} +.vs +.DE +.LP +Notice here that the declaration of the remote procedure +.I printmessage_1() +differs from that of the local procedure +.I printmessage() +in three ways: +.IP 1. +It takes a pointer to a string instead of a string itself. This +is true of all remote procedures: they always take pointers to their +arguments rather than the arguments themselves. +.IP 2. +It returns a pointer to an integer instead of an integer itself. This is +also generally true of remote procedures: they always return a pointer +to their results. +.IP 3. +It has an \*Q_1\*U appended to its name. In general, all remote +procedures called by +.I rpcgen +are named by the following rule: the name in the program definition +(here +.I PRINTMESSAGE ) +is converted to all +lower-case letters, an underbar (\*Q_\*U) is appended to it, and +finally the version number (here 1) is appended. +.LP +The last thing to do is declare the main client program that will call +the remote procedure. Here it is: +.ie t .DS +.el .DS L +.ft I +/* + * rprintmsg.c: remote version of "printmsg.c" + */ +.ft CW +#include <stdio.h> +#include <rpc/rpc.h> /* \fIalways needed\fP */ +#include "msg.h" /* \fIneed this too: msg.h will be generated by rpcgen\fP */ + +main(argc, argv) + int argc; + char *argv[]; +{ + CLIENT *cl; + int *result; + char *server; + char *message; + + if (argc < 3) { + fprintf(stderr, "usage: %s host message\en", argv[0]); + exit(1); + } + +.ft I + /* + * Save values of command line arguments + */ +.ft CW + server = argv[1]; + message = argv[2]; + +.ft I + /* + * Create client "handle" used for calling \fIMESSAGEPROG\fP on the + * server designated on the command line. We tell the RPC package + * to use the "tcp" protocol when contacting the server. + */ +.ft CW + cl = clnt_create(server, MESSAGEPROG, MESSAGEVERS, "tcp"); + if (cl == NULL) { +.ft I + /* + * Couldn't establish connection with server. + * Print error message and die. + */ +.ft CW + clnt_pcreateerror(server); + exit(1); + } + +.ft I + /* + * Call the remote procedure "printmessage" on the server + */ +.ft CW + result = printmessage_1(&message, cl); + if (result == NULL) { +.ft I + /* + * An error occurred while calling the server. + * Print error message and die. + */ +.ft CW + clnt_perror(cl, server); + exit(1); + } + +.ft I + /* + * Okay, we successfully called the remote procedure. + */ +.ft CW + if (*result == 0) { +.ft I + /* + * Server was unable to print our message. + * Print error message and die. + */ +.ft CW + fprintf(stderr, "%s: %s couldn't print your message\en", + argv[0], server); + exit(1); + } + +.ft I + /* + * The message got printed on the server's console + */ +.ft CW + printf("Message delivered to %s!\en", server); +} +.DE +There are two things to note here: +.IP 1. +.IX "client handle, used by rpcgen" "" "client handle, used by \fIrpcgen\fP" +First a client \*Qhandle\*U is created using the RPC library routine +.I clnt_create (). +This client handle will be passed to the stub routines +which call the remote procedure. +.IP 2. +The remote procedure +.I printmessage_1() +is called exactly the same way as it is declared in +.I msg_proc.c +except for the inserted client handle as the first argument. +.LP +Here's how to put all of the pieces together: +.ie t .DS +.el .DS L +.ft CW +example% \fBrpcgen msg.x\fP +example% \fBcc rprintmsg.c msg_clnt.c -o rprintmsg\fP +example% \fBcc msg_proc.c msg_svc.c -o msg_server\fP +.DE +Two programs were compiled here: the client program +.I rprintmsg +and the server program +.I msg_server . +Before doing this though, +.I rpcgen +was used to fill in the missing pieces. +.LP +Here is what +.I rpcgen +did with the input file +.I msg.x : +.IP 1. +It created a header file called +.I msg.h +that contained +.I #define 's +for +.I MESSAGEPROG , +.I MESSAGEVERS +and +.I PRINTMESSAGE +for use in the other modules. +.IP 2. +It created client \*Qstub\*U routines in the +.I msg_clnt.c +file. In this case there is only one, the +.I printmessage_1() +that was referred to from the +.I printmsg +client program. The name of the output file for +client stub routines is always formed in this way: if the name of the +input file is +.I FOO.x , +the client stubs output file is called +.I FOO_clnt.c . +.IP 3. +It created the server program which calls +.I printmessage_1() +in +.I msg_proc.c . +This server program is named +.I msg_svc.c . +The rule for naming the server output file is similar to the +previous one: for an input file called +.I FOO.x , +the output server file is named +.I FOO_svc.c . +.LP +Now we're ready to have some fun. First, copy the server to a +remote machine and run it. For this example, the +machine is called \*Qmoon\*U. Server processes are run in the +background, because they never exit. +.ie t .DS +.el .DS L +.ft CW +moon% \fBmsg_server &\fP +.DE +Then on our local machine (\*Qsun\*U) we can print a message on \*Qmoon\*Us +console. +.ie t .DS +.el .DS L +.ft CW +sun% \fBprintmsg moon "Hello, moon."\fP +.DE +The message will get printed to \*Qmoon\*Us console. You can print a +message on anybody's console (including your own) with this program if +you are able to copy the server to their machine and run it. +.NH 1 +\&Generating XDR Routines +.IX RPC "generating XDR routines" +.LP +The previous example only demonstrated the automatic generation of +client and server RPC code. +.I rpcgen +may also be used to generate XDR routines, that is, the routines +necessary to convert local data +structures into network format and vice-versa. This example presents +a complete RPC service\(ema remote directory listing service, which uses +.I rpcgen +not only to generate stub routines, but also to generate the XDR +routines. Here is the protocol description file: +.ie t .DS +.el .DS L +.ft I +/* + * dir.x: Remote directory listing protocol + */ +.ft CW +const MAXNAMELEN = 255; /* \fImaximum length of a directory entry\fP */ + +typedef string nametype<MAXNAMELEN>; /* \fIa directory entry\fP */ + +typedef struct namenode *namelist; /* \fIa link in the listing\fP */ + +.ft I +/* + * A node in the directory listing + */ +.ft CW +struct namenode { + nametype name; /* \fIname of directory entry\fP */ + namelist next; /* \fInext entry\fP */ +}; + +.ft I +/* + * The result of a READDIR operation. + */ +.ft CW +union readdir_res switch (int errno) { +case 0: + namelist list; /* \fIno error: return directory listing\fP */ +default: + void; /* \fIerror occurred: nothing else to return\fP */ +}; + +.ft I +/* + * The directory program definition + */ +.ft CW +program DIRPROG { + version DIRVERS { + readdir_res + READDIR(nametype) = 1; + } = 1; +} = 76; +.DE +.SH +Note: +.I +Types (like +.I readdir_res +in the example above) can be defined using +the \*Qstruct\*U, \*Qunion\*U and \*Qenum\*U keywords, but those keywords +should not be used in subsequent declarations of variables of those types. +For example, if you define a union \*Qfoo\*U, you should declare using +only \*Qfoo\*U and not \*Qunion foo\*U. In fact, +.I rpcgen +compiles +RPC unions into C structures and it is an error to declare them using the +\*Qunion\*U keyword. +.LP +Running +.I rpcgen +on +.I dir.x +creates four output files. Three are the same as before: header file, +client stub routines and server skeleton. The fourth are the XDR routines +necessary for converting the data types we declared into XDR format and +vice-versa. These are output in the file +.I dir_xdr.c . +.LP +Here is the implementation of the +.I READDIR +procedure. +.ie t .DS +.el .DS L +.vs 11 +.ft I +/* + * dir_proc.c: remote readdir implementation + */ +.ft CW +#include <rpc/rpc.h> +#include <sys/dir.h> +#include "dir.h" + +extern int errno; +extern char *malloc(); +extern char *strdup(); + +readdir_res * +readdir_1(dirname) + nametype *dirname; +{ + DIR *dirp; + struct direct *d; + namelist nl; + namelist *nlp; + static readdir_res res; /* \fImust be static\fP! */ + +.ft I + /* + * Open directory + */ +.ft CW + dirp = opendir(*dirname); + if (dirp == NULL) { + res.errno = errno; + return (&res); + } + +.ft I + /* + * Free previous result + */ +.ft CW + xdr_free(xdr_readdir_res, &res); + +.ft I + /* + * Collect directory entries. + * Memory allocated here will be freed by \fIxdr_free\fP + * next time \fIreaddir_1\fP is called + */ +.ft CW + nlp = &res.readdir_res_u.list; + while (d = readdir(dirp)) { + nl = *nlp = (namenode *) malloc(sizeof(namenode)); + nl->name = strdup(d->d_name); + nlp = &nl->next; + } + *nlp = NULL; + +.ft I + /* + * Return the result + */ +.ft CW + res.errno = 0; + closedir(dirp); + return (&res); +} +.vs +.DE +Finally, there is the client side program to call the server: +.ie t .DS +.el .DS L +.ft I +/* + * rls.c: Remote directory listing client + */ +.ft CW +#include <stdio.h> +#include <rpc/rpc.h> /* \fIalways need this\fP */ +#include "dir.h" /* \fIwill be generated by rpcgen\fP */ + +extern int errno; + +main(argc, argv) + int argc; + char *argv[]; +{ + CLIENT *cl; + char *server; + char *dir; + readdir_res *result; + namelist nl; + + + if (argc != 3) { + fprintf(stderr, "usage: %s host directory\en", + argv[0]); + exit(1); + } + +.ft I + /* + * Remember what our command line arguments refer to + */ +.ft CW + server = argv[1]; + dir = argv[2]; + +.ft I + /* + * Create client "handle" used for calling \fIMESSAGEPROG\fP on the + * server designated on the command line. We tell the RPC package + * to use the "tcp" protocol when contacting the server. + */ +.ft CW + cl = clnt_create(server, DIRPROG, DIRVERS, "tcp"); + if (cl == NULL) { +.ft I + /* + * Couldn't establish connection with server. + * Print error message and die. + */ +.ft CW + clnt_pcreateerror(server); + exit(1); + } + +.ft I + /* + * Call the remote procedure \fIreaddir\fP on the server + */ +.ft CW + result = readdir_1(&dir, cl); + if (result == NULL) { +.ft I + /* + * An error occurred while calling the server. + * Print error message and die. + */ +.ft CW + clnt_perror(cl, server); + exit(1); + } + +.ft I + /* + * Okay, we successfully called the remote procedure. + */ +.ft CW + if (result->errno != 0) { +.ft I + /* + * A remote system error occurred. + * Print error message and die. + */ +.ft CW + errno = result->errno; + perror(dir); + exit(1); + } + +.ft I + /* + * Successfully got a directory listing. + * Print it out. + */ +.ft CW + for (nl = result->readdir_res_u.list; nl != NULL; + nl = nl->next) { + printf("%s\en", nl->name); + } + exit(0); +} +.DE +Compile everything, and run. +.DS +.ft CW +sun% \fBrpcgen dir.x\fP +sun% \fBcc rls.c dir_clnt.c dir_xdr.c -o rls\fP +sun% \fBcc dir_svc.c dir_proc.c dir_xdr.c -o dir_svc\fP + +sun% \fBdir_svc &\fP + +moon% \fBrls sun /usr/pub\fP +\&. +\&.. +ascii +eqnchar +greek +kbd +marg8 +tabclr +tabs +tabs4 +moon% +.DE +.LP +.IX "debugging with rpcgen" "" "debugging with \fIrpcgen\fP" +A final note about +.I rpcgen : +The client program and the server procedure can be tested together +as a single program by simply linking them with each other rather +than with the client and server stubs. The procedure calls will be +executed as ordinary local procedure calls and the program can be +debugged with a local debugger such as +.I dbx . +When the program is working, the client program can be linked to +the client stub produced by +.I rpcgen +and the server procedures can be linked to the server stub produced +by +.I rpcgen . +.SH +.I NOTE : +\fIIf you do this, you may want to comment out calls to RPC library +routines, and have client-side routines call server routines +directly.\fP +.LP +.NH 1 +\&The C-Preprocessor +.IX rpcgen "C-preprocessor" \fIrpcgen\fP +.LP +The C-preprocessor is run on all input files before they are +compiled, so all the preprocessor directives are legal within a \*Q.x\*U +file. Four symbols may be defined, depending upon which output file is +getting generated. The symbols are: +.TS +box tab (&); +lfI lfI +lfL l . +Symbol&Usage +_ +RPC_HDR&for header-file output +RPC_XDR&for XDR routine output +RPC_SVC&for server-skeleton output +RPC_CLNT&for client stub output +.TE +.LP +Also, +.I rpcgen +does a little preprocessing of its own. Any line that +begins with a percent sign is passed directly into the output file, +without any interpretation of the line. Here is a simple example that +demonstrates the preprocessing features. +.ie t .DS +.el .DS L +.ft I +/* + * time.x: Remote time protocol + */ +.ft CW +program TIMEPROG { + version TIMEVERS { + unsigned int TIMEGET(void) = 1; + } = 1; +} = 44; + +#ifdef RPC_SVC +%int * +%timeget_1() +%{ +% static int thetime; +% +% thetime = time(0); +% return (&thetime); +%} +#endif +.DE +The '%' feature is not generally recommended, as there is no guarantee +that the compiler will stick the output where you intended. +.NH 1 +\&\fBrpcgen\fP Programming Notes +.IX rpcgen "other operations" \fIrpcgen\fP +.sp +.NH 2 +\&Timeout Changes +.IX rpcgen "timeout changes" \fIrpcgen\fP +.LP +RPC sets a default timeout of 25 seconds for RPC calls when +.I clnt_create() +is used. This timeout may be changed using +.I clnt_control() +Here is a small code fragment to demonstrate use of +.I clnt_control (): +.ID +struct timeval tv; +CLIENT *cl; +.sp .5 +cl = clnt_create("somehost", SOMEPROG, SOMEVERS, "tcp"); +if (cl == NULL) { + exit(1); +} +tv.tv_sec = 60; /* \fIchange timeout to 1 minute\fP */ +tv.tv_usec = 0; +clnt_control(cl, CLSET_TIMEOUT, &tv); +.DE +.NH 2 +\&Handling Broadcast on the Server Side +.IX "broadcast RPC" +.IX rpcgen "broadcast RPC" \fIrpcgen\fP +.LP +When a procedure is known to be called via broadcast RPC, +it is usually wise for the server to not reply unless it can provide +some useful information to the client. This prevents the network +from getting flooded by useless replies. +.LP +To prevent the server from replying, a remote procedure can +return NULL as its result, and the server code generated by +.I rpcgen +will detect this and not send out a reply. +.LP +Here is an example of a procedure that replies only if it +thinks it is an NFS server: +.ID +void * +reply_if_nfsserver() +{ + char notnull; /* \fIjust here so we can use its address\fP */ +.sp .5 + if (access("/etc/exports", F_OK) < 0) { + return (NULL); /* \fIprevent RPC from replying\fP */ + } +.ft I + /* + * return non-null pointer so RPC will send out a reply + */ +.ft L + return ((void *)¬null); +} +.DE +Note that if procedure returns type \*Qvoid *\*U, they must return a non-NULL +pointer if they want RPC to reply for them. +.NH 2 +\&Other Information Passed to Server Procedures +.LP +Server procedures will often want to know more about an RPC call +than just its arguments. For example, getting authentication information +is important to procedures that want to implement some level of security. +This extra information is actually supplied to the server procedure as a +second argument. Here is an example to demonstrate its use. What we've +done here is rewrite the previous +.I printmessage_1() +procedure to only allow root users to print a message to the console. +.ID +int * +printmessage_1(msg, rq) + char **msg; + struct svc_req *rq; +{ + static in result; /* \fIMust be static\fP */ + FILE *f; + struct suthunix_parms *aup; +.sp .5 + aup = (struct authunix_parms *)rq->rq_clntcred; + if (aup->aup_uid != 0) { + result = 0; + return (&result); + } +.sp +.ft I + /* + * Same code as before. + */ +.ft L +} +.DE +.NH 1 +\&RPC Language +.IX RPCL +.IX rpcgen "RPC Language" \fIrpcgen\fP +.LP +RPC language is an extension of XDR language. The sole extension is +the addition of the +.I program +type. For a complete description of the XDR language syntax, see the +.I "External Data Representation Standard: Protocol Specification" +chapter. For a description of the RPC extensions to the XDR language, +see the +.I "Remote Procedure Calls: Protocol Specification" +chapter. +.LP +However, XDR language is so close to C that if you know C, you know most +of it already. We describe here the syntax of the RPC language, +showing a few examples along the way. We also show how the various +RPC and XDR type definitions get compiled into C type definitions in +the output header file. +.KS +.NH 2 +Definitions +\& +.IX rpcgen definitions \fIrpcgen\fP +.LP +An RPC language file consists of a series of definitions. +.DS L +.ft CW + definition-list: + definition ";" + definition ";" definition-list +.DE +.KE +It recognizes five types of definitions. +.DS L +.ft CW + definition: + enum-definition + struct-definition + union-definition + typedef-definition + const-definition + program-definition +.DE +.NH 2 +Structures +\& +.IX rpcgen structures \fIrpcgen\fP +.LP +An XDR struct is declared almost exactly like its C counterpart. It +looks like the following: +.DS L +.ft CW + struct-definition: + "struct" struct-ident "{" + declaration-list + "}" + + declaration-list: + declaration ";" + declaration ";" declaration-list +.DE +As an example, here is an XDR structure to a two-dimensional +coordinate, and the C structure that it gets compiled into in the +output header file. +.DS +.ft CW + struct coord { struct coord { + int x; --> int x; + int y; int y; + }; }; + typedef struct coord coord; +.DE +The output is identical to the input, except for the added +.I typedef +at the end of the output. This allows one to use \*Qcoord\*U instead of +\*Qstruct coord\*U when declaring items. +.NH 2 +Unions +\& +.IX rpcgen unions \fIrpcgen\fP +.LP +XDR unions are discriminated unions, and look quite different from C +unions. They are more analogous to Pascal variant records than they +are to C unions. +.DS L +.ft CW + union-definition: + "union" union-ident "switch" "(" declaration ")" "{" + case-list + "}" + + case-list: + "case" value ":" declaration ";" + "default" ":" declaration ";" + "case" value ":" declaration ";" case-list +.DE +Here is an example of a type that might be returned as the result of a +\*Qread data\*U operation. If there is no error, return a block of data. +Otherwise, don't return anything. +.DS L +.ft CW + union read_result switch (int errno) { + case 0: + opaque data[1024]; + default: + void; + }; +.DE +It gets compiled into the following: +.DS L +.ft CW + struct read_result { + int errno; + union { + char data[1024]; + } read_result_u; + }; + typedef struct read_result read_result; +.DE +Notice that the union component of the output struct has the name as +the type name, except for the trailing \*Q_u\*U. +.NH 2 +Enumerations +\& +.IX rpcgen enumerations \fIrpcgen\fP +.LP +XDR enumerations have the same syntax as C enumerations. +.DS L +.ft CW + enum-definition: + "enum" enum-ident "{" + enum-value-list + "}" + + enum-value-list: + enum-value + enum-value "," enum-value-list + + enum-value: + enum-value-ident + enum-value-ident "=" value +.DE +Here is a short example of an XDR enum, and the C enum that it gets +compiled into. +.DS L +.ft CW + enum colortype { enum colortype { + RED = 0, RED = 0, + GREEN = 1, --> GREEN = 1, + BLUE = 2 BLUE = 2, + }; }; + typedef enum colortype colortype; +.DE +.NH 2 +Typedef +\& +.IX rpcgen typedef \fIrpcgen\fP +.LP +XDR typedefs have the same syntax as C typedefs. +.DS L +.ft CW + typedef-definition: + "typedef" declaration +.DE +Here is an example that defines a +.I fname_type +used for declaring +file name strings that have a maximum length of 255 characters. +.DS L +.ft CW +typedef string fname_type<255>; --> typedef char *fname_type; +.DE +.NH 2 +Constants +\& +.IX rpcgen constants \fIrpcgen\fP +.LP +XDR constants symbolic constants that may be used wherever a +integer constant is used, for example, in array size specifications. +.DS L +.ft CW + const-definition: + "const" const-ident "=" integer +.DE +For example, the following defines a constant +.I DOZEN +equal to 12. +.DS L +.ft CW + const DOZEN = 12; --> #define DOZEN 12 +.DE +.NH 2 +Programs +\& +.IX rpcgen programs \fIrpcgen\fP +.LP +RPC programs are declared using the following syntax: +.DS L +.ft CW + program-definition: + "program" program-ident "{" + version-list + "}" "=" value + + version-list: + version ";" + version ";" version-list + + version: + "version" version-ident "{" + procedure-list + "}" "=" value + + procedure-list: + procedure ";" + procedure ";" procedure-list + + procedure: + type-ident procedure-ident "(" type-ident ")" "=" value +.DE +For example, here is the time protocol, revisited: +.ie t .DS +.el .DS L +.ft I +/* + * time.x: Get or set the time. Time is represented as number of seconds + * since 0:00, January 1, 1970. + */ +.ft CW +program TIMEPROG { + version TIMEVERS { + unsigned int TIMEGET(void) = 1; + void TIMESET(unsigned) = 2; + } = 1; +} = 44; +.DE +This file compiles into #defines in the output header file: +.ie t .DS +.el .DS L +.ft CW +#define TIMEPROG 44 +#define TIMEVERS 1 +#define TIMEGET 1 +#define TIMESET 2 +.DE +.NH 2 +Declarations +\& +.IX rpcgen declarations \fIrpcgen\fP +.LP +In XDR, there are only four kinds of declarations. +.DS L +.ft CW + declaration: + simple-declaration + fixed-array-declaration + variable-array-declaration + pointer-declaration +.DE +\fB1) Simple declarations\fP are just like simple C declarations. +.DS L +.ft CW + simple-declaration: + type-ident variable-ident +.DE +Example: +.DS L +.ft CW + colortype color; --> colortype color; +.DE +\fB2) Fixed-length Array Declarations\fP are just like C array declarations: +.DS L +.ft CW + fixed-array-declaration: + type-ident variable-ident "[" value "]" +.DE +Example: +.DS L +.ft CW + colortype palette[8]; --> colortype palette[8]; +.DE +\fB3) Variable-Length Array Declarations\fP have no explicit syntax +in C, so XDR invents its own using angle-brackets. +.DS L +.ft CW +variable-array-declaration: + type-ident variable-ident "<" value ">" + type-ident variable-ident "<" ">" +.DE +The maximum size is specified between the angle brackets. The size may +be omitted, indicating that the array may be of any size. +.DS L +.ft CW + int heights<12>; /* \fIat most 12 items\fP */ + int widths<>; /* \fIany number of items\fP */ +.DE +Since variable-length arrays have no explicit syntax in C, these +declarations are actually compiled into \*Qstruct\*Us. For example, the +\*Qheights\*U declaration gets compiled into the following struct: +.DS L +.ft CW + struct { + u_int heights_len; /* \fI# of items in array\fP */ + int *heights_val; /* \fIpointer to array\fP */ + } heights; +.DE +Note that the number of items in the array is stored in the \*Q_len\*U +component and the pointer to the array is stored in the \*Q_val\*U +component. The first part of each of these component's names is the +same as the name of the declared XDR variable. +.LP +\fB4) Pointer Declarations\fP are made in +XDR exactly as they are in C. You can't +really send pointers over the network, but you can use XDR pointers +for sending recursive data types such as lists and trees. The type is +actually called \*Qoptional-data\*U, not \*Qpointer\*U, in XDR language. +.DS L +.ft CW + pointer-declaration: + type-ident "*" variable-ident +.DE +Example: +.DS L +.ft CW + listitem *next; --> listitem *next; +.DE +.NH 2 +\&Special Cases +.IX rpcgen "special cases" \fIrpcgen\fP +.LP +There are a few exceptions to the rules described above. +.LP +.B Booleans: +C has no built-in boolean type. However, the RPC library does a +boolean type called +.I bool_t +that is either +.I TRUE +or +.I FALSE . +Things declared as type +.I bool +in XDR language are compiled into +.I bool_t +in the output header file. +.LP +Example: +.DS L +.ft CW + bool married; --> bool_t married; +.DE +.B Strings: +C has no built-in string type, but instead uses the null-terminated +\*Qchar *\*U convention. In XDR language, strings are declared using the +\*Qstring\*U keyword, and compiled into \*Qchar *\*Us in the output header +file. The maximum size contained in the angle brackets specifies the +maximum number of characters allowed in the strings (not counting the +.I NULL +character). The maximum size may be left off, indicating a string +of arbitrary length. +.LP +Examples: +.DS L +.ft CW + string name<32>; --> char *name; + string longname<>; --> char *longname; +.DE +.B "Opaque Data:" +Opaque data is used in RPC and XDR to describe untyped data, that is, +just sequences of arbitrary bytes. It may be declared either as a +fixed or variable length array. +.DS L +Examples: +.ft CW + opaque diskblock[512]; --> char diskblock[512]; + + opaque filedata<1024>; --> struct { + u_int filedata_len; + char *filedata_val; + } filedata; +.DE +.B Voids: +In a void declaration, the variable is not named. The declaration is +just \*Qvoid\*U and nothing else. Void declarations can only occur in two +places: union definitions and program definitions (as the argument or +result of a remote procedure). diff --git a/share/doc/psd/22.rpcgen/stubs b/share/doc/psd/22.rpcgen/stubs new file mode 100644 index 000000000000..78b0a2c815d3 --- /dev/null +++ b/share/doc/psd/22.rpcgen/stubs @@ -0,0 +1,3 @@ +.\" $FreeBSD$ +.\" +.if t .ftr L CR diff --git a/share/doc/psd/23.rpc/Makefile b/share/doc/psd/23.rpc/Makefile new file mode 100644 index 000000000000..930c6773e188 --- /dev/null +++ b/share/doc/psd/23.rpc/Makefile @@ -0,0 +1,9 @@ +# $FreeBSD$ + +VOLUME= psd/23.rpc +SRCS= stubs rpc.prog.ms +MACROS= -ms +USE_TBL= +USE_PIC= + +.include <bsd.doc.mk> diff --git a/share/doc/psd/23.rpc/rpc.prog.ms b/share/doc/psd/23.rpc/rpc.prog.ms new file mode 100644 index 000000000000..0bc8743de659 --- /dev/null +++ b/share/doc/psd/23.rpc/rpc.prog.ms @@ -0,0 +1,2686 @@ +.\" +.\" Must use -- tbl and pic -- with this one +.\" +.\" @(#)rpc.prog.ms 2.3 88/08/11 4.0 RPCSRC +.\" $FreeBSD$ +.\" +.de BT +.if \\n%=1 .tl ''- % -'' +.. +.nr OF 0 +.ND +.\" prevent excess underlining in nroff +.if n .fp 2 R +.OH 'Remote Procedure Call Programming Guide''Page %' +.EH 'Page %''Remote Procedure Call Programming Guide' +.SH +\&Remote Procedure Call Programming Guide +.nr OF 1 +.IX "Network Programming" "" "" "" PAGE MAJOR +.IX "RPC Programming Guide" +.LP +This document assumes a working knowledge of network theory. It is +intended for programmers who wish to write network applications using +remote procedure calls (explained below), and who want to understand +the RPC mechanisms usually hidden by the +.I rpcgen(1) +protocol compiler. +.I rpcgen +is described in detail in the previous chapter, the +.I "\fBrpcgen\fP \fIProgramming Guide\fP". +.SH +Note: +.I +.IX rpcgen "" \fIrpcgen\fP +Before attempting to write a network application, or to convert an +existing non-network application to run over the network, you may want to +understand the material in this chapter. However, for most applications, +you can circumvent the need to cope with the details presented here by using +.I rpcgen . +The +.I "Generating XDR Routines" +section of that chapter contains the complete source for a working RPC +service\(ema remote directory listing service which uses +.I rpcgen +to generate XDR routines as well as client and server stubs. +.LP +.LP +What are remote procedure calls? Simply put, they are the high-level +communications paradigm used in the operating system. +RPC presumes the existence of +low-level networking mechanisms (such as TCP/IP and UDP/IP), and upon them +it implements a logical client to server communications system designed +specifically for the support of network applications. With RPC, the client +makes a procedure call to send a data packet to the server. When the +packet arrives, the server calls a dispatch routine, performs whatever +service is requested, sends back the reply, and the procedure call returns +to the client. +.NH 0 +\&Layers of RPC +.IX "layers of RPC" +.IX "RPC" "layers" +.LP +The RPC interface can be seen as being divided into three layers.\** +.FS +For a complete specification of the routines in the remote procedure +call Library, see the +.I rpc(3N) +manual page. +.FE +.LP +.I "The Highest Layer:" +.IX RPC "The Highest Layer" +The highest layer is totally transparent to the operating system, +machine and network upon which it is run. It's probably best to +think of this level as a way of +.I using +RPC, rather than as +a \fIpart of\fP RPC proper. Programmers who write RPC routines +should (almost) always make this layer available to others by way +of a simple C front end that entirely hides the networking. +.LP +To illustrate, at this level a program can simply make a call to +.I rnusers (), +a C routine which returns the number of users on a remote machine. +The user is not explicitly aware of using RPC \(em they simply +call a procedure, just as they would call +.I malloc() . +.LP +.I "The Middle Layer:" +.IX RPC "The Middle Layer" +The middle layer is really \*QRPC proper.\*U Here, the user doesn't +need to consider details about sockets, the UNIX system, or other low-level +implementation mechanisms. They simply make remote procedure calls +to routines on other machines. The selling point here is simplicity. +It's this layer that allows RPC to pass the \*Qhello world\*U test \(em +simple things should be simple. The middle-layer routines are used +for most applications. +.LP +RPC calls are made with the system routines +.I registerrpc() +.I callrpc() +and +.I svc_run (). +The first two of these are the most fundamental: +.I registerrpc() +obtains a unique system-wide procedure-identification number, and +.I callrpc() +actually executes a remote procedure call. At the middle level, a +call to +.I rnusers() +is implemented by way of these two routines. +.LP +The middle layer is unfortunately rarely used in serious programming +due to its inflexibility (simplicity). It does not allow timeout +specifications or the choice of transport. It allows no UNIX +process control or flexibility in case of errors. It doesn't support +multiple kinds of call authentication. The programmer rarely needs +all these kinds of control, but one or two of them is often necessary. +.LP +.I "The Lowest Layer:" +.IX RPC "The Lowest Layer" +The lowest layer does allow these details to be controlled by the +programmer, and for that reason it is often necessary. Programs +written at this level are also most efficient, but this is rarely a +real issue \(em since RPC clients and servers rarely generate +heavy network loads. +.LP +Although this document only discusses the interface to C, +remote procedure calls can be made from any language. +Even though this document discusses RPC +when it is used to communicate +between processes on different machines, +it works just as well for communication +between different processes on the same machine. +.br +.KS +.NH 2 +\&The RPC Paradigm +.IX RPC paradigm +.LP +Here is a diagram of the RPC paradigm: +.LP +\fBFigure 1-1\fI Network Communication with the Remote Reocedure Call\fR +.LP +.PS +L1: arrow down 1i "client " rjust "program " rjust +L2: line right 1.5i "\fIcallrpc\fP" "function" +move up 1.5i; line dotted down 6i; move up 4.5i +arrow right 1i +L3: arrow down 1i "invoke " rjust "service " rjust +L4: arrow right 1.5i "call" "service" +L5: arrow down 1i " service" ljust " executes" ljust +L6: arrow left 1.5i "\fIreturn\fP" "answer" +L7: arrow down 1i "request " rjust "completed " rjust +L8: line left 1i +arrow left 1.5i "\fIreturn\fP" "reply" +L9: arrow down 1i "program " rjust "continues " rjust +line dashed down from L2 to L9 +line dashed down from L4 to L7 +line dashed up 1i from L3 "service " rjust "daemon " rjust +arrow dashed down 1i from L8 +move right 1i from L3 +box invis "Machine B" +move left 1.2i from L2; move down +box invis "Machine A" +.PE +.KE +.KS +.NH 1 +\&Higher Layers of RPC +.NH 2 +\&Highest Layer +.IX "highest layer of RPC" +.IX RPC "highest layer" +.LP +Imagine you're writing a program that needs to know +how many users are logged into a remote machine. +You can do this by calling the RPC library routine +.I rnusers() +as illustrated below: +.ie t .DS +.el .DS L +.ft CW +#include <stdio.h> + +main(argc, argv) + int argc; + char **argv; +{ + int num; + + if (argc != 2) { + fprintf(stderr, "usage: rnusers hostname\en"); + exit(1); + } + if ((num = rnusers(argv[1])) < 0) { + fprintf(stderr, "error: rnusers\en"); + exit(-1); + } + printf("%d users on %s\en", num, argv[1]); + exit(0); +} +.DE +.KE +RPC library routines such as +.I rnusers() +are in the RPC services library +.I librpcsvc.a +Thus, the program above should be compiled with +.DS +.ft CW +% cc \fIprogram.c -lrpcsvc\fP +.DE +.I rnusers (), +like the other RPC library routines, is documented in section 3R +of the +.I "System Interface Manual for the Sun Workstation" , +the same section which documents the standard Sun RPC services. +.IX "RPC Services" +See the +.I intro(3R) +manual page for an explanation of the documentation strategy +for these services and their RPC protocols. +.LP +Here are some of the RPC service library routines available to the +C programmer: +.LP +\fBTable 3-3\fI RPC Service Library Routines\fR +.TS +box tab (&) ; +cfI cfI +lfL l . +Routine&Description +_ +.sp .5 +rnusers&Return number of users on remote machine +rusers&Return information about users on remote machine +havedisk&Determine if remote machine has disk +rstats&Get performance data from remote kernel +rwall&Write to specified remote machines +yppasswd&Update user password in Yellow Pages +.TE +.LP +Other RPC services \(em for example +.I ether() +.I mount +.I rquota() +and +.I spray +\(em are not available to the C programmer as library routines. +They do, however, +have RPC program numbers so they can be invoked with +.I callrpc() +which will be discussed in the next section. Most of them also +have compilable +.I rpcgen(1) +protocol description files. (The +.I rpcgen +protocol compiler radically simplifies the process of developing +network applications. +See the \fBrpcgen\fI Programming Guide\fR +for detailed information about +.I rpcgen +and +.I rpcgen +protocol description files). +.KS +.NH 2 +\&Intermediate Layer +.IX "intermediate layer of RPC" +.IX "RPC" "intermediate layer" +.LP +The simplest interface, which explicitly makes RPC calls, uses the +functions +.I callrpc() +and +.I registerrpc() +Using this method, the number of remote users can be gotten as follows: +.ie t .DS +.el .DS L +#include <stdio.h> +#include <rpc/rpc.h> +#include <utmp.h> +#include <rpcsvc/rusers.h> + +main(argc, argv) + int argc; + char **argv; +{ + unsigned long nusers; + int stat; + + if (argc != 2) { + fprintf(stderr, "usage: nusers hostname\en"); + exit(-1); + } + if (stat = callrpc(argv[1], + RUSERSPROG, RUSERSVERS, RUSERSPROC_NUM, + xdr_void, 0, xdr_u_long, &nusers) != 0) { + clnt_perrno(stat); + exit(1); + } + printf("%d users on %s\en", nusers, argv[1]); + exit(0); +} +.DE +.KE +Each RPC procedure is uniquely defined by a program number, +version number, and procedure number. The program number +specifies a group of related remote procedures, each of +which has a different procedure number. Each program also +has a version number, so when a minor change is made to a +remote service (adding a new procedure, for example), a new +program number doesn't have to be assigned. When you want +to call a procedure to find the number of remote users, you +look up the appropriate program, version and procedure numbers +in a manual, just as you look up the name of a memory allocator +when you want to allocate memory. +.LP +The simplest way of making remote procedure calls is with the RPC +library routine +.I callrpc() +It has eight parameters. The first is the name of the remote server +machine. The next three parameters are the program, version, and procedure +numbers\(emtogether they identify the procedure to be called. +The fifth and sixth parameters are an XDR filter and an argument to +be encoded and passed to the remote procedure. +The final two parameters are a filter for decoding the results +returned by the remote procedure and a pointer to the place where +the procedure's results are to be stored. Multiple arguments and +results are handled by embedding them in structures. If +.I callrpc() +completes successfully, it returns zero; else it returns a nonzero +value. The return codes (of type +.IX "enum clnt_stat (in RPC programming)" "" "\fIenum clnt_stat\fP (in RPC programming)" +cast into an integer) are found in +.I <rpc/clnt.h> . +.LP +Since data types may be represented differently on different machines, +.I callrpc() +needs both the type of the RPC argument, as well as +a pointer to the argument itself (and similarly for the result). For +.I RUSERSPROC_NUM , +the return value is an +.I "unsigned long" +so +.I callrpc() +has +.I xdr_u_long() +as its first return parameter, which says +that the result is of type +.I "unsigned long" +and +.I &nusers +as its second return parameter, +which is a pointer to where the long result will be placed. Since +.I RUSERSPROC_NUM +takes no argument, the argument parameter of +.I callrpc() +is +.I xdr_void (). +.LP +After trying several times to deliver a message, if +.I callrpc() +gets no answer, it returns with an error code. +The delivery mechanism is UDP, +which stands for User Datagram Protocol. +Methods for adjusting the number of retries +or for using a different protocol require you to use the lower +layer of the RPC library, discussed later in this document. +The remote server procedure +corresponding to the above might look like this: +.ie t .DS +.el .DS L +.ft CW +.ft CW +char * +nuser(indata) + char *indata; +{ + unsigned long nusers; + +.ft I + /* + * Code here to compute the number of users + * and place result in variable \fInusers\fP. + */ +.ft CW + return((char *)&nusers); +} +.DE +.LP +It takes one argument, which is a pointer to the input +of the remote procedure call (ignored in our example), +and it returns a pointer to the result. +In the current version of C, +character pointers are the generic pointers, +so both the input argument and the return value are cast to +.I "char *" . +.LP +Normally, a server registers all of the RPC calls it plans +to handle, and then goes into an infinite loop waiting to service requests. +In this example, there is only a single procedure +to register, so the main body of the server would look like this: +.ie t .DS +.el .DS L +.ft CW +#include <stdio.h> +#include <rpc/rpc.h> +#include <utmp.h> +#include <rpcsvc/rusers.h> + +char *nuser(); + +main() +{ + registerrpc(RUSERSPROG, RUSERSVERS, RUSERSPROC_NUM, + nuser, xdr_void, xdr_u_long); + svc_run(); /* \fINever returns\fP */ + fprintf(stderr, "Error: svc_run returned!\en"); + exit(1); +} +.DE +.LP +The +.I registerrpc() +routine registers a C procedure as corresponding to a +given RPC procedure number. The first three parameters, +.I RUSERPROG , +.I RUSERSVERS , +and +.I RUSERSPROC_NUM +are the program, version, and procedure numbers +of the remote procedure to be registered; +.I nuser() +is the name of the local procedure that implements the remote +procedure; and +.I xdr_void() +and +.I xdr_u_long() +are the XDR filters for the remote procedure's arguments and +results, respectively. (Multiple arguments or multiple results +are passed as structures). +.LP +Only the UDP transport mechanism can use +.I registerrpc() +thus, it is always safe in conjunction with calls generated by +.I callrpc() . +.SH +.IX "UDP 8K warning" +Warning: the UDP transport mechanism can only deal with +arguments and results less than 8K bytes in length. +.LP +.LP +After registering the local procedure, the server program's +main procedure calls +.I svc_run (), +the RPC library's remote procedure dispatcher. It is this +function that calls the remote procedures in response to RPC +call messages. Note that the dispatcher takes care of decoding +remote procedure arguments and encoding results, using the XDR +filters specified when the remote procedure was registered. +.NH 2 +\&Assigning Program Numbers +.IX "program number assignment" +.IX "assigning program numbers" +.LP +Program numbers are assigned in groups of +.I 0x20000000 +according to the following chart: +.DS +.ft CW + 0x0 - 0x1fffffff \fRDefined by Sun\fP +0x20000000 - 0x3fffffff \fRDefined by user\fP +0x40000000 - 0x5fffffff \fRTransient\fP +0x60000000 - 0x7fffffff \fRReserved\fP +0x80000000 - 0x9fffffff \fRReserved\fP +0xa0000000 - 0xbfffffff \fRReserved\fP +0xc0000000 - 0xdfffffff \fRReserved\fP +0xe0000000 - 0xffffffff \fRReserved\fP +.ft R +.DE +Sun Microsystems administers the first group of numbers, which +should be identical for all Sun customers. If a customer +develops an application that might be of general interest, that +application should be given an assigned number in the first +range. The second group of numbers is reserved for specific +customer applications. This range is intended primarily for +debugging new programs. The third group is reserved for +applications that generate program numbers dynamically. The +final groups are reserved for future use, and should not be +used. +.LP +To register a protocol specification, send a request by network +mail to +.I rpc@sun +or write to: +.DS +RPC Administrator +Sun Microsystems +2550 Garcia Ave. +Mountain View, CA 94043 +.DE +Please include a compilable +.I rpcgen +\*Q.x\*U file describing your protocol. +You will be given a unique program number in return. +.IX RPC administration +.IX administration "of RPC" +.LP +The RPC program numbers and protocol specifications +of standard Sun RPC services can be +found in the include files in +.I "/usr/include/rpcsvc" . +These services, however, constitute only a small subset +of those which have been registered. The complete list of +registered programs, as of the time when this manual was +printed, is: +.LP +\fBTable 3-2\fI RPC Registered Programs\fR +.TS H +box tab (&) ; +lfBI lfBI lfBI +lfL lfL lfI . +RPC Number&Program&Description +_ +.TH +.sp .5 +100000&PMAPPROG&portmapper +100001&RSTATPROG&remote stats +100002&RUSERSPROG&remote users +100003&NFSPROG&nfs +100004&YPPROG&Yellow Pages +100005&MOUNTPROG&mount daemon +100006&DBXPROG&remote dbx +100007&YPBINDPROG&yp binder +100008&WALLPROG&shutdown msg +100009&YPPASSWDPROG&yppasswd server +100010ÐERSTATPROGðer stats +100011&RQUOTAPROG&disk quotas +100012&SPRAYPROG&spray packets +100013&IBM3270PROG&3270 mapper +100014&IBMRJEPROG&RJE mapper +100015&SELNSVCPROG&selection service +100016&RDATABASEPROG&remote database access +100017&REXECPROG&remote execution +100018&ALICEPROG&Alice Office Automation +100019&SCHEDPROG&scheduling service +100020&LOCKPROG&local lock manager +100021&NETLOCKPROG&network lock manager +100022&X25PROG&x.25 inr protocol +100023&STATMON1PROG&status monitor 1 +100024&STATMON2PROG&status monitor 2 +100025&SELNLIBPROG&selection library +100026&BOOTPARAMPROG&boot parameters service +100027&MAZEPROG&mazewars game +100028&YPUPDATEPROG&yp update +100029&KEYSERVEPROG&key server +100030&SECURECMDPROG&secure login +100031&NETFWDIPROG&nfs net forwarder init +100032&NETFWDTPROG&nfs net forwarder trans +100033&SUNLINKMAP_PROG&sunlink MAP +100034&NETMONPROG&network monitor +100035&DBASEPROG&lightweight database +100036&PWDAUTHPROG&password authorization +100037&TFSPROG&translucent file svc +100038&NSEPROG&nse server +100039&NSE_ACTIVATE_PROG&nse activate daemon +.sp .2i +150001&PCNFSDPROG&pc passwd authorization +.sp .2i +200000&PYRAMIDLOCKINGPROG&Pyramid-locking +200001&PYRAMIDSYS5&Pyramid-sys5 +200002&CADDS_IMAGE&CV cadds_image +.sp .2i +300001&ADT_RFLOCKPROG&ADT file locking +.TE +.NH 2 +\&Passing Arbitrary Data Types +.IX "arbitrary data types" +.LP +In the previous example, the RPC call passes a single +.I "unsigned long" +RPC can handle arbitrary data structures, regardless of +different machines' byte orders or structure layout conventions, +by always converting them to a network standard called +.I "External Data Representation" +(XDR) before +sending them over the wire. +The process of converting from a particular machine representation +to XDR format is called +.I serializing , +and the reverse process is called +.I deserializing . +The type field parameters of +.I callrpc() +and +.I registerrpc() +can be a built-in procedure like +.I xdr_u_long() +in the previous example, or a user supplied one. +XDR has these built-in type routines: +.IX RPC "built-in routines" +.DS +.ft CW +xdr_int() xdr_u_int() xdr_enum() +xdr_long() xdr_u_long() xdr_bool() +xdr_short() xdr_u_short() xdr_wrapstring() +xdr_char() xdr_u_char() +.DE +Note that the routine +.I xdr_string() +exists, but cannot be used with +.I callrpc() +and +.I registerrpc (), +which only pass two parameters to their XDR routines. +.I xdr_wrapstring() +has only two parameters, and is thus OK. It calls +.I xdr_string (). +.LP +As an example of a user-defined type routine, +if you wanted to send the structure +.DS +.ft CW +struct simple { + int a; + short b; +} simple; +.DE +then you would call +.I callrpc() +as +.DS +.ft CW +callrpc(hostname, PROGNUM, VERSNUM, PROCNUM, + xdr_simple, &simple ...); +.DE +where +.I xdr_simple() +is written as: +.ie t .DS +.el .DS L +.ft CW +#include <rpc/rpc.h> + +xdr_simple(xdrsp, simplep) + XDR *xdrsp; + struct simple *simplep; +{ + if (!xdr_int(xdrsp, &simplep->a)) + return (0); + if (!xdr_short(xdrsp, &simplep->b)) + return (0); + return (1); +} +.DE +.LP +An XDR routine returns nonzero (true in the sense of C) if it +completes successfully, and zero otherwise. +A complete description of XDR is in the +.I "XDR Protocol Specification" +section of this manual, only few implementation examples are +given here. +.LP +In addition to the built-in primitives, +there are also the prefabricated building blocks: +.DS +.ft CW +xdr_array() xdr_bytes() xdr_reference() +xdr_vector() xdr_union() xdr_pointer() +xdr_string() xdr_opaque() +.DE +To send a variable array of integers, +you might package them up as a structure like this +.DS +.ft CW +struct varintarr { + int *data; + int arrlnth; +} arr; +.DE +and make an RPC call such as +.DS +.ft CW +callrpc(hostname, PROGNUM, VERSNUM, PROCNUM, + xdr_varintarr, &arr...); +.DE +with +.I xdr_varintarr() +defined as: +.ie t .DS +.el .DS L +.ft CW +xdr_varintarr(xdrsp, arrp) + XDR *xdrsp; + struct varintarr *arrp; +{ + return (xdr_array(xdrsp, &arrp->data, &arrp->arrlnth, + MAXLEN, sizeof(int), xdr_int)); +} +.DE +This routine takes as parameters the XDR handle, +a pointer to the array, a pointer to the size of the array, +the maximum allowable array size, +the size of each array element, +and an XDR routine for handling each array element. +.KS +.LP +If the size of the array is known in advance, one can use +.I xdr_vector (), +which serializes fixed-length arrays. +.ie t .DS +.el .DS L +.ft CW +int intarr[SIZE]; + +xdr_intarr(xdrsp, intarr) + XDR *xdrsp; + int intarr[]; +{ + int i; + + return (xdr_vector(xdrsp, intarr, SIZE, sizeof(int), + xdr_int)); +} +.DE +.KE +.LP +XDR always converts quantities to 4-byte multiples when serializing. +Thus, if either of the examples above involved characters +instead of integers, each character would occupy 32 bits. +That is the reason for the XDR routine +.I xdr_bytes() +which is like +.I xdr_array() +except that it packs characters; +.I xdr_bytes() +has four parameters, similar to the first four parameters of +.I xdr_array (). +For null-terminated strings, there is also the +.I xdr_string() +routine, which is the same as +.I xdr_bytes() +without the length parameter. +On serializing it gets the string length from +.I strlen (), +and on deserializing it creates a null-terminated string. +.LP +Here is a final example that calls the previously written +.I xdr_simple() +as well as the built-in functions +.I xdr_string() +and +.I xdr_reference (), +which chases pointers: +.ie t .DS +.el .DS L +.ft CW +struct finalexample { + char *string; + struct simple *simplep; +} finalexample; + +xdr_finalexample(xdrsp, finalp) + XDR *xdrsp; + struct finalexample *finalp; +{ + + if (!xdr_string(xdrsp, &finalp->string, MAXSTRLEN)) + return (0); + if (!xdr_reference(xdrsp, &finalp->simplep, + sizeof(struct simple), xdr_simple); + return (0); + return (1); +} +.DE +Note that we could as easily call +.I xdr_simple() +here instead of +.I xdr_reference (). +.NH 1 +\&Lowest Layer of RPC +.IX "lowest layer of RPC" +.IX "RPC" "lowest layer" +.LP +In the examples given so far, +RPC takes care of many details automatically for you. +In this section, we'll show you how you can change the defaults +by using lower layers of the RPC library. +It is assumed that you are familiar with sockets +and the system calls for dealing with them. +.LP +There are several occasions when you may need to use lower layers of +RPC. First, you may need to use TCP, since the higher layer uses UDP, +which restricts RPC calls to 8K bytes of data. Using TCP permits calls +to send long streams of data. +For an example, see the +.I TCP +section below. Second, you may want to allocate and free memory +while serializing or deserializing with XDR routines. +There is no call at the higher level to let +you free memory explicitly. +For more explanation, see the +.I "Memory Allocation with XDR" +section below. +Third, you may need to perform authentication +on either the client or server side, by supplying +credentials or verifying them. +See the explanation in the +.I Authentication +section below. +.NH 2 +\&More on the Server Side +.IX RPC "server side" +.LP +The server for the +.I nusers() +program shown below does the same thing as the one using +.I registerrpc() +above, but is written using a lower layer of the RPC package: +.ie t .DS +.el .DS L +.ft CW +#include <stdio.h> +#include <rpc/rpc.h> +#include <utmp.h> +#include <rpcsvc/rusers.h> + +main() +{ + SVCXPRT *transp; + int nuser(); + + transp = svcudp_create(RPC_ANYSOCK); + if (transp == NULL){ + fprintf(stderr, "can't create an RPC server\en"); + exit(1); + } + pmap_unset(RUSERSPROG, RUSERSVERS); + if (!svc_register(transp, RUSERSPROG, RUSERSVERS, + nuser, IPPROTO_UDP)) { + fprintf(stderr, "can't register RUSER service\en"); + exit(1); + } + svc_run(); /* \fINever returns\fP */ + fprintf(stderr, "should never reach this point\en"); +} + +nuser(rqstp, transp) + struct svc_req *rqstp; + SVCXPRT *transp; +{ + unsigned long nusers; + + switch (rqstp->rq_proc) { + case NULLPROC: + if (!svc_sendreply(transp, xdr_void, 0)) + fprintf(stderr, "can't reply to RPC call\en"); + return; + case RUSERSPROC_NUM: +.ft I + /* + * Code here to compute the number of users + * and assign it to the variable \fInusers\fP + */ +.ft CW + if (!svc_sendreply(transp, xdr_u_long, &nusers)) + fprintf(stderr, "can't reply to RPC call\en"); + return; + default: + svcerr_noproc(transp); + return; + } +} +.DE +.LP +First, the server gets a transport handle, which is used +for receiving and replying to RPC messages. +.I registerrpc() +uses +.I svcudp_create() +to get a UDP handle. +If you require a more reliable protocol, call +.I svctcp_create() +instead. +If the argument to +.I svcudp_create() +is +.I RPC_ANYSOCK +the RPC library creates a socket +on which to receive and reply to RPC calls. Otherwise, +.I svcudp_create() +expects its argument to be a valid socket number. +If you specify your own socket, it can be bound or unbound. +If it is bound to a port by the user, the port numbers of +.I svcudp_create() +and +.I clnttcp_create() +(the low-level client routine) must match. +.LP +If the user specifies the +.I RPC_ANYSOCK +argument, the RPC library routines will open sockets. +Otherwise they will expect the user to do so. The routines +.I svcudp_create() +and +.I clntudp_create() +will cause the RPC library routines to +.I bind() +their socket if it is not bound already. +.LP +A service may choose to register its port number with the +local portmapper service. This is done is done by specifying +a non-zero protocol number in +.I svc_register (). +Incidently, a client can discover the server's port number by +consulting the portmapper on their server's machine. This can +be done automatically by specifying a zero port number in +.I clntudp_create() +or +.I clnttcp_create (). +.LP +After creating an +.I SVCXPRT , +the next step is to call +.I pmap_unset() +so that if the +.I nusers() +server crashed earlier, +any previous trace of it is erased before restarting. +More precisely, +.I pmap_unset() +erases the entry for +.I RUSERSPROG +from the port mapper's tables. +.LP +Finally, we associate the program number for +.I nusers() +with the procedure +.I nuser (). +The final argument to +.I svc_register() +is normally the protocol being used, +which, in this case, is +.I IPPROTO_UDP +Notice that unlike +.I registerrpc (), +there are no XDR routines involved +in the registration process. +Also, registration is done on the program, +rather than procedure, level. +.LP +The user routine +.I nuser() +must call and dispatch the appropriate XDR routines +based on the procedure number. +Note that +two things are handled by +.I nuser() +that +.I registerrpc() +handles automatically. +The first is that procedure +.I NULLPROC +(currently zero) returns with no results. +This can be used as a simple test +for detecting if a remote program is running. +Second, there is a check for invalid procedure numbers. +If one is detected, +.I svcerr_noproc() +is called to handle the error. +.KS +.LP +The user service routine serializes the results and returns +them to the RPC caller via +.I svc_sendreply() +Its first parameter is the +.I SVCXPRT +handle, the second is the XDR routine, +and the third is a pointer to the data to be returned. +Not illustrated above is how a server +handles an RPC program that receives data. +As an example, we can add a procedure +.I RUSERSPROC_BOOL +which has an argument +.I nusers (), +and returns +.I TRUE +or +.I FALSE +depending on whether there are nusers logged on. +It would look like this: +.ie t .DS +.el .DS L +.ft CW +case RUSERSPROC_BOOL: { + int bool; + unsigned nuserquery; + + if (!svc_getargs(transp, xdr_u_int, &nuserquery) { + svcerr_decode(transp); + return; + } +.ft I + /* + * Code to set \fInusers\fP = number of users + */ +.ft CW + if (nuserquery == nusers) + bool = TRUE; + else + bool = FALSE; + if (!svc_sendreply(transp, xdr_bool, &bool)) { + fprintf(stderr, "can't reply to RPC call\en"); + return (1); + } + return; +} +.DE +.KE +.LP +The relevant routine is +.I svc_getargs() +which takes an +.I SVCXPRT +handle, the XDR routine, +and a pointer to where the input is to be placed as arguments. +.NH 2 +\&Memory Allocation with XDR +.IX "memory allocation with XDR" +.IX XDR "memory allocation" +.LP +XDR routines not only do input and output, +they also do memory allocation. +This is why the second parameter of +.I xdr_array() +is a pointer to an array, rather than the array itself. +If it is +.I NULL , +then +.I xdr_array() +allocates space for the array and returns a pointer to it, +putting the size of the array in the third argument. +As an example, consider the following XDR routine +.I xdr_chararr1() +which deals with a fixed array of bytes with length +.I SIZE . +.ie t .DS +.el .DS L +.ft CW +xdr_chararr1(xdrsp, chararr) + XDR *xdrsp; + char chararr[]; +{ + char *p; + int len; + + p = chararr; + len = SIZE; + return (xdr_bytes(xdrsp, &p, &len, SIZE)); +} +.DE +If space has already been allocated in +.I chararr , +it can be called from a server like this: +.ie t .DS +.el .DS L +.ft CW +char chararr[SIZE]; + +svc_getargs(transp, xdr_chararr1, chararr); +.DE +If you want XDR to do the allocation, +you would have to rewrite this routine in the following way: +.ie t .DS +.el .DS L +.ft CW +xdr_chararr2(xdrsp, chararrp) + XDR *xdrsp; + char **chararrp; +{ + int len; + + len = SIZE; + return (xdr_bytes(xdrsp, charrarrp, &len, SIZE)); +} +.DE +Then the RPC call might look like this: +.ie t .DS +.el .DS L +.ft CW +char *arrptr; + +arrptr = NULL; +svc_getargs(transp, xdr_chararr2, &arrptr); +.ft I +/* + * Use the result here + */ +.ft CW +svc_freeargs(transp, xdr_chararr2, &arrptr); +.DE +Note that, after being used, the character array can be freed with +.I svc_freeargs() +.I svc_freeargs() +will not attempt to free any memory if the variable indicating it +is NULL. For example, in the routine +.I xdr_finalexample (), +given earlier, if +.I finalp->string +was NULL, then it would not be freed. The same is true for +.I finalp->simplep . +.LP +To summarize, each XDR routine is responsible +for serializing, deserializing, and freeing memory. +When an XDR routine is called from +.I callrpc() +the serializing part is used. +When called from +.I svc_getargs() +the deserializer is used. +And when called from +.I svc_freeargs() +the memory deallocator is used. When building simple examples like those +in this section, a user doesn't have to worry +about the three modes. +See the +.I "External Data Representation: Sun Technical Notes" +for examples of more sophisticated XDR routines that determine +which of the three modes they are in and adjust their behavior accordingly. +.KS +.NH 2 +\&The Calling Side +.IX RPC "calling side" +.LP +When you use +.I callrpc() +you have no control over the RPC delivery +mechanism or the socket used to transport the data. +To illustrate the layer of RPC that lets you adjust these +parameters, consider the following code to call the +.I nusers +service: +.ie t .DS +.el .DS L +.ft CW +.vs 11 +#include <stdio.h> +#include <rpc/rpc.h> +#include <utmp.h> +#include <rpcsvc/rusers.h> +#include <sys/socket.h> +#include <sys/time.h> +#include <netdb.h> + +main(argc, argv) + int argc; + char **argv; +{ + struct hostent *hp; + struct timeval pertry_timeout, total_timeout; + struct sockaddr_in server_addr; + int sock = RPC_ANYSOCK; + register CLIENT *client; + enum clnt_stat clnt_stat; + unsigned long nusers; + + if (argc != 2) { + fprintf(stderr, "usage: nusers hostname\en"); + exit(-1); + } + if ((hp = gethostbyname(argv[1])) == NULL) { + fprintf(stderr, "can't get addr for %s\en",argv[1]); + exit(-1); + } + pertry_timeout.tv_sec = 3; + pertry_timeout.tv_usec = 0; + bcopy(hp->h_addr, (caddr_t)&server_addr.sin_addr, + hp->h_length); + server_addr.sin_family = AF_INET; + server_addr.sin_port = 0; + if ((client = clntudp_create(&server_addr, RUSERSPROG, + RUSERSVERS, pertry_timeout, &sock)) == NULL) { + clnt_pcreateerror("clntudp_create"); + exit(-1); + } + total_timeout.tv_sec = 20; + total_timeout.tv_usec = 0; + clnt_stat = clnt_call(client, RUSERSPROC_NUM, xdr_void, + 0, xdr_u_long, &nusers, total_timeout); + if (clnt_stat != RPC_SUCCESS) { + clnt_perror(client, "rpc"); + exit(-1); + } + clnt_destroy(client); + close(sock); + exit(0); +} +.vs +.DE +.KE +The low-level version of +.I callrpc() +is +.I clnt_call() +which takes a +.I CLIENT +pointer rather than a host name. The parameters to +.I clnt_call() +are a +.I CLIENT +pointer, the procedure number, +the XDR routine for serializing the argument, +a pointer to the argument, +the XDR routine for deserializing the return value, +a pointer to where the return value will be placed, +and the time in seconds to wait for a reply. +.LP +The +.I CLIENT +pointer is encoded with the transport mechanism. +.I callrpc() +uses UDP, thus it calls +.I clntudp_create() +to get a +.I CLIENT +pointer. To get TCP (Transmission Control Protocol), you would use +.I clnttcp_create() . +.LP +The parameters to +.I clntudp_create() +are the server address, the program number, the version number, +a timeout value (between tries), and a pointer to a socket. +The final argument to +.I clnt_call() +is the total time to wait for a response. +Thus, the number of tries is the +.I clnt_call() +timeout divided by the +.I clntudp_create() +timeout. +.LP +Note that the +.I clnt_destroy() +call +always deallocates the space associated with the +.I CLIENT +handle. It closes the socket associated with the +.I CLIENT +handle, however, only if the RPC library opened it. It the +socket was opened by the user, it stays open. This makes it +possible, in cases where there are multiple client handles +using the same socket, to destroy one handle without closing +the socket that other handles are using. +.LP +To make a stream connection, the call to +.I clntudp_create() +is replaced with a call to +.I clnttcp_create() . +.DS +.ft CW +clnttcp_create(&server_addr, prognum, versnum, &sock, + inputsize, outputsize); +.DE +There is no timeout argument; instead, the receive and send buffer +sizes must be specified. When the +.I clnttcp_create() +call is made, a TCP connection is established. +All RPC calls using that +.I CLIENT +handle would use this connection. +The server side of an RPC call using TCP has +.I svcudp_create() +replaced by +.I svctcp_create() . +.DS +.ft CW +transp = svctcp_create(RPC_ANYSOCK, 0, 0); +.DE +The last two arguments to +.I svctcp_create() +are send and receive sizes respectively. If `0' is specified for +either of these, the system chooses a reasonable default. +.KS +.NH 1 +\&Other RPC Features +.IX "RPC" "miscellaneous features" +.IX "miscellaneous RPC features" +.LP +This section discusses some other aspects of RPC +that are occasionally useful. +.NH 2 +\&Select on the Server Side +.IX RPC select() RPC \fIselect()\fP +.IX select() "" \fIselect()\fP "on the server side" +.LP +Suppose a process is processing RPC requests +while performing some other activity. +If the other activity involves periodically updating a data structure, +the process can set an alarm signal before calling +.I svc_run() +But if the other activity +involves waiting on a file descriptor, the +.I svc_run() +call won't work. +The code for +.I svc_run() +is as follows: +.ie t .DS +.el .DS L +.ft CW +.vs 11 +void +svc_run() +{ + fd_set readfds; + int dtbsz = getdtablesize(); + + for (;;) { + readfds = svc_fds; + switch (select(dtbsz, &readfds, NULL,NULL,NULL)) { + + case -1: + if (errno == EINTR) + continue; + perror("select"); + return; + case 0: + break; + default: + svc_getreqset(&readfds); + } + } +} +.vs +.DE +.KE +.LP +You can bypass +.I svc_run() +and call +.I svc_getreqset() +yourself. +All you need to know are the file descriptors +of the socket(s) associated with the programs you are waiting on. +Thus you can have your own +.I select() +.IX select() "" \fIselect()\fP +that waits on both the RPC socket, +and your own descriptors. Note that +.I svc_fds() +is a bit mask of all the file descriptors that RPC is using for +services. It can change everytime that +.I any +RPC library routine is called, because descriptors are constantly +being opened and closed, for example for TCP connections. +.NH 2 +\&Broadcast RPC +.IX "broadcast RPC" +.IX RPC "broadcast" +.LP +The +.I portmapper +is a daemon that converts RPC program numbers +into DARPA protocol port numbers; see the +.I portmap +man page. You can't do broadcast RPC without the portmapper. +Here are the main differences between +broadcast RPC and normal RPC calls: +.IP 1. +Normal RPC expects one answer, whereas +broadcast RPC expects many answers +(one or more answer from each responding machine). +.IP 2. +Broadcast RPC can only be supported by packet-oriented (connectionless) +transport protocols like UPD/IP. +.IP 3. +The implementation of broadcast RPC +treats all unsuccessful responses as garbage by filtering them out. +Thus, if there is a version mismatch between the +broadcaster and a remote service, +the user of broadcast RPC never knows. +.IP 4. +All broadcast messages are sent to the portmap port. +Thus, only services that register themselves with their portmapper +are accessible via the broadcast RPC mechanism. +.IP 5. +Broadcast requests are limited in size to the MTU (Maximum Transfer +Unit) of the local network. For Ethernet, the MTU is 1500 bytes. +.KS +.NH 3 +\&Broadcast RPC Synopsis +.IX "broadcast RPC" synopsis +.IX "RPC" "broadcast synopsis" +.ie t .DS +.el .DS L +.ft CW +#include <rpc/pmap_clnt.h> + . . . +enum clnt_stat clnt_stat; + . . . +clnt_stat = clnt_broadcast(prognum, versnum, procnum, + inproc, in, outproc, out, eachresult) + u_long prognum; /* \fIprogram number\fP */ + u_long versnum; /* \fIversion number\fP */ + u_long procnum; /* \fIprocedure number\fP */ + xdrproc_t inproc; /* \fIxdr routine for args\fP */ + caddr_t in; /* \fIpointer to args\fP */ + xdrproc_t outproc; /* \fIxdr routine for results\fP */ + caddr_t out; /* \fIpointer to results\fP */ + bool_t (*eachresult)();/* \fIcall with each result gotten\fP */ +.DE +.KE +The procedure +.I eachresult() +is called each time a valid result is obtained. +It returns a boolean that indicates +whether or not the user wants more responses. +.ie t .DS +.el .DS L +.ft CW +bool_t done; + . . . +done = eachresult(resultsp, raddr) + caddr_t resultsp; + struct sockaddr_in *raddr; /* \fIAddr of responding machine\fP */ +.DE +If +.I done +is +.I TRUE , +then broadcasting stops and +.I clnt_broadcast() +returns successfully. +Otherwise, the routine waits for another response. +The request is rebroadcast +after a few seconds of waiting. +If no responses come back, +the routine returns with +.I RPC_TIMEDOUT . +.NH 2 +\&Batching +.IX "batching" +.IX RPC "batching" +.LP +The RPC architecture is designed so that clients send a call message, +and wait for servers to reply that the call succeeded. +This implies that clients do not compute +while servers are processing a call. +This is inefficient if the client does not want or need +an acknowledgement for every message sent. +It is possible for clients to continue computing +while waiting for a response, +using RPC batch facilities. +.LP +RPC messages can be placed in a \*Qpipeline\*U of calls +to a desired server; this is called batching. +Batching assumes that: +1) each RPC call in the pipeline requires no response from the server, +and the server does not send a response message; and +2) the pipeline of calls is transported on a reliable +byte stream transport such as TCP/IP. +Since the server does not respond to every call, +the client can generate new calls in parallel +with the server executing previous calls. +Furthermore, the TCP/IP implementation can buffer up +many call messages, and send them to the server in one +.I write() +system call. This overlapped execution +greatly decreases the interprocess communication overhead of +the client and server processes, +and the total elapsed time of a series of calls. +.LP +Since the batched calls are buffered, +the client should eventually do a nonbatched call +in order to flush the pipeline. +.LP +A contrived example of batching follows. +Assume a string rendering service (like a window system) +has two similar calls: one renders a string and returns void results, +while the other renders a string and remains silent. +The service (using the TCP/IP transport) may look like: +.ie t .DS +.el .DS L +.ft CW +#include <stdio.h> +#include <rpc/rpc.h> +#include <suntool/windows.h> + +void windowdispatch(); + +main() +{ + SVCXPRT *transp; + + transp = svctcp_create(RPC_ANYSOCK, 0, 0); + if (transp == NULL){ + fprintf(stderr, "can't create an RPC server\en"); + exit(1); + } + pmap_unset(WINDOWPROG, WINDOWVERS); + if (!svc_register(transp, WINDOWPROG, WINDOWVERS, + windowdispatch, IPPROTO_TCP)) { + fprintf(stderr, "can't register WINDOW service\en"); + exit(1); + } + svc_run(); /* \fINever returns\fP */ + fprintf(stderr, "should never reach this point\en"); +} + +void +windowdispatch(rqstp, transp) + struct svc_req *rqstp; + SVCXPRT *transp; +{ + char *s = NULL; + + switch (rqstp->rq_proc) { + case NULLPROC: + if (!svc_sendreply(transp, xdr_void, 0)) + fprintf(stderr, "can't reply to RPC call\en"); + return; + case RENDERSTRING: + if (!svc_getargs(transp, xdr_wrapstring, &s)) { + fprintf(stderr, "can't decode arguments\en"); +.ft I + /* + * Tell caller he screwed up + */ +.ft CW + svcerr_decode(transp); + break; + } +.ft I + /* + * Code here to render the string \fIs\fP + */ +.ft CW + if (!svc_sendreply(transp, xdr_void, NULL)) + fprintf(stderr, "can't reply to RPC call\en"); + break; + case RENDERSTRING_BATCHED: + if (!svc_getargs(transp, xdr_wrapstring, &s)) { + fprintf(stderr, "can't decode arguments\en"); +.ft I + /* + * We are silent in the face of protocol errors + */ +.ft CW + break; + } +.ft I + /* + * Code here to render string s, but send no reply! + */ +.ft CW + break; + default: + svcerr_noproc(transp); + return; + } +.ft I + /* + * Now free string allocated while decoding arguments + */ +.ft CW + svc_freeargs(transp, xdr_wrapstring, &s); +} +.DE +Of course the service could have one procedure +that takes the string and a boolean +to indicate whether or not the procedure should respond. +.LP +In order for a client to take advantage of batching, +the client must perform RPC calls on a TCP-based transport +and the actual calls must have the following attributes: +1) the result's XDR routine must be zero +.I NULL ), +and 2) the RPC call's timeout must be zero. +.KS +.LP +Here is an example of a client that uses batching to render a +bunch of strings; the batching is flushed when the client gets +a null string (EOF): +.ie t .DS +.el .DS L +.ft CW +.vs 11 +#include <stdio.h> +#include <rpc/rpc.h> +#include <sys/socket.h> +#include <sys/time.h> +#include <netdb.h> +#include <suntool/windows.h> + +main(argc, argv) + int argc; + char **argv; +{ + struct hostent *hp; + struct timeval pertry_timeout, total_timeout; + struct sockaddr_in server_addr; + int sock = RPC_ANYSOCK; + register CLIENT *client; + enum clnt_stat clnt_stat; + char buf[1000], *s = buf; + + if ((client = clnttcp_create(&server_addr, + WINDOWPROG, WINDOWVERS, &sock, 0, 0)) == NULL) { + perror("clnttcp_create"); + exit(-1); + } + total_timeout.tv_sec = 0; + total_timeout.tv_usec = 0; + while (scanf("%s", s) != EOF) { + clnt_stat = clnt_call(client, RENDERSTRING_BATCHED, + xdr_wrapstring, &s, NULL, NULL, total_timeout); + if (clnt_stat != RPC_SUCCESS) { + clnt_perror(client, "batched rpc"); + exit(-1); + } + } + + /* \fINow flush the pipeline\fP */ + + total_timeout.tv_sec = 20; + clnt_stat = clnt_call(client, NULLPROC, xdr_void, NULL, + xdr_void, NULL, total_timeout); + if (clnt_stat != RPC_SUCCESS) { + clnt_perror(client, "rpc"); + exit(-1); + } + clnt_destroy(client); + exit(0); +} +.vs +.DE +.KE +Since the server sends no message, +the clients cannot be notified of any of the failures that may occur. +Therefore, clients are on their own when it comes to handling errors. +.LP +The above example was completed to render +all of the (2000) lines in the file +.I /etc/termcap . +The rendering service did nothing but throw the lines away. +The example was run in the following four configurations: +1) machine to itself, regular RPC; +2) machine to itself, batched RPC; +3) machine to another, regular RPC; and +4) machine to another, batched RPC. +The results are as follows: +1) 50 seconds; +2) 16 seconds; +3) 52 seconds; +4) 10 seconds. +Running +.I fscanf() +on +.I /etc/termcap +only requires six seconds. +These timings show the advantage of protocols +that allow for overlapped execution, +though these protocols are often hard to design. +.NH 2 +\&Authentication +.IX "authentication" +.IX "RPC" "authentication" +.LP +In the examples presented so far, +the caller never identified itself to the server, +and the server never required an ID from the caller. +Clearly, some network services, such as a network filesystem, +require stronger security than what has been presented so far. +.LP +In reality, every RPC call is authenticated by +the RPC package on the server, and similarly, +the RPC client package generates and sends authentication parameters. +Just as different transports (TCP/IP or UDP/IP) +can be used when creating RPC clients and servers, +different forms of authentication can be associated with RPC clients; +the default authentication type used as a default is type +.I none . +.LP +The authentication subsystem of the RPC package is open ended. +That is, numerous types of authentication are easy to support. +.NH 3 +\&UNIX Authentication +.IX "UNIX Authentication" +.IP "\fIThe Client Side\fP" +.LP +When a caller creates a new RPC client handle as in: +.DS +.ft CW +clnt = clntudp_create(address, prognum, versnum, + wait, sockp) +.DE +the appropriate transport instance defaults +the associate authentication handle to be +.DS +.ft CW +clnt->cl_auth = authnone_create(); +.DE +The RPC client can choose to use +.I UNIX +style authentication by setting +.I clnt\->cl_auth +after creating the RPC client handle: +.DS +.ft CW +clnt->cl_auth = authunix_create_default(); +.DE +This causes each RPC call associated with +.I clnt +to carry with it the following authentication credentials structure: +.ie t .DS +.el .DS L +.ft I +/* + * UNIX style credentials. + */ +.ft CW +struct authunix_parms { + u_long aup_time; /* \fIcredentials creation time\fP */ + char *aup_machname; /* \fIhost name where client is\fP */ + int aup_uid; /* \fIclient's UNIX effective uid\fP */ + int aup_gid; /* \fIclient's current group id\fP */ + u_int aup_len; /* \fIelement length of aup_gids\fP */ + int *aup_gids; /* \fIarray of groups user is in\fP */ +}; +.DE +These fields are set by +.I authunix_create_default() +by invoking the appropriate system calls. +Since the RPC user created this new style of authentication, +the user is responsible for destroying it with: +.DS +.ft CW +auth_destroy(clnt->cl_auth); +.DE +This should be done in all cases, to conserve memory. +.sp +.IP "\fIThe Server Side\fP" +.LP +Service implementors have a harder time dealing with authentication issues +since the RPC package passes the service dispatch routine a request +that has an arbitrary authentication style associated with it. +Consider the fields of a request handle passed to a service dispatch routine: +.ie t .DS +.el .DS L +.ft I +/* + * An RPC Service request + */ +.ft CW +struct svc_req { + u_long rq_prog; /* \fIservice program number\fP */ + u_long rq_vers; /* \fIservice protocol vers num\fP */ + u_long rq_proc; /* \fIdesired procedure number\fP */ + struct opaque_auth rq_cred; /* \fIraw credentials from wire\fP */ + caddr_t rq_clntcred; /* \fIcredentials (read only)\fP */ +}; +.DE +The +.I rq_cred +is mostly opaque, except for one field of interest: +the style or flavor of authentication credentials: +.ie t .DS +.el .DS L +.ft I +/* + * Authentication info. Mostly opaque to the programmer. + */ +.ft CW +struct opaque_auth { + enum_t oa_flavor; /* \fIstyle of credentials\fP */ + caddr_t oa_base; /* \fIaddress of more auth stuff\fP */ + u_int oa_length; /* \fInot to exceed \fIMAX_AUTH_BYTES */ +}; +.DE +.IX RPC guarantees +The RPC package guarantees the following +to the service dispatch routine: +.IP 1. +That the request's +.I rq_cred +is well formed. Thus the service implementor may inspect the request's +.I rq_cred.oa_flavor +to determine which style of authentication the caller used. +The service implementor may also wish to inspect the other fields of +.I rq_cred +if the style is not one of the styles supported by the RPC package. +.IP 2. +That the request's +.I rq_clntcred +field is either +.I NULL +or points to a well formed structure +that corresponds to a supported style of authentication credentials. +Remember that only +.I unix +style is currently supported, so (currently) +.I rq_clntcred +could be cast to a pointer to an +.I authunix_parms +structure. If +.I rq_clntcred +is +.I NULL , +the service implementor may wish to inspect the other (opaque) fields of +.I rq_cred +in case the service knows about a new type of authentication +that the RPC package does not know about. +.LP +Our remote users service example can be extended so that +it computes results for all users except UID 16: +.ie t .DS +.el .DS L +.ft CW +.vs 11 +nuser(rqstp, transp) + struct svc_req *rqstp; + SVCXPRT *transp; +{ + struct authunix_parms *unix_cred; + int uid; + unsigned long nusers; + +.ft I + /* + * we don't care about authentication for null proc + */ +.ft CW + if (rqstp->rq_proc == NULLPROC) { + if (!svc_sendreply(transp, xdr_void, 0)) { + fprintf(stderr, "can't reply to RPC call\en"); + return (1); + } + return; + } +.ft I + /* + * now get the uid + */ +.ft CW + switch (rqstp->rq_cred.oa_flavor) { + case AUTH_UNIX: + unix_cred = + (struct authunix_parms *)rqstp->rq_clntcred; + uid = unix_cred->aup_uid; + break; + case AUTH_NULL: + default: + svcerr_weakauth(transp); + return; + } + switch (rqstp->rq_proc) { + case RUSERSPROC_NUM: +.ft I + /* + * make sure caller is allowed to call this proc + */ +.ft CW + if (uid == 16) { + svcerr_systemerr(transp); + return; + } +.ft I + /* + * Code here to compute the number of users + * and assign it to the variable \fInusers\fP + */ +.ft CW + if (!svc_sendreply(transp, xdr_u_long, &nusers)) { + fprintf(stderr, "can't reply to RPC call\en"); + return (1); + } + return; + default: + svcerr_noproc(transp); + return; + } +} +.vs +.DE +A few things should be noted here. +First, it is customary not to check +the authentication parameters associated with the +.I NULLPROC +(procedure number zero). +Second, if the authentication parameter's type is not suitable +for your service, you should call +.I svcerr_weakauth() . +And finally, the service protocol itself should return status +for access denied; in the case of our example, the protocol +does not have such a status, so we call the service primitive +.I svcerr_systemerr() +instead. +.LP +The last point underscores the relation between +the RPC authentication package and the services; +RPC deals only with +.I authentication +and not with individual services' +.I "access control" . +The services themselves must implement their own access control policies +and reflect these policies as return statuses in their protocols. +.NH 2 +\&DES Authentication +.IX RPC DES +.IX RPC authentication +.LP +UNIX authentication is quite easy to defeat. Instead of using +.I authunix_create_default (), +one can call +.I authunix_create() +and then modify the RPC authentication handle it returns by filling in +whatever user ID and hostname they wish the server to think they have. +DES authentication is thus recommended for people who want more security +than UNIX authentication offers. +.LP +The details of the DES authentication protocol are complicated and +are not explained here. +See +.I "Remote Procedure Calls: Protocol Specification" +for the details. +.LP +In order for DES authentication to work, the +.I keyserv(8c) +daemon must be running on both the server and client machines. The +users on these machines need public keys assigned by the network +administrator in the +.I publickey(5) +database. And, they need to have decrypted their secret keys +using their login password. This automatically happens when one +logs in using +.I login(1) , +or can be done manually using +.I keylogin(1) . +The +.I "Network Services" +chapter +.\" XXX +explains more how to setup secure networking. +.sp +.IP "\fIClient Side\fP" +.LP +If a client wishes to use DES authentication, it must set its +authentication handle appropriately. Here is an example: +.DS +cl->cl_auth = + authdes_create(servername, 60, &server_addr, NULL); +.DE +The first argument is the network name or \*Qnetname\*U of the owner of +the server process. Typically, server processes are root processes +and their netname can be derived using the following call: +.DS +char servername[MAXNETNAMELEN]; + +host2netname(servername, rhostname, NULL); +.DE +Here, +.I rhostname +is the hostname of the machine the server process is running on. +.I host2netname() +fills in +.I servername +to contain this root process's netname. If the +server process was run by a regular user, one could use the call +.I user2netname() +instead. Here is an example for a server process with the same user +ID as the client: +.DS +char servername[MAXNETNAMELEN]; + +user2netname(servername, getuid(), NULL); +.DE +The last argument to both of these calls, +.I user2netname() +and +.I host2netname (), +is the name of the naming domain where the server is located. The +.I NULL +used here means \*Quse the local domain name.\*U +.LP +The second argument to +.I authdes_create() +is a lifetime for the credential. Here it is set to sixty +seconds. What that means is that the credential will expire 60 +seconds from now. If some mischievous user tries to reuse the +credential, the server RPC subsystem will recognize that it has +expired and not grant any requests. If the same mischievous user +tries to reuse the credential within the sixty second lifetime, +he will still be rejected because the server RPC subsystem +remembers which credentials it has already seen in the near past, +and will not grant requests to duplicates. +.LP +The third argument to +.I authdes_create() +is the address of the host to synchronize with. In order for DES +authentication to work, the server and client must agree upon the +time. Here we pass the address of the server itself, so the +client and server will both be using the same time: the server's +time. The argument can be +.I NULL , +which means \*Qdon't bother synchronizing.\*U You should only do this +if you are sure the client and server are already synchronized. +.LP +The final argument to +.I authdes_create() +is the address of a DES encryption key to use for encrypting +timestamps and data. If this argument is +.I NULL , +as it is in this example, a random key will be chosen. The client +may find out the encryption key being used by consulting the +.I ah_key +field of the authentication handle. +.sp +.IP "\fIServer Side\fP" +.LP +The server side is a lot simpler than the client side. Here is the +previous example rewritten to use +.I AUTH_DES +instead of +.I AUTH_UNIX : +.ie t .DS +.el .DS L +.ft CW +.vs 11 +#include <sys/time.h> +#include <rpc/auth_des.h> + . . . + . . . +nuser(rqstp, transp) + struct svc_req *rqstp; + SVCXPRT *transp; +{ + struct authdes_cred *des_cred; + int uid; + int gid; + int gidlen; + int gidlist[10]; +.ft I + /* + * we don't care about authentication for null proc + */ +.ft CW + + if (rqstp->rq_proc == NULLPROC) { + /* \fIsame as before\fP */ + } + +.ft I + /* + * now get the uid + */ +.ft CW + switch (rqstp->rq_cred.oa_flavor) { + case AUTH_DES: + des_cred = + (struct authdes_cred *) rqstp->rq_clntcred; + if (! netname2user(des_cred->adc_fullname.name, + &uid, &gid, &gidlen, gidlist)) + { + fprintf(stderr, "unknown user: %s\en", + des_cred->adc_fullname.name); + svcerr_systemerr(transp); + return; + } + break; + case AUTH_NULL: + default: + svcerr_weakauth(transp); + return; + } + +.ft I + /* + * The rest is the same as before + */ +.ft CW +.vs +.DE +Note the use of the routine +.I netname2user (), +the inverse of +.I user2netname (): +it takes a network ID and converts to a unix ID. +.I netname2user () +also supplies the group IDs which we don't use in this example, +but which may be useful to other UNIX programs. +.NH 2 +\&Using Inetd +.IX inetd "" "using \fIinetd\fP" +.LP +An RPC server can be started from +.I inetd +The only difference from the usual code is that the service +creation routine should be called in the following form: +.ie t .DS +.el .DS L +.ft CW +transp = svcudp_create(0); /* \fIFor UDP\fP */ +transp = svctcp_create(0,0,0); /* \fIFor listener TCP sockets\fP */ +transp = svcfd_create(0,0,0); /* \fIFor connected TCP sockets\fP */ +.DE +since +.I inet +passes a socket as file descriptor 0. +Also, +.I svc_register() +should be called as +.ie t .DS +.el .DS L +.ft CW +svc_register(transp, PROGNUM, VERSNUM, service, 0); +.DE +with the final flag as 0, +since the program would already be registered by +.I inetd +Remember that if you want to exit +from the server process and return control to +.I inet +you need to explicitly exit, since +.I svc_run() +never returns. +.LP +The format of entries in +.I /etc/inetd.conf +for RPC services is in one of the following two forms: +.ie t .DS +.el .DS L +.ft CW +p_name/version dgram rpc/udp wait/nowait user server args +p_name/version stream rpc/tcp wait/nowait user server args +.DE +where +.I p_name +is the symbolic name of the program as it appears in +.I rpc(5) , +.I server +is the program implementing the server, +and +.I program +and +.I version +are the program and version numbers of the service. +For more information, see +.I inetd.conf(5) . +.LP +If the same program handles multiple versions, +then the version number can be a range, +as in this example: +.ie t .DS +.el .DS L +.ft CW +rstatd/1-2 dgram rpc/udp wait root /usr/etc/rpc.rstatd +.DE +.NH 1 +\&More Examples +.sp 1 +.NH 2 +\&Versions +.IX "versions" +.IX "RPC" "versions" +.LP +By convention, the first version number of program +.I PROG +is +.I PROGVERS_ORIG +and the most recent version is +.I PROGVERS +Suppose there is a new version of the +.I user +program that returns an +.I "unsigned short" +rather than a +.I long . +If we name this version +.I RUSERSVERS_SHORT +then a server that wants to support both versions +would do a double register. +.ie t .DS +.el .DS L +.ft CW +if (!svc_register(transp, RUSERSPROG, RUSERSVERS_ORIG, + nuser, IPPROTO_TCP)) { + fprintf(stderr, "can't register RUSER service\en"); + exit(1); +} +if (!svc_register(transp, RUSERSPROG, RUSERSVERS_SHORT, + nuser, IPPROTO_TCP)) { + fprintf(stderr, "can't register RUSER service\en"); + exit(1); +} +.DE +Both versions can be handled by the same C procedure: +.ie t .DS +.el .DS L +.ft CW +.vs 11 +nuser(rqstp, transp) + struct svc_req *rqstp; + SVCXPRT *transp; +{ + unsigned long nusers; + unsigned short nusers2; + + switch (rqstp->rq_proc) { + case NULLPROC: + if (!svc_sendreply(transp, xdr_void, 0)) { + fprintf(stderr, "can't reply to RPC call\en"); + return (1); + } + return; + case RUSERSPROC_NUM: +.ft I + /* + * Code here to compute the number of users + * and assign it to the variable \fInusers\fP + */ +.ft CW + nusers2 = nusers; + switch (rqstp->rq_vers) { + case RUSERSVERS_ORIG: + if (!svc_sendreply(transp, xdr_u_long, + &nusers)) { + fprintf(stderr,"can't reply to RPC call\en"); + } + break; + case RUSERSVERS_SHORT: + if (!svc_sendreply(transp, xdr_u_short, + &nusers2)) { + fprintf(stderr,"can't reply to RPC call\en"); + } + break; + } + default: + svcerr_noproc(transp); + return; + } +} +.vs +.DE +.KS +.NH 2 +\&TCP +.IX "TCP" +.LP +Here is an example that is essentially +.I rcp. +The initiator of the RPC +.I snd +call takes its standard input and sends it to the server +.I rcv +which prints it on standard output. +The RPC call uses TCP. +This also illustrates an XDR procedure that behaves differently +on serialization than on deserialization. +.ie t .DS +.el .DS L +.vs 11 +.ft I +/* + * The xdr routine: + * on decode, read from wire, write onto fp + * on encode, read from fp, write onto wire + */ +.ft CW +#include <stdio.h> +#include <rpc/rpc.h> + +xdr_rcp(xdrs, fp) + XDR *xdrs; + FILE *fp; +{ + unsigned long size; + char buf[BUFSIZ], *p; + + if (xdrs->x_op == XDR_FREE)/* nothing to free */ + return 1; + while (1) { + if (xdrs->x_op == XDR_ENCODE) { + if ((size = fread(buf, sizeof(char), BUFSIZ, + fp)) == 0 && ferror(fp)) { + fprintf(stderr, "can't fread\en"); + return (1); + } + } + p = buf; + if (!xdr_bytes(xdrs, &p, &size, BUFSIZ)) + return 0; + if (size == 0) + return 1; + if (xdrs->x_op == XDR_DECODE) { + if (fwrite(buf, sizeof(char), size, + fp) != size) { + fprintf(stderr, "can't fwrite\en"); + return (1); + } + } + } +} +.vs +.DE +.KE +.ie t .DS +.el .DS L +.vs 11 +.ft I +/* + * The sender routines + */ +.ft CW +#include <stdio.h> +#include <netdb.h> +#include <rpc/rpc.h> +#include <sys/socket.h> +#include <sys/time.h> + +main(argc, argv) + int argc; + char **argv; +{ + int xdr_rcp(); + int err; + + if (argc < 2) { + fprintf(stderr, "usage: %s servername\en", argv[0]); + exit(-1); + } + if ((err = callrpctcp(argv[1], RCPPROG, RCPPROC, + RCPVERS, xdr_rcp, stdin, xdr_void, 0) != 0)) { + clnt_perrno(err); + fprintf(stderr, "can't make RPC call\en"); + exit(1); + } + exit(0); +} + +callrpctcp(host, prognum, procnum, versnum, + inproc, in, outproc, out) + char *host, *in, *out; + xdrproc_t inproc, outproc; +{ + struct sockaddr_in server_addr; + int socket = RPC_ANYSOCK; + enum clnt_stat clnt_stat; + struct hostent *hp; + register CLIENT *client; + struct timeval total_timeout; + + if ((hp = gethostbyname(host)) == NULL) { + fprintf(stderr, "can't get addr for '%s'\en", host); + return (-1); + } + bcopy(hp->h_addr, (caddr_t)&server_addr.sin_addr, + hp->h_length); + server_addr.sin_family = AF_INET; + server_addr.sin_port = 0; + if ((client = clnttcp_create(&server_addr, prognum, + versnum, &socket, BUFSIZ, BUFSIZ)) == NULL) { + perror("rpctcp_create"); + return (-1); + } + total_timeout.tv_sec = 20; + total_timeout.tv_usec = 0; + clnt_stat = clnt_call(client, procnum, + inproc, in, outproc, out, total_timeout); + clnt_destroy(client); + return (int)clnt_stat; +} +.vs +.DE +.ie t .DS +.el .DS L +.vs 11 +.ft I +/* + * The receiving routines + */ +.ft CW +#include <stdio.h> +#include <rpc/rpc.h> + +main() +{ + register SVCXPRT *transp; + int rcp_service(), xdr_rcp(); + + if ((transp = svctcp_create(RPC_ANYSOCK, + BUFSIZ, BUFSIZ)) == NULL) { + fprintf("svctcp_create: error\en"); + exit(1); + } + pmap_unset(RCPPROG, RCPVERS); + if (!svc_register(transp, + RCPPROG, RCPVERS, rcp_service, IPPROTO_TCP)) { + fprintf(stderr, "svc_register: error\en"); + exit(1); + } + svc_run(); /* \fInever returns\fP */ + fprintf(stderr, "svc_run should never return\en"); +} + +rcp_service(rqstp, transp) + register struct svc_req *rqstp; + register SVCXPRT *transp; +{ + switch (rqstp->rq_proc) { + case NULLPROC: + if (svc_sendreply(transp, xdr_void, 0) == 0) { + fprintf(stderr, "err: rcp_service"); + return (1); + } + return; + case RCPPROC_FP: + if (!svc_getargs(transp, xdr_rcp, stdout)) { + svcerr_decode(transp); + return; + } + if (!svc_sendreply(transp, xdr_void, 0)) { + fprintf(stderr, "can't reply\en"); + return; + } + return (0); + default: + svcerr_noproc(transp); + return; + } +} +.vs +.DE +.NH 2 +\&Callback Procedures +.IX RPC "callback procedures" +.LP +Occasionally, it is useful to have a server become a client, +and make an RPC call back to the process which is its client. +An example is remote debugging, +where the client is a window system program, +and the server is a debugger running on the remote machine. +Most of the time, +the user clicks a mouse button at the debugging window, +which converts this to a debugger command, +and then makes an RPC call to the server +(where the debugger is actually running), +telling it to execute that command. +However, when the debugger hits a breakpoint, the roles are reversed, +and the debugger wants to make an rpc call to the window program, +so that it can inform the user that a breakpoint has been reached. +.LP +In order to do an RPC callback, +you need a program number to make the RPC call on. +Since this will be a dynamically generated program number, +it should be in the transient range, +.I "0x40000000 - 0x5fffffff" . +The routine +.I gettransient() +returns a valid program number in the transient range, +and registers it with the portmapper. +It only talks to the portmapper running on the same machine as the +.I gettransient() +routine itself. The call to +.I pmap_set() +is a test and set operation, +in that it indivisibly tests whether a program number +has already been registered, +and if it has not, then reserves it. On return, the +.I sockp +argument will contain a socket that can be used +as the argument to an +.I svcudp_create() +or +.I svctcp_create() +call. +.ie t .DS +.el .DS L +.ft CW +.vs 11 +#include <stdio.h> +#include <rpc/rpc.h> +#include <sys/socket.h> + +gettransient(proto, vers, sockp) + int proto, vers, *sockp; +{ + static int prognum = 0x40000000; + int s, len, socktype; + struct sockaddr_in addr; + + switch(proto) { + case IPPROTO_UDP: + socktype = SOCK_DGRAM; + break; + case IPPROTO_TCP: + socktype = SOCK_STREAM; + break; + default: + fprintf(stderr, "unknown protocol type\en"); + return 0; + } + if (*sockp == RPC_ANYSOCK) { + if ((s = socket(AF_INET, socktype, 0)) < 0) { + perror("socket"); + return (0); + } + *sockp = s; + } + else + s = *sockp; + addr.sin_addr.s_addr = 0; + addr.sin_family = AF_INET; + addr.sin_port = 0; + len = sizeof(addr); +.ft I + /* + * may be already bound, so don't check for error + */ +.ft CW + bind(s, &addr, len); + if (getsockname(s, &addr, &len)< 0) { + perror("getsockname"); + return (0); + } + while (!pmap_set(prognum++, vers, proto, + ntohs(addr.sin_port))) continue; + return (prognum-1); +} +.vs +.DE +.SH +Note: +.I +The call to +.I ntohs() +is necessary to ensure that the port number in +.I "addr.sin_port" , +which is in +.I network +byte order, is passed in +.I host +byte order (as +.I pmap_set() +expects). See the +.I byteorder(3N) +man page for more details on the conversion of network +addresses from network to host byte order. +.KS +.LP +The following pair of programs illustrate how to use the +.I gettransient() +routine. +The client makes an RPC call to the server, +passing it a transient program number. +Then the client waits around to receive a callback +from the server at that program number. +The server registers the program +.I EXAMPLEPROG +so that it can receive the RPC call +informing it of the callback program number. +Then at some random time (on receiving an +.I ALRM +signal in this example), it sends a callback RPC call, +using the program number it received earlier. +.ie t .DS +.el .DS L +.vs 11 +.ft I +/* + * client + */ +.ft CW +#include <stdio.h> +#include <rpc/rpc.h> + +int callback(); +char hostname[256]; + +main() +{ + int x, ans, s; + SVCXPRT *xprt; + + gethostname(hostname, sizeof(hostname)); + s = RPC_ANYSOCK; + x = gettransient(IPPROTO_UDP, 1, &s); + fprintf(stderr, "client gets prognum %d\en", x); + if ((xprt = svcudp_create(s)) == NULL) { + fprintf(stderr, "rpc_server: svcudp_create\en"); + exit(1); + } +.ft I + /* protocol is 0 - gettransient does registering + */ +.ft CW + (void)svc_register(xprt, x, 1, callback, 0); + ans = callrpc(hostname, EXAMPLEPROG, EXAMPLEVERS, + EXAMPLEPROC_CALLBACK, xdr_int, &x, xdr_void, 0); + if ((enum clnt_stat) ans != RPC_SUCCESS) { + fprintf(stderr, "call: "); + clnt_perrno(ans); + fprintf(stderr, "\en"); + } + svc_run(); + fprintf(stderr, "Error: svc_run shouldn't return\en"); +} + +callback(rqstp, transp) + register struct svc_req *rqstp; + register SVCXPRT *transp; +{ + switch (rqstp->rq_proc) { + case 0: + if (!svc_sendreply(transp, xdr_void, 0)) { + fprintf(stderr, "err: exampleprog\en"); + return (1); + } + return (0); + case 1: + if (!svc_getargs(transp, xdr_void, 0)) { + svcerr_decode(transp); + return (1); + } + fprintf(stderr, "client got callback\en"); + if (!svc_sendreply(transp, xdr_void, 0)) { + fprintf(stderr, "err: exampleprog"); + return (1); + } + } +} +.vs +.DE +.KE +.ie t .DS +.el .DS L +.vs 11 +.ft I +/* + * server + */ +.ft CW +#include <stdio.h> +#include <rpc/rpc.h> +#include <sys/signal.h> + +char *getnewprog(); +char hostname[256]; +int docallback(); +int pnum; /* \fIprogram number for callback routine\fP */ + +main() +{ + gethostname(hostname, sizeof(hostname)); + registerrpc(EXAMPLEPROG, EXAMPLEVERS, + EXAMPLEPROC_CALLBACK, getnewprog, xdr_int, xdr_void); + fprintf(stderr, "server going into svc_run\en"); + signal(SIGALRM, docallback); + alarm(10); + svc_run(); + fprintf(stderr, "Error: svc_run shouldn't return\en"); +} + +char * +getnewprog(pnump) + char *pnump; +{ + pnum = *(int *)pnump; + return NULL; +} + +docallback() +{ + int ans; + + ans = callrpc(hostname, pnum, 1, 1, xdr_void, 0, + xdr_void, 0); + if (ans != 0) { + fprintf(stderr, "server: "); + clnt_perrno(ans); + fprintf(stderr, "\en"); + } +} +.vs +.DE diff --git a/share/doc/psd/23.rpc/stubs b/share/doc/psd/23.rpc/stubs new file mode 100644 index 000000000000..78b0a2c815d3 --- /dev/null +++ b/share/doc/psd/23.rpc/stubs @@ -0,0 +1,3 @@ +.\" $FreeBSD$ +.\" +.if t .ftr L CR diff --git a/share/doc/psd/24.xdr/Makefile b/share/doc/psd/24.xdr/Makefile new file mode 100644 index 000000000000..8e3071184545 --- /dev/null +++ b/share/doc/psd/24.xdr/Makefile @@ -0,0 +1,8 @@ +# $FreeBSD$ + +VOLUME= psd/24.xdr +SRCS= stubs xdr.nts.ms +MACROS= -ms +USE_EQN= + +.include <bsd.doc.mk> diff --git a/share/doc/psd/24.xdr/stubs b/share/doc/psd/24.xdr/stubs new file mode 100644 index 000000000000..78b0a2c815d3 --- /dev/null +++ b/share/doc/psd/24.xdr/stubs @@ -0,0 +1,3 @@ +.\" $FreeBSD$ +.\" +.if t .ftr L CR diff --git a/share/doc/psd/24.xdr/xdr.nts.ms b/share/doc/psd/24.xdr/xdr.nts.ms new file mode 100644 index 000000000000..f7c495d6a6f3 --- /dev/null +++ b/share/doc/psd/24.xdr/xdr.nts.ms @@ -0,0 +1,1968 @@ +.\" +.\" Must use -- eqn -- with this one +.\" +.\" @(#)xdr.nts.ms 2.2 88/08/05 4.0 RPCSRC +.\" $FreeBSD$ +.\" +.EQ +delim $$ +.EN +.de BT +.if \\n%=1 .tl ''- % -'' +.. +.ND +.\" prevent excess underlining in nroff +.if n .fp 2 R +.OH 'External Data Representation: Sun Technical Notes''Page %' +.EH 'Page %''External Data Representation: Sun Technical Notes' +.if \n%=1 .bp +.SH +\&External Data Representation: Sun Technical Notes +.IX XDR "Sun technical notes" +.LP +This chapter contains technical notes on Sun's implementation of the +External Data Representation (XDR) standard, a set of library routines +that allow a C programmer to describe arbitrary data structures in a +machinex-independent fashion. +For a formal specification of the XDR +standard, see the +.I "External Data Representation Standard: Protocol Specification". +XDR is the backbone of Sun's Remote Procedure Call package, in the +sense that data for remote procedure calls is transmitted using the +standard. XDR library routines should be used to transmit data +that is accessed (read or written) by more than one type of machine.\** +.FS +.IX XDR "system routines" +For a compete specification of the system External Data Representation +routines, see the +.I xdr(3N) +manual page. +.FE +.LP +This chapter contains a short tutorial overview of the XDR library +routines, a guide to accessing currently available XDR streams, and +information on defining new streams and data types. XDR was designed +to work across different languages, operating systems, and machine +architectures. Most users (particularly RPC users) will only need +the information in the +.I "Number Filters", +.I "Floating Point Filters", +and +.I "Enumeration Filters" +sections. +Programmers wishing to implement RPC and XDR on new machines +will be interested in the rest of the chapter, as well as the +.I "External Data Representaiton Standard: Protocol Specification", +which will be their primary reference. +.SH +Note: +.I +.I rpcgen +can be used to write XDR routines even in cases where no RPC calls are +being made. +.LP +On Sun systems, +C programs that want to use XDR routines +must include the file +.I <rpc/rpc.h> , +which contains all the necessary interfaces to the XDR system. +Since the C library +.I libc.a +contains all the XDR routines, +compile as normal. +.DS +example% \fBcc\0\fIprogram\fP.c\fI +.DE +.ne 3i +.NH 0 +\&Justification +.IX XDR justification +.LP +Consider the following two programs, +.I writer : +.ie t .DS +.el .DS L +.ft CW +#include <stdio.h> +.sp .5 +main() /* \fIwriter.c\fP */ +{ + long i; +.sp .5 + for (i = 0; i < 8; i++) { + if (fwrite((char *)&i, sizeof(i), 1, stdout) != 1) { + fprintf(stderr, "failed!\en"); + exit(1); + } + } + exit(0); +} +.DE +and +.I reader : +.ie t .DS +.el .DS L +.ft CW +#include <stdio.h> +.sp .5 +main() /* \fIreader.c\fP */ +{ + long i, j; +.sp .5 + for (j = 0; j < 8; j++) { + if (fread((char *)&i, sizeof (i), 1, stdin) != 1) { + fprintf(stderr, "failed!\en"); + exit(1); + } + printf("%ld ", i); + } + printf("\en"); + exit(0); +} +.DE +The two programs appear to be portable, because (a) they pass +.I lint +checking, and (b) they exhibit the same behavior when executed +on two different hardware architectures, a Sun and a VAX. +.LP +Piping the output of the +.I writer +program to the +.I reader +program gives identical results on a Sun or a VAX. +.DS +.ft CW +sun% \fBwriter | reader\fP +0 1 2 3 4 5 6 7 +sun% + + +vax% \fBwriter | reader\fP +0 1 2 3 4 5 6 7 +vax% +.DE +With the advent of local area networks and 4.2BSD came the concept +of \*Qnetwork pipes\*U \(em a process produces data on one machine, +and a second process consumes data on another machine. +A network pipe can be constructed with +.I writer +and +.I reader . +Here are the results if the first produces data on a Sun, +and the second consumes data on a VAX. +.DS +.ft CW +sun% \fBwriter | rsh vax reader\fP +0 16777216 33554432 50331648 67108864 83886080 100663296 +117440512 +sun% +.DE +Identical results can be obtained by executing +.I writer +on the VAX and +.I reader +on the Sun. These results occur because the byte ordering +of long integers differs between the VAX and the Sun, +even though word size is the same. +Note that $16777216$ is $2 sup 24$ \(em +when four bytes are reversed, the 1 winds up in the 24th bit. +.LP +Whenever data is shared by two or more machine types, there is +a need for portable data. Programs can be made data-portable by +replacing the +.I read() +and +.I write() +calls with calls to an XDR library routine +.I xdr_long() , +a filter that knows the standard representation +of a long integer in its external form. +Here are the revised versions of +.I writer : +.ie t .DS +.el .DS L +.ft CW +#include <stdio.h> +#include <rpc/rpc.h> /* \fIxdr is a sub-library of rpc\fP */ +.sp .5 +main() /* \fIwriter.c\fP */ +{ + XDR xdrs; + long i; +.sp .5 + xdrstdio_create(&xdrs, stdout, XDR_ENCODE); + for (i = 0; i < 8; i++) { + if (!xdr_long(&xdrs, &i)) { + fprintf(stderr, "failed!\en"); + exit(1); + } + } + exit(0); +} +.DE +and +.I reader : +.ie t .DS +.el .DS L +.ft CW +#include <stdio.h> +#include <rpc/rpc.h> /* \fIxdr is a sub-library of rpc\fP */ +.sp .5 +main() /* \fIreader.c\fP */ +{ + XDR xdrs; + long i, j; +.sp .5 + xdrstdio_create(&xdrs, stdin, XDR_DECODE); + for (j = 0; j < 8; j++) { + if (!xdr_long(&xdrs, &i)) { + fprintf(stderr, "failed!\en"); + exit(1); + } + printf("%ld ", i); + } + printf("\en"); + exit(0); +} +.DE +The new programs were executed on a Sun, +on a VAX, and from a Sun to a VAX; +the results are shown below. +.DS +.ft CW +sun% \fBwriter | reader\fP +0 1 2 3 4 5 6 7 +sun% + +vax% \fBwriter | reader\fP +0 1 2 3 4 5 6 7 +vax% + +sun% \fBwriter | rsh vax reader\fP +0 1 2 3 4 5 6 7 +sun% +.DE +.SH +Note: +.I +.IX XDR "portable data" +Integers are just the tip of the portable-data iceberg. Arbitrary +data structures present portability problems, particularly with +respect to alignment and pointers. Alignment on word boundaries +may cause the size of a structure to vary from machine to machine. +And pointers, which are very convenient to use, have no meaning +outside the machine where they are defined. +.LP +.NH 1 +\&A Canonical Standard +.IX XDR "canonical standard" +.LP +XDR's approach to standardizing data representations is +.I canonical . +That is, XDR defines a single byte order (Big Endian), a single +floating-point representation (IEEE), and so on. Any program running on +any machine can use XDR to create portable data by translating its +local representation to the XDR standard representations; similarly, any +program running on any machine can read portable data by translating the +XDR standard representaions to its local equivalents. The single standard +completely decouples programs that create or send portable data from those +that use or receive portable data. The advent of a new machine or a new +language has no effect upon the community of existing portable data creators +and users. A new machine joins this community by being \*Qtaught\*U how to +convert the standard representations and its local representations; the +local representations of other machines are irrelevant. Conversely, to +existing programs running on other machines, the local representations of +the new machine are also irrelevant; such programs can immediately read +portable data produced by the new machine because such data conforms to the +canonical standards that they already understand. +.LP +There are strong precedents for XDR's canonical approach. For example, +TCP/IP, UDP/IP, XNS, Ethernet, and, indeed, all protocols below layer five +of the ISO model, are canonical protocols. The advantage of any canonical +approach is simplicity; in the case of XDR, a single set of conversion +routines is written once and is never touched again. The canonical approach +has a disadvantage, but it is unimportant in real-world data transfer +applications. Suppose two Little-Endian machines are transferring integers +according to the XDR standard. The sending machine converts the integers +from Little-Endian byte order to XDR (Big-Endian) byte order; the receiving +machine performs the reverse conversion. Because both machines observe the +same byte order, their conversions are unnecessary. The point, however, is +not necessity, but cost as compared to the alternative. +.LP +The time spent converting to and from a canonical representation is +insignificant, especially in networking applications. Most of the time +required to prepare a data structure for transfer is not spent in conversion +but in traversing the elements of the data structure. To transmit a tree, +for example, each leaf must be visited and each element in a leaf record must +be copied to a buffer and aligned there; storage for the leaf may have to be +deallocated as well. Similarly, to receive a tree, storage must be +allocated for each leaf, data must be moved from the buffer to the leaf and +properly aligned, and pointers must be constructed to link the leaves +together. Every machine pays the cost of traversing and copying data +structures whether or not conversion is required. In networking +applications, communications overhead\(emthe time required to move the data +down through the sender's protocol layers, across the network and up through +the receiver's protocol layers\(emdwarfs conversion overhead. +.NH 1 +\&The XDR Library +.IX "XDR" "library" +.LP +The XDR library not only solves data portability problems, it also +allows you to write and read arbitrary C constructs in a consistent, +specified, well-documented manner. Thus, it can make sense to use the +library even when the data is not shared among machines on a network. +.LP +The XDR library has filter routines for +strings (null-terminated arrays of bytes), +structures, unions, and arrays, to name a few. +Using more primitive routines, +you can write your own specific XDR routines +to describe arbitrary data structures, +including elements of arrays, arms of unions, +or objects pointed at from other structures. +The structures themselves may contain arrays of arbitrary elements, +or pointers to other structures. +.LP +Let's examine the two programs more closely. +There is a family of XDR stream creation routines +in which each member treats the stream of bits differently. +In our example, data is manipulated using standard I/O routines, +so we use +.I xdrstdio_create (). +.IX xdrstdio_create() "" "\fIxdrstdio_create()\fP" +The parameters to XDR stream creation routines +vary according to their function. +In our example, +.I xdrstdio_create() +takes a pointer to an XDR structure that it initializes, +a pointer to a +.I FILE +that the input or output is performed on, and the operation. +The operation may be +.I XDR_ENCODE +for serializing in the +.I writer +program, or +.I XDR_DECODE +for deserializing in the +.I reader +program. +.LP +Note: RPC users never need to create XDR streams; +the RPC system itself creates these streams, +which are then passed to the users. +.LP +The +.I xdr_long() +.IX xdr_long() "" "\fIxdr_long()\fP" +primitive is characteristic of most XDR library +primitives and all client XDR routines. +First, the routine returns +.I FALSE +(0) if it fails, and +.I TRUE +(1) if it succeeds. +Second, for each data type, +.I xxx , +there is an associated XDR routine of the form: +.DS +.ft CW +xdr_xxx(xdrs, xp) + XDR *xdrs; + xxx *xp; +{ +} +.DE +In our case, +.I xxx +is long, and the corresponding XDR routine is +a primitive, +.I xdr_long() . +The client could also define an arbitrary structure +.I xxx +in which case the client would also supply the routine +.I xdr_xxx (), +describing each field by calling XDR routines +of the appropriate type. +In all cases the first parameter, +.I xdrs +can be treated as an opaque handle, +and passed to the primitive routines. +.LP +XDR routines are direction independent; +that is, the same routines are called to serialize or deserialize data. +This feature is critical to software engineering of portable data. +The idea is to call the same routine for either operation \(em +this almost guarantees that serialized data can also be deserialized. +One routine is used by both producer and consumer of networked data. +This is implemented by always passing the address +of an object rather than the object itself \(em +only in the case of deserialization is the object modified. +This feature is not shown in our trivial example, +but its value becomes obvious when nontrivial data structures +are passed among machines. +If needed, the user can obtain the +direction of the XDR operation. +See the +.I "XDR Operation Directions" +section below for details. +.LP +Let's look at a slightly more complicated example. +Assume that a person's gross assets and liabilities +are to be exchanged among processes. +Also assume that these values are important enough +to warrant their own data type: +.ie t .DS +.el .DS L +.ft CW +struct gnumbers { + long g_assets; + long g_liabilities; +}; +.DE +The corresponding XDR routine describing this structure would be: +.ie t .DS +.el .DS L +.ft CW +bool_t /* \fITRUE is success, FALSE is failure\fP */ +xdr_gnumbers(xdrs, gp) + XDR *xdrs; + struct gnumbers *gp; +{ + if (xdr_long(xdrs, &gp->g_assets) && + xdr_long(xdrs, &gp->g_liabilities)) + return(TRUE); + return(FALSE); +} +.DE +Note that the parameter +.I xdrs +is never inspected or modified; +it is only passed on to the subcomponent routines. +It is imperative to inspect the return value of each XDR routine call, +and to give up immediately and return +.I FALSE +if the subroutine fails. +.LP +This example also shows that the type +.I bool_t +is declared as an integer whose only values are +.I TRUE +(1) and +.I FALSE +(0). This document uses the following definitions: +.ie t .DS +.el .DS L +.ft CW +#define bool_t int +#define TRUE 1 +#define FALSE 0 +.DE +.LP +Keeping these conventions in mind, +.I xdr_gnumbers() +can be rewritten as follows: +.ie t .DS +.el .DS L +.ft CW +xdr_gnumbers(xdrs, gp) + XDR *xdrs; + struct gnumbers *gp; +{ + return(xdr_long(xdrs, &gp->g_assets) && + xdr_long(xdrs, &gp->g_liabilities)); +} +.DE +This document uses both coding styles. +.NH 1 +\&XDR Library Primitives +.IX "library primitives for XDR" +.IX XDR "library primitives" +.LP +This section gives a synopsis of each XDR primitive. +It starts with basic data types and moves on to constructed data types. +Finally, XDR utilities are discussed. +The interface to these primitives +and utilities is defined in the include file +.I <rpc/xdr.h> , +automatically included by +.I <rpc/rpc.h> . +.NH 2 +\&Number Filters +.IX "XDR library" "number filters" +.LP +The XDR library provides primitives to translate between numbers +and their corresponding external representations. +Primitives cover the set of numbers in: +.DS +.ft CW +[signed, unsigned] * [short, int, long] +.DE +.ne 2i +Specifically, the eight primitives are: +.DS +.ft CW +bool_t xdr_char(xdrs, cp) + XDR *xdrs; + char *cp; +.sp .5 +bool_t xdr_u_char(xdrs, ucp) + XDR *xdrs; + unsigned char *ucp; +.sp .5 +bool_t xdr_int(xdrs, ip) + XDR *xdrs; + int *ip; +.sp .5 +bool_t xdr_u_int(xdrs, up) + XDR *xdrs; + unsigned *up; +.sp .5 +bool_t xdr_long(xdrs, lip) + XDR *xdrs; + long *lip; +.sp .5 +bool_t xdr_u_long(xdrs, lup) + XDR *xdrs; + u_long *lup; +.sp .5 +bool_t xdr_short(xdrs, sip) + XDR *xdrs; + short *sip; +.sp .5 +bool_t xdr_u_short(xdrs, sup) + XDR *xdrs; + u_short *sup; +.DE +The first parameter, +.I xdrs , +is an XDR stream handle. +The second parameter is the address of the number +that provides data to the stream or receives data from it. +All routines return +.I TRUE +if they complete successfully, and +.I FALSE +otherwise. +.NH 2 +\&Floating Point Filters +.IX "XDR library" "floating point filters" +.LP +The XDR library also provides primitive routines +for C's floating point types: +.DS +.ft CW +bool_t xdr_float(xdrs, fp) + XDR *xdrs; + float *fp; +.sp .5 +bool_t xdr_double(xdrs, dp) + XDR *xdrs; + double *dp; +.DE +The first parameter, +.I xdrs +is an XDR stream handle. +The second parameter is the address +of the floating point number that provides data to the stream +or receives data from it. +Both routines return +.I TRUE +if they complete successfully, and +.I FALSE +otherwise. +.LP +Note: Since the numbers are represented in IEEE floating point, +routines may fail when decoding a valid IEEE representation +into a machine-specific representation, or vice-versa. +.NH 2 +\&Enumeration Filters +.IX "XDR library" "enumeration filters" +.LP +The XDR library provides a primitive for generic enumerations. +The primitive assumes that a C +.I enum +has the same representation inside the machine as a C integer. +The boolean type is an important instance of the +.I enum . +The external representation of a boolean is always +.I TRUE +(1) or +.I FALSE +(0). +.DS +.ft CW +#define bool_t int +#define FALSE 0 +#define TRUE 1 +.sp .5 +#define enum_t int +.sp .5 +bool_t xdr_enum(xdrs, ep) + XDR *xdrs; + enum_t *ep; +.sp .5 +bool_t xdr_bool(xdrs, bp) + XDR *xdrs; + bool_t *bp; +.DE +The second parameters +.I ep +and +.I bp +are addresses of the associated type that provides data to, or +receives data from, the stream +.I xdrs . +.NH 2 +\&No Data +.IX "XDR library" "no data" +.LP +Occasionally, an XDR routine must be supplied to the RPC system, +even when no data is passed or required. +The library provides such a routine: +.DS +.ft CW +bool_t xdr_void(); /* \fIalways returns TRUE\fP */ +.DE +.NH 2 +\&Constructed Data Type Filters +.IX "XDR library" "constructed data type filters" +.LP +Constructed or compound data type primitives +require more parameters and perform more complicated functions +then the primitives discussed above. +This section includes primitives for +strings, arrays, unions, and pointers to structures. +.LP +Constructed data type primitives may use memory management. +In many cases, memory is allocated when deserializing data with +.I XDR_DECODE +Therefore, the XDR package must provide means to deallocate memory. +This is done by an XDR operation, +.I XDR_FREE +To review, the three XDR directional operations are +.I XDR_ENCODE , +.I XDR_DECODE +and +.I XDR_FREE . +.NH 3 +\&Strings +.IX "XDR library" "strings" +.LP +In C, a string is defined as a sequence of bytes +terminated by a null byte, +which is not considered when calculating string length. +However, when a string is passed or manipulated, +a pointer to it is employed. +Therefore, the XDR library defines a string to be a +.I "char *" +and not a sequence of characters. +The external representation of a string is drastically different +from its internal representation. +Externally, strings are represented as +sequences of ASCII characters, +while internally, they are represented with character pointers. +Conversion between the two representations +is accomplished with the routine +.I xdr_string (): +.IX xdr_string() "" \fIxdr_string()\fP +.DS +.ft CW +bool_t xdr_string(xdrs, sp, maxlength) + XDR *xdrs; + char **sp; + u_int maxlength; +.DE +The first parameter +.I xdrs +is the XDR stream handle. +The second parameter +.I sp +is a pointer to a string (type +.I "char **" . +The third parameter +.I maxlength +specifies the maximum number of bytes allowed during encoding or decoding. +its value is usually specified by a protocol. For example, a protocol +specification may say that a file name may be no longer than 255 characters. +.LP +The routine returns +.I FALSE +if the number of characters exceeds +.I maxlength , +and +.I TRUE +if it doesn't. +.SH +Keep +.I maxlength +small. If it is too big you can blow the heap, since +.I xdr_string() +will call +.I malloc() +for space. +.LP +The behavior of +.I xdr_string() +.IX xdr_string() "" \fIxdr_string()\fP +is similar to the behavior of other routines +discussed in this section. The direction +.I XDR_ENCODE +is easiest to understand. The parameter +.I sp +points to a string of a certain length; +if the string does not exceed +.I maxlength , +the bytes are serialized. +.LP +The effect of deserializing a string is subtle. +First the length of the incoming string is determined; +it must not exceed +.I maxlength . +Next +.I sp +is dereferenced; if the value is +.I NULL , +then a string of the appropriate length is allocated and +.I *sp +is set to this string. +If the original value of +.I *sp +is non-null, then the XDR package assumes +that a target area has been allocated, +which can hold strings no longer than +.I maxlength . +In either case, the string is decoded into the target area. +The routine then appends a null character to the string. +.LP +In the +.I XDR_FREE +operation, the string is obtained by dereferencing +.I sp . +If the string is not +.I NULL , +it is freed and +.I *sp +is set to +.I NULL . +In this operation, +.I xdr_string() +ignores the +.I maxlength +parameter. +.NH 3 +\&Byte Arrays +.IX "XDR library" "byte arrays" +.LP +Often variable-length arrays of bytes are preferable to strings. +Byte arrays differ from strings in the following three ways: +1) the length of the array (the byte count) is explicitly +located in an unsigned integer, +2) the byte sequence is not terminated by a null character, and +3) the external representation of the bytes is the same as their +internal representation. +The primitive +.I xdr_bytes() +.IX xdr_bytes() "" \fIxdr_bytes()\fP +converts between the internal and external +representations of byte arrays: +.DS +.ft CW +bool_t xdr_bytes(xdrs, bpp, lp, maxlength) + XDR *xdrs; + char **bpp; + u_int *lp; + u_int maxlength; +.DE +The usage of the first, second and fourth parameters +are identical to the first, second and third parameters of +.I xdr_string (), +respectively. +The length of the byte area is obtained by dereferencing +.I lp +when serializing; +.I *lp +is set to the byte length when deserializing. +.NH 3 +\&Arrays +.IX "XDR library" "arrays" +.LP +The XDR library package provides a primitive +for handling arrays of arbitrary elements. +The +.I xdr_bytes() +routine treats a subset of generic arrays, +in which the size of array elements is known to be 1, +and the external description of each element is built-in. +The generic array primitive, +.I xdr_array() , +.IX xdr_array() "" \fIxdr_array()\fP +requires parameters identical to those of +.I xdr_bytes() +plus two more: +the size of array elements, +and an XDR routine to handle each of the elements. +This routine is called to encode or decode +each element of the array. +.DS +.ft CW +bool_t +xdr_array(xdrs, ap, lp, maxlength, elementsiz, xdr_element) + XDR *xdrs; + char **ap; + u_int *lp; + u_int maxlength; + u_int elementsiz; + bool_t (*xdr_element)(); +.DE +The parameter +.I ap +is the address of the pointer to the array. +If +.I *ap +is +.I NULL +when the array is being deserialized, +XDR allocates an array of the appropriate size and sets +.I *ap +to that array. +The element count of the array is obtained from +.I *lp +when the array is serialized; +.I *lp +is set to the array length when the array is deserialized. +The parameter +.I maxlength +is the maximum number of elements that the array is allowed to have; +.I elementsiz +is the byte size of each element of the array +(the C function +.I sizeof() +can be used to obtain this value). +The +.I xdr_element() +.IX xdr_element() "" \fIxdr_element()\fP +routine is called to serialize, deserialize, or free +each element of the array. +.br +.LP +Before defining more constructed data types, it is appropriate to +present three examples. +.LP +.I "Example A:" +.br +A user on a networked machine can be identified by +(a) the machine name, such as +.I krypton : +see the +.I gethostname +man page; (b) the user's UID: see the +.I geteuid +man page; and (c) the group numbers to which the user belongs: +see the +.I getgroups +man page. A structure with this information and its associated +XDR routine could be coded like this: +.ie t .DS +.el .DS L +.ft CW +struct netuser { + char *nu_machinename; + int nu_uid; + u_int nu_glen; + int *nu_gids; +}; +#define NLEN 255 /* \fImachine names < 256 chars\fP */ +#define NGRPS 20 /* \fIuser can't be in > 20 groups\fP */ +.sp .5 +bool_t +xdr_netuser(xdrs, nup) + XDR *xdrs; + struct netuser *nup; +{ + return(xdr_string(xdrs, &nup->nu_machinename, NLEN) && + xdr_int(xdrs, &nup->nu_uid) && + xdr_array(xdrs, &nup->nu_gids, &nup->nu_glen, + NGRPS, sizeof (int), xdr_int)); +} +.DE +.LP +.I "Example B:" +.br +A party of network users could be implemented +as an array of +.I netuser +structure. +The declaration and its associated XDR routines +are as follows: +.ie t .DS +.el .DS L +.ft CW +struct party { + u_int p_len; + struct netuser *p_nusers; +}; +#define PLEN 500 /* \fImax number of users in a party\fP */ +.sp .5 +bool_t +xdr_party(xdrs, pp) + XDR *xdrs; + struct party *pp; +{ + return(xdr_array(xdrs, &pp->p_nusers, &pp->p_len, PLEN, + sizeof (struct netuser), xdr_netuser)); +} +.DE +.LP +.I "Example C:" +.br +The well-known parameters to +.I main , +.I argc +and +.I argv +can be combined into a structure. +An array of these structures can make up a history of commands. +The declarations and XDR routines might look like: +.ie t .DS +.el .DS L +.ft CW +struct cmd { + u_int c_argc; + char **c_argv; +}; +#define ALEN 1000 /* \fIargs cannot be > 1000 chars\fP */ +#define NARGC 100 /* \fIcommands cannot have > 100 args\fP */ + +struct history { + u_int h_len; + struct cmd *h_cmds; +}; +#define NCMDS 75 /* \fIhistory is no more than 75 commands\fP */ + +bool_t +xdr_wrap_string(xdrs, sp) + XDR *xdrs; + char **sp; +{ + return(xdr_string(xdrs, sp, ALEN)); +} +.DE +.ie t .DS +.el .DS L +.ft CW +bool_t +xdr_cmd(xdrs, cp) + XDR *xdrs; + struct cmd *cp; +{ + return(xdr_array(xdrs, &cp->c_argv, &cp->c_argc, NARGC, + sizeof (char *), xdr_wrap_string)); +} +.DE +.ie t .DS +.el .DS L +.ft CW +bool_t +xdr_history(xdrs, hp) + XDR *xdrs; + struct history *hp; +{ + return(xdr_array(xdrs, &hp->h_cmds, &hp->h_len, NCMDS, + sizeof (struct cmd), xdr_cmd)); +} +.DE +The most confusing part of this example is that the routine +.I xdr_wrap_string() +is needed to package the +.I xdr_string() +routine, because the implementation of +.I xdr_array() +only passes two parameters to the array element description routine; +.I xdr_wrap_string() +supplies the third parameter to +.I xdr_string (). +.LP +By now the recursive nature of the XDR library should be obvious. +Let's continue with more constructed data types. +.NH 3 +\&Opaque Data +.IX "XDR library" "opaque data" +.LP +In some protocols, handles are passed from a server to client. +The client passes the handle back to the server at some later time. +Handles are never inspected by clients; +they are obtained and submitted. +That is to say, handles are opaque. +The +.I xdr_opaque() +.IX xdr_opaque() "" \fIxdr_opaque()\fP +primitive is used for describing fixed sized, opaque bytes. +.DS +.ft CW +bool_t xdr_opaque(xdrs, p, len) + XDR *xdrs; + char *p; + u_int len; +.DE +The parameter +.I p +is the location of the bytes; +.I len +is the number of bytes in the opaque object. +By definition, the actual data +contained in the opaque object are not machine portable. +.NH 3 +\&Fixed Sized Arrays +.IX "XDR library" "fixed sized arrays" +.LP +The XDR library provides a primitive, +.I xdr_vector (), +for fixed-length arrays. +.ie t .DS +.el .DS L +.ft CW +#define NLEN 255 /* \fImachine names must be < 256 chars\fP */ +#define NGRPS 20 /* \fIuser belongs to exactly 20 groups\fP */ +.sp .5 +struct netuser { + char *nu_machinename; + int nu_uid; + int nu_gids[NGRPS]; +}; +.sp .5 +bool_t +xdr_netuser(xdrs, nup) + XDR *xdrs; + struct netuser *nup; +{ + int i; +.sp .5 + if (!xdr_string(xdrs, &nup->nu_machinename, NLEN)) + return(FALSE); + if (!xdr_int(xdrs, &nup->nu_uid)) + return(FALSE); + if (!xdr_vector(xdrs, nup->nu_gids, NGRPS, sizeof(int), + xdr_int)) { + return(FALSE); + } + return(TRUE); +} +.DE +.NH 3 +\&Discriminated Unions +.IX "XDR library" "discriminated unions" +.LP +The XDR library supports discriminated unions. +A discriminated union is a C union and an +.I enum_t +value that selects an \*Qarm\*U of the union. +.DS +.ft CW +struct xdr_discrim { + enum_t value; + bool_t (*proc)(); +}; +.sp .5 +bool_t xdr_union(xdrs, dscmp, unp, arms, defaultarm) + XDR *xdrs; + enum_t *dscmp; + char *unp; + struct xdr_discrim *arms; + bool_t (*defaultarm)(); /* \fImay equal NULL\fP */ +.DE +First the routine translates the discriminant of the union located at +.I *dscmp . +The discriminant is always an +.I enum_t . +Next the union located at +.I *unp +is translated. +The parameter +.I arms +is a pointer to an array of +.I xdr_discrim +structures. +Each structure contains an ordered pair of +.I [value,proc] . +If the union's discriminant is equal to the associated +.I value , +then the +.I proc +is called to translate the union. +The end of the +.I xdr_discrim +structure array is denoted by a routine of value +.I NULL +(0). If the discriminant is not found in the +.I arms +array, then the +.I defaultarm +procedure is called if it is non-null; +otherwise the routine returns +.I FALSE . +.LP +.I "Example D:" +Suppose the type of a union may be integer, +character pointer (a string), or a +.I gnumbers +structure. +Also, assume the union and its current type +are declared in a structure. +The declaration is: +.ie t .DS +.el .DS L +.ft CW +enum utype { INTEGER=1, STRING=2, GNUMBERS=3 }; +.sp .5 +struct u_tag { + enum utype utype; /* \fIthe union's discriminant\fP */ + union { + int ival; + char *pval; + struct gnumbers gn; + } uval; +}; +.DE +The following constructs and XDR procedure (de)serialize +the discriminated union: +.ie t .DS +.el .DS L +.ft CW +struct xdr_discrim u_tag_arms[4] = { + { INTEGER, xdr_int }, + { GNUMBERS, xdr_gnumbers } + { STRING, xdr_wrap_string }, + { __dontcare__, NULL } + /* \fIalways terminate arms with a NULL xdr_proc\fP */ +} +.sp .5 +bool_t +xdr_u_tag(xdrs, utp) + XDR *xdrs; + struct u_tag *utp; +{ + return(xdr_union(xdrs, &utp->utype, &utp->uval, + u_tag_arms, NULL)); +} +.DE +The routine +.I xdr_gnumbers() +was presented above in +.I "The XDR Library" +section. +.I xdr_wrap_string() +was presented in example C. +The default +.I arm +parameter to +.I xdr_union() +(the last parameter) is +.I NULL +in this example. Therefore the value of the union's discriminant +may legally take on only values listed in the +.I u_tag_arms +array. This example also demonstrates that +the elements of the arm's array do not need to be sorted. +.LP +It is worth pointing out that the values of the discriminant +may be sparse, though in this example they are not. +It is always good +practice to assign explicitly integer values to each element of the +discriminant's type. +This practice both documents the external +representation of the discriminant and guarantees that different +C compilers emit identical discriminant values. +.LP +Exercise: Implement +.I xdr_union() +using the other primitives in this section. +.NH 3 +\&Pointers +.IX "XDR library" "pointers" +.LP +In C it is often convenient to put pointers +to another structure within a structure. +The +.I xdr_reference() +.IX xdr_reference() "" \fIxdr_reference()\fP +primitive makes it easy to serialize, deserialize, and free +these referenced structures. +.DS +.ft CW +bool_t xdr_reference(xdrs, pp, size, proc) + XDR *xdrs; + char **pp; + u_int ssize; + bool_t (*proc)(); +.DE +.LP +Parameter +.I pp +is the address of +the pointer to the structure; +parameter +.I ssize +is the size in bytes of the structure (use the C function +.I sizeof() +to obtain this value); and +.I proc +is the XDR routine that describes the structure. +When decoding data, storage is allocated if +.I *pp +is +.I NULL . +.LP +There is no need for a primitive +.I xdr_struct() +to describe structures within structures, +because pointers are always sufficient. +.LP +Exercise: Implement +.I xdr_reference() +using +.I xdr_array (). +Warning: +.I xdr_reference() +and +.I xdr_array() +are NOT interchangeable external representations of data. +.LP +.I "Example E:" +Suppose there is a structure containing a person's name +and a pointer to a +.I gnumbers +structure containing the person's gross assets and liabilities. +The construct is: +.DS +.ft CW +struct pgn { + char *name; + struct gnumbers *gnp; +}; +.DE +The corresponding XDR routine for this structure is: +.DS +.ft CW +bool_t +xdr_pgn(xdrs, pp) + XDR *xdrs; + struct pgn *pp; +{ + if (xdr_string(xdrs, &pp->name, NLEN) && + xdr_reference(xdrs, &pp->gnp, + sizeof(struct gnumbers), xdr_gnumbers)) + return(TRUE); + return(FALSE); +} +.DE +.IX "pointer semantics and XDR" +.I "Pointer Semantics and XDR" +.LP +In many applications, C programmers attach double meaning to +the values of a pointer. Typically the value +.I NULL +(or zero) means data is not needed, +yet some application-specific interpretation applies. +In essence, the C programmer is encoding +a discriminated union efficiently +by overloading the interpretation of the value of a pointer. +For instance, in example E a +.I NULL +pointer value for +.I gnp +could indicate that +the person's assets and liabilities are unknown. +That is, the pointer value encodes two things: +whether or not the data is known; +and if it is known, where it is located in memory. +Linked lists are an extreme example of the use +of application-specific pointer interpretation. +.LP +The primitive +.I xdr_reference() +.IX xdr_reference() "" \fIxdr_reference()\fP +cannot and does not attach any special +meaning to a null-value pointer during serialization. +That is, passing an address of a pointer whose value is +.I NULL +to +.I xdr_reference() +when serialing data will most likely cause a memory fault and, on the UNIX +system, a core dump. +.LP +.I xdr_pointer() +correctly handles +.I NULL +pointers. For more information about its use, see +the +.I "Linked Lists" +topics below. +.LP +.I Exercise: +After reading the section on +.I "Linked Lists" , +return here and extend example E so that +it can correctly deal with +.I NULL +pointer values. +.LP +.I Exercise: +Using the +.I xdr_union (), +.I xdr_reference() +and +.I xdr_void() +primitives, implement a generic pointer handling primitive +that implicitly deals with +.I NULL +pointers. That is, implement +.I xdr_pointer (). +.NH 2 +\&Non-filter Primitives +.IX "XDR" "non-filter primitives" +.LP +XDR streams can be manipulated with +the primitives discussed in this section. +.DS +.ft CW +u_int xdr_getpos(xdrs) + XDR *xdrs; +.sp .5 +bool_t xdr_setpos(xdrs, pos) + XDR *xdrs; + u_int pos; +.sp .5 +xdr_destroy(xdrs) + XDR *xdrs; +.DE +The routine +.I xdr_getpos() +.IX xdr_getpos() "" \fIxdr_getpos()\fP +returns an unsigned integer +that describes the current position in the data stream. +Warning: In some XDR streams, the returned value of +.I xdr_getpos() +is meaningless; +the routine returns a \-1 in this case +(though \-1 should be a legitimate value). +.LP +The routine +.I xdr_setpos() +.IX xdr_setpos() "" \fIxdr_setpos()\fP +sets a stream position to +.I pos . +Warning: In some XDR streams, setting a position is impossible; +in such cases, +.I xdr_setpos() +will return +.I FALSE . +This routine will also fail if the requested position is out-of-bounds. +The definition of bounds varies from stream to stream. +.LP +The +.I xdr_destroy() +.IX xdr_destroy() "" \fIxdr_destroy()\fP +primitive destroys the XDR stream. +Usage of the stream +after calling this routine is undefined. +.NH 2 +\&XDR Operation Directions +.IX XDR "operation directions" +.IX "direction of XDR operations" +.LP +At times you may wish to optimize XDR routines by taking +advantage of the direction of the operation \(em +.I XDR_ENCODE +.I XDR_DECODE +or +.I XDR_FREE +The value +.I xdrs->x_op +always contains the direction of the XDR operation. +Programmers are not encouraged to take advantage of this information. +Therefore, no example is presented here. However, an example in the +.I "Linked Lists" +topic below, demonstrates the usefulness of the +.I xdrs->x_op +field. +.NH 2 +\&XDR Stream Access +.IX "XDR" "stream access" +.LP +An XDR stream is obtained by calling the appropriate creation routine. +These creation routines take arguments that are tailored to the +specific properties of the stream. +.LP +Streams currently exist for (de)serialization of data to or from +standard I/O +.I FILE +streams, TCP/IP connections and UNIX files, and memory. +.NH 3 +\&Standard I/O Streams +.IX "XDR" "standard I/O streams" +.LP +XDR streams can be interfaced to standard I/O using the +.I xdrstdio_create() +.IX xdrstdio_create() "" \fIxdrstdio_create()\fP +routine as follows: +.DS +.ft CW +#include <stdio.h> +#include <rpc/rpc.h> /* \fIxdr streams part of rpc\fP */ +.sp .5 +void +xdrstdio_create(xdrs, fp, x_op) + XDR *xdrs; + FILE *fp; + enum xdr_op x_op; +.DE +The routine +.I xdrstdio_create() +initializes an XDR stream pointed to by +.I xdrs . +The XDR stream interfaces to the standard I/O library. +Parameter +.I fp +is an open file, and +.I x_op +is an XDR direction. +.NH 3 +\&Memory Streams +.IX "XDR" "memory streams" +.LP +Memory streams allow the streaming of data into or out of +a specified area of memory: +.DS +.ft CW +#include <rpc/rpc.h> +.sp .5 +void +xdrmem_create(xdrs, addr, len, x_op) + XDR *xdrs; + char *addr; + u_int len; + enum xdr_op x_op; +.DE +The routine +.I xdrmem_create() +.IX xdrmem_create() "" \fIxdrmem_create()\fP +initializes an XDR stream in local memory. +The memory is pointed to by parameter +.I addr ; +parameter +.I len +is the length in bytes of the memory. +The parameters +.I xdrs +and +.I x_op +are identical to the corresponding parameters of +.I xdrstdio_create (). +Currently, the UDP/IP implementation of RPC uses +.I xdrmem_create (). +Complete call or result messages are built in memory before calling the +.I sendto() +system routine. +.NH 3 +\&Record (TCP/IP) Streams +.IX "XDR" "record (TCP/IP) streams" +.LP +A record stream is an XDR stream built on top of +a record marking standard that is built on top of the +UNIX file or 4.2 BSD connection interface. +.DS +.ft CW +#include <rpc/rpc.h> /* \fIxdr streams part of rpc\fP */ +.sp .5 +xdrrec_create(xdrs, + sendsize, recvsize, iohandle, readproc, writeproc) + XDR *xdrs; + u_int sendsize, recvsize; + char *iohandle; + int (*readproc)(), (*writeproc)(); +.DE +The routine +.I xdrrec_create() +provides an XDR stream interface that allows for a bidirectional, +arbitrarily long sequence of records. +The contents of the records are meant to be data in XDR form. +The stream's primary use is for interfacing RPC to TCP connections. +However, it can be used to stream data into or out of normal +UNIX files. +.LP +The parameter +.I xdrs +is similar to the corresponding parameter described above. +The stream does its own data buffering similar to that of standard I/O. +The parameters +.I sendsize +and +.I recvsize +determine the size in bytes of the output and input buffers, respectively; +if their values are zero (0), then predetermined defaults are used. +When a buffer needs to be filled or flushed, the routine +.I readproc() +or +.I writeproc() +is called, respectively. +The usage and behavior of these +routines are similar to the UNIX system calls +.I read() +and +.I write (). +However, +the first parameter to each of these routines is the opaque parameter +.I iohandle . +The other two parameters +.I buf "" +and +.I nbytes ) +and the results +(byte count) are identical to the system routines. +If +.I xxx +is +.I readproc() +or +.I writeproc (), +then it has the following form: +.DS +.ft CW +.ft I +/* + * returns the actual number of bytes transferred. + * -1 is an error + */ +.ft CW +int +xxx(iohandle, buf, len) + char *iohandle; + char *buf; + int nbytes; +.DE +The XDR stream provides means for delimiting records in the byte stream. +The implementation details of delimiting records in a stream are +discussed in the +.I "Advanced Topics" +topic below. +The primitives that are specific to record streams are as follows: +.DS +.ft CW +bool_t +xdrrec_endofrecord(xdrs, flushnow) + XDR *xdrs; + bool_t flushnow; +.sp .5 +bool_t +xdrrec_skiprecord(xdrs) + XDR *xdrs; +.sp .5 +bool_t +xdrrec_eof(xdrs) + XDR *xdrs; +.DE +The routine +.I xdrrec_endofrecord() +.IX xdrrec_endofrecord() "" \fIxdrrec_endofrecord()\fP +causes the current outgoing data to be marked as a record. +If the parameter +.I flushnow +is +.I TRUE , +then the stream's +.I writeproc +will be called; otherwise, +.I writeproc +will be called when the output buffer has been filled. +.LP +The routine +.I xdrrec_skiprecord() +.IX xdrrec_skiprecord() "" \fIxdrrec_skiprecord()\fP +causes an input stream's position to be moved past +the current record boundary and onto the +beginning of the next record in the stream. +.LP +If there is no more data in the stream's input buffer, +then the routine +.I xdrrec_eof() +.IX xdrrec_eof() "" \fIxdrrec_eof()\fP +returns +.I TRUE . +That is not to say that there is no more data +in the underlying file descriptor. +.NH 2 +\&XDR Stream Implementation +.IX "XDR" "stream implementation" +.IX "stream implementation in XDR" +.LP +This section provides the abstract data types needed +to implement new instances of XDR streams. +.NH 3 +\&The XDR Object +.IX "XDR" "object" +.LP +The following structure defines the interface to an XDR stream: +.ie t .DS +.el .DS L +.ft CW +enum xdr_op { XDR_ENCODE=0, XDR_DECODE=1, XDR_FREE=2 }; +.sp .5 +typedef struct { + enum xdr_op x_op; /* \fIoperation; fast added param\fP */ + struct xdr_ops { + bool_t (*x_getlong)(); /* \fIget long from stream\fP */ + bool_t (*x_putlong)(); /* \fIput long to stream\fP */ + bool_t (*x_getbytes)(); /* \fIget bytes from stream\fP */ + bool_t (*x_putbytes)(); /* \fIput bytes to stream\fP */ + u_int (*x_getpostn)(); /* \fIreturn stream offset\fP */ + bool_t (*x_setpostn)(); /* \fIreposition offset\fP */ + caddr_t (*x_inline)(); /* \fIptr to buffered data\fP */ + VOID (*x_destroy)(); /* \fIfree private area\fP */ + } *x_ops; + caddr_t x_public; /* \fIusers' data\fP */ + caddr_t x_private; /* \fIpointer to private data\fP */ + caddr_t x_base; /* \fIprivate for position info\fP */ + int x_handy; /* \fIextra private word\fP */ +} XDR; +.DE +The +.I x_op +field is the current operation being performed on the stream. +This field is important to the XDR primitives, +but should not affect a stream's implementation. +That is, a stream's implementation should not depend +on this value. +The fields +.I x_private , +.I x_base , +and +.I x_handy +are private to the particular +stream's implementation. +The field +.I x_public +is for the XDR client and should never be used by +the XDR stream implementations or the XDR primitives. +.I x_getpostn() , +.I x_setpostn() +and +.I x_destroy() +are macros for accessing operations. The operation +.I x_inline() +takes two parameters: +an XDR *, and an unsigned integer, which is a byte count. +The routine returns a pointer to a piece of +the stream's internal buffer. +The caller can then use the buffer segment for any purpose. +From the stream's point of view, the bytes in the +buffer segment have been consumed or put. +The routine may return +.I NULL +if it cannot return a buffer segment of the requested size. +(The +.I x_inline() +routine is for cycle squeezers. +Use of the resulting buffer is not data-portable. +Users are encouraged not to use this feature.) +.LP +The operations +.I x_getbytes() +and +.I x_putbytes() +blindly get and put sequences of bytes +from or to the underlying stream; +they return +.I TRUE +if they are successful, and +.I FALSE +otherwise. The routines have identical parameters (replace +.I xxx ): +.DS +.ft CW +bool_t +xxxbytes(xdrs, buf, bytecount) + XDR *xdrs; + char *buf; + u_int bytecount; +.DE +The operations +.I x_getlong() +and +.I x_putlong() +receive and put +long numbers from and to the data stream. +It is the responsibility of these routines +to translate the numbers between the machine representation +and the (standard) external representation. +The UNIX primitives +.I htonl() +and +.I ntohl() +can be helpful in accomplishing this. +The higher-level XDR implementation assumes that +signed and unsigned long integers contain the same number of bits, +and that nonnegative integers +have the same bit representations as unsigned integers. +The routines return +.I TRUE +if they succeed, and +.I FALSE +otherwise. They have identical parameters: +.DS +.ft CW +bool_t +xxxlong(xdrs, lp) + XDR *xdrs; + long *lp; +.DE +Implementors of new XDR streams must make an XDR structure +(with new operation routines) available to clients, +using some kind of create routine. +.NH 1 +\&Advanced Topics +.IX XDR "advanced topics" +.LP +This section describes techniques for passing data structures that +are not covered in the preceding sections. Such structures include +linked lists (of arbitrary lengths). Unlike the simpler examples +covered in the earlier sections, the following examples are written +using both the XDR C library routines and the XDR data description +language. +The +.I "External Data Representation Standard: Protocol Specification" +describes this +language in complete detail. +.NH 2 +\&Linked Lists +.IX XDR "linked lists" +.LP +The last example in the +.I Pointers +topic earlier in this chapter +presented a C data structure and its associated XDR +routines for an individual's gross assets and liabilities. +The example is duplicated below: +.ie t .DS +.el .DS L +.ft CW +struct gnumbers { + long g_assets; + long g_liabilities; +}; +.sp .5 +bool_t +xdr_gnumbers(xdrs, gp) + XDR *xdrs; + struct gnumbers *gp; +{ + if (xdr_long(xdrs, &(gp->g_assets))) + return(xdr_long(xdrs, &(gp->g_liabilities))); + return(FALSE); +} +.DE +.LP +Now assume that we wish to implement a linked list of such information. +A data structure could be constructed as follows: +.ie t .DS +.el .DS L +.ft CW +struct gnumbers_node { + struct gnumbers gn_numbers; + struct gnumbers_node *gn_next; +}; +.sp .5 +typedef struct gnumbers_node *gnumbers_list; +.DE +.LP +The head of the linked list can be thought of as the data object; +that is, the head is not merely a convenient shorthand for a +structure. Similarly the +.I gn_next +field is used to indicate whether or not the object has terminated. +Unfortunately, if the object continues, the +.I gn_next +field is also the address of where it continues. The link addresses +carry no useful information when the object is serialized. +.LP +The XDR data description of this linked list is described by the +recursive declaration of +.I gnumbers_list : +.ie t .DS +.el .DS L +.ft CW +struct gnumbers { + int g_assets; + int g_liabilities; +}; +.sp .5 +struct gnumbers_node { + gnumbers gn_numbers; + gnumbers_node *gn_next; +}; +.DE +.LP +In this description, the boolean indicates whether there is more data +following it. If the boolean is +.I FALSE , +then it is the last data field of the structure. If it is +.I TRUE , +then it is followed by a gnumbers structure and (recursively) by a +.I gnumbers_list . +Note that the C declaration has no boolean explicitly declared in it +(though the +.I gn_next +field implicitly carries the information), while the XDR data +description has no pointer explicitly declared in it. +.LP +Hints for writing the XDR routines for a +.I gnumbers_list +follow easily from the XDR description above. Note how the primitive +.I xdr_pointer() +is used to implement the XDR union above. +.ie t .DS +.el .DS L +.ft CW +bool_t +xdr_gnumbers_node(xdrs, gn) + XDR *xdrs; + gnumbers_node *gn; +{ + return(xdr_gnumbers(xdrs, &gn->gn_numbers) && + xdr_gnumbers_list(xdrs, &gp->gn_next)); +} +.sp .5 +bool_t +xdr_gnumbers_list(xdrs, gnp) + XDR *xdrs; + gnumbers_list *gnp; +{ + return(xdr_pointer(xdrs, gnp, + sizeof(struct gnumbers_node), + xdr_gnumbers_node)); +} +.DE +.LP +The unfortunate side effect of XDR'ing a list with these routines +is that the C stack grows linearly with respect to the number of +node in the list. This is due to the recursion. The following +routine collapses the above two mutually recursive into a single, +non-recursive one. +.ie t .DS +.el .DS L +.ft CW +bool_t +xdr_gnumbers_list(xdrs, gnp) + XDR *xdrs; + gnumbers_list *gnp; +{ + bool_t more_data; + gnumbers_list *nextp; +.sp .5 + for (;;) { + more_data = (*gnp != NULL); + if (!xdr_bool(xdrs, &more_data)) { + return(FALSE); + } + if (! more_data) { + break; + } + if (xdrs->x_op == XDR_FREE) { + nextp = &(*gnp)->gn_next; + } + if (!xdr_reference(xdrs, gnp, + sizeof(struct gnumbers_node), xdr_gnumbers)) { + + return(FALSE); + } + gnp = (xdrs->x_op == XDR_FREE) ? + nextp : &(*gnp)->gn_next; + } + *gnp = NULL; + return(TRUE); +} +.DE +.LP +The first task is to find out whether there is more data or not, +so that this boolean information can be serialized. Notice that +this statement is unnecessary in the +.I XDR_DECODE +case, since the value of more_data is not known until we +deserialize it in the next statement. +.LP +The next statement XDR's the more_data field of the XDR union. +Then if there is truly no more data, we set this last pointer to +.I NULL +to indicate the end of the list, and return +.I TRUE +because we are done. Note that setting the pointer to +.I NULL +is only important in the +.I XDR_DECODE +case, since it is already +.I NULL +in the +.I XDR_ENCODE +and +XDR_FREE +cases. +.LP +Next, if the direction is +.I XDR_FREE , +the value of +.I nextp +is set to indicate the location of the next pointer in the list. +We do this now because we need to dereference gnp to find the +location of the next item in the list, and after the next +statement the storage pointed to by +.I gnp +will be freed up and no be longer valid. We can't do this for all +directions though, because in the +.I XDR_DECODE +direction the value of +.I gnp +won't be set until the next statement. +.LP +Next, we XDR the data in the node using the primitive +.I xdr_reference (). +.I xdr_reference() +is like +.I xdr_pointer() +which we used before, but it does not +send over the boolean indicating whether there is more data. +We use it instead of +.I xdr_pointer() +because we have already XDR'd this information ourselves. Notice +that the xdr routine passed is not the same type as an element +in the list. The routine passed is +.I xdr_gnumbers (), +for XDR'ing gnumbers, but each element in the list is actually of +type +.I gnumbers_node . +We don't pass +.I xdr_gnumbers_node() +because it is recursive, and instead use +.I xdr_gnumbers() +which XDR's all of the non-recursive part. Note that this trick +will work only if the +.I gn_numbers +field is the first item in each element, so that their addresses +are identical when passed to +.I xdr_reference (). +.LP +Finally, we update +.I gnp +to point to the next item in the list. If the direction is +.I XDR_FREE , +we set it to the previously saved value, otherwise we can +dereference +.I gnp +to get the proper value. Though harder to understand than the +recursive version, this non-recursive routine is far less likely +to blow the C stack. It will also run more efficiently since +a lot of procedure call overhead has been removed. Most lists +are small though (in the hundreds of items or less) and the +recursive version should be sufficient for them. +.EQ +delim off +.EN diff --git a/share/doc/psd/25.xdrrfc/Makefile b/share/doc/psd/25.xdrrfc/Makefile new file mode 100644 index 000000000000..7045d4572d90 --- /dev/null +++ b/share/doc/psd/25.xdrrfc/Makefile @@ -0,0 +1,8 @@ +# $FreeBSD$ + +VOLUME= psd/25.xdrrfc +SRCS= stubs xdr.rfc.ms +MACROS= -ms +USE_TBL= + +.include <bsd.doc.mk> diff --git a/share/doc/psd/25.xdrrfc/stubs b/share/doc/psd/25.xdrrfc/stubs new file mode 100644 index 000000000000..78b0a2c815d3 --- /dev/null +++ b/share/doc/psd/25.xdrrfc/stubs @@ -0,0 +1,3 @@ +.\" $FreeBSD$ +.\" +.if t .ftr L CR diff --git a/share/doc/psd/25.xdrrfc/xdr.rfc.ms b/share/doc/psd/25.xdrrfc/xdr.rfc.ms new file mode 100644 index 000000000000..480a339bce57 --- /dev/null +++ b/share/doc/psd/25.xdrrfc/xdr.rfc.ms @@ -0,0 +1,1060 @@ +.\" +.\" Must use -- tbl -- with this one +.\" +.\" @(#)xdr.rfc.ms 2.2 88/08/05 4.0 RPCSRC +.\" $FreeBSD$ +.\" +.de BT +.if \\n%=1 .tl ''- % -'' +.. +.ND +.\" prevent excess underlining in nroff +.if n .fp 2 R +.OH 'External Data Representation Standard''Page %' +.EH 'Page %''External Data Representation Standard' +.if \n%=1 .bp +.SH +\&External Data Representation Standard: Protocol Specification +.IX "External Data Representation" +.IX XDR RFC +.IX XDR "protocol specification" +.LP +.NH 0 +\&Status of this Standard +.nr OF 1 +.IX XDR "RFC status" +.LP +Note: This chapter specifies a protocol that Sun Microsystems, Inc., and +others are using. It has been designated RFC1014 by the ARPA Network +Information Center. +.NH 1 +Introduction +\& +.LP +XDR is a standard for the description and encoding of data. It is +useful for transferring data between different computer +architectures, and has been used to communicate data between such +diverse machines as the Sun Workstation, VAX, IBM-PC, and Cray. +XDR fits into the ISO presentation layer, and is roughly analogous in +purpose to X.409, ISO Abstract Syntax Notation. The major difference +between these two is that XDR uses implicit typing, while X.409 uses +explicit typing. +.LP +XDR uses a language to describe data formats. The language can only +be used only to describe data; it is not a programming language. +This language allows one to describe intricate data formats in a +concise manner. The alternative of using graphical representations +(itself an informal language) quickly becomes incomprehensible when +faced with complexity. The XDR language itself is similar to the C +language [1], just as Courier [4] is similar to Mesa. Protocols such +as Sun RPC (Remote Procedure Call) and the NFS (Network File System) +use XDR to describe the format of their data. +.LP +The XDR standard makes the following assumption: that bytes (or +octets) are portable, where a byte is defined to be 8 bits of data. +A given hardware device should encode the bytes onto the various +media in such a way that other hardware devices may decode the bytes +without loss of meaning. For example, the Ethernet standard +suggests that bytes be encoded in "little-endian" style [2], or least +significant bit first. +.NH 2 +\&Basic Block Size +.IX XDR "basic block size" +.IX XDR "block size" +.LP +The representation of all items requires a multiple of four bytes (or +32 bits) of data. The bytes are numbered 0 through n-1. The bytes +are read or written to some byte stream such that byte m always +precedes byte m+1. If the n bytes needed to contain the data are not +a multiple of four, then the n bytes are followed by enough (0 to 3) +residual zero bytes, r, to make the total byte count a multiple of 4. +.LP +We include the familiar graphic box notation for illustration and +comparison. In most illustrations, each box (delimited by a plus +sign at the 4 corners and vertical bars and dashes) depicts a byte. +Ellipses (...) between boxes show zero or more additional bytes where +required. +.ie t .DS +.el .DS L +\fIA Block\fP + +\f(CW+--------+--------+...+--------+--------+...+--------+ +| byte 0 | byte 1 |...|byte n-1| 0 |...| 0 | ++--------+--------+...+--------+--------+...+--------+ +|<-----------n bytes---------->|<------r bytes------>| +|<-----------n+r (where (n+r) mod 4 = 0)>----------->|\fP + +.DE +.NH 1 +\&XDR Data Types +.IX XDR "data types" +.IX "XDR data types" +.LP +Each of the sections that follow describes a data type defined in the +XDR standard, shows how it is declared in the language, and includes +a graphic illustration of its encoding. +.LP +For each data type in the language we show a general paradigm +declaration. Note that angle brackets (< and >) denote +variable length sequences of data and square brackets ([ and ]) denote +fixed-length sequences of data. "n", "m" and "r" denote integers. +For the full language specification and more formal definitions of +terms such as "identifier" and "declaration", refer to +.I "The XDR Language Specification" , +below. +.LP +For some data types, more specific examples are included. +A more extensive example of a data description is in +.I "An Example of an XDR Data Description" +below. +.NH 2 +\&Integer +.IX XDR integer +.LP +An XDR signed integer is a 32-bit datum that encodes an integer in +the range [-2147483648,2147483647]. The integer is represented in +two's complement notation. The most and least significant bytes are +0 and 3, respectively. Integers are declared as follows: +.ie t .DS +.el .DS L +\fIInteger\fP + +\f(CW(MSB) (LSB) ++-------+-------+-------+-------+ +|byte 0 |byte 1 |byte 2 |byte 3 | ++-------+-------+-------+-------+ +<------------32 bits------------>\fP +.DE +.NH 2 +\&Unsigned Integer +.IX XDR "unsigned integer" +.IX XDR "integer, unsigned" +.LP +An XDR unsigned integer is a 32-bit datum that encodes a nonnegative +integer in the range [0,4294967295]. It is represented by an +unsigned binary number whose most and least significant bytes are 0 +and 3, respectively. An unsigned integer is declared as follows: +.ie t .DS +.el .DS L +\fIUnsigned Integer\fP + +\f(CW(MSB) (LSB) ++-------+-------+-------+-------+ +|byte 0 |byte 1 |byte 2 |byte 3 | ++-------+-------+-------+-------+ +<------------32 bits------------>\fP +.DE +.NH 2 +\&Enumeration +.IX XDR enumeration +.LP +Enumerations have the same representation as signed integers. +Enumerations are handy for describing subsets of the integers. +Enumerated data is declared as follows: +.ft CW +.DS +enum { name-identifier = constant, ... } identifier; +.DE +For example, the three colors red, yellow, and blue could be +described by an enumerated type: +.DS +.ft CW +enum { RED = 2, YELLOW = 3, BLUE = 5 } colors; +.DE +It is an error to encode as an enum any other integer than those that +have been given assignments in the enum declaration. +.NH 2 +\&Boolean +.IX XDR boolean +.LP +Booleans are important enough and occur frequently enough to warrant +their own explicit type in the standard. Booleans are declared as +follows: +.DS +.ft CW +bool identifier; +.DE +This is equivalent to: +.DS +.ft CW +enum { FALSE = 0, TRUE = 1 } identifier; +.DE +.NH 2 +\&Hyper Integer and Unsigned Hyper Integer +.IX XDR "hyper integer" +.IX XDR "integer, hyper" +.LP +The standard also defines 64-bit (8-byte) numbers called hyper +integer and unsigned hyper integer. Their representations are the +obvious extensions of integer and unsigned integer defined above. +They are represented in two's complement notation. The most and +least significant bytes are 0 and 7, respectively. Their +declarations: +.ie t .DS +.el .DS L +\fIHyper Integer\fP +\fIUnsigned Hyper Integer\fP + +\f(CW(MSB) (LSB) ++-------+-------+-------+-------+-------+-------+-------+-------+ +|byte 0 |byte 1 |byte 2 |byte 3 |byte 4 |byte 5 |byte 6 |byte 7 | ++-------+-------+-------+-------+-------+-------+-------+-------+ +<----------------------------64 bits---------------------------->\fP +.DE +.NH 2 +\&Floating-point +.IX XDR "integer, floating point" +.IX XDR "floating-point integer" +.LP +The standard defines the floating-point data type "float" (32 bits or +4 bytes). The encoding used is the IEEE standard for normalized +single-precision floating-point numbers [3]. The following three +fields describe the single-precision floating-point number: +.RS +.IP \fBS\fP: +The sign of the number. Values 0 and 1 represent positive and +negative, respectively. One bit. +.IP \fBE\fP: +The exponent of the number, base 2. 8 bits are devoted to this +field. The exponent is biased by 127. +.IP \fBF\fP: +The fractional part of the number's mantissa, base 2. 23 bits +are devoted to this field. +.RE +.LP +Therefore, the floating-point number is described by: +.DS +(-1)**S * 2**(E-Bias) * 1.F +.DE +It is declared as follows: +.ie t .DS +.el .DS L +\fISingle-Precision Floating-Point\fP + +\f(CW+-------+-------+-------+-------+ +|byte 0 |byte 1 |byte 2 |byte 3 | +S| E | F | ++-------+-------+-------+-------+ +1|<- 8 ->|<-------23 bits------>| +<------------32 bits------------>\fP +.DE +Just as the most and least significant bytes of a number are 0 and 3, +the most and least significant bits of a single-precision floating- +point number are 0 and 31. The beginning bit (and most significant +bit) offsets of S, E, and F are 0, 1, and 9, respectively. Note that +these numbers refer to the mathematical positions of the bits, and +NOT to their actual physical locations (which vary from medium to +medium). +.LP +The IEEE specifications should be consulted concerning the encoding +for signed zero, signed infinity (overflow), and denormalized numbers +(underflow) [3]. According to IEEE specifications, the "NaN" (not a +number) is system dependent and should not be used externally. +.NH 2 +\&Double-precision Floating-point +.IX XDR "integer, double-precision floating point" +.IX XDR "double-precision floating-point integer" +.LP +The standard defines the encoding for the double-precision floating- +point data type "double" (64 bits or 8 bytes). The encoding used is +the IEEE standard for normalized double-precision floating-point +numbers [3]. The standard encodes the following three fields, which +describe the double-precision floating-point number: +.RS +.IP \fBS\fP: +The sign of the number. Values 0 and 1 represent positive and +negative, respectively. One bit. +.IP \fBE\fP: +The exponent of the number, base 2. 11 bits are devoted to this +field. The exponent is biased by 1023. +.IP \fBF\fP: +The fractional part of the number's mantissa, base 2. 52 bits +are devoted to this field. +.RE +.LP +Therefore, the floating-point number is described by: +.DS +(-1)**S * 2**(E-Bias) * 1.F +.DE +It is declared as follows: +.ie t .DS +.el .DS L +\fIDouble-Precision Floating-Point\fP + +\f(CW+------+------+------+------+------+------+------+------+ +|byte 0|byte 1|byte 2|byte 3|byte 4|byte 5|byte 6|byte 7| +S| E | F | ++------+------+------+------+------+------+------+------+ +1|<--11-->|<-----------------52 bits------------------->| +<-----------------------64 bits------------------------->\fP +.DE +Just as the most and least significant bytes of a number are 0 and 3, +the most and least significant bits of a double-precision floating- +point number are 0 and 63. The beginning bit (and most significant +bit) offsets of S, E , and F are 0, 1, and 12, respectively. Note +that these numbers refer to the mathematical positions of the bits, +and NOT to their actual physical locations (which vary from medium to +medium). +.LP +The IEEE specifications should be consulted concerning the encoding +for signed zero, signed infinity (overflow), and denormalized numbers +(underflow) [3]. According to IEEE specifications, the "NaN" (not a +number) is system dependent and should not be used externally. +.NH 2 +\&Fixed-length Opaque Data +.IX XDR "fixed-length opaque data" +.IX XDR "opaque data, fixed length" +.LP +At times, fixed-length uninterpreted data needs to be passed among +machines. This data is called "opaque" and is declared as follows: +.DS +.ft CW +opaque identifier[n]; +.DE +where the constant n is the (static) number of bytes necessary to +contain the opaque data. If n is not a multiple of four, then the n +bytes are followed by enough (0 to 3) residual zero bytes, r, to make +the total byte count of the opaque object a multiple of four. +.ie t .DS +.el .DS L +\fIFixed-Length Opaque\fP + +\f(CW0 1 ... ++--------+--------+...+--------+--------+...+--------+ +| byte 0 | byte 1 |...|byte n-1| 0 |...| 0 | ++--------+--------+...+--------+--------+...+--------+ +|<-----------n bytes---------->|<------r bytes------>| +|<-----------n+r (where (n+r) mod 4 = 0)------------>|\fP +.DE +.NH 2 +\&Variable-length Opaque Data +.IX XDR "variable-length opaque data" +.IX XDR "opaque data, variable length" +.LP +The standard also provides for variable-length (counted) opaque data, +defined as a sequence of n (numbered 0 through n-1) arbitrary bytes +to be the number n encoded as an unsigned integer (as described +below), and followed by the n bytes of the sequence. +.LP +Byte m of the sequence always precedes byte m+1 of the sequence, and +byte 0 of the sequence always follows the sequence's length (count). +enough (0 to 3) residual zero bytes, r, to make the total byte count +a multiple of four. Variable-length opaque data is declared in the +following way: +.DS +.ft CW +opaque identifier<m>; +.DE +or +.DS +.ft CW +opaque identifier<>; +.DE +The constant m denotes an upper bound of the number of bytes that the +sequence may contain. If m is not specified, as in the second +declaration, it is assumed to be (2**32) - 1, the maximum length. +The constant m would normally be found in a protocol specification. +For example, a filing protocol may state that the maximum data +transfer size is 8192 bytes, as follows: +.DS +.ft CW +opaque filedata<8192>; +.DE +This can be illustrated as follows: +.ie t .DS +.el .DS L +\fIVariable-Length Opaque\fP + +\f(CW0 1 2 3 4 5 ... ++-----+-----+-----+-----+-----+-----+...+-----+-----+...+-----+ +| length n |byte0|byte1|...| n-1 | 0 |...| 0 | ++-----+-----+-----+-----+-----+-----+...+-----+-----+...+-----+ +|<-------4 bytes------->|<------n bytes------>|<---r bytes--->| +|<----n+r (where (n+r) mod 4 = 0)---->|\fP +.DE +.LP +It is an error to encode a length greater than the maximum +described in the specification. +.NH 2 +\&String +.IX XDR string +.LP +The standard defines a string of n (numbered 0 through n-1) ASCII +bytes to be the number n encoded as an unsigned integer (as described +above), and followed by the n bytes of the string. Byte m of the +string always precedes byte m+1 of the string, and byte 0 of the +string always follows the string's length. If n is not a multiple of +four, then the n bytes are followed by enough (0 to 3) residual zero +bytes, r, to make the total byte count a multiple of four. Counted +byte strings are declared as follows: +.DS +.ft CW +string object<m>; +.DE +or +.DS +.ft CW +string object<>; +.DE +The constant m denotes an upper bound of the number of bytes that a +string may contain. If m is not specified, as in the second +declaration, it is assumed to be (2**32) - 1, the maximum length. +The constant m would normally be found in a protocol specification. +For example, a filing protocol may state that a file name can be no +longer than 255 bytes, as follows: +.DS +.ft CW +string filename<255>; +.DE +Which can be illustrated as: +.ie t .DS +.el .DS L +\fIA String\fP + +\f(CW0 1 2 3 4 5 ... ++-----+-----+-----+-----+-----+-----+...+-----+-----+...+-----+ +| length n |byte0|byte1|...| n-1 | 0 |...| 0 | ++-----+-----+-----+-----+-----+-----+...+-----+-----+...+-----+ +|<-------4 bytes------->|<------n bytes------>|<---r bytes--->| +|<----n+r (where (n+r) mod 4 = 0)---->|\fP +.DE +.LP +It is an error to encode a length greater than the maximum +described in the specification. +.NH 2 +\&Fixed-length Array +.IX XDR "fixed-length array" +.IX XDR "array, fixed length" +.LP +Declarations for fixed-length arrays of homogeneous elements are in +the following form: +.DS +.ft CW +type-name identifier[n]; +.DE +Fixed-length arrays of elements numbered 0 through n-1 are encoded by +individually encoding the elements of the array in their natural +order, 0 through n-1. Each element's size is a multiple of four +bytes. Though all elements are of the same type, the elements may +have different sizes. For example, in a fixed-length array of +strings, all elements are of type "string", yet each element will +vary in its length. +.ie t .DS +.el .DS L +\fIFixed-Length Array\fP + +\f(CW+---+---+---+---+---+---+---+---+...+---+---+---+---+ +| element 0 | element 1 |...| element n-1 | ++---+---+---+---+---+---+---+---+...+---+---+---+---+ +|<--------------------n elements------------------->|\fP +.DE +.NH 2 +\&Variable-length Array +.IX XDR "variable-length array" +.IX XDR "array, variable length" +.LP +Counted arrays provide the ability to encode variable-length arrays +of homogeneous elements. The array is encoded as the element count n +(an unsigned integer) followed by the encoding of each of the array's +elements, starting with element 0 and progressing through element n- +1. The declaration for variable-length arrays follows this form: +.DS +.ft CW +type-name identifier<m>; +.DE +or +.DS +.ft CW +type-name identifier<>; +.DE +The constant m specifies the maximum acceptable element count of an +array; if m is not specified, as in the second declaration, it is +assumed to be (2**32) - 1. +.ie t .DS +.el .DS L +\fICounted Array\fP + +\f(CW0 1 2 3 ++--+--+--+--+--+--+--+--+--+--+--+--+...+--+--+--+--+ +| n | element 0 | element 1 |...|element n-1| ++--+--+--+--+--+--+--+--+--+--+--+--+...+--+--+--+--+ +|<-4 bytes->|<--------------n elements------------->|\fP +.DE +It is an error to encode a value of n that is greater than the +maximum described in the specification. +.NH 2 +\&Structure +.IX XDR structure +.LP +Structures are declared as follows: +.DS +.ft CW +struct { + component-declaration-A; + component-declaration-B; + \&... +} identifier; +.DE +The components of the structure are encoded in the order of their +declaration in the structure. Each component's size is a multiple of +four bytes, though the components may be different sizes. +.ie t .DS +.el .DS L +\fIStructure\fP + +\f(CW+-------------+-------------+... +| component A | component B |... ++-------------+-------------+...\fP +.DE +.NH 2 +\&Discriminated Union +.IX XDR "discriminated union" +.IX XDR union discriminated +.LP +A discriminated union is a type composed of a discriminant followed +by a type selected from a set of prearranged types according to the +value of the discriminant. The type of discriminant is either "int", +"unsigned int", or an enumerated type, such as "bool". The component +types are called "arms" of the union, and are preceded by the value +of the discriminant which implies their encoding. Discriminated +unions are declared as follows: +.DS +.ft CW +union switch (discriminant-declaration) { + case discriminant-value-A: + arm-declaration-A; + case discriminant-value-B: + arm-declaration-B; + \&... + default: default-declaration; +} identifier; +.DE +Each "case" keyword is followed by a legal value of the discriminant. +The default arm is optional. If it is not specified, then a valid +encoding of the union cannot take on unspecified discriminant values. +The size of the implied arm is always a multiple of four bytes. +.LP +The discriminated union is encoded as its discriminant followed by +the encoding of the implied arm. +.ie t .DS +.el .DS L +\fIDiscriminated Union\fP + +\f(CW0 1 2 3 ++---+---+---+---+---+---+---+---+ +| discriminant | implied arm | ++---+---+---+---+---+---+---+---+ +|<---4 bytes--->|\fP +.DE +.NH 2 +\&Void +.IX XDR void +.LP +An XDR void is a 0-byte quantity. Voids are useful for describing +operations that take no data as input or no data as output. They are +also useful in unions, where some arms may contain data and others do +not. The declaration is simply as follows: +.DS +.ft CW +void; +.DE +Voids are illustrated as follows: +.ie t .DS +.el .DS L +\fIVoid\fP + +\f(CW ++ + || + ++ +--><-- 0 bytes\fP +.DE +.NH 2 +\&Constant +.IX XDR constant +.LP +The data declaration for a constant follows this form: +.DS +.ft CW +const name-identifier = n; +.DE +"const" is used to define a symbolic name for a constant; it does not +declare any data. The symbolic constant may be used anywhere a +regular constant may be used. For example, the following defines a +symbolic constant DOZEN, equal to 12. +.DS +.ft CW +const DOZEN = 12; +.DE +.NH 2 +\&Typedef +.IX XDR typedef +.LP +"typedef" does not declare any data either, but serves to define new +identifiers for declaring data. The syntax is: +.DS +.ft CW +typedef declaration; +.DE +The new type name is actually the variable name in the declaration +part of the typedef. For example, the following defines a new type +called "eggbox" using an existing type called "egg": +.DS +.ft CW +typedef egg eggbox[DOZEN]; +.DE +Variables declared using the new type name have the same type as the +new type name would have in the typedef, if it was considered a +variable. For example, the following two declarations are equivalent +in declaring the variable "fresheggs": +.DS +.ft CW +eggbox fresheggs; +egg fresheggs[DOZEN]; +.DE +When a typedef involves a struct, enum, or union definition, there is +another (preferred) syntax that may be used to define the same type. +In general, a typedef of the following form: +.DS +.ft CW +typedef <<struct, union, or enum definition>> identifier; +.DE +may be converted to the alternative form by removing the "typedef" +part and placing the identifier after the "struct", "union", or +"enum" keyword, instead of at the end. For example, here are the two +ways to define the type "bool": +.DS +.ft CW +typedef enum { /* \fIusing typedef\fP */ + FALSE = 0, + TRUE = 1 + } bool; + +enum bool { /* \fIpreferred alternative\fP */ + FALSE = 0, + TRUE = 1 + }; +.DE +The reason this syntax is preferred is one does not have to wait +until the end of a declaration to figure out the name of the new +type. +.NH 2 +\&Optional-data +.IX XDR "optional data" +.IX XDR "data, optional" +.LP +Optional-data is one kind of union that occurs so frequently that we +give it a special syntax of its own for declaring it. It is declared +as follows: +.DS +.ft CW +type-name *identifier; +.DE +This is equivalent to the following union: +.DS +.ft CW +union switch (bool opted) { + case TRUE: + type-name element; + case FALSE: + void; +} identifier; +.DE +It is also equivalent to the following variable-length array +declaration, since the boolean "opted" can be interpreted as the +length of the array: +.DS +.ft CW +type-name identifier<1>; +.DE +Optional-data is not so interesting in itself, but it is very useful +for describing recursive data-structures such as linked-lists and +trees. For example, the following defines a type "stringlist" that +encodes lists of arbitrary length strings: +.DS +.ft CW +struct *stringlist { + string item<>; + stringlist next; +}; +.DE +It could have been equivalently declared as the following union: +.DS +.ft CW +union stringlist switch (bool opted) { + case TRUE: + struct { + string item<>; + stringlist next; + } element; + case FALSE: + void; +}; +.DE +or as a variable-length array: +.DS +.ft CW +struct stringlist<1> { + string item<>; + stringlist next; +}; +.DE +Both of these declarations obscure the intention of the stringlist +type, so the optional-data declaration is preferred over both of +them. The optional-data type also has a close correlation to how +recursive data structures are represented in high-level languages +such as Pascal or C by use of pointers. In fact, the syntax is the +same as that of the C language for pointers. +.NH 2 +\&Areas for Future Enhancement +.IX XDR futures +.LP +The XDR standard lacks representations for bit fields and bitmaps, +since the standard is based on bytes. Also missing are packed (or +binary-coded) decimals. +.LP +The intent of the XDR standard was not to describe every kind of data +that people have ever sent or will ever want to send from machine to +machine. Rather, it only describes the most commonly used data-types +of high-level languages such as Pascal or C so that applications +written in these languages will be able to communicate easily over +some medium. +.LP +One could imagine extensions to XDR that would let it describe almost +any existing protocol, such as TCP. The minimum necessary for this +are support for different block sizes and byte-orders. The XDR +discussed here could then be considered the 4-byte big-endian member +of a larger XDR family. +.NH 1 +\&Discussion +.sp 2 +.NH 2 +\&Why a Language for Describing Data? +.IX XDR language +.LP +There are many advantages in using a data-description language such +as XDR versus using diagrams. Languages are more formal than +diagrams and lead to less ambiguous descriptions of data. +Languages are also easier to understand and allow one to think of +other issues instead of the low-level details of bit-encoding. +Also, there is a close analogy between the types of XDR and a +high-level language such as C or Pascal. This makes the +implementation of XDR encoding and decoding modules an easier task. +Finally, the language specification itself is an ASCII string that +can be passed from machine to machine to perform on-the-fly data +interpretation. +.NH 2 +\&Why Only one Byte-Order for an XDR Unit? +.IX XDR "byte order" +.LP +Supporting two byte-orderings requires a higher level protocol for +determining in which byte-order the data is encoded. Since XDR is +not a protocol, this can't be done. The advantage of this, though, +is that data in XDR format can be written to a magnetic tape, for +example, and any machine will be able to interpret it, since no +higher level protocol is necessary for determining the byte-order. +.NH 2 +\&Why does XDR use Big-Endian Byte-Order? +.LP +Yes, it is unfair, but having only one byte-order means you have to +be unfair to somebody. Many architectures, such as the Motorola +68000 and IBM 370, support the big-endian byte-order. +.NH 2 +\&Why is the XDR Unit Four Bytes Wide? +.LP +There is a tradeoff in choosing the XDR unit size. Choosing a small +size such as two makes the encoded data small, but causes alignment +problems for machines that aren't aligned on these boundaries. A +large size such as eight means the data will be aligned on virtually +every machine, but causes the encoded data to grow too big. We chose +four as a compromise. Four is big enough to support most +architectures efficiently, except for rare machines such as the +eight-byte aligned Cray. Four is also small enough to keep the +encoded data restricted to a reasonable size. +.NH 2 +\&Why must Variable-Length Data be Padded with Zeros? +.IX XDR "variable-length data" +.LP +It is desirable that the same data encode into the same thing on all +machines, so that encoded data can be meaningfully compared or +checksummed. Forcing the padded bytes to be zero ensures this. +.NH 2 +\&Why is there No Explicit Data-Typing? +.LP +Data-typing has a relatively high cost for what small advantages it +may have. One cost is the expansion of data due to the inserted type +fields. Another is the added cost of interpreting these type fields +and acting accordingly. And most protocols already know what type +they expect, so data-typing supplies only redundant information. +However, one can still get the benefits of data-typing using XDR. One +way is to encode two things: first a string which is the XDR data +description of the encoded data, and then the encoded data itself. +Another way is to assign a value to all the types in XDR, and then +define a universal type which takes this value as its discriminant +and for each value, describes the corresponding data type. +.NH 1 +\&The XDR Language Specification +.IX XDR language +.sp 1 +.NH 2 +\&Notational Conventions +.IX "XDR language" notation +.LP +This specification uses an extended Backus-Naur Form notation for +describing the XDR language. Here is a brief description of the +notation: +.IP 1. +The characters +.I | , +.I ( , +.I ) , +.I [ , +.I ] , +.I " , +and +.I * +are special. +.IP 2. +Terminal symbols are strings of any characters surrounded by +double quotes. +.IP 3. +Non-terminal symbols are strings of non-special characters. +.IP 4. +Alternative items are separated by a vertical bar ("\fI|\fP"). +.IP 5. +Optional items are enclosed in brackets. +.IP 6. +Items are grouped together by enclosing them in parentheses. +.IP 7. +A +.I * +following an item means 0 or more occurrences of that item. +.LP +For example, consider the following pattern: +.DS L +"a " "very" (", " " very")* [" cold " "and"] " rainy " ("day" | "night") +.DE +.LP +An infinite number of strings match this pattern. A few of them +are: +.DS +"a very rainy day" +"a very, very rainy day" +"a very cold and rainy day" +"a very, very, very cold and rainy night" +.DE +.NH 2 +\&Lexical Notes +.IP 1. +Comments begin with '/*' and terminate with '*/'. +.IP 2. +White space serves to separate items and is otherwise ignored. +.IP 3. +An identifier is a letter followed by an optional sequence of +letters, digits or underbar ('_'). The case of identifiers is +not ignored. +.IP 4. +A constant is a sequence of one or more decimal digits, +optionally preceded by a minus-sign ('-'). +.NH 2 +\&Syntax Information +.IX "XDR language" syntax +.DS +.ft CW +declaration: + type-specifier identifier + | type-specifier identifier "[" value "]" + | type-specifier identifier "<" [ value ] ">" + | "opaque" identifier "[" value "]" + | "opaque" identifier "<" [ value ] ">" + | "string" identifier "<" [ value ] ">" + | type-specifier "*" identifier + | "void" +.DE +.DS +.ft CW +value: + constant + | identifier + +type-specifier: + [ "unsigned" ] "int" + | [ "unsigned" ] "hyper" + | "float" + | "double" + | "bool" + | enum-type-spec + | struct-type-spec + | union-type-spec + | identifier +.DE +.DS +.ft CW +enum-type-spec: + "enum" enum-body + +enum-body: + "{" + ( identifier "=" value ) + ( "," identifier "=" value )* + "}" +.DE +.DS +.ft CW +struct-type-spec: + "struct" struct-body + +struct-body: + "{" + ( declaration ";" ) + ( declaration ";" )* + "}" +.DE +.DS +.ft CW +union-type-spec: + "union" union-body + +union-body: + "switch" "(" declaration ")" "{" + ( "case" value ":" declaration ";" ) + ( "case" value ":" declaration ";" )* + [ "default" ":" declaration ";" ] + "}" + +constant-def: + "const" identifier "=" constant ";" +.DE +.DS +.ft CW +type-def: + "typedef" declaration ";" + | "enum" identifier enum-body ";" + | "struct" identifier struct-body ";" + | "union" identifier union-body ";" + +definition: + type-def + | constant-def + +specification: + definition * +.DE +.NH 3 +\&Syntax Notes +.IX "XDR language" syntax +.LP +.IP 1. +The following are keywords and cannot be used as identifiers: +"bool", "case", "const", "default", "double", "enum", "float", +"hyper", "opaque", "string", "struct", "switch", "typedef", "union", +"unsigned" and "void". +.IP 2. +Only unsigned constants may be used as size specifications for +arrays. If an identifier is used, it must have been declared +previously as an unsigned constant in a "const" definition. +.IP 3. +Constant and type identifiers within the scope of a specification +are in the same name space and must be declared uniquely within this +scope. +.IP 4. +Similarly, variable names must be unique within the scope of +struct and union declarations. Nested struct and union declarations +create new scopes. +.IP 5. +The discriminant of a union must be of a type that evaluates to +an integer. That is, "int", "unsigned int", "bool", an enumerated +type or any typedefed type that evaluates to one of these is legal. +Also, the case values must be one of the legal values of the +discriminant. Finally, a case value may not be specified more than +once within the scope of a union declaration. +.NH 1 +\&An Example of an XDR Data Description +.LP +Here is a short XDR data description of a thing called a "file", +which might be used to transfer files from one machine to another. +.ie t .DS +.el .DS L +.ft CW + +const MAXUSERNAME = 32; /*\fI max length of a user name \fP*/ +const MAXFILELEN = 65535; /*\fI max length of a file \fP*/ +const MAXNAMELEN = 255; /*\fI max length of a file name \fP*/ + +.ft I +/* + * Types of files: + */ +.ft CW + +enum filekind { + TEXT = 0, /*\fI ascii data \fP*/ + DATA = 1, /*\fI raw data \fP*/ + EXEC = 2 /*\fI executable \fP*/ +}; + +.ft I +/* + * File information, per kind of file: + */ +.ft CW + +union filetype switch (filekind kind) { + case TEXT: + void; /*\fI no extra information \fP*/ + case DATA: + string creator<MAXNAMELEN>; /*\fI data creator \fP*/ + case EXEC: + string interpretor<MAXNAMELEN>; /*\fI program interpretor \fP*/ +}; + +.ft I +/* + * A complete file: + */ +.ft CW + +struct file { + string filename<MAXNAMELEN>; /*\fI name of file \fP*/ + filetype type; /*\fI info about file \fP*/ + string owner<MAXUSERNAME>; /*\fI owner of file \fP*/ + opaque data<MAXFILELEN>; /*\fI file data \fP*/ +}; +.DE +.LP +Suppose now that there is a user named "john" who wants to store +his lisp program "sillyprog" that contains just the data "(quit)". +His file would be encoded as follows: +.TS +box tab (&) ; +lfI lfI lfI lfI +rfL rfL rfL l . +Offset&Hex Bytes&ASCII&Description +_ +0&00 00 00 09&....&Length of filename = 9 +4&73 69 6c 6c&sill&Filename characters +8&79 70 72 6f&ypro& ... and more characters ... +12&67 00 00 00&g...& ... and 3 zero-bytes of fill +16&00 00 00 02&....&Filekind is EXEC = 2 +20&00 00 00 04&....&Length of interpretor = 4 +24&6c 69 73 70&lisp&Interpretor characters +28&00 00 00 04&....&Length of owner = 4 +32&6a 6f 68 6e&john&Owner characters +36&00 00 00 06&....&Length of file data = 6 +40&28 71 75 69&(qui&File data bytes ... +44&74 29 00 00&t)..& ... and 2 zero-bytes of fill +.TE +.NH 1 +\&References +.LP +[1] Brian W. Kernighan & Dennis M. Ritchie, "The C Programming +Language", Bell Laboratories, Murray Hill, New Jersey, 1978. +.LP +[2] Danny Cohen, "On Holy Wars and a Plea for Peace", IEEE Computer, +October 1981. +.LP +[3] "IEEE Standard for Binary Floating-Point Arithmetic", ANSI/IEEE +Standard 754-1985, Institute of Electrical and Electronics +Engineers, August 1985. +.LP +[4] "Courier: The Remote Procedure Call Protocol", XEROX +Corporation, XSIS 038112, December 1981. diff --git a/share/doc/psd/26.rpcrfc/Makefile b/share/doc/psd/26.rpcrfc/Makefile new file mode 100644 index 000000000000..3ffdc27a7bae --- /dev/null +++ b/share/doc/psd/26.rpcrfc/Makefile @@ -0,0 +1,8 @@ +# $FreeBSD$ + +VOLUME= psd/26.rpcrfc +SRCS= stubs rpc.rfc.ms +MACROS= -ms +USE_TBL= + +.include <bsd.doc.mk> diff --git a/share/doc/psd/26.rpcrfc/rpc.rfc.ms b/share/doc/psd/26.rpcrfc/rpc.rfc.ms new file mode 100644 index 000000000000..9a948bd445dc --- /dev/null +++ b/share/doc/psd/26.rpcrfc/rpc.rfc.ms @@ -0,0 +1,1304 @@ +.\" +.\" Must use -- tbl -- with this one +.\" +.\" @(#)rpc.rfc.ms 2.2 88/08/05 4.0 RPCSRC +.\" $FreeBSD$ +.\" +.de BT +.if \\n%=1 .tl ''- % -'' +.. +.ND +.\" prevent excess underlining in nroff +.if n .fp 2 R +.OH 'Remote Procedure Calls: Protocol Specification''Page %' +.EH 'Page %''Remote Procedure Calls: Protocol Specification' +.if \n%=1 .bp +.SH +\&Remote Procedure Calls: Protocol Specification +.LP +.NH 0 +\&Status of this Memo +.LP +Note: This chapter specifies a protocol that Sun Microsystems, Inc., +and others are using. +It has been designated RFC1050 by the ARPA Network +Information Center. +.LP +.NH 1 +\&Introduction +.LP +This chapter specifies a message protocol used in implementing +Sun's Remote Procedure Call (RPC) package. (The message protocol is +specified with the External Data Representation (XDR) language. +See the +.I "External Data Representation Standard: Protocol Specification" +for the details. Here, we assume that the reader is familiar +with XDR and do not attempt to justify it or its uses). The paper +by Birrell and Nelson [1] is recommended as an excellent background +to and justification of RPC. +.NH 2 +\&Terminology +.LP +This chapter discusses servers, services, programs, procedures, +clients, and versions. A server is a piece of software where network +services are implemented. A network service is a collection of one +or more remote programs. A remote program implements one or more +remote procedures; the procedures, their parameters, and results are +documented in the specific program's protocol specification (see the +\fIPort Mapper Program Protocol\fP\, below, for an example). Network +clients are pieces of software that initiate remote procedure calls +to services. A server may support more than one version of a remote +program in order to be forward compatible with changing protocols. +.LP +For example, a network file service may be composed of two programs. +One program may deal with high-level applications such as file system +access control and locking. The other may deal with low-level file +IO and have procedures like "read" and "write". A client machine of +the network file service would call the procedures associated with +the two programs of the service on behalf of some user on the client +machine. +.NH 2 +\&The RPC Model +.LP +The remote procedure call model is similar to the local procedure +call model. In the local case, the caller places arguments to a +procedure in some well-specified location (such as a result +register). It then transfers control to the procedure, and +eventually gains back control. At that point, the results of the +procedure are extracted from the well-specified location, and the +caller continues execution. +.LP +The remote procedure call is similar, in that one thread of control +logically winds through two processes\(emone is the caller's process, +the other is a server's process. That is, the caller process sends a +call message to the server process and waits (blocks) for a reply +message. The call message contains the procedure's parameters, among +other things. The reply message contains the procedure's results, +among other things. Once the reply message is received, the results +of the procedure are extracted, and caller's execution is resumed. +.LP +On the server side, a process is dormant awaiting the arrival of a +call message. When one arrives, the server process extracts the +procedure's parameters, computes the results, sends a reply message, +and then awaits the next call message. +.LP +Note that in this model, only one of the two processes is active at +any given time. However, this model is only given as an example. +The RPC protocol makes no restrictions on the concurrency model +implemented, and others are possible. For example, an implementation +may choose to have RPC calls be asynchronous, so that the client may +do useful work while waiting for the reply from the server. Another +possibility is to have the server create a task to process an +incoming request, so that the server can be free to receive other +requests. +.NH 2 +\&Transports and Semantics +.LP +The RPC protocol is independent of transport protocols. That is, RPC +does not care how a message is passed from one process to another. +The protocol deals only with specification and interpretation of +messages. +.LP +It is important to point out that RPC does not try to implement any +kind of reliability and that the application must be aware of the +type of transport protocol underneath RPC. If it knows it is running +on top of a reliable transport such as TCP/IP[6], then most of the +work is already done for it. On the other hand, if it is running on +top of an unreliable transport such as UDP/IP[7], it must implement +is own retransmission and time-out policy as the RPC layer does not +provide this service. +.LP +Because of transport independence, the RPC protocol does not attach +specific semantics to the remote procedures or their execution. +Semantics can be inferred from (but should be explicitly specified +by) the underlying transport protocol. For example, consider RPC +running on top of an unreliable transport such as UDP/IP. If an +application retransmits RPC messages after short time-outs, the only +thing it can infer if it receives no reply is that the procedure was +executed zero or more times. If it does receive a reply, then it can +infer that the procedure was executed at least once. +.LP +A server may wish to remember previously granted requests from a +client and not regrant them in order to insure some degree of +execute-at-most-once semantics. A server can do this by taking +advantage of the transaction ID that is packaged with every RPC +request. The main use of this transaction is by the client RPC layer +in matching replies to requests. However, a client application may +choose to reuse its previous transaction ID when retransmitting a +request. The server application, knowing this fact, may choose to +remember this ID after granting a request and not regrant requests +with the same ID in order to achieve some degree of +execute-at-most-once semantics. The server is not allowed to examine +this ID in any other way except as a test for equality. +.LP +On the other hand, if using a reliable transport such as TCP/IP, the +application can infer from a reply message that the procedure was +executed exactly once, but if it receives no reply message, it cannot +assume the remote procedure was not executed. Note that even if a +connection-oriented protocol like TCP is used, an application still +needs time-outs and reconnection to handle server crashes. +.LP +There are other possibilities for transports besides datagram- or +connection-oriented protocols. For example, a request-reply protocol +such as VMTP[2] is perhaps the most natural transport for RPC. +.SH +.I +NOTE: At Sun, RPC is currently implemented on top of both TCP/IP +and UDP/IP transports. +.LP +.NH 2 +\&Binding and Rendezvous Independence +.LP +The act of binding a client to a service is NOT part of the remote +procedure call specification. This important and necessary function +is left up to some higher-level software. (The software may use RPC +itself\(emsee the \fIPort Mapper Program Protocol\fP\, below). +.LP +Implementors should think of the RPC protocol as the jump-subroutine +instruction ("JSR") of a network; the loader (binder) makes JSR +useful, and the loader itself uses JSR to accomplish its task. +Likewise, the network makes RPC useful, using RPC to accomplish this +task. +.NH 2 +\&Authentication +.LP +The RPC protocol provides the fields necessary for a client to +identify itself to a service and vice-versa. Security and access +control mechanisms can be built on top of the message authentication. +Several different authentication protocols can be supported. A field +in the RPC header indicates which protocol is being used. More +information on specific authentication protocols can be found in the +\fIAuthentication Protocols\fP\, +below. +.KS +.NH 1 +\&RPC Protocol Requirements +.LP +The RPC protocol must provide for the following: +.IP 1. +Unique specification of a procedure to be called. +.IP 2. +Provisions for matching response messages to request messages. +.KE +.IP 3. +Provisions for authenticating the caller to service and vice-versa. +.LP +Besides these requirements, features that detect the following are +worth supporting because of protocol roll-over errors, implementation +bugs, user error, and network administration: +.IP 1. +RPC protocol mismatches. +.IP 2. +Remote program protocol version mismatches. +.IP 3. +Protocol errors (such as misspecification of a procedure's parameters). +.IP 4. +Reasons why remote authentication failed. +.IP 5. +Any other reasons why the desired procedure was not called. +.NH 2 +\&Programs and Procedures +.LP +The RPC call message has three unsigned fields: remote program +number, remote program version number, and remote procedure number. +The three fields uniquely identify the procedure to be called. +Program numbers are administered by some central authority (like +Sun). Once an implementor has a program number, he can implement his +remote program; the first implementation would most likely have the +version number of 1. Because most new protocols evolve into better, +stable, and mature protocols, a version field of the call message +identifies which version of the protocol the caller is using. +Version numbers make speaking old and new protocols through the same +server process possible. +.LP +The procedure number identifies the procedure to be called. These +numbers are documented in the specific program's protocol +specification. For example, a file service's protocol specification +may state that its procedure number 5 is "read" and procedure number +12 is "write". +.LP +Just as remote program protocols may change over several versions, +the actual RPC message protocol could also change. Therefore, the +call message also has in it the RPC version number, which is always +equal to two for the version of RPC described here. +.LP +The reply message to a request message has enough information to +distinguish the following error conditions: +.IP 1. +The remote implementation of RPC does speak protocol version 2. +The lowest and highest supported RPC version numbers are returned. +.IP 2. +The remote program is not available on the remote system. +.IP 3. +The remote program does not support the requested version number. +The lowest and highest supported remote program version numbers are +returned. +.IP 4. +The requested procedure number does not exist. (This is usually a +caller side protocol or programming error.) +.IP 5. +The parameters to the remote procedure appear to be garbage from the +server's point of view. (Again, this is usually caused by a +disagreement about the protocol between client and service.) +.NH 2 +\&Authentication +.LP +Provisions for authentication of caller to service and vice-versa are +provided as a part of the RPC protocol. The call message has two +authentication fields, the credentials and verifier. The reply +message has one authentication field, the response verifier. The RPC +protocol specification defines all three fields to be the following +opaque type: +.DS +.ft CW +.vs 11 +enum auth_flavor { + AUTH_NULL = 0, + AUTH_UNIX = 1, + AUTH_SHORT = 2, + AUTH_DES = 3 + /* \fIand more to be defined\fP */ +}; + +struct opaque_auth { + auth_flavor flavor; + opaque body<400>; +}; +.DE +.LP +In simple English, any +.I opaque_auth +structure is an +.I auth_flavor +enumeration followed by bytes which are opaque to the RPC protocol +implementation. +.LP +The interpretation and semantics of the data contained within the +authentication fields is specified by individual, independent +authentication protocol specifications. (See +\fIAuthentication Protocols\fP\, +below, for definitions of the various authentication protocols.) +.LP +If authentication parameters were rejected, the response message +contains information stating why they were rejected. +.NH 2 +\&Program Number Assignment +.LP +Program numbers are given out in groups of +.I 0x20000000 +(decimal 536870912) according to the following chart: +.TS +box tab (&) ; +lfI lfI +rfL cfI . +Program Numbers&Description +_ +.sp .5 +0 - 1fffffff&Defined by Sun +20000000 - 3fffffff&Defined by user +40000000 - 5fffffff&Transient +60000000 - 7fffffff&Reserved +80000000 - 9fffffff&Reserved +a0000000 - bfffffff&Reserved +c0000000 - dfffffff&Reserved +e0000000 - ffffffff&Reserved +.TE +.LP +The first group is a range of numbers administered by Sun +Microsystems and should be identical for all sites. The second range +is for applications peculiar to a particular site. This range is +intended primarily for debugging new programs. When a site develops +an application that might be of general interest, that application +should be given an assigned number in the first range. The third +group is for applications that generate program numbers dynamically. +The final groups are reserved for future use, and should not be used. +.NH 2 +\&Other Uses of the RPC Protocol +.LP +The intended use of this protocol is for calling remote procedures. +That is, each call message is matched with a response message. +However, the protocol itself is a message-passing protocol with which +other (non-RPC) protocols can be implemented. Sun currently uses, or +perhaps abuses, the RPC message protocol for the following two +(non-RPC) protocols: batching (or pipelining) and broadcast RPC. +These two protocols are discussed but not defined below. +.NH 3 +\&Batching +.LP +Batching allows a client to send an arbitrarily large sequence of +call messages to a server; batching typically uses reliable byte +stream protocols (like TCP/IP) for its transport. In the case of +batching, the client never waits for a reply from the server, and the +server does not send replies to batch requests. A sequence of batch +calls is usually terminated by a legitimate RPC in order to flush the +pipeline (with positive acknowledgement). +.NH 3 +\&Broadcast RPC +.LP +In broadcast RPC-based protocols, the client sends a broadcast packet +to the network and waits for numerous replies. Broadcast RPC uses +unreliable, packet-based protocols (like UDP/IP) as its transports. +Servers that support broadcast protocols only respond when the +request is successfully processed, and are silent in the face of +errors. Broadcast RPC uses the Port Mapper RPC service to achieve +its semantics. See the \fIPort Mapper Program Protocol\fP\, below, +for more information. +.KS +.NH 1 +\&The RPC Message Protocol +.LP +This section defines the RPC message protocol in the XDR data +description language. The message is defined in a top-down style. +.ie t .DS +.el .DS L +.ft CW +enum msg_type { + CALL = 0, + REPLY = 1 +}; + +.ft I +/* +* A reply to a call message can take on two forms: +* The message was either accepted or rejected. +*/ +.ft CW +enum reply_stat { + MSG_ACCEPTED = 0, + MSG_DENIED = 1 +}; + +.ft I +/* +* Given that a call message was accepted, the following is the +* status of an attempt to call a remote procedure. +*/ +.ft CW +enum accept_stat { + SUCCESS = 0, /* \fIRPC executed successfully \fP*/ + PROG_UNAVAIL = 1, /* \fIremote hasn't exported program \fP*/ + PROG_MISMATCH = 2, /* \fIremote can't support version # \fP*/ + PROC_UNAVAIL = 3, /* \fIprogram can't support procedure \fP*/ + GARBAGE_ARGS = 4 /* \fIprocedure can't decode params \fP*/ +}; +.DE +.ie t .DS +.el .DS L +.ft I +/* +* Reasons why a call message was rejected: +*/ +.ft CW +enum reject_stat { + RPC_MISMATCH = 0, /* \fIRPC version number != 2 \fP*/ + AUTH_ERROR = 1 /* \fIremote can't authenticate caller \fP*/ +}; + +.ft I +/* +* Why authentication failed: +*/ +.ft CW +enum auth_stat { + AUTH_BADCRED = 1, /* \fIbad credentials \fP*/ + AUTH_REJECTEDCRED = 2, /* \fIclient must begin new session \fP*/ + AUTH_BADVERF = 3, /* \fIbad verifier \fP*/ + AUTH_REJECTEDVERF = 4, /* \fIverifier expired or replayed \fP*/ + AUTH_TOOWEAK = 5 /* \fIrejected for security reasons \fP*/ +}; +.DE +.KE +.ie t .DS +.el .DS L +.ft I +/* +* The RPC message: +* All messages start with a transaction identifier, xid, +* followed by a two-armed discriminated union. The union's +* discriminant is a msg_type which switches to one of the two +* types of the message. The xid of a \fIREPLY\fP message always +* matches that of the initiating \fICALL\fP message. NB: The xid +* field is only used for clients matching reply messages with +* call messages or for servers detecting retransmissions; the +* service side cannot treat this id as any type of sequence +* number. +*/ +.ft CW +struct rpc_msg { + unsigned int xid; + union switch (msg_type mtype) { + case CALL: + call_body cbody; + case REPLY: + reply_body rbody; + } body; +}; +.DE +.ie t .DS +.el .DS L +.ft I +/* +* Body of an RPC request call: +* In version 2 of the RPC protocol specification, rpcvers must +* be equal to 2. The fields prog, vers, and proc specify the +* remote program, its version number, and the procedure within +* the remote program to be called. After these fields are two +* authentication parameters: cred (authentication credentials) +* and verf (authentication verifier). The two authentication +* parameters are followed by the parameters to the remote +* procedure, which are specified by the specific program +* protocol. +*/ +.ft CW +struct call_body { + unsigned int rpcvers; /* \fImust be equal to two (2) \fP*/ + unsigned int prog; + unsigned int vers; + unsigned int proc; + opaque_auth cred; + opaque_auth verf; + /* \fIprocedure specific parameters start here \fP*/ +}; +.DE +.ie t .DS +.el .DS L +.ft I +/* +* Body of a reply to an RPC request: +* The call message was either accepted or rejected. +*/ +.ft CW +union reply_body switch (reply_stat stat) { + case MSG_ACCEPTED: + accepted_reply areply; + case MSG_DENIED: + rejected_reply rreply; +} reply; +.DE +.ie t .DS +.el .DS L +.ft I +/* +* Reply to an RPC request that was accepted by the server: +* there could be an error even though the request was accepted. +* The first field is an authentication verifier that the server +* generates in order to validate itself to the caller. It is +* followed by a union whose discriminant is an enum +* accept_stat. The \fISUCCESS\fP arm of the union is protocol +* specific. The \fIPROG_UNAVAIL\fP, \fIPROC_UNAVAIL\fP, and \fIGARBAGE_ARGP\fP +* arms of the union are void. The \fIPROG_MISMATCH\fP arm specifies +* the lowest and highest version numbers of the remote program +* supported by the server. +*/ +.ft CW +struct accepted_reply { + opaque_auth verf; + union switch (accept_stat stat) { + case SUCCESS: + opaque results[0]; + /* \fIprocedure-specific results start here\fP */ + case PROG_MISMATCH: + struct { + unsigned int low; + unsigned int high; + } mismatch_info; + default: +.ft I + /* + * Void. Cases include \fIPROG_UNAVAIL, PROC_UNAVAIL\fP, + * and \fIGARBAGE_ARGS\fP. + */ +.ft CW + void; + } reply_data; +}; +.DE +.ie t .DS +.el .DS L +.ft I +/* +* Reply to an RPC request that was rejected by the server: +* The request can be rejected for two reasons: either the +* server is not running a compatible version of the RPC +* protocol (\fIRPC_MISMATCH\fP), or the server refuses to +* authenticate the caller (\fIAUTH_ERROR\fP). In case of an RPC +* version mismatch, the server returns the lowest and highest +* supported RPC version numbers. In case of refused +* authentication, failure status is returned. +*/ +.ft CW +union rejected_reply switch (reject_stat stat) { + case RPC_MISMATCH: + struct { + unsigned int low; + unsigned int high; + } mismatch_info; + case AUTH_ERROR: + auth_stat stat; +}; +.DE +.NH 1 +\&Authentication Protocols +.LP +As previously stated, authentication parameters are opaque, but +open-ended to the rest of the RPC protocol. This section defines +some "flavors" of authentication implemented at (and supported by) +Sun. Other sites are free to invent new authentication types, with +the same rules of flavor number assignment as there is for program +number assignment. +.NH 2 +\&Null Authentication +.LP +Often calls must be made where the caller does not know who he is or +the server does not care who the caller is. In this case, the flavor +value (the discriminant of the \fIopaque_auth\fP's union) of the RPC +message's credentials, verifier, and response verifier is +.I AUTH_NULL . +The bytes of the opaque_auth's body are undefined. +It is recommended that the opaque length be zero. +.NH 2 +\&UNIX Authentication +.LP +The caller of a remote procedure may wish to identify himself as he +is identified on a UNIX system. The value of the credential's +discriminant of an RPC call message is +.I AUTH_UNIX . +The bytes of +the credential's opaque body encode the following structure: +.DS +.ft CW +struct auth_unix { + unsigned int stamp; + string machinename<255>; + unsigned int uid; + unsigned int gid; + unsigned int gids<10>; +}; +.DE +The +.I stamp +is an arbitrary ID which the caller machine may +generate. The +.I machinename +is the name of the caller's machine (like "krypton"). The +.I uid +is the caller's effective user ID. The +.I gid +is the caller's effective group ID. The +.I gids +is a +counted array of groups which contain the caller as a member. The +verifier accompanying the credentials should be of +.I AUTH_NULL +(defined above). +.LP +The value of the discriminant of the response verifier received in +the reply message from the server may be +.I AUTH_NULL +or +.I AUTH_SHORT . +In the case of +.I AUTH_SHORT , +the bytes of the response verifier's string encode an opaque +structure. This new opaque structure may now be passed to the server +instead of the original +.I AUTH_UNIX +flavor credentials. The server keeps a cache which maps shorthand +opaque structures (passed back by way of an +.I AUTH_SHORT +style response verifier) to the original credentials of the caller. +The caller can save network bandwidth and server cpu cycles by using +the new credentials. +.LP +The server may flush the shorthand opaque structure at any time. If +this happens, the remote procedure call message will be rejected due +to an authentication error. The reason for the failure will be +.I AUTH_REJECTEDCRED . +At this point, the caller may wish to try the original +.I AUTH_UNIX +style of credentials. +.KS +.NH 2 +\&DES Authentication +.LP +UNIX authentication suffers from two major problems: +.IP 1. +The naming is too UNIX-system oriented. +.IP 2. +There is no verifier, so credentials can easily be faked. +.LP +DES authentication attempts to fix these two problems. +.KE +.NH 3 +\&Naming +.LP +The first problem is handled by addressing the caller by a simple +string of characters instead of by an operating system specific +integer. This string of characters is known as the "netname" or +network name of the caller. The server is not allowed to interpret +the contents of the caller's name in any other way except to +identify the caller. Thus, netnames should be unique for every +caller in the internet. +.LP +It is up to each operating system's implementation of DES +authentication to generate netnames for its users that insure this +uniqueness when they call upon remote servers. Operating systems +already know how to distinguish users local to their systems. It is +usually a simple matter to extend this mechanism to the network. +For example, a UNIX user at Sun with a user ID of 515 might be +assigned the following netname: "unix.515@sun.com". This netname +contains three items that serve to insure it is unique. Going +backwards, there is only one naming domain called "sun.com" in the +internet. Within this domain, there is only one UNIX user with +user ID 515. However, there may be another user on another +operating system, for example VMS, within the same naming domain +that, by coincidence, happens to have the same user ID. To insure +that these two users can be distinguished we add the operating +system name. So one user is "unix.515@sun.com" and the other is +"vms.515@sun.com". +.LP +The first field is actually a naming method rather than an +operating system name. It just happens that today there is almost +a one-to-one correspondence between naming methods and operating +systems. If the world could agree on a naming standard, the first +field could be the name of that standard, instead of an operating +system name. +.LP +.NH 3 +\&DES Authentication Verifiers +.LP +Unlike UNIX authentication, DES authentication does have a verifier +so the server can validate the client's credential (and +vice-versa). The contents of this verifier is primarily an +encrypted timestamp. The server can decrypt this timestamp, and if +it is close to what the real time is, then the client must have +encrypted it correctly. The only way the client could encrypt it +correctly is to know the "conversation key" of the RPC session. And +if the client knows the conversation key, then it must be the real +client. +.LP +The conversation key is a DES [5] key which the client generates +and notifies the server of in its first RPC call. The conversation +key is encrypted using a public key scheme in this first +transaction. The particular public key scheme used in DES +authentication is Diffie-Hellman [3] with 192-bit keys. The +details of this encryption method are described later. +.LP +The client and the server need the same notion of the current time +in order for all of this to work. If network time synchronization +cannot be guaranteed, then client can synchronize with the server +before beginning the conversation, perhaps by consulting the +Internet Time Server (TIME[4]). +.LP +The way a server determines if a client timestamp is valid is +somewhat complicated. For any other transaction but the first, the +server just checks for two things: +.IP 1. +the timestamp is greater than the one previously seen from the +same client. +.IP 2. +the timestamp has not expired. +.LP +A timestamp is expired if the server's time is later than the sum +of the client's timestamp plus what is known as the client's +"window". The "window" is a number the client passes (encrypted) +to the server in its first transaction. You can think of it as a +lifetime for the credential. +.LP +This explains everything but the first transaction. In the first +transaction, the server checks only that the timestamp has not +expired. If this was all that was done though, then it would be +quite easy for the client to send random data in place of the +timestamp with a fairly good chance of succeeding. As an added +check, the client sends an encrypted item in the first transaction +known as the "window verifier" which must be equal to the window +minus 1, or the server will reject the credential. +.LP +The client too must check the verifier returned from the server to +be sure it is legitimate. The server sends back to the client the +encrypted timestamp it received from the client, minus one second. +If the client gets anything different than this, it will reject it. +.LP +.NH 3 +\&Nicknames and Clock Synchronization +.LP +After the first transaction, the server's DES authentication +subsystem returns in its verifier to the client an integer +"nickname" which the client may use in its further transactions +instead of passing its netname, encrypted DES key and window every +time. The nickname is most likely an index into a table on the +server which stores for each client its netname, decrypted DES key +and window. +.LP +Though they originally were synchronized, the client's and server's +clocks can get out of sync again. When this happens the client RPC +subsystem most likely will get back +.I RPC_AUTHERROR +at which point it should resynchronize. +.LP +A client may still get the +.I RPC_AUTHERROR +error even though it is +synchronized with the server. The reason is that the server's +nickname table is a limited size, and it may flush entries whenever +it wants. A client should resend its original credential in this +case and the server will give it a new nickname. If a server +crashes, the entire nickname table gets flushed, and all clients +will have to resend their original credentials. +.KS +.NH 3 +\&DES Authentication Protocol (in XDR language) +.ie t .DS +.el .DS L +.ft I +/* +* There are two kinds of credentials: one in which the client uses +* its full network name, and one in which it uses its "nickname" +* (just an unsigned integer) given to it by the server. The +* client must use its fullname in its first transaction with the +* server, in which the server will return to the client its +* nickname. The client may use its nickname in all further +* transactions with the server. There is no requirement to use the +* nickname, but it is wise to use it for performance reasons. +*/ +.ft CW +enum authdes_namekind { + ADN_FULLNAME = 0, + ADN_NICKNAME = 1 +}; + +.ft I +/* +* A 64-bit block of encrypted DES data +*/ +.ft CW +typedef opaque des_block[8]; + +.ft I +/* +* Maximum length of a network user's name +*/ +.ft CW +const MAXNETNAMELEN = 255; + +.ft I +/* +* A fullname contains the network name of the client, an encrypted +* conversation key and the window. The window is actually a +* lifetime for the credential. If the time indicated in the +* verifier timestamp plus the window has past, then the server +* should expire the request and not grant it. To insure that +* requests are not replayed, the server should insist that +* timestamps are greater than the previous one seen, unless it is +* the first transaction. In the first transaction, the server +* checks instead that the window verifier is one less than the +* window. +*/ +.ft CW +struct authdes_fullname { +string name<MAXNETNAMELEN>; /* \fIname of client \f(CW*/ +des_block key; /* \fIPK encrypted conversation key \f(CW*/ +unsigned int window; /* \fIencrypted window \f(CW*/ +}; + +.ft I +/* +* A credential is either a fullname or a nickname +*/ +.ft CW +union authdes_cred switch (authdes_namekind adc_namekind) { + case ADN_FULLNAME: + authdes_fullname adc_fullname; + case ADN_NICKNAME: + unsigned int adc_nickname; +}; + +.ft I +/* +* A timestamp encodes the time since midnight, January 1, 1970. +*/ +.ft CW +struct timestamp { + unsigned int seconds; /* \fIseconds \fP*/ + unsigned int useconds; /* \fIand microseconds \fP*/ +}; + +.ft I +/* +* Verifier: client variety +* The window verifier is only used in the first transaction. In +* conjunction with a fullname credential, these items are packed +* into the following structure before being encrypted: +* +* \f(CWstruct {\fP +* \f(CWadv_timestamp; \fP-- one DES block +* \f(CWadc_fullname.window; \fP-- one half DES block +* \f(CWadv_winverf; \fP-- one half DES block +* \f(CW}\fP +* This structure is encrypted using CBC mode encryption with an +* input vector of zero. All other encryptions of timestamps use +* ECB mode encryption. +*/ +.ft CW +struct authdes_verf_clnt { + timestamp adv_timestamp; /* \fIencrypted timestamp \fP*/ + unsigned int adv_winverf; /* \fIencrypted window verifier \fP*/ +}; + +.ft I +/* +* Verifier: server variety +* The server returns (encrypted) the same timestamp the client +* gave it minus one second. It also tells the client its nickname +* to be used in future transactions (unencrypted). +*/ +.ft CW +struct authdes_verf_svr { +timestamp adv_timeverf; /* \fIencrypted verifier \fP*/ +unsigned int adv_nickname; /* \fInew nickname for client \fP*/ +}; +.DE +.KE +.NH 3 +\&Diffie-Hellman Encryption +.LP +In this scheme, there are two constants, +.I BASE +and +.I MODULUS . +The +particular values Sun has chosen for these for the DES +authentication protocol are: +.ie t .DS +.el .DS L +.ft CW +const BASE = 3; +const MODULUS = + "d4a0ba0250b6fd2ec626e7efd637df76c716e22d0944b88b"; /* \fIhex \fP*/ +.DE +.ft R +The way this scheme works is best explained by an example. Suppose +there are two people "A" and "B" who want to send encrypted +messages to each other. So, A and B both generate "secret" keys at +random which they do not reveal to anyone. Let these keys be +represented as SK(A) and SK(B). They also publish in a public +directory their "public" keys. These keys are computed as follows: +.ie t .DS +.el .DS L +.ft CW +PK(A) = ( BASE ** SK(A) ) mod MODULUS +PK(B) = ( BASE ** SK(B) ) mod MODULUS +.DE +.ft R +The "**" notation is used here to represent exponentiation. Now, +both A and B can arrive at the "common" key between them, +represented here as CK(A, B), without revealing their secret keys. +.LP +A computes: +.ie t .DS +.el .DS L +.ft CW +CK(A, B) = ( PK(B) ** SK(A)) mod MODULUS +.DE +.ft R +while B computes: +.ie t .DS +.el .DS L +.ft CW +CK(A, B) = ( PK(A) ** SK(B)) mod MODULUS +.DE +.ft R +These two can be shown to be equivalent: +.ie t .DS +.el .DS L +.ft CW +(PK(B) ** SK(A)) mod MODULUS = (PK(A) ** SK(B)) mod MODULUS +.DE +.ft R +We drop the "mod MODULUS" parts and assume modulo arithmetic to +simplify things: +.ie t .DS +.el .DS L +.ft CW +PK(B) ** SK(A) = PK(A) ** SK(B) +.DE +.ft R +Then, replace PK(B) by what B computed earlier and likewise for +PK(A). +.ie t .DS +.el .DS L +.ft CW +((BASE ** SK(B)) ** SK(A) = (BASE ** SK(A)) ** SK(B) +.DE +.ft R +which leads to: +.ie t .DS +.el .DS L +.ft CW +BASE ** (SK(A) * SK(B)) = BASE ** (SK(A) * SK(B)) +.DE +.ft R +This common key CK(A, B) is not used to encrypt the timestamps used +in the protocol. Rather, it is used only to encrypt a conversation +key which is then used to encrypt the timestamps. The reason for +doing this is to use the common key as little as possible, for fear +that it could be broken. Breaking the conversation key is a far +less serious offense, since conversations are relatively +short-lived. +.LP +The conversation key is encrypted using 56-bit DES keys, yet the +common key is 192 bits. To reduce the number of bits, 56 bits are +selected from the common key as follows. The middle-most 8-bytes +are selected from the common key, and then parity is added to the +lower order bit of each byte, producing a 56-bit key with 8 bits of +parity. +.KS +.NH 1 +\&Record Marking Standard +.LP +When RPC messages are passed on top of a byte stream protocol (like +TCP/IP), it is necessary, or at least desirable, to delimit one +message from another in order to detect and possibly recover from +user protocol errors. This is called record marking (RM). Sun uses +this RM/TCP/IP transport for passing RPC messages on TCP streams. +One RPC message fits into one RM record. +.LP +A record is composed of one or more record fragments. A record +fragment is a four-byte header followed by 0 to (2**31) - 1 bytes of +fragment data. The bytes encode an unsigned binary number; as with +XDR integers, the byte order is from highest to lowest. The number +encodes two values\(ema boolean which indicates whether the fragment +is the last fragment of the record (bit value 1 implies the fragment +is the last fragment) and a 31-bit unsigned binary value which is the +length in bytes of the fragment's data. The boolean value is the +highest-order bit of the header; the length is the 31 low-order bits. +(Note that this record specification is NOT in XDR standard form!) +.KE +.KS +.NH 1 +\&The RPC Language +.LP +Just as there was a need to describe the XDR data-types in a formal +language, there is also need to describe the procedures that operate +on these XDR data-types in a formal language as well. We use the RPC +Language for this purpose. It is an extension to the XDR language. +The following example is used to describe the essence of the +language. +.NH 2 +\&An Example Service Described in the RPC Language +.LP +Here is an example of the specification of a simple ping program. +.ie t .DS +.el .DS L +.vs 11 +.ft I +/* +* Simple ping program +*/ +.ft CW +program PING_PROG { + /* \fILatest and greatest version\fP */ + version PING_VERS_PINGBACK { + void + PINGPROC_NULL(void) = 0; + +.ft I + /* + * Ping the caller, return the round-trip time + * (in microseconds). Returns -1 if the operation + * timed out. + */ +.ft CW + int + PINGPROC_PINGBACK(void) = 1; +} = 2; + +.ft I +/* +* Original version +*/ +.ft CW +version PING_VERS_ORIG { + void + PINGPROC_NULL(void) = 0; + } = 1; +} = 1; + +const PING_VERS = 2; /* \fIlatest version \fP*/ +.vs +.DE +.KE +.LP +The first version described is +.I PING_VERS_PINGBACK +with two procedures, +.I PINGPROC_NULL +and +.I PINGPROC_PINGBACK . +.I PINGPROC_NULL +takes no arguments and returns no results, but it is useful for +computing round-trip times from the client to the server and back +again. By convention, procedure 0 of any RPC protocol should have +the same semantics, and never require any kind of authentication. +The second procedure is used for the client to have the server do a +reverse ping operation back to the client, and it returns the amount +of time (in microseconds) that the operation used. The next version, +.I PING_VERS_ORIG , +is the original version of the protocol +and it does not contain +.I PINGPROC_PINGBACK +procedure. It is useful +for compatibility with old client programs, and as this program +matures it may be dropped from the protocol entirely. +.KS +.NH 2 +\&The RPC Language Specification +.LP +The RPC language is identical to the XDR language, except for the +added definition of a +.I program-def +described below. +.DS +.ft CW +program-def: + "program" identifier "{" + version-def + version-def * + "}" "=" constant ";" + +version-def: + "version" identifier "{" + procedure-def + procedure-def * + "}" "=" constant ";" + +procedure-def: + type-specifier identifier "(" type-specifier ")" + "=" constant ";" +.DE +.KE +.NH 2 +\&Syntax Notes +.IP 1. +The following keywords are added and cannot be used as +identifiers: "program" and "version"; +.IP 2. +A version name cannot occur more than once within the scope of +a program definition. Nor can a version number occur more than once +within the scope of a program definition. +.IP 3. +A procedure name cannot occur more than once within the scope +of a version definition. Nor can a procedure number occur more than +once within the scope of version definition. +.IP 4. +Program identifiers are in the same name space as constant and +type identifiers. +.IP 5. +Only unsigned constants can be assigned to programs, versions +and procedures. +.NH 1 +\&Port Mapper Program Protocol +.LP +The port mapper program maps RPC program and version numbers to +transport-specific port numbers. This program makes dynamic binding +of remote programs possible. +.LP +This is desirable because the range of reserved port numbers is very +small and the number of potential remote programs is very large. By +running only the port mapper on a reserved port, the port numbers of +other remote programs can be ascertained by querying the port mapper. +.LP +The port mapper also aids in broadcast RPC. A given RPC program will +usually have different port number bindings on different machines, so +there is no way to directly broadcast to all of these programs. The +port mapper, however, does have a fixed port number. So, to +broadcast to a given program, the client actually sends its message +to the port mapper located at the broadcast address. Each port +mapper that picks up the broadcast then calls the local service +specified by the client. When the port mapper gets the reply from +the local service, it sends the reply on back to the client. +.KS +.NH 2 +\&Port Mapper Protocol Specification (in RPC Language) +.ie t .DS +.el .DS L +.ft CW +.vs 11 +const PMAP_PORT = 111; /* \fIportmapper port number \fP*/ + +.ft I +/* +* A mapping of (program, version, protocol) to port number +*/ +.ft CW +struct mapping { + unsigned int prog; + unsigned int vers; + unsigned int prot; + unsigned int port; +}; + +.ft I +/* +* Supported values for the "prot" field +*/ +.ft CW +const IPPROTO_TCP = 6; /* \fIprotocol number for TCP/IP \fP*/ +const IPPROTO_UDP = 17; /* \fIprotocol number for UDP/IP \fP*/ + +.ft I +/* +* A list of mappings +*/ +.ft CW +struct *pmaplist { + mapping map; + pmaplist next; +}; +.vs +.DE +.ie t .DS +.el .DS L +.vs 11 +.ft I +/* +* Arguments to callit +*/ +.ft CW +struct call_args { + unsigned int prog; + unsigned int vers; + unsigned int proc; + opaque args<>; +}; + +.ft I +/* +* Results of callit +*/ +.ft CW +struct call_result { + unsigned int port; + opaque res<>; +}; +.vs +.DE +.KE +.ie t .DS +.el .DS L +.vs 11 +.ft I +/* +* Port mapper procedures +*/ +.ft CW +program PMAP_PROG { + version PMAP_VERS { + void + PMAPPROC_NULL(void) = 0; + + bool + PMAPPROC_SET(mapping) = 1; + + bool + PMAPPROC_UNSET(mapping) = 2; + + unsigned int + PMAPPROC_GETPORT(mapping) = 3; + + pmaplist + PMAPPROC_DUMP(void) = 4; + + call_result + PMAPPROC_CALLIT(call_args) = 5; + } = 2; +} = 100000; +.vs +.DE +.NH 2 +\&Port Mapper Operation +.LP +The portmapper program currently supports two protocols (UDP/IP and +TCP/IP). The portmapper is contacted by talking to it on assigned +port number 111 (SUNRPC [8]) on either of these protocols. The +following is a description of each of the portmapper procedures: +.IP \fBPMAPPROC_NULL:\fP +This procedure does no work. By convention, procedure zero of any +protocol takes no parameters and returns no results. +.IP \fBPMAPPROC_SET:\fP +When a program first becomes available on a machine, it registers +itself with the port mapper program on the same machine. The program +passes its program number "prog", version number "vers", transport +protocol number "prot", and the port "port" on which it awaits +service request. The procedure returns a boolean response whose +value is +.I TRUE +if the procedure successfully established the mapping and +.I FALSE +otherwise. The procedure refuses to establish +a mapping if one already exists for the tuple "(prog, vers, prot)". +.IP \fBPMAPPROC_UNSET:\fP +When a program becomes unavailable, it should unregister itself with +the port mapper program on the same machine. The parameters and +results have meanings identical to those of +.I PMAPPROC_SET . +The protocol and port number fields of the argument are ignored. +.IP \fBPMAPPROC_GETPORT:\fP +Given a program number "prog", version number "vers", and transport +protocol number "prot", this procedure returns the port number on +which the program is awaiting call requests. A port value of zeros +means the program has not been registered. The "port" field of the +argument is ignored. +.IP \fBPMAPPROC_DUMP:\fP +This procedure enumerates all entries in the port mapper's database. +The procedure takes no parameters and returns a list of program, +version, protocol, and port values. +.IP \fBPMAPPROC_CALLIT:\fP +This procedure allows a caller to call another remote procedure on +the same machine without knowing the remote procedure's port number. +It is intended for supporting broadcasts to arbitrary remote programs +via the well-known port mapper's port. The parameters "prog", +"vers", "proc", and the bytes of "args" are the program number, +version number, procedure number, and parameters of the remote +procedure. +.LP +.B Note: +.RS +.IP 1. +This procedure only sends a response if the procedure was +successfully executed and is silent (no response) otherwise. +.IP 2. +The port mapper communicates with the remote program using UDP/IP +only. +.RE +.LP +The procedure returns the remote program's port number, and the bytes +of results are the results of the remote procedure. +.bp +.NH 1 +\&References +.LP +[1] Birrell, Andrew D. & Nelson, Bruce Jay; "Implementing Remote +Procedure Calls"; XEROX CSL-83-7, October 1983. +.LP +[2] Cheriton, D.; "VMTP: Versatile Message Transaction Protocol", +Preliminary Version 0.3; Stanford University, January 1987. +.LP +[3] Diffie & Hellman; "New Directions in Cryptography"; IEEE +Transactions on Information Theory IT-22, November 1976. +.LP +[4] Harrenstien, K.; "Time Server", RFC 738; Information Sciences +Institute, October 1977. +.LP +[5] National Bureau of Standards; "Data Encryption Standard"; Federal +Information Processing Standards Publication 46, January 1977. +.LP +[6] Postel, J.; "Transmission Control Protocol - DARPA Internet +Program Protocol Specification", RFC 793; Information Sciences +Institute, September 1981. +.LP +[7] Postel, J.; "User Datagram Protocol", RFC 768; Information Sciences +Institute, August 1980. +.LP +[8] Reynolds, J. & Postel, J.; "Assigned Numbers", RFC 923; Information +Sciences Institute, October 1984. diff --git a/share/doc/psd/26.rpcrfc/stubs b/share/doc/psd/26.rpcrfc/stubs new file mode 100644 index 000000000000..78b0a2c815d3 --- /dev/null +++ b/share/doc/psd/26.rpcrfc/stubs @@ -0,0 +1,3 @@ +.\" $FreeBSD$ +.\" +.if t .ftr L CR diff --git a/share/doc/psd/27.nfsrpc/Makefile b/share/doc/psd/27.nfsrpc/Makefile new file mode 100644 index 000000000000..c9d4f2308c4a --- /dev/null +++ b/share/doc/psd/27.nfsrpc/Makefile @@ -0,0 +1,8 @@ +# $FreeBSD$ + +VOLUME= psd/27.nfsrfc +SRCS= stubs nfs.rfc.ms +MACROS= -ms +USE_TBL= + +.include <bsd.doc.mk> diff --git a/share/doc/psd/27.nfsrpc/nfs.rfc.ms b/share/doc/psd/27.nfsrpc/nfs.rfc.ms new file mode 100644 index 000000000000..5a2d9492d39f --- /dev/null +++ b/share/doc/psd/27.nfsrpc/nfs.rfc.ms @@ -0,0 +1,1374 @@ +.\" +.\" Must use -- tbl -- with this one +.\" +.\" @(#)nfs.rfc.ms 2.2 88/08/05 4.0 RPCSRC +.\" $FreeBSD$ +.\" +.de BT +.if \\n%=1 .tl ''- % -'' +.. +.ND +.\" prevent excess underlining in nroff +.if n .fp 2 R +.OH 'Network File System: Version 2 Protocol Specification''Page %' +.EH 'Page %''Network File System: Version 2 Protocol Specification' +.if \n%=1 .bp +.SH +\&Network File System: Version 2 Protocol Specification +.IX NFS "" "" "" PAGE MAJOR +.IX "Network File System" "" "" "" PAGE MAJOR +.IX NFS "version-2 protocol specification" +.IX "Network File System" "version-2 protocol specification" +.LP +.NH 0 +\&Status of this Standard +.LP +Note: This document specifies a protocol that Sun Microsystems, Inc., +and others are using. It specifies it in standard ARPA RFC form. +.NH 1 +\&Introduction +.IX NFS introduction +.LP +The Sun Network Filesystem (NFS) protocol provides transparent remote +access to shared filesystems over local area networks. The NFS +protocol is designed to be machine, operating system, network architecture, +and transport protocol independent. This independence is +achieved through the use of Remote Procedure Call (RPC) primitives +built on top of an External Data Representation (XDR). Implementations +exist for a variety of machines, from personal computers to +supercomputers. +.LP +The supporting mount protocol allows the server to hand out remote +access privileges to a restricted set of clients. It performs the +operating system-specific functions that allow, for example, to +attach remote directory trees to some local file system. +.NH 2 +\&Remote Procedure Call +.IX "Remote Procedure Call" +.LP +Sun's remote procedure call specification provides a procedure- +oriented interface to remote services. Each server supplies a +program that is a set of procedures. NFS is one such "program". +The combination of host address, program number, and procedure +number specifies one remote service procedure. RPC does not depend +on services provided by specific protocols, so it can be used with +any underlying transport protocol. See the +.I "Remote Procedure Calls: Protocol Specification" +chapter of this manual. +.NH 2 +\&External Data Representation +.IX "External Data Representation" +.LP +The External Data Representation (XDR) standard provides a common +way of representing a set of data types over a network. +The NFS +Protocol Specification is written using the RPC data description +language. +For more information, see the +.I " External Data Representation Standard: Protocol Specification." +Sun provides implementations of XDR and +RPC, but NFS does not require their use. Any software that +provides equivalent functionality can be used, and if the encoding +is exactly the same it can interoperate with other implementations +of NFS. +.NH 2 +\&Stateless Servers +.IX "stateless servers" +.IX servers stateless +.LP +The NFS protocol is stateless. That is, a server does not need to +maintain any extra state information about any of its clients in +order to function correctly. Stateless servers have a distinct +advantage over stateful servers in the event of a failure. With +stateless servers, a client need only retry a request until the +server responds; it does not even need to know that the server has +crashed, or the network temporarily went down. The client of a +stateful server, on the other hand, needs to either detect a server +crash and rebuild the server's state when it comes back up, or +cause client operations to fail. +.LP +This may not sound like an important issue, but it affects the +protocol in some unexpected ways. We feel that it is worth a bit +of extra complexity in the protocol to be able to write very simple +servers that do not require fancy crash recovery. +.LP +On the other hand, NFS deals with objects such as files and +directories that inherently have state -- what good would a file be +if it did not keep its contents intact? The goal is to not +introduce any extra state in the protocol itself. Another way to +simplify recovery is by making operations "idempotent" whenever +possible (so that they can potentially be repeated). +.NH 1 +\&NFS Protocol Definition +.IX NFS "protocol definition" +.IX NFS protocol +.LP +Servers have been known to change over time, and so can the +protocol that they use. So RPC provides a version number with each +RPC request. This RFC describes version two of the NFS protocol. +Even in the second version, there are various obsolete procedures +and parameters, which will be removed in later versions. An RFC +for version three of the NFS protocol is currently under +preparation. +.NH 2 +\&File System Model +.IX filesystem model +.LP +NFS assumes a file system that is hierarchical, with directories as +all but the bottom-level files. Each entry in a directory (file, +directory, device, etc.) has a string name. Different operating +systems may have restrictions on the depth of the tree or the names +used, as well as using different syntax to represent the "pathname", +which is the concatenation of all the "components" (directory and +file names) in the name. A "file system" is a tree on a single +server (usually a single disk or physical partition) with a specified +"root". Some operating systems provide a "mount" operation to make +all file systems appear as a single tree, while others maintain a +"forest" of file systems. Files are unstructured streams of +uninterpreted bytes. Version 3 of NFS uses a slightly more general +file system model. +.LP +NFS looks up one component of a pathname at a time. It may not be +obvious why it does not just take the whole pathname, traipse down +the directories, and return a file handle when it is done. There are +several good reasons not to do this. First, pathnames need +separators between the directory components, and different operating +systems use different separators. We could define a Network Standard +Pathname Representation, but then every pathname would have to be +parsed and converted at each end. Other issues are discussed in +\fINFS Implementation Issues\fP below. +.LP +Although files and directories are similar objects in many ways, +different procedures are used to read directories and files. This +provides a network standard format for representing directories. The +same argument as above could have been used to justify a procedure +that returns only one directory entry per call. The problem is +efficiency. Directories can contain many entries, and a remote call +to return each would be just too slow. +.NH 2 +\&RPC Information +.IX NFS "RPC information" +.IP \fIAuthentication\fP +The NFS service uses +.I AUTH_UNIX , +.I AUTH_DES , +or +.I AUTH_SHORT +style +authentication, except in the NULL procedure where +.I AUTH_NONE +is also allowed. +.IP "\fITransport Protocols\fP" +NFS currently is supported on UDP/IP only. +.IP "\fIPort Number\fP" +The NFS protocol currently uses the UDP port number 2049. This is +not an officially assigned port, so later versions of the protocol +use the \*QPortmapping\*U facility of RPC. +.NH 2 +\&Sizes of XDR Structures +.IX "XDR structure sizes" +.LP +These are the sizes, given in decimal bytes, of various XDR +structures used in the protocol: +.DS +/* \fIThe maximum number of bytes of data in a READ or WRITE request\fP */ +const MAXDATA = 8192; + +/* \fIThe maximum number of bytes in a pathname argument\fP */ +const MAXPATHLEN = 1024; + +/* \fIThe maximum number of bytes in a file name argument\fP */ +const MAXNAMLEN = 255; + +/* \fIThe size in bytes of the opaque "cookie" passed by READDIR\fP */ +const COOKIESIZE = 4; + +/* \fIThe size in bytes of the opaque file handle\fP */ +const FHSIZE = 32; +.DE +.NH 2 +\&Basic Data Types +.IX "NFS data types" +.IX NFS "basic data types" +.LP +The following XDR definitions are basic structures and types used +in other structures described further on. +.KS +.NH 3 +\&stat +.IX "NFS data types" stat "" \fIstat\fP +.DS +enum stat { + NFS_OK = 0, + NFSERR_PERM=1, + NFSERR_NOENT=2, + NFSERR_IO=5, + NFSERR_NXIO=6, + NFSERR_ACCES=13, + NFSERR_EXIST=17, + NFSERR_NODEV=19, + NFSERR_NOTDIR=20, + NFSERR_ISDIR=21, + NFSERR_FBIG=27, + NFSERR_NOSPC=28, + NFSERR_ROFS=30, + NFSERR_NAMETOOLONG=63, + NFSERR_NOTEMPTY=66, + NFSERR_DQUOT=69, + NFSERR_STALE=70, + NFSERR_WFLUSH=99 +}; +.DE +.KE +.LP +The +.I stat +type is returned with every procedure's results. A +value of +.I NFS_OK +indicates that the call completed successfully and +the results are valid. The other values indicate some kind of +error occurred on the server side during the servicing of the +procedure. The error values are derived from UNIX error numbers. +.IP \fBNFSERR_PERM\fP: +Not owner. The caller does not have correct ownership +to perform the requested operation. +.IP \fBNFSERR_NOENT\fP: +No such file or directory. The file or directory +specified does not exist. +.IP \fBNFSERR_IO\fP: +Some sort of hard error occurred when the operation was +in progress. This could be a disk error, for example. +.IP \fBNFSERR_NXIO\fP: +No such device or address. +.IP \fBNFSERR_ACCES\fP: +Permission denied. The caller does not have the +correct permission to perform the requested operation. +.IP \fBNFSERR_EXIST\fP: +File exists. The file specified already exists. +.IP \fBNFSERR_NODEV\fP: +No such device. +.IP \fBNFSERR_NOTDIR\fP: +Not a directory. The caller specified a +non-directory in a directory operation. +.IP \fBNFSERR_ISDIR\fP: +Is a directory. The caller specified a directory in +a non- directory operation. +.IP \fBNFSERR_FBIG\fP: +File too large. The operation caused a file to grow +beyond the server's limit. +.IP \fBNFSERR_NOSPC\fP: +No space left on device. The operation caused the +server's filesystem to reach its limit. +.IP \fBNFSERR_ROFS\fP: +Read-only filesystem. Write attempted on a read-only filesystem. +.IP \fBNFSERR_NAMETOOLONG\fP: +File name too long. The file name in an operation was too long. +.IP \fBNFSERR_NOTEMPTY\fP: +Directory not empty. Attempted to remove a +directory that was not empty. +.IP \fBNFSERR_DQUOT\fP: +Disk quota exceeded. The client's disk quota on the +server has been exceeded. +.IP \fBNFSERR_STALE\fP: +The "fhandle" given in the arguments was invalid. +That is, the file referred to by that file handle no longer exists, +or access to it has been revoked. +.IP \fBNFSERR_WFLUSH\fP: +The server's write cache used in the +.I WRITECACHE +call got flushed to disk. +.LP +.KS +.NH 3 +\&ftype +.IX "NFS data types" ftype "" \fIftype\fP +.DS +enum ftype { + NFNON = 0, + NFREG = 1, + NFDIR = 2, + NFBLK = 3, + NFCHR = 4, + NFLNK = 5 +}; +.DE +.KE +The enumeration +.I ftype +gives the type of a file. The type +.I NFNON +indicates a non-file, +.I NFREG +is a regular file, +.I NFDIR +is a directory, +.I NFBLK +is a block-special device, +.I NFCHR +is a character-special device, and +.I NFLNK +is a symbolic link. +.KS +.NH 3 +\&fhandle +.IX "NFS data types" fhandle "" \fIfhandle\fP +.DS +typedef opaque fhandle[FHSIZE]; +.DE +.KE +The +.I fhandle +is the file handle passed between the server and the client. +All file operations are done using file handles to refer to a file or +directory. The file handle can contain whatever information the server +needs to distinguish an individual file. +.KS +.NH 3 +\&timeval +.IX "NFS data types" timeval "" \fItimeval\fP +.DS +struct timeval { + unsigned int seconds; + unsigned int useconds; +}; +.DE +.KE +The +.I timeval +structure is the number of seconds and microseconds +since midnight January 1, 1970, Greenwich Mean Time. It is used to +pass time and date information. +.KS +.NH 3 +\&fattr +.IX "NFS data types" fattr "" \fIfattr\fP +.DS +struct fattr { + ftype type; + unsigned int mode; + unsigned int nlink; + unsigned int uid; + unsigned int gid; + unsigned int size; + unsigned int blocksize; + unsigned int rdev; + unsigned int blocks; + unsigned int fsid; + unsigned int fileid; + timeval atime; + timeval mtime; + timeval ctime; +}; +.DE +.KE +The +.I fattr +structure contains the attributes of a file; "type" is the type of +the file; "nlink" is the number of hard links to the file (the number +of different names for the same file); "uid" is the user +identification number of the owner of the file; "gid" is the group +identification number of the group of the file; "size" is the size in +bytes of the file; "blocksize" is the size in bytes of a block of the +file; "rdev" is the device number of the file if it is type +.I NFCHR +or +.I NFBLK ; +"blocks" is the number of blocks the file takes up on disk; "fsid" is +the file system identifier for the filesystem containing the file; +"fileid" is a number that uniquely identifies the file within its +filesystem; "atime" is the time when the file was last accessed for +either read or write; "mtime" is the time when the file data was last +modified (written); and "ctime" is the time when the status of the +file was last changed. Writing to the file also changes "ctime" if +the size of the file changes. +.LP +"mode" is the access mode encoded as a set of bits. Notice that the +file type is specified both in the mode bits and in the file type. +This is really a bug in the protocol and will be fixed in future +versions. The descriptions given below specify the bit positions +using octal numbers. +.TS +box tab (&) ; +cfI cfI +lfL l . +Bit&Description +_ +0040000&This is a directory; "type" field should be NFDIR. +0020000&This is a character special file; "type" field should be NFCHR. +0060000&This is a block special file; "type" field should be NFBLK. +0100000&This is a regular file; "type" field should be NFREG. +0120000&This is a symbolic link file; "type" field should be NFLNK. +0140000&This is a named socket; "type" field should be NFNON. +0004000&Set user id on execution. +0002000&Set group id on execution. +0001000&Save swapped text even after use. +0000400&Read permission for owner. +0000200&Write permission for owner. +0000100&Execute and search permission for owner. +0000040&Read permission for group. +0000020&Write permission for group. +0000010&Execute and search permission for group. +0000004&Read permission for others. +0000002&Write permission for others. +0000001&Execute and search permission for others. +.TE +.KS +Notes: +.IP +The bits are the same as the mode bits returned by the +.I stat(2) +system call in the UNIX system. The file type is specified both in +the mode bits and in the file type. This is fixed in future +versions. +.IP +The "rdev" field in the attributes structure is an operating system +specific device specifier. It will be removed and generalized in +the next revision of the protocol. +.KE +.LP +.KS +.NH 3 +\&sattr +.IX "NFS data types" sattr "" \fIsattr\fP +.DS +struct sattr { + unsigned int mode; + unsigned int uid; + unsigned int gid; + unsigned int size; + timeval atime; + timeval mtime; +}; +.DE +.KE +The +.I sattr +structure contains the file attributes which can be set +from the client. The fields are the same as for +.I fattr +above. A "size" of zero means the file should be truncated. +A value of -1 indicates a field that should be ignored. +.LP +.KS +.NH 3 +\&filename +.IX "NFS data types" filename "" \fIfilename\fP +.DS +typedef string filename<MAXNAMLEN>; +.DE +.KE +The type +.I filename +is used for passing file names or pathname components. +.LP +.KS +.NH 3 +\&path +.IX "NFS data types" path "" \fIpath\fP +.DS +typedef string path<MAXPATHLEN>; +.DE +.KE +The type +.I path +is a pathname. The server considers it as a string +with no internal structure, but to the client it is the name of a +node in a filesystem tree. +.LP +.KS +.NH 3 +\&attrstat +.IX "NFS data types" attrstat "" \fIattrstat\fP +.DS +union attrstat switch (stat status) { + case NFS_OK: + fattr attributes; + default: + void; +}; +.DE +.KE +The +.I attrstat +structure is a common procedure result. It contains +a "status" and, if the call succeeded, it also contains the +attributes of the file on which the operation was done. +.LP +.KS +.NH 3 +\&diropargs +.IX "NFS data types" diropargs "" \fIdiropargs\fP +.DS +struct diropargs { + fhandle dir; + filename name; +}; +.DE +.KE +The +.I diropargs +structure is used in directory operations. The +"fhandle" "dir" is the directory in which to find the file "name". +A directory operation is one in which the directory is affected. +.LP +.KS +.NH 3 +\&diropres +.IX "NFS data types" diropres "" \fIdiropres\fP +.DS +union diropres switch (stat status) { + case NFS_OK: + struct { + fhandle file; + fattr attributes; + } diropok; + default: + void; +}; +.DE +.KE +The results of a directory operation are returned in a +.I diropres +structure. If the call succeeded, a new file handle "file" and the +"attributes" associated with that file are returned along with the +"status". +.NH 2 +\&Server Procedures +.IX "NFS server procedures" "" "" "" PAGE MAJOR +.LP +The protocol definition is given as a set of procedures with +arguments and results defined using the RPC language. A brief +description of the function of each procedure should provide enough +information to allow implementation. +.LP +All of the procedures in the NFS protocol are assumed to be +synchronous. When a procedure returns to the client, the client +can assume that the operation has completed and any data associated +with the request is now on stable storage. For example, a client +.I WRITE +request may cause the server to update data blocks, +filesystem information blocks (such as indirect blocks), and file +attribute information (size and modify times). When the +.I WRITE +returns to the client, it can assume that the write is safe, even +in case of a server crash, and it can discard the data written. +This is a very important part of the statelessness of the server. +If the server waited to flush data from remote requests, the client +would have to save those requests so that it could resend them in +case of a server crash. +.ie t .DS +.el .DS L + +.ft I +/* +* Remote file service routines +*/ +.ft CW +program NFS_PROGRAM { + version NFS_VERSION { + void NFSPROC_NULL(void) = 0; + attrstat NFSPROC_GETATTR(fhandle) = 1; + attrstat NFSPROC_SETATTR(sattrargs) = 2; + void NFSPROC_ROOT(void) = 3; + diropres NFSPROC_LOOKUP(diropargs) = 4; + readlinkres NFSPROC_READLINK(fhandle) = 5; + readres NFSPROC_READ(readargs) = 6; + void NFSPROC_WRITECACHE(void) = 7; + attrstat NFSPROC_WRITE(writeargs) = 8; + diropres NFSPROC_CREATE(createargs) = 9; + stat NFSPROC_REMOVE(diropargs) = 10; + stat NFSPROC_RENAME(renameargs) = 11; + stat NFSPROC_LINK(linkargs) = 12; + stat NFSPROC_SYMLINK(symlinkargs) = 13; + diropres NFSPROC_MKDIR(createargs) = 14; + stat NFSPROC_RMDIR(diropargs) = 15; + readdirres NFSPROC_READDIR(readdirargs) = 16; + statfsres NFSPROC_STATFS(fhandle) = 17; + } = 2; +} = 100003; +.DE +.KS +.NH 3 +\&Do Nothing +.IX "NFS server procedures" NFSPROC_NULL() "" \fINFSPROC_NULL()\fP +.DS +void +NFSPROC_NULL(void) = 0; +.DE +.KE +This procedure does no work. It is made available in all RPC +services to allow server response testing and timing. +.KS +.NH 3 +\&Get File Attributes +.IX "NFS server procedures" NFSPROC_GETATTR() "" \fINFSPROC_GETATTR()\fP +.DS +attrstat +NFSPROC_GETATTR (fhandle) = 1; +.DE +.KE +If the reply status is +.I NFS_OK , +then the reply attributes contains +the attributes for the file given by the input fhandle. +.KS +.NH 3 +\&Set File Attributes +.IX "NFS server procedures" NFSPROC_SETATTR() "" \fINFSPROC_SETATTR()\fP +.DS +struct sattrargs { + fhandle file; + sattr attributes; + }; + +attrstat +NFSPROC_SETATTR (sattrargs) = 2; +.DE +.KE +The "attributes" argument contains fields which are either -1 or +are the new value for the attributes of "file". If the reply +status is +.I NFS_OK , +then the reply attributes have the attributes of +the file after the "SETATTR" operation has completed. +.LP +Note: The use of -1 to indicate an unused field in "attributes" is +changed in the next version of the protocol. +.KS +.NH 3 +\&Get Filesystem Root +.IX "NFS server procedures" NFSPROC_ROOT "" \fINFSPROC_ROOT\fP +.DS +void +NFSPROC_ROOT(void) = 3; +.DE +.KE +Obsolete. This procedure is no longer used because finding the +root file handle of a filesystem requires moving pathnames between +client and server. To do this right we would have to define a +network standard representation of pathnames. Instead, the +function of looking up the root file handle is done by the +.I MNTPROC_MNT() +procedure. (See the +.I "Mount Protocol Definition" +later in this chapter for details). +.KS +.NH 3 +\&Look Up File Name +.IX "NFS server procedures" NFSPROC_LOOKUP() "" \fINFSPROC_LOOKUP()\fP +.DS +diropres +NFSPROC_LOOKUP(diropargs) = 4; +.DE +.KE +If the reply "status" is +.I NFS_OK , +then the reply "file" and reply +"attributes" are the file handle and attributes for the file "name" +in the directory given by "dir" in the argument. +.KS +.NH 3 +\&Read From Symbolic Link +.IX "NFS server procedures" NFSPROC_READLINK() "" \fINFSPROC_READLINK()\fP +.DS +union readlinkres switch (stat status) { + case NFS_OK: + path data; + default: + void; +}; + +readlinkres +NFSPROC_READLINK(fhandle) = 5; +.DE +.KE +If "status" has the value +.I NFS_OK , +then the reply "data" is the data in +the symbolic link given by the file referred to by the fhandle argument. +.LP +Note: since NFS always parses pathnames on the client, the +pathname in a symbolic link may mean something different (or be +meaningless) on a different client or on the server if a different +pathname syntax is used. +.KS +.NH 3 +\&Read From File +.IX "NFS server procedures" NFSPROC_READ "" \fINFSPROC_READ\fP +.DS +struct readargs { + fhandle file; + unsigned offset; + unsigned count; + unsigned totalcount; +}; + +union readres switch (stat status) { + case NFS_OK: + fattr attributes; + opaque data<NFS_MAXDATA>; + default: + void; +}; + +readres +NFSPROC_READ(readargs) = 6; +.DE +.KE +Returns up to "count" bytes of "data" from the file given by +"file", starting at "offset" bytes from the beginning of the file. +The first byte of the file is at offset zero. The file attributes +after the read takes place are returned in "attributes". +.LP +Note: The argument "totalcount" is unused, and is removed in the +next protocol revision. +.KS +.NH 3 +\&Write to Cache +.IX "NFS server procedures" NFSPROC_WRITECACHE() "" \fINFSPROC_WRITECACHE()\fP +.DS +void +NFSPROC_WRITECACHE(void) = 7; +.DE +.KE +To be used in the next protocol revision. +.KS +.NH 3 +\&Write to File +.IX "NFS server procedures" NFSPROC_WRITE() "" \fINFSPROC_WRITE()\fP +.DS +struct writeargs { + fhandle file; + unsigned beginoffset; + unsigned offset; + unsigned totalcount; + opaque data<NFS_MAXDATA>; +}; + +attrstat +NFSPROC_WRITE(writeargs) = 8; +.DE +.KE +Writes "data" beginning "offset" bytes from the beginning of +"file". The first byte of the file is at offset zero. If the +reply "status" is NFS_OK, then the reply "attributes" contains the +attributes of the file after the write has completed. The write +operation is atomic. Data from this call to +.I WRITE +will not be mixed with data from another client's calls. +.LP +Note: The arguments "beginoffset" and "totalcount" are ignored and +are removed in the next protocol revision. +.KS +.NH 3 +\&Create File +.IX "NFS server procedures" NFSPROC_CREATE() "" \fINFSPROC_CREATE()\fP +.DS +struct createargs { + diropargs where; + sattr attributes; +}; + +diropres +NFSPROC_CREATE(createargs) = 9; +.DE +.KE +The file "name" is created in the directory given by "dir". The +initial attributes of the new file are given by "attributes". A +reply "status" of NFS_OK indicates that the file was created, and +reply "file" and reply "attributes" are its file handle and +attributes. Any other reply "status" means that the operation +failed and no file was created. +.LP +Note: This routine should pass an exclusive create flag, meaning +"create the file only if it is not already there". +.KS +.NH 3 +\&Remove File +.IX "NFS server procedures" NFSPROC_REMOVE() "" \fINFSPROC_REMOVE()\fP +.DS +stat +NFSPROC_REMOVE(diropargs) = 10; +.DE +.KE +The file "name" is removed from the directory given by "dir". A +reply of NFS_OK means the directory entry was removed. +.LP +Note: possibly non-idempotent operation. +.KS +.NH 3 +\&Rename File +.IX "NFS server procedures" NFSPROC_RENAME() "" \fINFSPROC_RENAME()\fP +.DS +struct renameargs { + diropargs from; + diropargs to; +}; + +stat +NFSPROC_RENAME(renameargs) = 11; +.DE +.KE +The existing file "from.name" in the directory given by "from.dir" +is renamed to "to.name" in the directory given by "to.dir". If the +reply is +.I NFS_OK , +the file was renamed. The +RENAME +operation is +atomic on the server; it cannot be interrupted in the middle. +.LP +Note: possibly non-idempotent operation. +.KS +.NH 3 +\&Create Link to File +.IX "NFS server procedures" NFSPROC_LINK() "" \fINFSPROC_LINK()\fP +.DS +struct linkargs { + fhandle from; + diropargs to; +}; + +stat +NFSPROC_LINK(linkargs) = 12; +.DE +.KE +Creates the file "to.name" in the directory given by "to.dir", +which is a hard link to the existing file given by "from". If the +return value is +.I NFS_OK , +a link was created. Any other return value +indicates an error, and the link was not created. +.LP +A hard link should have the property that changes to either of the +linked files are reflected in both files. When a hard link is made +to a file, the attributes for the file should have a value for +"nlink" that is one greater than the value before the link. +.LP +Note: possibly non-idempotent operation. +.KS +.NH 3 +\&Create Symbolic Link +.IX "NFS server procedures" NFSPROC_SYMLINK() "" \fINFSPROC_SYMLINK()\fP +.DS +struct symlinkargs { + diropargs from; + path to; + sattr attributes; +}; + +stat +NFSPROC_SYMLINK(symlinkargs) = 13; +.DE +.KE +Creates the file "from.name" with ftype +.I NFLNK +in the directory +given by "from.dir". The new file contains the pathname "to" and +has initial attributes given by "attributes". If the return value +is +.I NFS_OK , +a link was created. Any other return value indicates an +error, and the link was not created. +.LP +A symbolic link is a pointer to another file. The name given in +"to" is not interpreted by the server, only stored in the newly +created file. When the client references a file that is a symbolic +link, the contents of the symbolic link are normally transparently +reinterpreted as a pathname to substitute. A +.I READLINK +operation returns the data to the client for interpretation. +.LP +Note: On UNIX servers the attributes are never used, since +symbolic links always have mode 0777. +.KS +.NH 3 +\&Create Directory +.IX "NFS server procedures" NFSPROC_MKDIR() "" \fINFSPROC_MKDIR()\fP +.DS +diropres +NFSPROC_MKDIR (createargs) = 14; +.DE +.KE +The new directory "where.name" is created in the directory given by +"where.dir". The initial attributes of the new directory are given +by "attributes". A reply "status" of NFS_OK indicates that the new +directory was created, and reply "file" and reply "attributes" are +its file handle and attributes. Any other reply "status" means +that the operation failed and no directory was created. +.LP +Note: possibly non-idempotent operation. +.KS +.NH 3 +\&Remove Directory +.IX "NFS server procedures" NFSPROC_RMDIR() "" \fINFSPROC_RMDIR()\fP +.DS +stat +NFSPROC_RMDIR(diropargs) = 15; +.DE +.KE +The existing empty directory "name" in the directory given by "dir" +is removed. If the reply is +.I NFS_OK , +the directory was removed. +.LP +Note: possibly non-idempotent operation. +.KS +.NH 3 +\&Read From Directory +.IX "NFS server procedures" NFSPROC_READDIR() "" \fINFSPROC_READDIR()\fP +.DS +struct readdirargs { + fhandle dir; + nfscookie cookie; + unsigned count; +}; + +struct entry { + unsigned fileid; + filename name; + nfscookie cookie; + entry *nextentry; +}; + +union readdirres switch (stat status) { + case NFS_OK: + struct { + entry *entries; + bool eof; + } readdirok; + default: + void; +}; + +readdirres +NFSPROC_READDIR (readdirargs) = 16; +.DE +.KE +Returns a variable number of directory entries, with a total size +of up to "count" bytes, from the directory given by "dir". If the +returned value of "status" is +.I NFS_OK , +then it is followed by a +variable number of "entry"s. Each "entry" contains a "fileid" +which consists of a unique number to identify the file within a +filesystem, the "name" of the file, and a "cookie" which is an +opaque pointer to the next entry in the directory. The cookie is +used in the next +.I READDIR +call to get more entries starting at a +given point in the directory. The special cookie zero (all bits +zero) can be used to get the entries starting at the beginning of +the directory. The "fileid" field should be the same number as the +"fileid" in the attributes of the file. (See the +.I "Basic Data Types" +section.) +The "eof" flag has a value of +.I TRUE +if there are no more entries in the directory. +.KS +.NH 3 +\&Get Filesystem Attributes +.IX "NFS server procedures" NFSPROC_STATFS() "" \fINFSPROC_STATFS()\fP +.DS +union statfsres (stat status) { + case NFS_OK: + struct { + unsigned tsize; + unsigned bsize; + unsigned blocks; + unsigned bfree; + unsigned bavail; + } info; + default: + void; +}; + +statfsres +NFSPROC_STATFS(fhandle) = 17; +.DE +.KE +If the reply "status" is +.I NFS_OK , +then the reply "info" gives the +attributes for the filesystem that contains file referred to by the +input fhandle. The attribute fields contain the following values: +.IP tsize: +The optimum transfer size of the server in bytes. This is +the number of bytes the server would like to have in the +data part of READ and WRITE requests. +.IP bsize: +The block size in bytes of the filesystem. +.IP blocks: +The total number of "bsize" blocks on the filesystem. +.IP bfree: +The number of free "bsize" blocks on the filesystem. +.IP bavail: +The number of "bsize" blocks available to non-privileged users. +.LP +Note: This call does not work well if a filesystem has variable +size blocks. +.NH 1 +\&NFS Implementation Issues +.IX NFS implementation +.LP +The NFS protocol is designed to be operating system independent, but +since this version was designed in a UNIX environment, many +operations have semantics similar to the operations of the UNIX file +system. This section discusses some of the implementation-specific +semantic issues. +.NH 2 +\&Server/Client Relationship +.IX NFS "server/client relationship" +.LP +The NFS protocol is designed to allow servers to be as simple and +general as possible. Sometimes the simplicity of the server can be a +problem, if the client wants to implement complicated filesystem +semantics. +.LP +For example, some operating systems allow removal of open files. A +process can open a file and, while it is open, remove it from the +directory. The file can be read and written as long as the process +keeps it open, even though the file has no name in the filesystem. +It is impossible for a stateless server to implement these semantics. +The client can do some tricks such as renaming the file on remove, +and only removing it on close. We believe that the server provides +enough functionality to implement most file system semantics on the +client. +.LP +Every NFS client can also potentially be a server, and remote and +local mounted filesystems can be freely intermixed. This leads to +some interesting problems when a client travels down the directory +tree of a remote filesystem and reaches the mount point on the server +for another remote filesystem. Allowing the server to follow the +second remote mount would require loop detection, server lookup, and +user revalidation. Instead, we decided not to let clients cross a +server's mount point. When a client does a LOOKUP on a directory on +which the server has mounted a filesystem, the client sees the +underlying directory instead of the mounted directory. A client can +do remote mounts that match the server's mount points to maintain the +server's view. +.LP +.NH 2 +\&Pathname Interpretation +.IX NFS "pathname interpretation" +.LP +There are a few complications to the rule that pathnames are always +parsed on the client. For example, symbolic links could have +different interpretations on different clients. Another common +problem for non-UNIX implementations is the special interpretation of +the pathname ".." to mean the parent of a given directory. The next +revision of the protocol uses an explicit flag to indicate the parent +instead. +.NH 2 +\&Permission Issues +.IX NFS "permission issues" +.LP +The NFS protocol, strictly speaking, does not define the permission +checking used by servers. However, it is expected that a server +will do normal operating system permission checking using +.I AUTH_UNIX +style authentication as the basis of its protection mechanism. The +server gets the client's effective "uid", effective "gid", and groups +on each call and uses them to check permission. There are various +problems with this method that can been resolved in interesting ways. +.LP +Using "uid" and "gid" implies that the client and server share the +same "uid" list. Every server and client pair must have the same +mapping from user to "uid" and from group to "gid". Since every +client can also be a server, this tends to imply that the whole +network shares the same "uid/gid" space. +.I AUTH_DES +(and the next +revision of the NFS protocol) uses string names instead of numbers, +but there are still complex problems to be solved. +.LP +Another problem arises due to the usually stateful open operation. +Most operating systems check permission at open time, and then check +that the file is open on each read and write request. With stateless +servers, the server has no idea that the file is open and must do +permission checking on each read and write call. On a local +filesystem, a user can open a file and then change the permissions so +that no one is allowed to touch it, but will still be able to write +to the file because it is open. On a remote filesystem, by contrast, +the write would fail. To get around this problem, the server's +permission checking algorithm should allow the owner of a file to +access it regardless of the permission setting. +.LP +A similar problem has to do with paging in from a file over the +network. The operating system usually checks for execute permission +before opening a file for demand paging, and then reads blocks from +the open file. The file may not have read permission, but after it +is opened it doesn't matter. An NFS server can not tell the +difference between a normal file read and a demand page-in read. To +make this work, the server allows reading of files if the "uid" given +in the call has execute or read permission on the file. +.LP +In most operating systems, a particular user (on the user ID zero) +has access to all files no matter what permission and ownership they +have. This "super-user" permission may not be allowed on the server, +since anyone who can become super-user on their workstation could +gain access to all remote files. The UNIX server by default maps +user id 0 to -2 before doing its access checking. This works except +for NFS root filesystems, where super-user access cannot be avoided. +.NH 2 +\&Setting RPC Parameters +.IX NFS "setting RPC parameters" +.LP +Various file system parameters and options should be set at mount +time. The mount protocol is described in the appendix below. For +example, "Soft" mounts as well as "Hard" mounts are usually both +provided. Soft mounted file systems return errors when RPC +operations fail (after a given number of optional retransmissions), +while hard mounted file systems continue to retransmit forever. +Clients and servers may need to keep caches of recent operations to +help avoid problems with non-idempotent operations. +.NH 1 +\&Mount Protocol Definition +.IX "mount protocol" "" "" "" PAGE MAJOR +.sp 1 +.NH 2 +\&Introduction +.IX "mount protocol" introduction +.LP +The mount protocol is separate from, but related to, the NFS +protocol. It provides operating system specific services to get the +NFS off the ground -- looking up server path names, validating user +identity, and checking access permissions. Clients use the mount +protocol to get the first file handle, which allows them entry into a +remote filesystem. +.LP +The mount protocol is kept separate from the NFS protocol to make it +easy to plug in new access checking and validation methods without +changing the NFS server protocol. +.LP +Notice that the protocol definition implies stateful servers because +the server maintains a list of client's mount requests. The mount +list information is not critical for the correct functioning of +either the client or the server. It is intended for advisory use +only, for example, to warn possible clients when a server is going +down. +.LP +Version one of the mount protocol is used with version two of the NFS +protocol. The only connecting point is the +.I fhandle +structure, which is the same for both protocols. +.NH 2 +\&RPC Information +.IX "mount protocol" "RPC information" +.IP \fIAuthentication\fP +The mount service uses +.I AUTH_UNIX +and +.I AUTH_DES +style authentication only. +.IP "\fITransport Protocols\fP" +The mount service is currently supported on UDP/IP only. +.IP "\fIPort Number\fP" +Consult the server's portmapper, described in the chapter +.I "Remote Procedure Calls: Protocol Specification", +to find the port number on which the mount service is registered. +.NH 2 +\&Sizes of XDR Structures +.IX "mount protocol" "XDR structure sizes" +.LP +These are the sizes, given in decimal bytes, of various XDR +structures used in the protocol: +.DS +/* \fIThe maximum number of bytes in a pathname argument\fP */ +const MNTPATHLEN = 1024; + +/* \fIThe maximum number of bytes in a name argument\fP */ +const MNTNAMLEN = 255; + +/* \fIThe size in bytes of the opaque file handle\fP */ +const FHSIZE = 32; +.DE +.NH 2 +\&Basic Data Types +.IX "mount protocol" "basic data types" +.IX "mount data types" +.LP +This section presents the data types used by the mount protocol. +In many cases they are similar to the types used in NFS. +.KS +.NH 3 +\&fhandle +.IX "mount data types" fhandle "" \fIfhandle\fP +.DS +typedef opaque fhandle[FHSIZE]; +.DE +.KE +The type +.I fhandle +is the file handle that the server passes to the +client. All file operations are done using file handles to refer +to a file or directory. The file handle can contain whatever +information the server needs to distinguish an individual file. +.LP +This is the same as the "fhandle" XDR definition in version 2 of +the NFS protocol; see +.I "Basic Data Types" +in the definition of the NFS protocol, above. +.KS +.NH 3 +\&fhstatus +.IX "mount data types" fhstatus "" \fIfhstatus\fP +.DS +union fhstatus switch (unsigned status) { + case 0: + fhandle directory; + default: + void; +}; +.DE +.KE +The type +.I fhstatus +is a union. If a "status" of zero is returned, +the call completed successfully, and a file handle for the +"directory" follows. A non-zero status indicates some sort of +error. In this case the status is a UNIX error number. +.KS +.NH 3 +\&dirpath +.IX "mount data types" dirpath "" \fIdirpath\fP +.DS +typedef string dirpath<MNTPATHLEN>; +.DE +.KE +The type +.I dirpath +is a server pathname of a directory. +.KS +.NH 3 +\&name +.IX "mount data types" name "" \fIname\fP +.DS +typedef string name<MNTNAMLEN>; +.DE +.KE +The type +.I name +is an arbitrary string used for various names. +.NH 2 +\&Server Procedures +.IX "mount server procedures" +.LP +The following sections define the RPC procedures supplied by a +mount server. +.ie t .DS +.el .DS L +.ft I +/* +* Protocol description for the mount program +*/ +.ft CW + +program MOUNTPROG { +.ft I +/* +* Version 1 of the mount protocol used with +* version 2 of the NFS protocol. +*/ +.ft CW + version MOUNTVERS { + void MOUNTPROC_NULL(void) = 0; + fhstatus MOUNTPROC_MNT(dirpath) = 1; + mountlist MOUNTPROC_DUMP(void) = 2; + void MOUNTPROC_UMNT(dirpath) = 3; + void MOUNTPROC_UMNTALL(void) = 4; + exportlist MOUNTPROC_EXPORT(void) = 5; + } = 1; +} = 100005; +.DE +.KS +.NH 3 +\&Do Nothing +.IX "mount server procedures" MNTPROC_NULL() "" \fIMNTPROC_NULL()\fP +.DS +void +MNTPROC_NULL(void) = 0; +.DE +.KE +This procedure does no work. It is made available in all RPC +services to allow server response testing and timing. +.KS +.NH 3 +\&Add Mount Entry +.IX "mount server procedures" MNTPROC_MNT() "" \fIMNTPROC_MNT()\fP +.DS +fhstatus +MNTPROC_MNT(dirpath) = 1; +.DE +.KE +If the reply "status" is 0, then the reply "directory" contains the +file handle for the directory "dirname". This file handle may be +used in the NFS protocol. This procedure also adds a new entry to +the mount list for this client mounting "dirname". +.KS +.NH 3 +\&Return Mount Entries +.IX "mount server procedures" MNTPROC_DUMP() "" \fIMNTPROC_DUMP()\fP +.DS +struct *mountlist { + name hostname; + dirpath directory; + mountlist nextentry; +}; + +mountlist +MNTPROC_DUMP(void) = 2; +.DE +.KE +Returns the list of remote mounted filesystems. The "mountlist" +contains one entry for each "hostname" and "directory" pair. +.KS +.NH 3 +\&Remove Mount Entry +.IX "mount server procedures" MNTPROC_UMNT() "" \fIMNTPROC_UMNT()\fP +.DS +void +MNTPROC_UMNT(dirpath) = 3; +.DE +.KE +Removes the mount list entry for the input "dirpath". +.KS +.NH 3 +\&Remove All Mount Entries +.IX "mount server procedures" MNTPROC_UMNTALL() "" \fIMNTPROC_UMNTALL()\fP +.DS +void +MNTPROC_UMNTALL(void) = 4; +.DE +.KE +Removes all of the mount list entries for this client. +.KS +.NH 3 +\&Return Export List +.IX "mount server procedures" MNTPROC_EXPORT() "" \fIMNTPROC_EXPORT()\fP +.DS +struct *groups { + name grname; + groups grnext; +}; + +struct *exportlist { + dirpath filesys; + groups groups; + exportlist next; +}; + +exportlist +MNTPROC_EXPORT(void) = 5; +.DE +.KE +Returns a variable number of export list entries. Each entry +contains a filesystem name and a list of groups that are allowed to +import it. The filesystem name is in "filesys", and the group name +is in the list "groups". +.LP +Note: The exportlist should contain +more information about the status of the filesystem, such as a +read-only flag. diff --git a/share/doc/psd/27.nfsrpc/stubs b/share/doc/psd/27.nfsrpc/stubs new file mode 100644 index 000000000000..78b0a2c815d3 --- /dev/null +++ b/share/doc/psd/27.nfsrpc/stubs @@ -0,0 +1,3 @@ +.\" $FreeBSD$ +.\" +.if t .ftr L CR diff --git a/share/doc/psd/28.cvs/Makefile b/share/doc/psd/28.cvs/Makefile new file mode 100644 index 000000000000..a62473284e36 --- /dev/null +++ b/share/doc/psd/28.cvs/Makefile @@ -0,0 +1,10 @@ +# $FreeBSD$ + +VOLUME= psd/28.cvs +SRCS= cvs-paper.ms +MACROS= -ms +USE_PIC= +USE_TBL= +SRCDIR= ${.CURDIR}/../../../../contrib/cvs/doc + +.include <bsd.doc.mk> diff --git a/share/doc/psd/Makefile b/share/doc/psd/Makefile new file mode 100644 index 000000000000..d50f05b1b2c6 --- /dev/null +++ b/share/doc/psd/Makefile @@ -0,0 +1,41 @@ +# From: @(#)Makefile 8.1 (Berkeley) 6/8/93 +# $FreeBSD$ + +# The following modules do not build/install: +# 10.gdb + +# The following modules do not apply to FreeBSD: +# 07.pascal 08.f77 09.f77io + +# The following previously encumbered files have not yet been added to +# the tree: +# 11.adb + +SUBDIR= title \ + contents \ + 01.cacm \ + 02.implement \ + 03.iosys \ + 04.uprog \ + 05.sysman \ + 06.Clang \ + 12.make \ + 13.rcs \ + 15.yacc \ + 16.lex \ + 17.m4 \ + 18.gprof \ + 20.ipctut \ + 21.ipc + +# The following modules don't appear in the O'Reilly book, but +# are in the 4.4BSD distribution. +SUBDIR+=22.rpcgen \ + 23.rpc \ + 24.xdr \ + 25.xdrrfc \ + 26.rpcrfc \ + 27.nfsrpc \ + 28.cvs + +.include <bsd.subdir.mk> diff --git a/share/doc/psd/contents/Makefile b/share/doc/psd/contents/Makefile new file mode 100644 index 000000000000..38864e96a4fe --- /dev/null +++ b/share/doc/psd/contents/Makefile @@ -0,0 +1,8 @@ +# $FreeBSD$ + +VOLUME= psd +DOC= contents +SRCS= contents.ms +MACROS= -ms + +.include <bsd.doc.mk> diff --git a/share/doc/psd/contents/contents.ms b/share/doc/psd/contents/contents.ms new file mode 100644 index 000000000000..9f374dc47cdd --- /dev/null +++ b/share/doc/psd/contents/contents.ms @@ -0,0 +1,289 @@ +.\" Copyright (c) 1986, 1993 +.\" The Regents of the University of California. All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)00.contents 8.1 (Berkeley) 6/8/93 +.\" $FreeBSD$ +.\" +.OH '''PSD Contents' +.EH 'PSD Contents''' +.TL +UNIX Programmer's Supplementary Documents (PSD) +.if !r.U .nr .U 0 +.if \n(.U \{\ +.br +.>> <a href="Title.html">Title.html</a> +.\} +.sp +\s-2 4.4 Berkeley Software Distribution\s+2 +.sp +\fRJune, 1993\fR +.PP +This volume contains documents which supplement the manual pages in +.I +The +.UX +Programmer's Reference Manual +.R +for the FreeBSD system as distributed by the FreeBSD Project. +.SH +Documents of Historical Interest +.IP +.tl 'The Unix Time\-Sharing System''PSD:1' +.QP +Dennis Ritchie and Ken Thompson's original paper about UNIX, reprinted +from Communications of the ACM. +.sp +.IP +.tl 'Unix Implementation''PSD:2' +.QP +Ken Thompson's description of the implementation of the Version 7 +kernel and file system. +.sp +.IP +.tl 'The Unix I/O System''PSD:3' +.QP +Dennis Ritchie's overview of the I/O System of Version 7; still helpful for +those writing device drivers. +.sp +.IP +.tl 'Unix Programming \- Second Edition ''PSD:4' +.QP +Describes the programming interface to the UNIX version 7 operating +system and the standard I/O library. Should be supplemented by +Kernighan and Pike, ``The UNIX Programming Environment'', +Prentice-Hall, 1984 and especially by the Programmer Reference Manual +section 2 (system calls) and 3 (library routines). +.sp +.IP +.tl 'Berkeley Software Architecture Manual (4.4 Edition)''PSD:5' +.if \n(.U \{\ +.br +.>> <a href="05.sysman/paper.html">05.sysman/paper.html</a> +.\} +.QP +A concise and terse description of the system call interface +provided in Berkeley Unix, as revised for 4.4BSD. +This will never be a best seller. + +.SH +Languages in common use +.IP +.tl 'The C Programming Language \- Reference Manual''PSD:6' +.QP +Official statement of the syntax of C. +Should be supplemented by ``The C Programming Language,'' +B.W. Kernighan and D.M. Ritchie, Prentice-Hall, 1978, that +contains a tutorial introduction and many examples. +.sp +.IP +.tl 'Berkeley Pascal User\'s Manual''PSD:7' +.QP +An implementation of this language popular for learning to program. +(Not provided in FreeBSD.) +.sp +.IP +.tl 'A Portable Fortran 77 Compiler''PSD:8' +.QP +A revised version of the document which originally appeared in +Volume 2b of the Bell Labs documentation; +this version reflects the work done at Berkeley. +(Not provided in FreeBSD.) +.sp +.IP +.tl 'Introduction to the f77 I/O Library''PSD:9' +.QP +A description of the revised input/output library for Fortran 77, +reflecting work carried out at Berkeley. (Not provided in FreeBSD.) + +.SH +Programming Tools +.IP +.tl 'Debugging with GDB: The GNU Source-Level Debugger''PSD:10' +.QP +How to debug programs using the source level \fIgdb\fP debugger +(or how to debug programs without having to know much about machine language). +(A TeXinfo version is provided separately.) +.sp +.IP +.tl 'A Tutorial Introduction to ADB''PSD:11' +.QP +How to debug programs using the assembly-language level \fIadb\fP debugger. +(Not provided in FreeBSD.) +.sp +.IP +.tl 'Make \- A Program for Maintaining Computer Programs''PSD:12' +.if \n(.U \{\ +.br +.>> <a href="12.make/paper.html">12.make/paper.html</a> +.\} +.QP +Indispensable tool for making sure large programs are properly +compiled with minimal effort. +.sp +.IP +.tl 'An Introduction to the Revision Control System''PSD:13' +.if \n(.U \{\ +.br +.>> <a href="13.rcs/paper.html">13.rcs/paper.html</a> +.\} +.QP +RCS is a user-contributed tool for working together with other people +without stepping on each other's toes. +An alternative to \fIsccs\fR for controlling software changes. +.sp +.IP +.tl 'An Introduction to the Source Code Control System''PSD:14' +.QP +A useful introductory article for those users with +installations licensed for SCCS. +.sp +.IP +.tl 'YACC: Yet Another Compiler-Compiler''PSD:15' +.QP +Converts a BNF specification of a language and semantic actions +written in C into a compiler for that language. +.sp +.IP +.tl 'LEX \- A Lexical Analyzer Generator''PSD:16' +.QP +Creates a recognizer for a set of regular expressions: +each regular expression can be followed by arbitrary C code +to be executed upon finding the regular expression. +.sp +.IP +.tl 'The M4 Macro Processor''PSD:17' +.QP +M4 is a macro processor useful in its own right and as a +front-end for C, Ratfor, and Cobol. +.sp +.IP +.tl 'gprof: a Call Graph Execution Profiler''PSD:18' +.if \n(.U \{\ +.br +.>> <a href="18.gprof/paper.html">18.gprof/paper.html +.\} +.QP +A program to show the call graph and execution time of a program. +Indispensable aid for improving the running time of almost everything. + +.SH +General Reference +.IP +.tl 'An Introductory 4.4BSD Interprocess Communication Tutorial''PSD:20' +.if \n(.U \{\ +.br +.>> <a href="20.ipctut/paper.html">20.ipctut/paper.html +.\} +.QP +How to write programs that use the Interprocess Communication Facilities +of 4.4BSD. +.sp +.IP +.tl 'An Advanced 4.4BSD Interprocess Communication Tutorial''PSD:21' +.if \n(.U \{\ +.br +.>> <a href="21.ipc/paper.html">21.ipc/paper.html +.\} +.QP +The reference document (with some examples) for the Interprocess Communication +Facilities of 4.4BSD. +.sp +.IP +.tl 'RPCGEN Programming Guide''PSD:22' +.if \n(.U \{\ +.br +.>> <a href="22.rpcgen/paper.html">22.rpcgen/paper.html +.\} +.QP +Manual for the ONC RPC stub-generating program, provided by Sun Microsystems. +.sp +.IP +.tl 'Remote Procedure Call Programming Guide''PSD:23' +.if \n(.U \{\ +.br +.>> <a href="23.rpc/paper.html">23.rpc/paper.html +.\} +.QP +A tutorial introduction to programming the ONC RPC system, provided by +Sun Microsystems. +.sp +.IP +.tl 'External Data Representation: Sun Technical Notes''PSD:24' +.if \n(.U \{\ +.br +.>> <a href="24.xdr/paper.html">24.xdr/paper.html +.\} +.QP +Technical details about the design of the XDR component of ONC RPC, +provided by Sun Microsystems. +.sp +.IP +.tl 'External Data Representation Standard: Protocol Specification''PSD:25' +.if \n(.U \{\ +.br +.>> <a href="25.xdrrfc/paper.html">25.xdrrfc/paper.html +.\} +.QP +The Internet RFC specifying ONC XDR, provided by Sun Microsystems. +.sp +.IP +.tl 'Remote Procedure Calls: Protocol Specification''PSD:26' +.if \n(.U \{\ +.br +.>> <a href="26.rpcrfc/paper.html">26.rpcrfc/paper.html +.\} +.QP +The Internet RFC specifying ONC RPC, RFC 1050, as provided by Sun +Microsystems. +.sp +.IP +.tl 'Network File System: Version 2 Protocol Specification''PSD:27' +.if \n(.U \{\ +.br +.>> <a href="27.nfsrpc/paper.html">27.nfsrpc/paper.html +.\} +.QP +The Internet RFC specifying NFS, as provided by Sun Microsystems. +Note that the NFS-compatible filesystem itself, while +compliant with this specification, was not provided by Sun. +.sp +.IP +.tl 'CVS II: Parallelizing Software Development''PSD:28' +.if \n(.U \{\ +.br +.>> <a href="28.cvs/paper.html">28.cvs/paper.html +.\} +.QP +CVS (Concurrent Versions System) is a front end to the +RCS revision control system which extends the notion of +revision control from a collection of files in a single +directory to a hierarchical collection of directories each +containing revision controlled files. diff --git a/share/doc/psd/title/Makefile b/share/doc/psd/title/Makefile new file mode 100644 index 000000000000..d073730b322c --- /dev/null +++ b/share/doc/psd/title/Makefile @@ -0,0 +1,7 @@ +# $FreeBSD$ + +VOLUME= psd +DOC= Title +SRCS= Title + +.include <bsd.doc.mk> diff --git a/share/doc/psd/title/Title b/share/doc/psd/title/Title new file mode 100644 index 000000000000..014b3d54dcdb --- /dev/null +++ b/share/doc/psd/title/Title @@ -0,0 +1,132 @@ +.\" Copyright (c) 1986, 1993 The Regents of the University of California. +.\" All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" 3. All advertising materials mentioning features or use of this software +.\" must display the following acknowledgement: +.\" This product includes software developed by the University of +.\" California, Berkeley and its contributors. +.\" 4. Neither the name of the University nor the names of its contributors +.\" may be used to endorse or promote products derived from this software +.\" without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" @(#)Title 8.2 (Berkeley) 4/19/94 +.\" $FreeBSD$ +.\" +.ps 18 +.vs 22 +.sp 2.75i +.ft B +.ce 3 +UNIX Programmer's Supplementary Documents +(PSD) +.ps 14 +.vs 16 +.sp |4i +.ce 2 +4.4 Berkeley Software Distribution +.sp |5.75i +.ft R +.ps 12 +.vs 16 +.ce +June, 1993 +.sp |8.2i +.ce 5 +Computer Systems Research Group +Computer Science Division +Department of Electrical Engineering and Computer Science +University of California +Berkeley, California 94720 +.bp +\& +.sp |1i +.hy 0 +.ps 10 +.vs 12p +Copyright 1979, 1980, 1983, 1986, 1993 +The Regents of the University of California. All rights reserved. +.sp 2 +Other than the specific documents listed below as copyrighted by AT&T, +redistribution and use of this manual in source and binary forms, +with or without modification, are permitted provided that the +following conditions are met: +.sp 0.5 +.in +0.2i +.ta 0.2i +.ti -0.2i +1) Redistributions of this manual must retain the copyright +notices on this page, this list of conditions and the following disclaimer. +.ti -0.2i +2) Software or documentation that incorporates part of this manual must +reproduce the copyright notices on this page, this list of conditions and +the following disclaimer in the documentation and/or other materials +provided with the distribution. +.ti -0.2i +3) All advertising materials mentioning features or use of this software +must display the following acknowledgement: +``This product includes software developed by the University of +California, Berkeley and its contributors.'' +.ti -0.2i +4) Neither the name of the University nor the names of its contributors +may be used to endorse or promote products derived from this software +without specific prior written permission. +.in -0.2i +.sp +\fB\s-1THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND +ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE +FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +SUCH DAMAGE.\s+1\fP +.sp 2 +Documents PSD:1, 2, 3, 4, 6, 11, 15, 16, and 17 +are copyright 1979, AT&T Bell Laboratories, Incorporated. +Document PSD:8 is a modification of an earlier document that +is copyrighted 1979 by AT&T Bell Laboratories, Incorporated. +Holders of \x'-1p'UNIX\v'-4p'\s-3TM\s0\v'4p'/32V, +System III, or System V software licenses are +permitted to copy these documents, or any portion of them, +as necessary for licensed use of the software, +provided this copyright notice and statement of permission +are included. +.sp 2 +Document PSD:10 is part of the user contributed software and is +copyright 1992 by the Free Software Foundation, Inc. +Permission is granted to make and distribute verbatim copies of +this document provided the copyright notice and this permission notice +are preserved on all copies. +.sp 2 +Document PSD:13 is part of the user contributed software and is +copyright 1983 by Walter F. Tichy. +Permission to copy the RCS documentation or any portion thereof as +necessary for licensed use of the software is granted to licensees +of this software, provided this copyright notice is included. +.sp 2 +The views and conclusions contained in this manual are those of the +authors and should not be interpreted as representing official policies, +either expressed or implied, of the Regents of the University of California. |