aboutsummaryrefslogtreecommitdiff
path: root/en_US.ISO8859-1/books/handbook/jails/chapter.sgml
blob: 8f28a49105b82653857877e82255e6f7b9b5099c (plain) (blame)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
<!--
    The FreeBSD Documentation Project

    $FreeBSD$
-->
<chapter id="jails">
  <chapterinfo>
    <authorgroup>
      <author>
	<firstname>Matteo</firstname>
	<surname>Riondato</surname>
	<contrib>Contributed by </contrib>
      </author>
    </authorgroup>
  </chapterinfo>

  <title>Jails</title>

  <indexterm><primary>jails</primary></indexterm>

  <sect1 id="jails-synopsis">
    <title>Synopsis</title>

    <para>This chapter will provide an explanation of what &os; jails
      are and how to use them.  Jails, sometimes referred to as an
      enhanced replacement of <emphasis>chroot environments</emphasis>,
      are a very powerful tool for system administrators, but their basic
      usage can also be useful for advanced users.</para>

    <para>After reading this chapter, you will know:</para>

    <itemizedlist>
      <listitem>
	<para>What a jail is, and what purpose it may serve in &os;
	  installations.</para>
      </listitem>

      <listitem>
	<para>How to build, start, and stop a jail.</para>
      </listitem>

      <listitem>
	<para>The basics of jail administration, both from inside
	  and outside the jail.</para>
      </listitem>
    </itemizedlist>

    <para>Other sources of useful information about jails are:</para>

    <itemizedlist>
      <listitem>
	<para>The &man.jail.8; manual page.  This is the full reference
	  of the <command>jail</command> utility &mdash; the
	  administrative tool which can be used in &os; to start, stop,
	  and control &os; jails.</para>
      </listitem>

      <listitem>
	<para>The mailing lists and their archives.  The archives of the
	  &a.questions; and other mailing lists hosted by the
	  &a.mailman.lists; already contain a wealth of material for
	  jails.  It should always be engaging to search the archives,
	  or post a new question to the &a.questions.name; mailing
	  list.</para>
      </listitem>
    </itemizedlist>

  </sect1>

  <sect1 id="jails-terms">
    <title>Terms Related to Jails</title>

    <para>To facilitate better understanding of parts of the &os; system
      related to jails, their internals and the way they interact with
      the rest of &os;, the following terms are used further in this
      chapter:</para>

    <variablelist>
      <varlistentry>
        <term>&man.chroot.8; (command)</term>
        <listitem>
	  <para>Utility, which uses &man.chroot.2; &os; system call to change
	    the root directory of a process and all its descendants.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
        <term>&man.chroot.2; (environment)</term>
        <listitem>
	  <para>The environment of processes running in
	    a <quote>chroot</quote>.  This includes resources such as the part
	    of the file system which is visible, user and group IDs which are
	    available, network interfaces and other IPC mechanisms,
	    etc.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
        <term>&man.jail.8; (command)</term>
        <listitem>
	  <para>The system administration utility which allows launching of
	    processes within a jail environment.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
        <term>host (system, process, user, etc.)</term>
        <listitem>
	  <para>The controlling system of a jail environment.  The host system
	    has access to all the hardware resources available, and can
	    control processes both outside of and inside a jail environment.
	    One of the important differences of the host system from a jail is
	    that the limitations which apply to superuser processes inside a
	    jail are not enforced for processes of the host system.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
        <term>hosted (system, process, user, etc.)</term>
        <listitem>
	  <para>A process, user or other entity, whose access to resources is
	    restricted by a &os; jail.</para>
	</listitem>
      </varlistentry>
    </variablelist>
  </sect1>

  <sect1 id="jails-intro">
    <title>Introduction</title>

    <para>Since system administration is a difficult and perplexing
      task, many powerful tools were developed to make life easier for
      the administrator.  These tools mostly provide enhancements of some sort
      to the way systems are installed, configured and maintained.
      Part of the tasks which an administrator is
      expected to do is to properly configure the security of a system,
      so that it can continue serving its real purpose, without allowing
      security violations.</para>

    <para>One of the tools which can be used to enhance the security of
      a &os; system are <emphasis>jails</emphasis>.  Jails were
      introduced in &os;&nbsp;4.X by &a.phk;, but were greatly improved in
      &os;&nbsp;5.X to make them a powerful and flexible subsystem.  Their
      development still goes on, enhancing their usefulness, performance, reliability,
      and security.</para>

    <sect2 id="jails-what">
      <title>What is a Jail</title>

      <para>BSD-like operating systems have had &man.chroot.2; since the
	time of 4.2BSD.  The &man.chroot.8; utility can be used to
	change the root directory
	of a set of processes, creating a safe environment, separate
	from the rest of the system.  Processes created in the chrooted
	environment can not access files or resources outside of it.
	For that reason, compromising a service running in a chrooted
	environment should not allow the attacker to compromise the
	entire system.  The &man.chroot.8; utility is good for easy
	tasks, which do not require a lot of flexibility or complex and
	advanced features.  Since the inception of the
	chroot concept, however, many ways have been found to escape from a
	chrooted environment and, although they have been fixed in
	modern versions of the &os; kernel, it was clear that
	&man.chroot.2; was not the ideal solution for securing services.
	A new subsystem had to be implemented.</para>

      <para>This is one of the main reasons why
	<emphasis>jails</emphasis> were developed.</para>

      <para>Jails improve on the concept of the traditional
	&man.chroot.2; environment, in several ways.  In a traditional
	&man.chroot.2; environment, processes are only limited in the
	part of the file system they can access.  The rest of the system
	resources (like the set of system users, the running processes,
	or the networking subsystem) are shared by the chrooted
	processes and the processes of the host system.  Jails expand
	this model by virtualizing not only access to the file system,
	but also the set of users, the networking subsystem of the &os;
	kernel and a few other things.  A more complete set of
	fine-grained controls available for tuning the access of a
	jailed environment is described in <xref
	  linkend="jails-tuning">.</para>

      <para>A jail is characterized by four elements:</para>

      <itemizedlist>
	<listitem>
	  <para>A directory subtree &mdash; the starting point from
	    which a jail is entered.  Once inside the jail, a process
	    is not permitted to escape outside of this subtree.
	    Traditional security issues which plagued the original
	    &man.chroot.2; design will not affect &os; jails.</para>
	</listitem>

	<listitem>
	  <para>A hostname &mdash; the hostname which will be used
	    within the jail.  Jails are mainly used for hosting network
	    services, therefore having a descriptive hostname for each
	    jail can really help the system administrator.</para>
	</listitem>

	<listitem>
	  <para>An <acronym>IP</acronym> address &mdash; this will be
	    assigned to the jail and cannot be changed in any way during
	    the jail's life span.  The IP address of a jail is usually an alias address
	    for an existing network interface, but this is not strictly necessary.</para>
	</listitem>

	<listitem>
	  <para>A command &mdash; the path name of an executable to run
	    inside the jail.  This is relative to the root directory of
	    the jail environment, and may vary a lot, depending on the
	    type of the specific jail environment.</para>
	</listitem>
      </itemizedlist>

      <para>Apart from these, jails can have their own set of users and
	their own <username>root</username> user.  Naturally, the powers
	of the <username>root</username> user are limited within the
	jail environment and, from the point of view of the host system,
	the jail <username>root</username> user is not an omnipotent user.
	In addition, the <username>root</username> user of a jail is not
	allowed to perform critical operations to the system outside of
	the associated &man.jail.8; environment.  More information
	about capabilities and restrictions of the
	<username>root</username> user will be discussed in <xref
	  linkend="jails-tuning"> below.</para>
    </sect2>
  </sect1>

  <sect1 id="jails-build">
    <title>Creating and Controlling Jails</title>

    <para>Some administrators divide jails into the following two types:
      <quote>complete</quote> jails, which resemble a real &os; system,
      and <quote>service</quote> jails, dedicated to one application or
      service, possibly running with privileges.  This is only a
      conceptual division and the process of building a jail is not
      affected by it.  The &man.jail.8; manual page is quite clear about
      the procedure for building a jail:</para>

      <screen>&prompt.root; <userinput>setenv D <replaceable>/here/is/the/jail</replaceable></userinput>
&prompt.root; <userinput>mkdir -p $D</userinput> <co id="jailpath">
&prompt.root; <userinput>cd /usr/src</userinput>
&prompt.root; <userinput>make buildworld</userinput> <co id="jailbuildworld">
&prompt.root; <userinput>make installworld DESTDIR=$D</userinput> <co id="jailinstallworld">
&prompt.root; <userinput>make distribution DESTDIR=$D</userinput> <co id="jaildistrib">
&prompt.root; <userinput>mount -t devfs devfs $D/dev</userinput> <co id="jaildevfs"></screen>

    <calloutlist>
      <callout arearefs="jailpath">
	<para>Selecting a location for a jail is the best starting point.
	  This is where the jail will physically reside within the file system of the jail's host.
	  A good choice can be <filename
	    class="directory">/usr/jail/<replaceable>jailname</replaceable></filename>,
	  where <replaceable>jailname</replaceable> is the hostname
	  identifying the jail.  The <filename
	    class="directory">/usr/</filename> file system usually has
	  enough space for the jail file system, which for <quote>complete</quote> jails is, essentially,
	  a replication of every file present in a default installation
	  of the &os; base system.</para>
      </callout>

      <callout arearefs="jailbuildworld">
	<para>If you have already rebuilt your userland using
	  <command>make world</command> or <command>make buildworld</command>,
	  you can skip this step and install your existing userland into the
	  new jail.</para>
      </callout>

      <callout arearefs="jailinstallworld">
	<para>This command will populate the directory subtree chosen
	  as jail's physical location on the file system with the
	  necessary binaries, libraries, manual pages and so on.</para>
      </callout>

      <callout arearefs="jaildistrib">
	<para>The <maketarget>distribution</maketarget> target for
	  <application>make</application> installs every needed
	  configuration file.  In simple words, it installs every installable file of
	  <filename class="directory">/usr/src/etc/</filename> to the
	  <filename class="directory">/etc</filename> directory of the jail
	  environment:
	  <filename class="directory">$D/etc/</filename>.</para>
      </callout>

      <callout arearefs="jaildevfs">
	<para>Mounting the &man.devfs.8; file system inside a jail is
	  not required.  On the other hand, any, or almost any
	  application requires access to at least one device, depending
	  on the purpose of the given application.  It is very important
	  to control access to devices from inside a jail, as improper
	  settings could permit an attacker to do nasty things in the
	  jail.  Control over &man.devfs.8; is managed through rulesets
	  which are described in the &man.devfs.8; and
	  &man.devfs.conf.5; manual pages.</para>
      </callout>
    </calloutlist>

    <para>Once a jail is installed, it can be started by using the
      &man.jail.8; utility.  The  &man.jail.8; utility takes four
      mandatory arguments which are described in the <xref
	linkend="jails-what">.  Other arguments may be
      specified too, e.g., to run the jailed process with the credentials of a specific
      user.  The <option><replaceable>command</replaceable></option> argument depends on
      the type of the jail; for a <emphasis>virtual system</emphasis>,
      <filename>/etc/rc</filename> is a good choice, since it will
      replicate the startup sequence of a real &os; system.  For a
      <emphasis>service</emphasis> jail, it depends on the service or
      application that will run within the jail.</para>

    <para>Jails are often started at boot time and the &os;
      <filename>rc</filename> mechanism provides an easy way to do
      this.</para>

    <procedure>
      <step>
	<para>A list of the jails which are enabled to start at boot
	  time should be added to the &man.rc.conf.5; file:</para>

	<programlisting>jail_enable="YES"   # Set to NO to disable starting of any jails
jail_list="<replaceable>www</replaceable>"     # Space separated list of names of jails</programlisting>

	<note>
	  <para>Jail names in <varname>jail_list</varname> should
	    contain alphanumeric characters only.</para>
	</note>
      </step>

      <step>
	<para>For each jail listed in <varname>jail_list</varname>, a
	  group of &man.rc.conf.5; settings, which describe the
	  particular jail, should be added:</para>

	<programlisting>jail_<replaceable>www</replaceable>_rootdir="/usr/jail/www"     # jail's root directory
jail_<replaceable>www</replaceable>_hostname="<replaceable>www</replaceable>.example.org"  # jail's hostname
jail_<replaceable>www</replaceable>_ip="192.168.0.10"           # jail's IP address
jail_<replaceable>www</replaceable>_devfs_enable="YES"          # mount devfs in the jail
jail_<replaceable>www</replaceable>_devfs_ruleset="<replaceable>www_ruleset</replaceable>" # devfs ruleset to apply to jail</programlisting>

	<para>The default startup of jails configured in
	  &man.rc.conf.5;, will run the <filename>/etc/rc</filename>
	  script of the jail, which assumes the jail is a complete
	  virtual system.  For service jails, the default startup
	  command of the jail should be changed, by setting the
	  <varname>jail_<replaceable>jailname</replaceable>_exec_start</varname>
	  option appropriately.</para>

	<note>
	  <para>For a full list of available options, please see the
	    &man.rc.conf.5; manual page.</para>
	</note>
      </step>
    </procedure>

    <para>The <filename>/etc/rc.d/jail</filename> script can be used to
      start or stop a jail by hand, if an entry for it exists in
      <filename>rc.conf</filename>:</para>

    <screen>&prompt.root; <userinput>/etc/rc.d/jail start <replaceable>www</replaceable></userinput>
&prompt.root; <userinput>/etc/rc.d/jail stop <replaceable>www</replaceable></userinput></screen>

    <para>A clean way to shut down a &man.jail.8; is not available at
      the moment.  This is because commands normally used to accomplish
      a clean system shutdown cannot be used inside a jail.  The best
      way to shut down a jail is to run the following command from
      within the jail itself or using the &man.jexec.8; utility from
      outside the jail:</para>

    <screen>&prompt.root; <userinput>sh /etc/rc.shutdown</userinput></screen>

    <para>More information about this can be found in the &man.jail.8;
      manual page.</para>
  </sect1>

  <sect1 id="jails-tuning">
    <title>Fine Tuning and Administration</title>

    <para>There are several options which can be set for any jail, and
      various ways of combining a host &os; system with jails, to produce
      higher level applications.  This section presents:</para>

    <itemizedlist>
      <listitem>
	<para>Some of the options available for tuning the behavior and
	  security restrictions implemented by a jail
	  installation.</para>
      </listitem>

      <listitem>
	<para>Some of the high-level applications for jail management,
	  which are available through the &os; Ports Collection, and can
	  be used to implement overall jail-based solutions.</para>
    </itemizedlist>

    <sect2 id="jails-tuning-utilities">
      <title>System Tools for Jail Tuning in &os;</title>

      <para>Fine tuning of a jail's configuration is mostly done by
	setting &man.sysctl.8; variables.  A special subtree of sysctl
	exists as a basis for organizing all the relevant options: the
	<varname>security.jail.*</varname> hierarchy of &os; kernel
	options.  Here is a list of the main jail-related sysctls,
	complete with their default value.  Names should be
	self-explanatory, but for more information about them, please
	refer to the &man.jail.8; and &man.sysctl.8; manual
	pages.</para>

      <itemizedlist>
	<listitem>
	  <para><varname>security.jail.set_hostname_allowed:
	     1</varname></para>
	</listitem>

	<listitem>
	  <para><varname>security.jail.socket_unixiproute_only:
	     1</varname></para>
	</listitem>

	<listitem>
	  <para><varname>security.jail.sysvipc_allowed:
	     0</varname></para>
	</listitem>

	<listitem>
	  <para><varname>security.jail.enforce_statfs:
	     2</varname></para>
	</listitem>

	<listitem>
	  <para><varname>security.jail.allow_raw_sockets:
	     0</varname></para>
	</listitem>

	<listitem>
	  <para><varname>security.jail.chflags_allowed:
	    0</varname></para>
	</listitem>

	<listitem>
	  <para><varname>security.jail.jailed: 0</varname></para>
	</listitem>
      </itemizedlist>

      <para>These variables can be used by the system administrator of
	the <emphasis>host system</emphasis> to add or remove some of
	the limitations imposed by default on the
	<username>root</username> user.  Note that there are some
	limitations which cannot be removed.  The
	<username>root</username> user is not allowed to mount or
	unmount file systems from within a &man.jail.8;.  The
	<username>root</username> inside a jail may not load or unload
	&man.devfs.8; rulesets, set firewall rules, or do many other
	administrative tasks which require modifications of in-kernel
	data, such as setting the <varname>securelevel</varname> of the
	kernel.</para>

      <para>The base system of &os; contains a basic set of tools for
	viewing information about the active jails, and attaching to a
	jail to run administrative commands.  The &man.jls.8; and
	&man.jexec.8; commands are part of the base &os; system, and can be used
	to perform the following simple tasks:</para>

      <itemizedlist>
	<listitem>
	  <para>Print a list of active jails and their corresponding
	    jail identifier (<acronym>JID</acronym>),
	    <acronym>IP</acronym> address, hostname and path.</para>
	</listitem>

	<listitem>
	  <para>Attach to a running jail, from its host system, and run
	    a command inside the jail or perform administrative tasks inside the
	    jail itself.  This is especially useful when the
	    <username>root</username> user wants to cleanly shut down a
	    jail.  The &man.jexec.8; utility can also be used to start a
	    shell in a jail to do administration in it; for
	    example:</para>

	  <screen>&prompt.root; <userinput>jexec <replaceable>1</replaceable> tcsh</userinput></screen>
	</listitem>
      </itemizedlist>
    </sect2>

    <sect2 id="jails-tuning-admintools">
      <title>High-Level Administrative Tools in &os; Ports
	Collection</title>

    <para>Among the many third-party utilities for jail administration,
      one of the most complete and useful is <filename
	role="package">sysutils/jailutils</filename>.  It is a set of
      small applications that contribute to &man.jail.8; management.
      Please refer to its web page for more information.</para>
    </sect2>
  </sect1>

  <sect1 id="jails-application">
    <title>Application of Jails</title>

    <sect2 id="jails-service-jails">
      <sect2info>
	<authorgroup>
	  <author>
	    <firstname>Daniel</firstname>
	    <surname>Gerzo</surname>
	    <contrib>Contributed by </contrib>
	    <!-- 15. May 2007 -->
	  </author>
	</authorgroup>
      </sect2info>

      <title>Service Jails</title>

      <para>This section is based upon an idea originally presented by
	&a.simon; at <ulink
	url="http://simon.nitro.dk/service-jails.html"></ulink>, and an
	updated article written by Ken Tom
	<email>locals@gmail.com</email>.  This section illustrates how
	to set up a &os; system that adds an additional layer of
	security, using the &man.jail.8; feature.  It is also assumed
	that the given system is at least running RELENG_6_0 and the
	information provided earlier in this chapter has been well
	understood.</para>

      <sect3 id="jails-service-jails-design">
       <title>Design</title>

	<para>One of the major problems with jails is the management of
	  their upgrade process.  This tends to be a problem because
	  every jail has to be rebuilt from scratch whenever it is
	  updated.  This is usually not a problem for a single jail,
	  since the update process is fairly simple, but can be quite
	  time consuming and tedious if a lot of jails are
	  created.</para>

	<warning>
	  <para>This setup requires advanced experience with &os; and
	    usage of its features.  If the presented steps below look
	    too complicated, it is advised to take a look at a simpler
	    system such as <filename
	      role="package">sysutils/ezjail</filename>, which provides
	    an easier method of administering &os; jails and is not as
	    sophisticated as this setup.</para>
	</warning>

	<para>This idea has been presented to resolve such issues by
	  sharing as much as is possible between jails, in a safe way
	  &mdash; using read-only &man.mount.nullfs.8; mounts, so that
	  updating will be simpler, and putting single services into
	  individual jails will become more attractive.  Additionally,
	  it provides a simple way to add or remove jails as well as a
	  way to upgrade them.</para>

	<note>
	  <para>Examples of services in this context are: an
	  <acronym>HTTP</acronym> server, a <acronym>DNS</acronym>
	  server, a <acronym>SMTP</acronym> server, and so forth.</para>
	</note>

	<para>The goals of the setup described in this section 
	  are:</para>

	<itemizedlist>
	  <listitem>
	    <para>Create a simple and easy to understand jail structure.
	      This implies <emphasis>not</emphasis> having to run a full
	      installworld on each and every jail.</para>
	  </listitem>
	  <listitem>
	    <para>Make it easy to add new jails or remove existing
	      ones.</para>
	  </listitem>
	  <listitem>
	    <para>Make it easy to update or upgrade existing
	      jails.</para>
	  </listitem>
	  <listitem>
	    <para>Make it possible to run a customized &os;
	      branch.</para>
	  </listitem>
	  <listitem>
	    <para>Be paranoid about security, reducing as much as
	      possible the possibility of compromise.</para>
	  </listitem>
	  <listitem>
	    <para>Save space and inodes, as much as possible.</para>
	  </listitem>
	</itemizedlist>

	<para>As it has been already mentioned, this design relies
	  heavily on having a single master template which is read-only
	  (known as <application>nullfs</application>) mounted into each
	  jail and one read-write device per jail.  A device can be a
	  separate physical disc, a partition, or a vnode backed
	  &man.md.4; device.  In this example, we will use read-write
	  <application>nullfs</application> mounts.</para>

	<para>The file system layout is described in the following
	  list:</para>

	<itemizedlist>
	  <listitem>
	    <para>Each jail will be mounted under the <filename
	      class="directory">/home/j</filename> directory.</para>
	  </listitem>
	  <listitem>
	    <para><filename class="directory">/home/j/mroot</filename> is
	      the template for each jail and the read-only partition for
	      all of the jails.</para>
	  </listitem>
	  <listitem>
	    <para>A blank directory will be created for each jail under
	      the  <filename class="directory">/home/j</filename>
	      directory.</para>
	  </listitem>
	  <listitem>
	    <para>Each jail will have a <filename
		class="directory">/s</filename> directory, that will be
	      linked to the read-write portion of the system.</para>
	  </listitem>
	  <listitem>
	    <para>Each jail shall have its own read-write system that is
	      based upon <filename
		class="directory">/home/j/skel</filename>.</para>
	  </listitem>
	  <listitem>
	    <para>Each jailspace (read-write portion of each jail) shall
	      be created in <filename
		class="directory">/home/js</filename>.</para>
	  </listitem>
	</itemizedlist>

	<note>
	  <para>This assumes that the jails are based under the
	    <filename class="directory">/home</filename> partition.  This
	    can, of course, be changed to anything else, but this change
	    will have to be reflected in each of the examples
	    below.</para>
	</note>
	<!-- Insert an image or drawing here to illustrate the example. -->
      </sect3>

      <sect3 id="jails-service-jails-template">
	<title>Creating the Template</title>

	<para>This section will describe the steps needed to create the
	  master template that will be the read-only portion for the
	  jails to use.</para>

	<para>It is always a good idea to update the &os; system to the
	  latest -RELEASE branch.  Check the corresponding Handbook
	  <ulink url="&url.books.handbook;/makeworld.html">Chapter</ulink>
	  to accomplish this task.  In the case the update is not
	  feasible, the buildworld will be required in order to be able
	  to proceed.  Additionally, the <filename
	    role="package">sysutils/cpdup</filename> package will be
	  required.  We will use the &man.portsnap.8; utility to
	  download the &os; Ports Collection.  The Handbook <ulink
	    url="&url.books.handbook;/portsnap.html">Portsnap Chapter</ulink>
	  is always good reading for newcomers.</para>

	<procedure>
	  <step>
	    <para>First, create a directory structure for the read-only
	      file system which will contain the &os; binaries for our
	      jails, then change directory to the &os; source tree and
	      install the read-only file system to the jail
	      template:</para>

	    <screen>&prompt.root; <userinput>mkdir /home/j /home/j/mroot</userinput>
&prompt.root; <userinput>cd /usr/src</userinput>
&prompt.root; <userinput>make installworld DESTDIR=/home/j/mroot</userinput></screen>
	  </step>
	  <step>
	    <para>Next, prepare a &os; Ports Collection for the jails as
	      well as a &os; source tree, which is required for
	      <application>mergemaster</application>:</para>

	    <screen>&prompt.root; <userinput>cd /home/j/mroot</userinput>
&prompt.root; <userinput>mkdir usr/ports</userinput>
&prompt.root; <userinput>portsnap -p /home/j/mroot/usr/ports fetch extract</userinput>
&prompt.root; <userinput>cpdup /usr/src /home/j/mroot/usr/src</userinput></screen>
	  </step>
	  <step>
	    <para>Create a skeleton for the read-write portion of the
	      system:</para>

	    <screen>&prompt.root; <userinput>mkdir /home/j/skel /home/j/skel/home /home/j/skel/usr-X11R6 /home/j/skel/distfiles</userinput>
&prompt.root; <userinput>mv etc /home/j/skel</userinput>
&prompt.root; <userinput>mv usr/local /home/j/skel/usr-local</userinput>
&prompt.root; <userinput>mv tmp /home/j/skel</userinput>
&prompt.root; <userinput>mv var /home/j/skel</userinput>
&prompt.root; <userinput>mv root /home/j/skel</userinput></screen>
	  </step>
	  <step>
	    <para>Use <application>mergemaster</application> to install
	      missing configuration files.  Then get rid of the extra
	      directories that <application>mergemaster</application>
	      creates:</para>

	    <screen>&prompt.root; <userinput>mergemaster -t /home/j/skel/var/tmp/temproot -D /home/j/skel -i</userinput>
&prompt.root; <userinput>cd /home/j/skel</userinput>
&prompt.root; <userinput>rm -R bin boot lib libexec mnt proc rescue sbin sys usr dev</userinput></screen>
	  </step>
	  <step>
	    <para>Now, symlink the read-write file system to the
	      read-only file system.  Please make sure that the symlinks
	      are created in the correct <filename
		class="directory">s/</filename> locations.  Real
	      directories or the creation of directories in the wrong
	      locations will cause the installation to  fail.</para>

	    <screen>&prompt.root; <userinput>cd /home/j/mroot</userinput>
&prompt.root; <userinput>mkdir s</userinput>
&prompt.root; <userinput>ln -s s/etc etc</userinput>
&prompt.root; <userinput>ln -s s/home home</userinput>
&prompt.root; <userinput>ln -s s/root root</userinput>
&prompt.root; <userinput>ln -s ../s/usr-local usr/local</userinput>
&prompt.root; <userinput>ln -s ../s/usr-X11R6 usr/X11R6</userinput>
&prompt.root; <userinput>ln -s ../../s/distfiles usr/ports/distfiles</userinput>
&prompt.root; <userinput>ln -s s/tmp tmp</userinput>
&prompt.root; <userinput>ln -s s/var var</userinput></screen>
	  </step>
	  <step>
	    <para>As a last step, create a generic
	      <filename>/home/j/skel/etc/make.conf</filename> with its
	      contents as shown below:</para>

	      <programlisting>WRKDIRPREFIX?=  /s/portbuild</programlisting>


	      <para>Having <literal>WRKDIRPREFIX</literal> set up this
	      way will make it possible to compile &os; ports inside
	      each jail.  Remember that the ports directory is part of
	      the read-only system.  The custom path for
	      <literal>WRKDIRPREFIX</literal> allows builds to be done
	      in the read-write portion of every jail.</para>
	  </step>
	</procedure>
      </sect3>

      <sect3 id="jails-service-jails-creating">
	<title>Creating Jails</title>

	<para>Now that we have a complete &os; jail template, we can
	  setup and configure the jails in
	  <filename>/etc/rc.conf</filename>.  This example demonstrates
	  the creation of 3 jails: <quote>NS</quote>,
	  <quote>MAIL</quote> and <quote>WWW</quote>.</para>

	<procedure>
	  <step>
	    <para>Put the following lines into the
	      <filename>/etc/fstab</filename> file, so that the
	      read-only template for the jails and the read-write space
	      will be available in the respective jails:</para>

	    <programlisting>/home/j/mroot   /home/j/ns     nullfs  ro  0   0
/home/j/mroot   /home/j/mail   nullfs  ro  0   0
/home/j/mroot   /home/j/www    nullfs  ro  0   0
/home/js/ns     /home/j/ns/s   nullfs  rw  0   0
/home/js/mail   /home/j/mail/s nullfs  rw  0   0
/home/js/www    /home/j/www/s  nullfs  rw  0   0</programlisting>
	
	    <note>
	      <para>Partitions marked with a 0 pass number are not
		checked by &man.fsck.8; during boot, and partitions
		marked with a 0 dump number are not backed up by
		&man.dump.8;.  We do not want
		<application>fsck</application> to check
		<application>nullfs</application> mounts or
		<application>dump</application> to back up the read-only
		nullfs mounts of the jails.  This is why they are marked
		with <quote>0&nbsp;0</quote> in the last two columns of
		each <filename>fstab</filename> entry above.</para>
	    </note>
	  </step>
	  <step>
	    <para>Configure the jails in
	      <filename>/etc/rc.conf</filename>:</para>

	    <programlisting>jail_enable="YES"
jail_set_hostname_allow="NO"
jail_list="ns mail www"
jail_ns_hostname="ns.example.org"
jail_ns_ip="192.168.3.17"
jail_ns_rootdir="/usr/home/j/ns"
jail_ns_devfs_enable="YES"
jail_mail_hostname="mail.example.org"
jail_mail_ip="192.168.3.18"
jail_mail_rootdir="/usr/home/j/mail"
jail_mail_devfs_enable="YES"
jail_www_hostname="www.example.org"
jail_www_ip="62.123.43.14"
jail_www_rootdir="/usr/home/j/www"
jail_www_devfs_enable="YES"</programlisting>

	    <warning>
	      <para>The reason why the
		<varname>jail_<replaceable>name</replaceable>_rootdir</varname>
		variable is set to <filename
		  class="directory">/usr/home</filename> instead of
		<filename class="directory">/home</filename> is that the
		physical path of the <filename
		  class="directory">/home</filename> directory on a
		default &os; installation is <filename
		  class="directory">/usr/home</filename>.  The
		<varname>jail_<replaceable>name</replaceable>_rootdir</varname>
		variable must <emphasis>not</emphasis> be set to a path
		which includes a symbolic link, otherwise the jails will
		refuse to start.  Use the &man.realpath.1; utility to
		determine a value which should be set to this variable.
		Please see the &os;-SA-07:01.jail Security Advisory for
		more information.</para>
	    </warning>
	  </step>
	  <step>
	    <para>Create the required mount points for the read-only
	      file system of each jail:</para>

	    <screen>&prompt.root; <userinput>mkdir /home/j/ns /home/j/mail /home/j/www</userinput></screen>
	  </step>
	  <step>
	    <para>Install the read-write template into each jail.  Note
	      the use of <filename
		role="package">sysutils/cpdup</filename>, which helps to
	      ensure that a correct copy is done of each
	      directory:</para>
	    <!-- keramida: Why is cpdup required here?  Doesn't cpio(1)
	     already include adequate functionality for performing this
	     job *and* have the advantage of being part of the base
	     system of FreeBSD? -->

	    <screen>&prompt.root; <userinput>mkdir /home/js</userinput>
&prompt.root; <userinput>cpdup /home/j/skel /home/js/ns</userinput>
&prompt.root; <userinput>cpdup /home/j/skel /home/js/mail</userinput>
&prompt.root; <userinput>cpdup /home/j/skel /home/js/www</userinput></screen>
	  </step>
	  <step>
	    <para>In this phase, the jails are built and prepared to
	      run.  First, mount the required file systems for each
	      jail, and then start them using the
	      <filename>/etc/rc.d/jail</filename> script:</para>

	    <screen>&prompt.root; <userinput>mount -a</userinput>
&prompt.root; <userinput>/etc/rc.d/jail start</userinput></screen>
	  </step>
	</procedure>

	<para>The jails should be running now.  To check if they have
	  started correctly, use the &man.jls.8; command.  Its output
	  should be similar to the following:</para>

	<screen>&prompt.root; <userinput>jls</userinput>
   JID  IP Address      Hostname                      Path
     3  192.168.3.17    ns.example.org                /home/j/ns
     2  192.168.3.18    mail.example.org              /home/j/mail
     1  62.123.43.14    www.example.org               /home/j/www</screen>

	<para>At this point, it should be possible to log onto each
	  jail, add new users or configure daemons.  The
	  <literal>JID</literal> column indicates the jail
	  identification number of each running jail.  Use the
	  following command in order to perform administrative tasks in
	  the jail whose <literal>JID</literal> is 3:</para>

	<screen>&prompt.root; <userinput>jexec 3 tcsh</userinput></screen>
      </sect3>

      <sect3 id="jails-service-jails-upgrading">
	<title>Upgrading</title>

	<para>In time, there will be a need to upgrade the system to a
	  newer version of &os;, either because of a security issue, or
	  because new features have been implemented which are useful
	  for the existing jails.  The design of this setup provides an
	  easy way to upgrade existing jails.  Additionally, it
	  minimizes their downtime, as the jails will be brought down
	  only in the very last minute.  Also, it provides a way to roll
	  back to the older versions should any problems occur.</para>

	<procedure>
	  <step>
	    <para>The first step is to upgrade the host system in the
	      usual manner.  Then create a new temporary read-only
	      template in <filename
		class="directory">/home/j/mroot2</filename>.</para>

	    <screen>&prompt.root; <userinput>mkdir /home/j/mroot2</userinput>
&prompt.root; <userinput>cd /usr/src</userinput>
&prompt.root; <userinput>make installworld DESTDIR=/home/j/mroot2</userinput>
&prompt.root; <userinput>cd /home/j/mroot2</userinput>
&prompt.root; <userinput>cpdup /usr/src usr/src</userinput>
&prompt.root; <userinput>mkdir s</userinput></screen>

	    <para>The <maketarget>installworld</maketarget> run creates
	      a few unnecessary directories, which should be
	      removed:</para>

	    <screen>&prompt.root; <userinput>chflags -R 0 var</userinput>
&prompt.root; <userinput>rm -R etc var root usr/local tmp</userinput></screen>
	  </step>
	  <step>
	    <para>Recreate the read-write symlinks for the master file
	      system:</para>

	    <screen>&prompt.root; <userinput>ln -s s/etc etc</userinput>
&prompt.root; <userinput>ln -s s/root root</userinput>
&prompt.root; <userinput>ln -s s/home home</userinput>
&prompt.root; <userinput>ln -s ../s/usr-local usr/local</userinput>
&prompt.root; <userinput>ln -s ../s/usr-X11R6 usr/X11R6</userinput>
&prompt.root; <userinput>ln -s s/tmp tmp</userinput>
&prompt.root; <userinput>ln -s s/var var</userinput></screen>
	  </step>
	  <step>
	    <para>The right time to stop the jails is now:</para>

	    <screen>&prompt.root; <userinput>/etc/rc.d/jail stop</userinput></screen>
	  </step>
	  <step>
	    <para>Unmount the original file systems:</para>
	    <!-- keramida: Shouldn't we suggest a short script-based
	     loop here, instead of tediously copying the same commands
	     multiple times? -->

	    <screen>&prompt.root; <userinput>umount /home/j/ns/s</userinput>
&prompt.root; <userinput>umount /home/j/ns</userinput>
&prompt.root; <userinput>umount /home/j/mail/s</userinput>
&prompt.root; <userinput>umount /home/j/mail</userinput>
&prompt.root; <userinput>umount /home/j/www/s</userinput>
&prompt.root; <userinput>umount /home/j/www</userinput></screen>

	    <note>
	      <para>The read-write systems are attached to the read-only
		system (<filename class="directory">/s</filename>) and
		must be unmounted first.</para>
	    </note>
	  </step>
	  <step>
	    <para>Move the old read-only file system and replace it with
	      the new one.  This will serve as a backup and archive of the
	      old read-only file system should something go wrong.  The
	      naming convention used here corresponds to when a new
	      read-only file system has been created.  Move the original
	      &os; Ports Collection over to the new file system to save
	      some space and inodes:</para>

	    <screen>&prompt.root; <userinput>cd /home/j</userinput>
&prompt.root; <userinput>mv mroot mroot.20060601</userinput>
&prompt.root; <userinput>mv mroot2 mroot</userinput>
&prompt.root; <userinput>mv mroot.20060601/usr/ports mroot/usr</userinput></screen>
	  </step>
	  <step>
	    <para>At this point the new read-only template is ready, so
	      the only remaining task is to remount the file systems and
	      start the jails:</para>

	    <screen>&prompt.root; <userinput>mount -a</userinput>
&prompt.root; <userinput>/etc/rc.d/jail start</userinput></screen>
	  </step>
	</procedure>

	<para>Use &man.jls.8; to check if the jails started correctly.
	  Do not forget to run mergemaster in each jail.  The
	  configuration files will need to be updated as well as the
	  rc.d scripts.</para>
      </sect3>
    </sect2>
  </sect1>
</chapter>