1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
|
<!--
The FreeBSD Documentation Project
$FreeBSD$
-->
<chapter id="jails">
<chapterinfo>
<authorgroup>
<author>
<firstname>Matteo</firstname>
<surname>Riondato</surname>
<contrib>Contributed by </contrib>
</author>
</authorgroup>
</chapterinfo>
<title>Jails</title>
<indexterm><primary>jails</primary></indexterm>
<sect1 id="jails-synopsis">
<title>Synopsis</title>
<para>This chapter will provide an explanation of what &os; jails
are and how to use them. Jails, sometimes referred to as an
enhanced replacement of <emphasis>chroot environments</emphasis>,
are a very powerful tool for system administrators, but their basic
usage can also be useful for advanced users.</para>
<para>After reading this chapter, you will know:</para>
<itemizedlist>
<listitem>
<para>What a jail is, and what purpose it may serve in &os;
installations.</para>
</listitem>
<listitem>
<para>How to build, start, and stop a jail.</para>
</listitem>
<listitem>
<para>The basics of jail administration, both from inside
and outside the jail.</para>
</listitem>
</itemizedlist>
<para>Other sources of useful information about jails are:</para>
<itemizedlist>
<listitem>
<para>The &man.jail.8; manual page. This is the full reference
of the <command>jail</command> utility — the
administrative tool which can be used in &os; to start, stop,
and control &os; jails.</para>
</listitem>
<listitem>
<para>The mailing lists and their archives. The archives of the
&a.questions; and other mailing lists hosted by the
&a.mailman.lists; already contain a wealth of material for
jails. It should always be engaging to search the archives,
or post a new question to the &a.questions.name; mailing
list.</para>
</listitem>
</itemizedlist>
</sect1>
<sect1 id="jails-terms">
<title>Terms Related to Jails</title>
<para>To facilitate better understanding of parts of the &os; system
related to jails, their internals and the way they interact with
the rest of &os;, the following terms are used further in this
chapter:</para>
<variablelist>
<varlistentry>
<term>&man.chroot.8; (command)</term>
<listitem>
<para>Utility, which uses &man.chroot.2; &os; system call to change
the root directory of a process and all its descendants.</para>
</listitem>
</varlistentry>
<varlistentry>
<term>&man.chroot.2; (environment)</term>
<listitem>
<para>The environment of processes running in
a <quote>chroot</quote>. This includes resources such as the part
of the file system which is visible, user and group IDs which are
available, network interfaces and other IPC mechanisms,
etc.</para>
</listitem>
</varlistentry>
<varlistentry>
<term>&man.jail.8; (command)</term>
<listitem>
<para>The system administration utility which allows launching of
processes within a jail environment.</para>
</listitem>
</varlistentry>
<varlistentry>
<term>host (system, process, user, etc.)</term>
<listitem>
<para>The controlling system of a jail environment. The host system
has access to all the hardware resources available, and can
control processes both outside of and inside a jail environment.
One of the important differences of the host system from a jail is
that the limitations which apply to superuser processes inside a
jail are not enforced for processes of the host system.</para>
</listitem>
</varlistentry>
<varlistentry>
<term>hosted (system, process, user, etc.)</term>
<listitem>
<para>A process, user or other entity, whose access to resources is
restricted by a &os; jail.</para>
</listitem>
</varlistentry>
</variablelist>
</sect1>
<sect1 id="jails-intro">
<title>Introduction</title>
<para>Since system administration is a difficult and perplexing
task, many powerful tools were developed to make life easier for
the administrator. These tools mostly provide enhancements of some sort
to the way systems are installed, configured and maintained.
Part of the tasks which an administrator is
expected to do is to properly configure the security of a system,
so that it can continue serving its real purpose, without allowing
security violations.</para>
<para>One of the tools which can be used to enhance the security of
a &os; system are <emphasis>jails</emphasis>. Jails were
introduced in &os; 4.X by &a.phk;, but were greatly improved in
&os; 5.X to make them a powerful and flexible subsystem. Their
development still goes on, enhancing their usefulness, performance, reliability,
and security.</para>
<sect2 id="jails-what">
<title>What is a Jail</title>
<para>BSD-like operating systems have had &man.chroot.2; since the
time of 4.2BSD. The &man.chroot.8; utility can be used to
change the root directory
of a set of processes, creating a safe environment, separate
from the rest of the system. Processes created in the chrooted
environment can not access files or resources outside of it.
For that reason, compromising a service running in a chrooted
environment should not allow the attacker to compromise the
entire system. The &man.chroot.8; utility is good for easy
tasks, which do not require a lot of flexibility or complex and
advanced features. Since the inception of the
chroot concept, however, many ways have been found to escape from a
chrooted environment and, although they have been fixed in
modern versions of the &os; kernel, it was clear that
&man.chroot.2; was not the ideal solution for securing services.
A new subsystem had to be implemented.</para>
<para>This is one of the main reasons why
<emphasis>jails</emphasis> were developed.</para>
<para>Jails improve on the concept of the traditional
&man.chroot.2; environment, in several ways. In a traditional
&man.chroot.2; environment, processes are only limited in the
part of the file system they can access. The rest of the system
resources (like the set of system users, the running processes,
or the networking subsystem) are shared by the chrooted
processes and the processes of the host system. Jails expand
this model by virtualizing not only access to the file system,
but also the set of users, the networking subsystem of the &os;
kernel and a few other things. A more complete set of
fine-grained controls available for tuning the access of a
jailed environment is described in <xref
linkend="jails-tuning">.</para>
<para>A jail is characterized by four elements:</para>
<itemizedlist>
<listitem>
<para>A directory subtree — the starting point from
which a jail is entered. Once inside the jail, a process
is not permitted to escape outside of this subtree.
Traditional security issues which plagued the original
&man.chroot.2; design will not affect &os; jails.</para>
</listitem>
<listitem>
<para>A hostname — the hostname which will be used
within the jail. Jails are mainly used for hosting network
services, therefore having a descriptive hostname for each
jail can really help the system administrator.</para>
</listitem>
<listitem>
<para>An <acronym>IP</acronym> address — this will be
assigned to the jail and cannot be changed in any way during
the jail's life span. The IP address of a jail is usually an alias address
for an existing network interface, but this is not strictly necessary.</para>
</listitem>
<listitem>
<para>A command — the path name of an executable to run
inside the jail. This is relative to the root directory of
the jail environment, and may vary a lot, depending on the
type of the specific jail environment.</para>
</listitem>
</itemizedlist>
<para>Apart from these, jails can have their own set of users and
their own <username>root</username> user. Naturally, the powers
of the <username>root</username> user are limited within the
jail environment and, from the point of view of the host system,
the jail <username>root</username> user is not an omnipotent user.
In addition, the <username>root</username> user of a jail is not
allowed to perform critical operations to the system outside of
the associated &man.jail.8; environment. More information
about capabilities and restrictions of the
<username>root</username> user will be discussed in <xref
linkend="jails-tuning"> below.</para>
</sect2>
</sect1>
<sect1 id="jails-build">
<title>Creating and Controlling Jails</title>
<para>Some administrators divide jails into the following two types:
<quote>complete</quote> jails, which resemble a real &os; system,
and <quote>service</quote> jails, dedicated to one application or
service, possibly running with privileges. This is only a
conceptual division and the process of building a jail is not
affected by it. The &man.jail.8; manual page is quite clear about
the procedure for building a jail:</para>
<screen>&prompt.root; <userinput>setenv D <replaceable>/here/is/the/jail</replaceable></userinput>
&prompt.root; <userinput>mkdir -p $D</userinput> <co id="jailpath">
&prompt.root; <userinput>cd /usr/src</userinput>
&prompt.root; <userinput>make buildworld</userinput> <co id="jailbuildworld">
&prompt.root; <userinput>make installworld DESTDIR=$D</userinput> <co id="jailinstallworld">
&prompt.root; <userinput>make distribution DESTDIR=$D</userinput> <co id="jaildistrib">
&prompt.root; <userinput>mount -t devfs devfs $D/dev</userinput> <co id="jaildevfs"></screen>
<calloutlist>
<callout arearefs="jailpath">
<para>Selecting a location for a jail is the best starting point.
This is where the jail will physically reside within the file system of the jail's host.
A good choice can be <filename
class="directory">/usr/jail/<replaceable>jailname</replaceable></filename>,
where <replaceable>jailname</replaceable> is the hostname
identifying the jail. The <filename
class="directory">/usr/</filename> file system usually has
enough space for the jail file system, which for <quote>complete</quote> jails is, essentially,
a replication of every file present in a default installation
of the &os; base system.</para>
</callout>
<callout arearefs="jailbuildworld">
<para>If you have already rebuilt your userland using
<command>make world</command> or <command>make buildworld</command>,
you can skip this step and install your existing userland into the
new jail.</para>
</callout>
<callout arearefs="jailinstallworld">
<para>This command will populate the directory subtree chosen
as jail's physical location on the file system with the
necessary binaries, libraries, manual pages and so on.</para>
</callout>
<callout arearefs="jaildistrib">
<para>The <maketarget>distribution</maketarget> target for
<application>make</application> installs every needed
configuration file. In simple words, it installs every installable file of
<filename class="directory">/usr/src/etc/</filename> to the
<filename class="directory">/etc</filename> directory of the jail
environment:
<filename class="directory">$D/etc/</filename>.</para>
</callout>
<callout arearefs="jaildevfs">
<para>Mounting the &man.devfs.8; file system inside a jail is
not required. On the other hand, any, or almost any
application requires access to at least one device, depending
on the purpose of the given application. It is very important
to control access to devices from inside a jail, as improper
settings could permit an attacker to do nasty things in the
jail. Control over &man.devfs.8; is managed through rulesets
which are described in the &man.devfs.8; and
&man.devfs.conf.5; manual pages.</para>
</callout>
</calloutlist>
<para>Once a jail is installed, it can be started by using the
&man.jail.8; utility. The &man.jail.8; utility takes four
mandatory arguments which are described in the <xref
linkend="jails-what">. Other arguments may be
specified too, e.g., to run the jailed process with the credentials of a specific
user. The <option><replaceable>command</replaceable></option> argument depends on
the type of the jail; for a <emphasis>virtual system</emphasis>,
<filename>/etc/rc</filename> is a good choice, since it will
replicate the startup sequence of a real &os; system. For a
<emphasis>service</emphasis> jail, it depends on the service or
application that will run within the jail.</para>
<para>Jails are often started at boot time and the &os;
<filename>rc</filename> mechanism provides an easy way to do
this.</para>
<procedure>
<step>
<para>A list of the jails which are enabled to start at boot
time should be added to the &man.rc.conf.5; file:</para>
<programlisting>jail_enable="YES" # Set to NO to disable starting of any jails
jail_list="<replaceable>www</replaceable>" # Space separated list of names of jails</programlisting>
<note>
<para>Jail names in <varname>jail_list</varname> should
contain alphanumeric characters only.</para>
</note>
</step>
<step>
<para>For each jail listed in <varname>jail_list</varname>, a
group of &man.rc.conf.5; settings, which describe the
particular jail, should be added:</para>
<programlisting>jail_<replaceable>www</replaceable>_rootdir="/usr/jail/www" # jail's root directory
jail_<replaceable>www</replaceable>_hostname="<replaceable>www</replaceable>.example.org" # jail's hostname
jail_<replaceable>www</replaceable>_ip="192.168.0.10" # jail's IP address
jail_<replaceable>www</replaceable>_devfs_enable="YES" # mount devfs in the jail
jail_<replaceable>www</replaceable>_devfs_ruleset="<replaceable>www_ruleset</replaceable>" # devfs ruleset to apply to jail</programlisting>
<para>The default startup of jails configured in
&man.rc.conf.5;, will run the <filename>/etc/rc</filename>
script of the jail, which assumes the jail is a complete
virtual system. For service jails, the default startup
command of the jail should be changed, by setting the
<varname>jail_<replaceable>jailname</replaceable>_exec_start</varname>
option appropriately.</para>
<note>
<para>For a full list of available options, please see the
&man.rc.conf.5; manual page.</para>
</note>
</step>
</procedure>
<para>The <filename>/etc/rc.d/jail</filename> script can be used to
start or stop a jail by hand, if an entry for it exists in
<filename>rc.conf</filename>:</para>
<screen>&prompt.root; <userinput>/etc/rc.d/jail start <replaceable>www</replaceable></userinput>
&prompt.root; <userinput>/etc/rc.d/jail stop <replaceable>www</replaceable></userinput></screen>
<para>A clean way to shut down a &man.jail.8; is not available at
the moment. This is because commands normally used to accomplish
a clean system shutdown cannot be used inside a jail. The best
way to shut down a jail is to run the following command from
within the jail itself or using the &man.jexec.8; utility from
outside the jail:</para>
<screen>&prompt.root; <userinput>sh /etc/rc.shutdown</userinput></screen>
<para>More information about this can be found in the &man.jail.8;
manual page.</para>
</sect1>
<sect1 id="jails-tuning">
<title>Fine Tuning and Administration</title>
<para>There are several options which can be set for any jail, and
various ways of combining a host &os; system with jails, to produce
higher level applications. This section presents:</para>
<itemizedlist>
<listitem>
<para>Some of the options available for tuning the behavior and
security restrictions implemented by a jail
installation.</para>
</listitem>
<listitem>
<para>Some of the high-level applications for jail management,
which are available through the &os; Ports Collection, and can
be used to implement overall jail-based solutions.</para>
</itemizedlist>
<sect2 id="jails-tuning-utilities">
<title>System Tools for Jail Tuning in &os;</title>
<para>Fine tuning of a jail's configuration is mostly done by
setting &man.sysctl.8; variables. A special subtree of sysctl
exists as a basis for organizing all the relevant options: the
<varname>security.jail.*</varname> hierarchy of &os; kernel
options. Here is a list of the main jail-related sysctls,
complete with their default value. Names should be
self-explanatory, but for more information about them, please
refer to the &man.jail.8; and &man.sysctl.8; manual
pages.</para>
<itemizedlist>
<listitem>
<para><varname>security.jail.set_hostname_allowed:
1</varname></para>
</listitem>
<listitem>
<para><varname>security.jail.socket_unixiproute_only:
1</varname></para>
</listitem>
<listitem>
<para><varname>security.jail.sysvipc_allowed:
0</varname></para>
</listitem>
<listitem>
<para><varname>security.jail.enforce_statfs:
2</varname></para>
</listitem>
<listitem>
<para><varname>security.jail.allow_raw_sockets:
0</varname></para>
</listitem>
<listitem>
<para><varname>security.jail.chflags_allowed:
0</varname></para>
</listitem>
<listitem>
<para><varname>security.jail.jailed: 0</varname></para>
</listitem>
</itemizedlist>
<para>These variables can be used by the system administrator of
the <emphasis>host system</emphasis> to add or remove some of
the limitations imposed by default on the
<username>root</username> user. Note that there are some
limitations which cannot be removed. The
<username>root</username> user is not allowed to mount or
unmount file systems from within a &man.jail.8;. The
<username>root</username> inside a jail may not load or unload
&man.devfs.8; rulesets, set firewall rules, or do many other
administrative tasks which require modifications of in-kernel
data, such as setting the <varname>securelevel</varname> of the
kernel.</para>
<para>The base system of &os; contains a basic set of tools for
viewing information about the active jails, and attaching to a
jail to run administrative commands. The &man.jls.8; and
&man.jexec.8; commands are part of the base &os; system, and can be used
to perform the following simple tasks:</para>
<itemizedlist>
<listitem>
<para>Print a list of active jails and their corresponding
jail identifier (<acronym>JID</acronym>),
<acronym>IP</acronym> address, hostname and path.</para>
</listitem>
<listitem>
<para>Attach to a running jail, from its host system, and run
a command inside the jail or perform administrative tasks inside the
jail itself. This is especially useful when the
<username>root</username> user wants to cleanly shut down a
jail. The &man.jexec.8; utility can also be used to start a
shell in a jail to do administration in it; for
example:</para>
<screen>&prompt.root; <userinput>jexec <replaceable>1</replaceable> tcsh</userinput></screen>
</listitem>
</itemizedlist>
</sect2>
<sect2 id="jails-tuning-admintools">
<title>High-Level Administrative Tools in the &os; Ports
Collection</title>
<para>Among the many third-party utilities for jail administration,
one of the most complete and useful is <filename
role="package">sysutils/jailutils</filename>. It is a set of
small applications that contribute to &man.jail.8; management.
Please refer to its web page for more information.</para>
</sect2>
</sect1>
<sect1 id="jails-application">
<title>Application of Jails</title>
<sect2 id="jails-service-jails">
<sect2info>
<authorgroup>
<author>
<firstname>Daniel</firstname>
<surname>Gerzo</surname>
<contrib>Contributed by </contrib>
<!-- 15. May 2007 -->
</author>
</authorgroup>
</sect2info>
<title>Service Jails</title>
<para>This section is based upon an idea originally presented by
&a.simon; at <ulink
url="http://simon.nitro.dk/service-jails.html"></ulink>, and an
updated article written by Ken Tom
<email>locals@gmail.com</email>. This section illustrates how
to set up a &os; system that adds an additional layer of
security, using the &man.jail.8; feature. It is also assumed
that the given system is at least running RELENG_6_0 and the
information provided earlier in this chapter has been well
understood.</para>
<sect3 id="jails-service-jails-design">
<title>Design</title>
<para>One of the major problems with jails is the management of
their upgrade process. This tends to be a problem because
every jail has to be rebuilt from scratch whenever it is
updated. This is usually not a problem for a single jail,
since the update process is fairly simple, but can be quite
time consuming and tedious if a lot of jails are
created.</para>
<warning>
<para>This setup requires advanced experience with &os; and
usage of its features. If the presented steps below look
too complicated, it is advised to take a look at a simpler
system such as <filename
role="package">sysutils/ezjail</filename>, which provides
an easier method of administering &os; jails and is not as
sophisticated as this setup.</para>
</warning>
<para>This idea has been presented to resolve such issues by
sharing as much as is possible between jails, in a safe way
— using read-only &man.mount.nullfs.8; mounts, so that
updating will be simpler, and putting single services into
individual jails will become more attractive. Additionally,
it provides a simple way to add or remove jails as well as a
way to upgrade them.</para>
<note>
<para>Examples of services in this context are: an
<acronym>HTTP</acronym> server, a <acronym>DNS</acronym>
server, a <acronym>SMTP</acronym> server, and so forth.</para>
</note>
<para>The goals of the setup described in this section
are:</para>
<itemizedlist>
<listitem>
<para>Create a simple and easy to understand jail structure.
This implies <emphasis>not</emphasis> having to run a full
installworld on each and every jail.</para>
</listitem>
<listitem>
<para>Make it easy to add new jails or remove existing
ones.</para>
</listitem>
<listitem>
<para>Make it easy to update or upgrade existing
jails.</para>
</listitem>
<listitem>
<para>Make it possible to run a customized &os;
branch.</para>
</listitem>
<listitem>
<para>Be paranoid about security, reducing as much as
possible the possibility of compromise.</para>
</listitem>
<listitem>
<para>Save space and inodes, as much as possible.</para>
</listitem>
</itemizedlist>
<para>As it has been already mentioned, this design relies
heavily on having a single master template which is read-only
(known as <application>nullfs</application>) mounted into each
jail and one read-write device per jail. A device can be a
separate physical disc, a partition, or a vnode backed
&man.md.4; device. In this example, we will use read-write
<application>nullfs</application> mounts.</para>
<para>The file system layout is described in the following
list:</para>
<itemizedlist>
<listitem>
<para>Each jail will be mounted under the <filename
class="directory">/home/j</filename> directory.</para>
</listitem>
<listitem>
<para><filename class="directory">/home/j/mroot</filename> is
the template for each jail and the read-only partition for
all of the jails.</para>
</listitem>
<listitem>
<para>A blank directory will be created for each jail under
the <filename class="directory">/home/j</filename>
directory.</para>
</listitem>
<listitem>
<para>Each jail will have a <filename
class="directory">/s</filename> directory, that will be
linked to the read-write portion of the system.</para>
</listitem>
<listitem>
<para>Each jail shall have its own read-write system that is
based upon <filename
class="directory">/home/j/skel</filename>.</para>
</listitem>
<listitem>
<para>Each jailspace (read-write portion of each jail) shall
be created in <filename
class="directory">/home/js</filename>.</para>
</listitem>
</itemizedlist>
<note>
<para>This assumes that the jails are based under the
<filename class="directory">/home</filename> partition. This
can, of course, be changed to anything else, but this change
will have to be reflected in each of the examples
below.</para>
</note>
<!-- Insert an image or drawing here to illustrate the example. -->
</sect3>
<sect3 id="jails-service-jails-template">
<title>Creating the Template</title>
<para>This section will describe the steps needed to create the
master template that will be the read-only portion for the
jails to use.</para>
<para>It is always a good idea to update the &os; system to the
latest -RELEASE branch. Check the corresponding Handbook
<ulink url="&url.books.handbook;/makeworld.html">Chapter</ulink>
to accomplish this task. In the case the update is not
feasible, the buildworld will be required in order to be able
to proceed. Additionally, the <filename
role="package">sysutils/cpdup</filename> package will be
required. We will use the &man.portsnap.8; utility to
download the &os; Ports Collection. The Handbook <ulink
url="&url.books.handbook;/portsnap.html">Portsnap Chapter</ulink>
is always good reading for newcomers.</para>
<procedure>
<step>
<para>First, create a directory structure for the read-only
file system which will contain the &os; binaries for our
jails, then change directory to the &os; source tree and
install the read-only file system to the jail
template:</para>
<screen>&prompt.root; <userinput>mkdir /home/j /home/j/mroot</userinput>
&prompt.root; <userinput>cd /usr/src</userinput>
&prompt.root; <userinput>make installworld DESTDIR=/home/j/mroot</userinput></screen>
</step>
<step>
<para>Next, prepare a &os; Ports Collection for the jails as
well as a &os; source tree, which is required for
<application>mergemaster</application>:</para>
<screen>&prompt.root; <userinput>cd /home/j/mroot</userinput>
&prompt.root; <userinput>mkdir usr/ports</userinput>
&prompt.root; <userinput>portsnap -p /home/j/mroot/usr/ports fetch extract</userinput>
&prompt.root; <userinput>cpdup /usr/src /home/j/mroot/usr/src</userinput></screen>
</step>
<step>
<para>Create a skeleton for the read-write portion of the
system:</para>
<screen>&prompt.root; <userinput>mkdir /home/j/skel /home/j/skel/home /home/j/skel/usr-X11R6 /home/j/skel/distfiles</userinput>
&prompt.root; <userinput>mv etc /home/j/skel</userinput>
&prompt.root; <userinput>mv usr/local /home/j/skel/usr-local</userinput>
&prompt.root; <userinput>mv tmp /home/j/skel</userinput>
&prompt.root; <userinput>mv var /home/j/skel</userinput>
&prompt.root; <userinput>mv root /home/j/skel</userinput></screen>
</step>
<step>
<para>Use <application>mergemaster</application> to install
missing configuration files. Then get rid of the extra
directories that <application>mergemaster</application>
creates:</para>
<screen>&prompt.root; <userinput>mergemaster -t /home/j/skel/var/tmp/temproot -D /home/j/skel -i</userinput>
&prompt.root; <userinput>cd /home/j/skel</userinput>
&prompt.root; <userinput>rm -R bin boot lib libexec mnt proc rescue sbin sys usr dev</userinput></screen>
</step>
<step>
<para>Now, symlink the read-write file system to the
read-only file system. Please make sure that the symlinks
are created in the correct <filename
class="directory">s/</filename> locations. Real
directories or the creation of directories in the wrong
locations will cause the installation to fail.</para>
<screen>&prompt.root; <userinput>cd /home/j/mroot</userinput>
&prompt.root; <userinput>mkdir s</userinput>
&prompt.root; <userinput>ln -s s/etc etc</userinput>
&prompt.root; <userinput>ln -s s/home home</userinput>
&prompt.root; <userinput>ln -s s/root root</userinput>
&prompt.root; <userinput>ln -s ../s/usr-local usr/local</userinput>
&prompt.root; <userinput>ln -s ../s/usr-X11R6 usr/X11R6</userinput>
&prompt.root; <userinput>ln -s ../../s/distfiles usr/ports/distfiles</userinput>
&prompt.root; <userinput>ln -s s/tmp tmp</userinput>
&prompt.root; <userinput>ln -s s/var var</userinput></screen>
</step>
<step>
<para>As a last step, create a generic
<filename>/home/j/skel/etc/make.conf</filename> with its
contents as shown below:</para>
<programlisting>WRKDIRPREFIX?= /s/portbuild</programlisting>
<para>Having <literal>WRKDIRPREFIX</literal> set up this
way will make it possible to compile &os; ports inside
each jail. Remember that the ports directory is part of
the read-only system. The custom path for
<literal>WRKDIRPREFIX</literal> allows builds to be done
in the read-write portion of every jail.</para>
</step>
</procedure>
</sect3>
<sect3 id="jails-service-jails-creating">
<title>Creating Jails</title>
<para>Now that we have a complete &os; jail template, we can
setup and configure the jails in
<filename>/etc/rc.conf</filename>. This example demonstrates
the creation of 3 jails: <quote>NS</quote>,
<quote>MAIL</quote> and <quote>WWW</quote>.</para>
<procedure>
<step>
<para>Put the following lines into the
<filename>/etc/fstab</filename> file, so that the
read-only template for the jails and the read-write space
will be available in the respective jails:</para>
<programlisting>/home/j/mroot /home/j/ns nullfs ro 0 0
/home/j/mroot /home/j/mail nullfs ro 0 0
/home/j/mroot /home/j/www nullfs ro 0 0
/home/js/ns /home/j/ns/s nullfs rw 0 0
/home/js/mail /home/j/mail/s nullfs rw 0 0
/home/js/www /home/j/www/s nullfs rw 0 0</programlisting>
<note>
<para>Partitions marked with a 0 pass number are not
checked by &man.fsck.8; during boot, and partitions
marked with a 0 dump number are not backed up by
&man.dump.8;. We do not want
<application>fsck</application> to check
<application>nullfs</application> mounts or
<application>dump</application> to back up the read-only
nullfs mounts of the jails. This is why they are marked
with <quote>0 0</quote> in the last two columns of
each <filename>fstab</filename> entry above.</para>
</note>
</step>
<step>
<para>Configure the jails in
<filename>/etc/rc.conf</filename>:</para>
<programlisting>jail_enable="YES"
jail_set_hostname_allow="NO"
jail_list="ns mail www"
jail_ns_hostname="ns.example.org"
jail_ns_ip="192.168.3.17"
jail_ns_rootdir="/usr/home/j/ns"
jail_ns_devfs_enable="YES"
jail_mail_hostname="mail.example.org"
jail_mail_ip="192.168.3.18"
jail_mail_rootdir="/usr/home/j/mail"
jail_mail_devfs_enable="YES"
jail_www_hostname="www.example.org"
jail_www_ip="62.123.43.14"
jail_www_rootdir="/usr/home/j/www"
jail_www_devfs_enable="YES"</programlisting>
<warning>
<para>The reason why the
<varname>jail_<replaceable>name</replaceable>_rootdir</varname>
variable is set to <filename
class="directory">/usr/home</filename> instead of
<filename class="directory">/home</filename> is that the
physical path of the <filename
class="directory">/home</filename> directory on a
default &os; installation is <filename
class="directory">/usr/home</filename>. The
<varname>jail_<replaceable>name</replaceable>_rootdir</varname>
variable must <emphasis>not</emphasis> be set to a path
which includes a symbolic link, otherwise the jails will
refuse to start. Use the &man.realpath.1; utility to
determine a value which should be set to this variable.
Please see the &os;-SA-07:01.jail Security Advisory for
more information.</para>
</warning>
</step>
<step>
<para>Create the required mount points for the read-only
file system of each jail:</para>
<screen>&prompt.root; <userinput>mkdir /home/j/ns /home/j/mail /home/j/www</userinput></screen>
</step>
<step>
<para>Install the read-write template into each jail. Note
the use of <filename
role="package">sysutils/cpdup</filename>, which helps to
ensure that a correct copy is done of each
directory:</para>
<!-- keramida: Why is cpdup required here? Doesn't cpio(1)
already include adequate functionality for performing this
job *and* have the advantage of being part of the base
system of FreeBSD? -->
<screen>&prompt.root; <userinput>mkdir /home/js</userinput>
&prompt.root; <userinput>cpdup /home/j/skel /home/js/ns</userinput>
&prompt.root; <userinput>cpdup /home/j/skel /home/js/mail</userinput>
&prompt.root; <userinput>cpdup /home/j/skel /home/js/www</userinput></screen>
</step>
<step>
<para>In this phase, the jails are built and prepared to
run. First, mount the required file systems for each
jail, and then start them using the
<filename>/etc/rc.d/jail</filename> script:</para>
<screen>&prompt.root; <userinput>mount -a</userinput>
&prompt.root; <userinput>/etc/rc.d/jail start</userinput></screen>
</step>
</procedure>
<para>The jails should be running now. To check if they have
started correctly, use the &man.jls.8; command. Its output
should be similar to the following:</para>
<screen>&prompt.root; <userinput>jls</userinput>
JID IP Address Hostname Path
3 192.168.3.17 ns.example.org /home/j/ns
2 192.168.3.18 mail.example.org /home/j/mail
1 62.123.43.14 www.example.org /home/j/www</screen>
<para>At this point, it should be possible to log onto each
jail, add new users or configure daemons. The
<literal>JID</literal> column indicates the jail
identification number of each running jail. Use the
following command in order to perform administrative tasks in
the jail whose <literal>JID</literal> is 3:</para>
<screen>&prompt.root; <userinput>jexec 3 tcsh</userinput></screen>
</sect3>
<sect3 id="jails-service-jails-upgrading">
<title>Upgrading</title>
<para>In time, there will be a need to upgrade the system to a
newer version of &os;, either because of a security issue, or
because new features have been implemented which are useful
for the existing jails. The design of this setup provides an
easy way to upgrade existing jails. Additionally, it
minimizes their downtime, as the jails will be brought down
only in the very last minute. Also, it provides a way to roll
back to the older versions should any problems occur.</para>
<procedure>
<step>
<para>The first step is to upgrade the host system in the
usual manner. Then create a new temporary read-only
template in <filename
class="directory">/home/j/mroot2</filename>.</para>
<screen>&prompt.root; <userinput>mkdir /home/j/mroot2</userinput>
&prompt.root; <userinput>cd /usr/src</userinput>
&prompt.root; <userinput>make installworld DESTDIR=/home/j/mroot2</userinput>
&prompt.root; <userinput>cd /home/j/mroot2</userinput>
&prompt.root; <userinput>cpdup /usr/src usr/src</userinput>
&prompt.root; <userinput>mkdir s</userinput></screen>
<para>The <maketarget>installworld</maketarget> run creates
a few unnecessary directories, which should be
removed:</para>
<screen>&prompt.root; <userinput>chflags -R 0 var</userinput>
&prompt.root; <userinput>rm -R etc var root usr/local tmp</userinput></screen>
</step>
<step>
<para>Recreate the read-write symlinks for the master file
system:</para>
<screen>&prompt.root; <userinput>ln -s s/etc etc</userinput>
&prompt.root; <userinput>ln -s s/root root</userinput>
&prompt.root; <userinput>ln -s s/home home</userinput>
&prompt.root; <userinput>ln -s ../s/usr-local usr/local</userinput>
&prompt.root; <userinput>ln -s ../s/usr-X11R6 usr/X11R6</userinput>
&prompt.root; <userinput>ln -s s/tmp tmp</userinput>
&prompt.root; <userinput>ln -s s/var var</userinput></screen>
</step>
<step>
<para>The right time to stop the jails is now:</para>
<screen>&prompt.root; <userinput>/etc/rc.d/jail stop</userinput></screen>
</step>
<step>
<para>Unmount the original file systems:</para>
<!-- keramida: Shouldn't we suggest a short script-based
loop here, instead of tediously copying the same commands
multiple times? -->
<screen>&prompt.root; <userinput>umount /home/j/ns/s</userinput>
&prompt.root; <userinput>umount /home/j/ns</userinput>
&prompt.root; <userinput>umount /home/j/mail/s</userinput>
&prompt.root; <userinput>umount /home/j/mail</userinput>
&prompt.root; <userinput>umount /home/j/www/s</userinput>
&prompt.root; <userinput>umount /home/j/www</userinput></screen>
<note>
<para>The read-write systems are attached to the read-only
system (<filename class="directory">/s</filename>) and
must be unmounted first.</para>
</note>
</step>
<step>
<para>Move the old read-only file system and replace it with
the new one. This will serve as a backup and archive of the
old read-only file system should something go wrong. The
naming convention used here corresponds to when a new
read-only file system has been created. Move the original
&os; Ports Collection over to the new file system to save
some space and inodes:</para>
<screen>&prompt.root; <userinput>cd /home/j</userinput>
&prompt.root; <userinput>mv mroot mroot.20060601</userinput>
&prompt.root; <userinput>mv mroot2 mroot</userinput>
&prompt.root; <userinput>mv mroot.20060601/usr/ports mroot/usr</userinput></screen>
</step>
<step>
<para>At this point the new read-only template is ready, so
the only remaining task is to remount the file systems and
start the jails:</para>
<screen>&prompt.root; <userinput>mount -a</userinput>
&prompt.root; <userinput>/etc/rc.d/jail start</userinput></screen>
</step>
</procedure>
<para>Use &man.jls.8; to check if the jails started correctly.
Do not forget to run mergemaster in each jail. The
configuration files will need to be updated as well as the
rc.d scripts.</para>
</sect3>
</sect2>
</sect1>
</chapter>
|