1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
|
<?xml version="1.0" encoding="ISO8859-1" standalone="no"?>
<!DOCTYPE article PUBLIC "-//FreeBSD//DTD DocBook XML V4.2-Based Extension//EN"
"../../../share/sgml/freebsd42.dtd" [
<!ENTITY % entities PUBLIC "-//FreeBSD//ENTITIES DocBook FreeBSD Entity Set//EN" "../../share/sgml/entities.ent">
%entities;
]>
<!-- Copyright (c) 2001 The FreeBSD Documentation Project
Redistribution and use in source (SGML DocBook) and 'compiled' forms
(SGML, HTML, PDF, PostScript, RTF and so forth) with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code (SGML DocBook) must retain the above
copyright notice, this list of conditions and the following
disclaimer as the first lines of this file unmodified.
2. Redistributions in compiled form (transformed to other DTDs,
converted to PDF, PostScript, RTF and other formats) must reproduce
the above copyright notice, this list of conditions and the
following disclaimer in the documentation and/or other materials
provided with the distribution.
THIS DOCUMENTATION IS PROVIDED BY THE FREEBSD DOCUMENTATION PROJECT "AS
IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL NIK CLAYTON BE LIABLE FOR ANY
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
ANY WAY OUT OF THE USE OF THIS DOCUMENTATION, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.
$FreeBSD$
-->
<article lang='en'>
<articleinfo>
<title>&os; and Solid State Devices</title>
<authorgroup>
<author>
<firstname>John</firstname>
<surname>Kozubik</surname>
<affiliation>
<address><email>john@kozubik.com</email></address>
</affiliation>
</author>
</authorgroup>
<copyright>
<year>2001</year>
<year>2009</year>
<holder>The FreeBSD Documentation Project</holder>
</copyright>
<legalnotice id="trademarks" role="trademarks">
&tm-attrib.freebsd;
&tm-attrib.general;
</legalnotice>
&legalnotice;
<pubdate>$FreeBSD$</pubdate>
<releaseinfo>$FreeBSD$</releaseinfo>
<abstract>
<para>This article covers the use of solid state disk devices in &os;
to create embedded systems.</para>
<para>Embedded systems have the advantage of increased stability due to
the lack of integral moving parts (hard drives). Account must be
taken, however, for the generally low disk space available in the
system and the durability of the storage medium.</para>
<para>Specific topics to be covered include the types and attributes of
solid state media suitable for disk use in &os;, kernel options
that are of interest in such an environment, the
<filename>rc.initdiskless</filename> mechanisms that automate the
initialization of such systems and the need for read-only filesystems,
and building filesystems from scratch. The article will conclude
with some general strategies for small and read-only &os;
environments.</para>
</abstract>
</articleinfo>
<sect1 id="intro">
<title>Solid State Disk Devices</title>
<para>The scope of this article will be limited to solid state disk
devices made from flash memory. Flash memory is a solid state memory
(no moving parts) that is non-volatile (the memory maintains data even
after all power sources have been disconnected). Flash memory can
withstand tremendous physical shock and is reasonably fast (the flash
memory solutions covered in this article are slightly slower than a EIDE
hard disk for write operations, and much faster for read operations).
One very important aspect of flash memory, the ramifications of which
will be discussed later in this article, is that each sector has a
limited rewrite capacity. You can only write, erase, and write again to
a sector of flash memory a certain number of times before the sector
becomes permanently unusable. Although many flash memory products
automatically map bad blocks, and although some even distribute write
operations evenly throughout the unit, the fact remains that there
exists a limit to the amount of writing that can be done to the device.
Competitive units have between 1,000,000 and 10,000,000 writes per
sector in their specification. This figure varies due to the
temperature of the environment.</para>
<para>Specifically, we will be discussing ATA compatible compact-flash
units, which are quite popular as storage media for digital
cameras. Of particular interest is the fact that they pin out directly
to the IDE bus and are compatible with the ATA command set. Therefore,
with a very simple and low-cost adaptor, these devices can be attached
directly to an IDE bus in a computer. Once implemented in this manner,
operating systems such as &os; see the device as a normal hard disk
(albeit small).</para>
<para>Other solid state disk solutions do exist, but their expense,
obscurity, and relative unease of use places them beyond the scope of
this article.</para>
</sect1>
<sect1 id="kernel">
<title>Kernel Options</title>
<para>A few kernel options are of specific interest to those creating
an embedded &os; system.</para>
<para>All embedded &os; systems that use flash memory as system
disk will be interested in memory disks and memory filesystems. Because
of the limited number of writes that can be done to flash memory, the
disk and the filesystems on the disk will most likely be mounted
read-only. In this environment, filesystems such as
<filename>/tmp</filename> and <filename>/var</filename> are mounted as
memory filesystems to allow the system to create logs and update
counters and temporary files. Memory filesystems are a critical
component to a successful solid state &os; implementation.</para>
<para>You should make sure the following lines exist in your kernel
configuration file:</para>
<programlisting>options MFS # Memory Filesystem
options MD_ROOT # md device usable as a potential root device
pseudo-device md # memory disk</programlisting>
</sect1>
<sect1 id="ro-fs">
<title>The <literal>rc</literal> Subsystem and Read-Only Filesystems</title>
<para>The post-boot initialization of an embedded &os; system is
controlled by <filename>/etc/rc.initdiskless</filename>.</para>
<para><filename>/etc/rc.d/var</filename> mounts <filename>/var</filename>
as a memory filesystem, makes a configurable list of directories in
<filename>/var</filename> with the &man.mkdir.1; command, and
changes modes on some of those directories. In the execution of
<filename>/etc/rc.d/var</filename>, one other
<filename>rc.conf</filename> variable comes into play –
<literal>varsize</literal>. The <filename>/etc/rc.d/var</filename>
file creates a <filename>/var</filename> partition based on the value of
this variable in <filename>rc.conf</filename>:</para>
<programlisting>varsize=8192</programlisting>
<para>Remember that this value is in sectors by default.</para>
<para>The fact that <filename>/var</filename>
is a read-write filesystem is an important
distinction, as the <filename>/</filename> partition (and any other
partitions you may have on your flash media) should be mounted
read-only. Remember that in <xref linkend="intro"/> we detailed the
limitations of flash memory - specifically the limited write capability.
The importance of not mounting filesystems on flash media read-write,
and the importance of not using a swap file, cannot be overstated. A
swap file on a busy system can burn through a piece of flash media in
less than one year. Heavy logging or temporary file creation and
destruction can do the same. Therefore, in addition to removing the
<literal>swap</literal> entry from your
<filename>/etc/fstab</filename> file, you should also change the Options
field for each filesystem to <literal>ro</literal> as follows:</para>
<programlisting># Device Mountpoint FStype Options Dump Pass#
/dev/ad0s1a / ufs ro 1 1</programlisting>
<para>A few applications in the average system will immediately begin to
fail as a result of this change. For instance, cron
will not run properly as a result of missing cron tabs in the
<filename>/var</filename> created by
<filename>/etc/rc.d/var</filename>, and syslog and dhcp will
encounter problems as well as a result of the read-only filesystem and
missing items in the <filename>/var</filename> that
<filename>/etc/rc.d/var</filename> has created. These are only
temporary problems though, and are addressed, along with solutions to
the execution of other common software packages in
<xref linkend="strategies"/>.</para>
<para>An important thing to remember is that a filesystem that was mounted
read-only with <filename>/etc/fstab</filename> can be made read-write at
any time by issuing the command:</para>
<screen>&prompt.root; <userinput>/sbin/mount -uw <replaceable>partition</replaceable></userinput></screen>
<para>and can be toggled back to read-only with the command:</para>
<screen>&prompt.root; <userinput>/sbin/mount -ur <replaceable>partition</replaceable></userinput></screen>
</sect1>
<sect1>
<title>Building a File System From Scratch</title>
<para>Because ATA compatible compact-flash cards are seen by &os; as
normal IDE hard drives, you could
theoretically install &os; from the network using the kern and
mfsroot floppies or from a CD.</para>
<para>However, even a small installation of &os; using normal
installation procedures can produce a system in size of greater than 200
megabytes. Because most people will be using smaller flash memory
devices (128 megabytes is considered fairly large - 32 or even 16
megabytes is common) an installation using normal mechanisms is not
possible—there is simply not enough disk space for even the
smallest of conventional installations.</para>
<para>The easiest way to overcome this space limitation is to install
&os; using conventional means to a normal hard disk. After the
installation is complete, pare down the operating system to a size that
will fit onto your flash media, then tar the entire filesystem. The
following steps will guide you through the process of preparing a piece
of flash memory for your tarred filesystem. Remember, because a normal
installation is not being performed, operations such as partitioning,
labeling, file-system creation, etc. need to be performed by hand. In
addition to the kern and mfsroot floppy disks, you will also need to use
the fixit floppy.</para>
<procedure>
<step>
<title>Partitioning your flash media device</title>
<para>After booting with the kern and mfsroot floppies, choose
<literal>custom</literal> from the installation menu. In the custom
installation menu, choose <literal>partition</literal>. In the
partition menu, you should delete all existing partitions using the
<keycap>d</keycap> key. After deleting all existing partitions,
create a partition using the <keycap>c</keycap> key and accept the
default value for the size of the partition. When asked for the
type of the partition, make sure the value is set to
<literal>165</literal>. Now write this partition table to the disk
by pressing the <keycap>w</keycap> key (this is a hidden option on
this screen). If you are using an ATA compatible compact
flash card, you should choose the &os; Boot Manager. Now press
the <keycap>q</keycap> key to quit the partition menu. You will be
shown the boot manager menu once more - repeat the choice you made
earlier.</para>
</step>
<step>
<title>Creating filesystems on your flash memory device</title>
<para>Exit the custom installation menu, and from the main
installation menu choose the <literal>fixit</literal> option. After
entering the fixit environment, enter the following command:</para>
<screen>&prompt.root; <userinput>disklabel -e /dev/ad0c</userinput></screen>
<para>At this point you will have entered the vi editor under the
auspices of the disklabel command. Next, you need to add
an <literal>a:</literal> line at the end of the file. This
<literal>a:</literal> line should look like:</para>
<programlisting>a: <replaceable>123456</replaceable> 0 4.2BSD 0 0</programlisting>
<para>Where <replaceable>123456</replaceable> is a number that is
exactly the same as the number in the existing <literal>c:</literal>
entry for size. Basically you are duplicating the existing
<literal>c:</literal> line as an <literal>a:</literal> line, making
sure that fstype is <literal>4.2BSD</literal>. Save the file and
exit.</para>
<screen>&prompt.root; <userinput>disklabel -B -r /dev/ad0c</userinput>
&prompt.root; <userinput>newfs /dev/ad0a</userinput></screen>
</step>
<step>
<title>Placing your filesystem on the flash media</title>
<para>Mount the newly prepared flash media:</para>
<screen>&prompt.root; <userinput>mount /dev/ad0a /flash</userinput></screen>
<para>Bring this machine up on the network so we may transfer our tar
file and explode it onto our flash media filesystem. One example of
how to do this is:</para>
<screen>&prompt.root; <userinput>ifconfig xl0 192.168.0.10 netmask 255.255.255.0</userinput>
&prompt.root; <userinput>route add default 192.168.0.1</userinput></screen>
<para>Now that the machine is on the network, transfer your tar file.
You may be faced with a bit of a dilemma at this point - if your
flash memory part is 128 megabytes, for instance, and your tar file
is larger than 64 megabytes, you cannot have your tar file on the
flash media at the same time as you explode it - you will run out of
space. One solution to this problem, if you are using FTP, is to
untar the file while it is transferred over FTP. If you perform
your transfer in this manner, you will never have the tar file and
the tar contents on your disk at the same time:</para>
<screen><prompt>ftp></prompt> <userinput>get tarfile.tar "| tar xvf -"</userinput></screen>
<para>If your tarfile is gzipped, you can accomplish this as
well:</para>
<screen><prompt>ftp></prompt> <userinput>get tarfile.tar "| zcat | tar xvf -"</userinput></screen>
<para>After the contents of your tarred filesystem are on your flash
memory filesystem, you can unmount the flash memory and
reboot:</para>
<screen>&prompt.root; <userinput>cd /</userinput>
&prompt.root; <userinput>umount /flash</userinput>
&prompt.root; <userinput>exit</userinput></screen>
<para>Assuming that you configured your filesystem correctly when it
was built on the normal hard disk (with your filesystems mounted
read-only, and with the necessary options compiled into the kernel)
you should now be successfully booting your &os; embedded
system.</para>
</step>
</procedure>
</sect1>
<sect1 id="strategies">
<title>System Strategies for Small and Read Only Environments</title>
<para>In <xref linkend="ro-fs"/>, it was pointed out that the
<filename>/var</filename> filesystem constructed by
<filename>/etc/rc.d/var</filename> and the presence of a read-only
root filesystem causes problems with many common software packages used
with &os;. In this article, suggestions for successfully running
cron, syslog, ports installations, and the Apache web server will be
provided.</para>
<sect2>
<title>cron</title>
<para>Upon boot, <filename class="directory">/var</filename> gets
populated by <filename>/etc/rc.d/var</filename> using the list from
<filename>/etc/mtree/BSD.var.dist</filename>, so the <filename
class="directory">cron</filename>, <filename
class="directory">cron/tabs</filename>, <filename
class="directory">at</filename>, and a few other standard
directories get created.</para>
<para>However, this does not solve the problem of maintaining cron
tabs across reboots. When the system reboots, the
<filename>/var</filename> filesystem that is in memory will disappear
and any cron tabs you may have had in it will also disappear.
Therefore, one solution would be to create cron tabs for the users
that need them, mount your <filename>/</filename> filesystem as
read-write and copy those cron tabs to somewhere safe, like
<filename>/etc/tabs</filename>, then add a line to the end of
<filename>/etc/rc.initdiskless</filename> that copies those crontabs into
<filename>/var/cron/tabs</filename> after that directory has been
created during system initialization. You may also need to add a line
that changes modes and permissions on the directories you create and
the files you copy with <filename>/etc/rc.initdiskless</filename>.</para>
</sect2>
<sect2>
<title>syslog</title>
<para><filename>syslog.conf</filename> specifies the locations of
certain log files that exist in <filename>/var/log</filename>. These
files are not created by <filename>/etc/rc.d/var</filename> upon
system initialization. Therefore, somewhere in
<filename>/etc/rc.d/var</filename>, after the section that creates
the directories in <filename>/var</filename>, you will need to add
something like this:</para>
<screen>&prompt.root; <userinput>touch /var/log/security /var/log/maillog /var/log/cron /var/log/messages</userinput>
&prompt.root; <userinput>chmod 0644 /var/log/*</userinput></screen>
</sect2>
<sect2>
<title>Ports Installation</title>
<para>Before discussing the changes necessary to successfully use the
ports tree, a reminder is necessary regarding the read-only nature of
your filesystems on the flash media. Since they are read-only, you
will need to temporarily mount them read-write using the mount syntax
shown in <xref linkend="ro-fs"/>. You should always remount those
filesystems read-only when you are done with any maintenance -
unnecessary writes to the flash media could considerably shorten its
lifespan.</para>
<para>To make it possible to enter a ports directory and successfully
run <command>make</command> <maketarget>install</maketarget>,
we must create a packages directory on
a non-memory filesystem that will keep track of our packages across
reboots. Because it is necessary to mount your filesystems as
read-write for the installation of a package anyway, it is sensible to
assume that an area on the flash media can also be used for package
information to be written to.</para>
<para>First, create a package database directory. This is normally in
<filename>/var/db/pkg</filename>, but we cannot place it there as it
will disappear every time the system is booted.</para>
<screen>&prompt.root; <userinput>mkdir /etc/pkg</userinput></screen>
<para>Now, add a line to <filename>/etc/rc.d/var</filename> that
links the <filename>/etc/pkg</filename> directory to
<filename>/var/db/pkg</filename>. An example:</para>
<screen>&prompt.root; <userinput>ln -s /etc/pkg /var/db/pkg</userinput></screen>
<para>Now, any time that you mount your filesystems as read-write and
install a package, the <command>make</command> <maketarget>install</maketarget> will work,
and package information
will be written successfully to <filename>/etc/pkg</filename> (because
the filesystem will, at that time, be mounted read-write) which will
always be available to the operating system as
<filename>/var/db/pkg</filename>.</para>
</sect2>
<sect2>
<title>Apache Web Server</title>
<note>
<para>The steps in this section are only necessary if Apache is
set up to write its pid or log information outside of
<filename class="directory">/var</filename>. By default,
Apache keeps its pid file in <filename
class="directory">/var/run/httpd.pid</filename> and its log
files in <filename class="directory">/var/log</filename>.</para>
</note>
<para>It is now assumed that Apache keeps its log files in a
directory <filename
class="directory"><replaceable>apache_log_dir</replaceable></filename>
outside of <filename class="directory">/var</filename>.
When this directory lives on a read-only filesystem, Apache will
not be able to save any log files, and may have problems working.
If so, it is necessary to add a new directory to the
list of directories in <filename>/etc/rc.d/var</filename> to
create in <filename>/var</filename>, and to link
<filename class="directory"><replaceable>apache_log_dir</replaceable></filename> to
<filename>/var/log/apache</filename>. It is also necessary to set
permissions and ownership on this new directory.</para>
<para>First, add the directory <literal>log/apache</literal> to the list
of directories to be created in
<filename>/etc/rc.d/var</filename>.</para>
<para>Second, add these commands to
<filename>/etc/rc.d/var</filename> after the directory creation
section:</para>
<screen>&prompt.root; <userinput>chmod 0774 /var/log/apache</userinput>
&prompt.root; <userinput>chown nobody:nobody /var/log/apache</userinput></screen>
<para>Finally, remove the existing
<filename class="directory"><replaceable>apache_log_dir</replaceable></filename>
directory, and replace it with a link:</para>
<screen>&prompt.root; <userinput>rm -rf <filename class="directory"><replaceable>apache_log_dir</replaceable></filename></userinput>
&prompt.root; <userinput>ln -s /var/log/apache <filename class="directory"><replaceable>apache_log_dir</replaceable></filename></userinput></screen>
</sect2>
</sect1>
</article>
|