VirtualizationMurrayStokelyContributed by AllanJudebhyve section by BenedictReuschlingXen section by SynopsisVirtualization software allows multiple operating systems to
run simultaneously on the same computer. Such software systems
for PCs often involve a host operating system
which runs the virtualization software and supports any number
of guest operating systems.After reading this chapter, you will know:The difference between a host operating system and a
guest operating system.How to install &os; on an &intel;-based &apple;
&mac; computer.How to install &os; on µsoft.windows; with
Virtual PC.How to install &os; as a guest in
bhyve.How to tune a &os; system for best performance under
virtualization.Before reading this chapter, you should:Understand the basics of &unix;
and &os;.Know how to install
&os;.Know how to set up a
network connection.Know how to install additional
third-party software.&os; as a Guest on Parallels for
&macos; XParallels Desktop for &mac; is
a commercial software product available for &intel; based
&apple; &mac; computers running &macos; 10.4.6 or higher. &os;
is a fully supported guest operating system. Once
Parallels has been installed on
&macos; X, the user must configure a virtual machine and then
install the desired guest operating system.Installing &os; on Parallels/&macos; XThe first step in installing &os; on
Parallels is to create a new
virtual machine for installing &os;. Select
&os; as the
Guest OS Type when prompted:Choose a reasonable amount of disk and memory
depending on the plans for this virtual &os; instance.
4GB of disk space and 512MB of RAM work well for most uses
of &os; under Parallels:Select the type of networking and a network
interface:Save and finish the configuration:After the &os; virtual machine has been created, &os;
can be installed on it. This is best done with an official
&os; CD/DVD or with an
ISO image downloaded from an official
FTP site. Copy the appropriate
ISO image to the local &mac; filesystem or
insert a CD/DVD in the
&mac;'s CD-ROM drive. Click on the disc
icon in the bottom right corner of the &os;
Parallels window. This will bring
up a window that can be used to associate the
CD-ROM drive in the virtual machine with
the ISO file on disk or with the real
CD-ROM drive.Once this association with the CD-ROM
source has been made, reboot the &os; virtual machine by
clicking the reboot icon.
Parallels will reboot with a
special BIOS that first checks if there is
a CD-ROM.In this case it will find the &os; installation media and
begin a normal &os; installation. Perform the installation,
but do not attempt to configure
&xorg; at this time.When the installation is finished, reboot into the newly
installed &os; virtual machine.Configuring &os; on
ParallelsAfter &os; has been successfully installed on &macos; X
with Parallels, there are a number
of configuration steps that can be taken to optimize the
system for virtualized operation.Set Boot Loader VariablesThe most important step is to reduce the
tunable to reduce the CPU
utilization of &os; under the
Parallels environment. This is
accomplished by adding the following line to
/boot/loader.conf:kern.hz=100Without this setting, an idle &os;
Parallels guest will use
roughly 15% of the CPU of a single processor &imac;.
After this change the usage will be closer to 5%.Create a New Kernel Configuration FileAll of the SCSI, FireWire, and USB device drivers
can be removed from a custom kernel configuration file.
Parallels provides a virtual
network adapter used by the &man.ed.4; driver, so all
network devices except for &man.ed.4; and &man.miibus.4;
can be removed from the kernel.Configure NetworkingThe most basic networking setup uses DHCP to connect
the virtual machine to the same local area network as the
host &mac;. This can be accomplished by adding
ifconfig_ed0="DHCP" to
/etc/rc.conf. More advanced
networking setups are described in
.&os; as a Guest on Virtual PC
for &windows;Virtual PC for &windows; is a
µsoft; software product available for free download. See
this website for the system
requirements. Once
Virtual PC has been installed on
µsoft.windows;, the user can configure a virtual machine
and then install the desired guest operating system.Installing &os; on
Virtual PCThe first step in installing &os; on
Virtual PC is to create a new
virtual machine for installing &os;. Select
Create a virtual machine when
prompted:Select Other as the
Operating system when
prompted:Then, choose a reasonable amount of disk and memory
depending on the plans for this virtual &os; instance.
4GB of disk space and 512MB of RAM work well for most uses
of &os; under Virtual PC:Save and finish the configuration:Select the &os; virtual machine and click
Settings, then set the type of networking
and a network interface:After the &os; virtual machine has been created, &os; can
be installed on it. This is best done with an official &os;
CD/DVD or with an
ISO image downloaded from an official
FTP site. Copy the appropriate
ISO image to the local &windows; filesystem
or insert a CD/DVD in
the CD drive, then double click on the &os;
virtual machine to boot. Then, click CD
and choose Capture ISO Image... on the
Virtual PC window. This will bring
up a window where the CD-ROM drive in the
virtual machine can be associated with an
ISO file on disk or with the real
CD-ROM drive.Once this association with the CD-ROM
source has been made, reboot the &os; virtual machine by
clicking Action and
Reset.
Virtual PC will reboot with a
special BIOS that first checks for a
CD-ROM.In this case it will find the &os; installation media
and begin a normal &os; installation. Continue with the
installation, but do not attempt to configure
&xorg; at this time.When the installation is finished, remember to eject the
CD/DVD or release the
ISO image. Finally, reboot into the newly
installed &os; virtual machine.Configuring &os; on Virtual
PCAfter &os; has been successfully installed on
µsoft.windows; with
Virtual PC, there are a number of
configuration steps that can be taken to optimize the system
for virtualized operation.Set Boot Loader VariablesThe most important step is to reduce the
tunable to reduce the CPU
utilization of &os; under the
Virtual PC environment. This
is accomplished by adding the following line to
/boot/loader.conf:kern.hz=100Without this setting, an idle &os;
Virtual PC guest OS will
use roughly 40% of the CPU of a single processor
computer. After this change, the usage will be
closer to 3%.Create a New Kernel Configuration FileAll of the SCSI, FireWire, and USB device drivers can
be removed from a custom kernel configuration file.
Virtual PC provides a virtual
network adapter used by the &man.de.4; driver, so all
network devices except for &man.de.4; and &man.miibus.4;
can be removed from the kernel.Configure NetworkingThe most basic networking setup uses DHCP to connect
the virtual machine to the same local area network as the
µsoft.windows; host. This can be accomplished by
adding ifconfig_de0="DHCP" to
/etc/rc.conf. More advanced
networking setups are described in
.&os; as a Guest on VMware Fusion
for &macos;VMware Fusion for &mac; is a
commercial software product available for &intel; based &apple;
&mac; computers running &macos; 10.4.9 or higher. &os; is a
fully supported guest operating system. Once
VMware Fusion has been installed on
&macos; X, the user can configure a virtual machine and then
install the desired guest operating system.Installing &os; on
VMware FusionThe first step is to start
VMware Fusion which will load the
Virtual Machine Library. Click New
to create the virtual machine:This will load the New Virtual Machine Assistant. Click
Continue to proceed:Select Other as the
Operating System and either
&os; or
&os; 64-bit, as the
Version when prompted:Choose the name of the virtual machine and the directory
where it should be saved:Choose the size of the Virtual Hard Disk for the virtual
machine:Choose the method to install the virtual machine, either
from an ISO image or from a
CD/DVD:Click Finish and the virtual
machine will boot:Install &os; as usual:Once the install is complete, the settings of the virtual
machine can be modified, such as memory usage:The System Hardware settings of the virtual machine
cannot be modified while the virtual machine is
running.The number of CPUs the virtual machine will have access
to:The status of the CD-ROM device.
Normally the
CD/DVD/ISO
is disconnected from the virtual machine when it is no longer
needed.The last thing to change is how the virtual machine will
connect to the network. To allow connections to the virtual
machine from other machines besides the host, choose
Connect directly to the physical network
(Bridged). Otherwise,
Share the host's internet connection
(NAT) is preferred so that the virtual machine
can have access to the Internet, but the network cannot access
the virtual machine.After modifying the settings, boot the newly installed
&os; virtual machine.Configuring &os; on VMware
FusionAfter &os; has been successfully installed on &macos; X
with VMware Fusion, there are a
number of configuration steps that can be taken to optimize
the system for virtualized operation.Set Boot Loader VariablesThe most important step is to reduce the
tunable to reduce the CPU
utilization of &os; under the
VMware Fusion environment.
This is accomplished by adding the following line to
/boot/loader.conf:kern.hz=100Without this setting, an idle &os;
VMware Fusion guest will use
roughly 15% of the CPU of a single processor &imac;.
After this change, the usage will be closer to 5%.Create a New Kernel Configuration FileAll of the FireWire, and USB device drivers can be
removed from a custom kernel configuration file.
VMware Fusion provides a
virtual network adapter used by the &man.em.4; driver, so
all network devices except for &man.em.4; can be removed
from the kernel.Configure NetworkingThe most basic networking setup uses DHCP to connect
the virtual machine to the same local area network as the
host &mac;. This can be accomplished by adding
ifconfig_em0="DHCP" to
/etc/rc.conf. More advanced
networking setups are described in
.&os; as a Guest on &virtualbox;&os; works well as a guest in
&virtualbox;. The virtualization
software is available for most common operating systems,
including &os; itself.The &virtualbox; guest additions
provide support for:Clipboard sharing.Mouse pointer integration.Host time synchronization.Window scaling.Seamless mode.These commands are run in the &os; guest.First, install the
emulators/virtualbox-ose-additions package
or port in the &os; guest. This will install the port:&prompt.root; cd /usr/ports/emulators/virtualbox-ose-additions && make install cleanAdd these lines to /etc/rc.conf:vboxguest_enable="YES"
vboxservice_enable="YES"If &man.ntpd.8; or &man.ntpdate.8; is used, disable host
time synchronization:vboxservice_flags="--disable-timesync"Xorg will automatically recognize
the vboxvideo driver. It can also be
manually entered in
/etc/X11/xorg.conf:Section "Device"
Identifier "Card0"
Driver "vboxvideo"
VendorName "InnoTek Systemberatung GmbH"
BoardName "VirtualBox Graphics Adapter"
EndSectionTo use the vboxmouse driver, adjust the
mouse section in /etc/X11/xorg.conf:Section "InputDevice"
Identifier "Mouse0"
Driver "vboxmouse"
EndSectionHAL users should create the following
/usr/local/etc/hal/fdi/policy/90-vboxguest.fdi
or copy it from
/usr/local/share/hal/fdi/policy/10osvendor/90-vboxguest.fdi:<?xml version="1.0" encoding="utf-8"?>
<!--
# Sun VirtualBox
# Hal driver description for the vboxmouse driver
# $Id: chapter.xml,v 1.33 2012-03-17 04:53:52 eadler Exp $
Copyright (C) 2008-2009 Sun Microsystems, Inc.
This file is part of VirtualBox Open Source Edition (OSE, as
available from http://www.virtualbox.org. This file is free software;
you can redistribute it and/or modify it under the terms of the GNU
General Public License (GPL) as published by the Free Software
Foundation, in version 2 as it comes in the "COPYING" file of the
VirtualBox OSE distribution. VirtualBox OSE is distributed in the
hope that it will be useful, but WITHOUT ANY WARRANTY of any kind.
Please contact Sun Microsystems, Inc., 4150 Network Circle, Santa
Clara, CA 95054 USA or visit http://www.sun.com if you need
additional information or have any questions.
-->
<deviceinfo version="0.2">
<device>
<match key="info.subsystem" string="pci">
<match key="info.product" string="VirtualBox guest Service">
<append key="info.capabilities" type="strlist">input</append>
<append key="info.capabilities" type="strlist">input.mouse</append>
<merge key="input.x11_driver" type="string">vboxmouse</merge>
<merge key="input.device" type="string">/dev/vboxguest</merge>
</match>
</match>
</device>
</deviceinfo>&os; as a Host with
VirtualBox&virtualbox; is an actively
developed, complete virtualization package, that is available
for most operating systems including &windows;, &macos;, &linux;
and &os;. It is equally capable of running &windows; or
&unix;-like guests. It is released as open source software, but
with closed-source components available in a separate extension
pack. These components include support for USB 2.0 devices.
More information may be found on the Downloads
page of the &virtualbox;
wiki. Currently, these extensions are not available
for &os;.Installing &virtualbox;&virtualbox; is available as a
&os; package or port in
emulators/virtualbox-ose. The port can be
installed using these commands:&prompt.root; cd /usr/ports/emulators/virtualbox-ose
&prompt.root; make install cleanOne useful option in the port's configuration menu is the
GuestAdditions suite of programs. These
provide a number of useful features in guest operating
systems, like mouse pointer integration (allowing the mouse to
be shared between host and guest without the need to press a
special keyboard shortcut to switch) and faster video
rendering, especially in &windows; guests. The guest
additions are available in the Devices
menu, after the installation of the guest is finished.A few configuration changes are needed before
&virtualbox; is started for the
first time. The port installs a kernel module in
/boot/modules which
must be loaded into the running kernel:&prompt.root; kldload vboxdrvTo ensure the module is always loaded after a reboot,
add this line to
/boot/loader.conf:vboxdrv_load="YES"To use the kernel modules that allow bridged or host-only
networking, add this line to
/etc/rc.conf and reboot the
computer:vboxnet_enable="YES"The vboxusers
group is created during installation of
&virtualbox;. All users that need
access to &virtualbox; will have to
be added as members of this group. pw can
be used to add new members:&prompt.root; pw groupmod vboxusers -m yourusernameThe default permissions for
/dev/vboxnetctl are restrictive and need
to be changed for bridged networking:&prompt.root; chown root:vboxusers /dev/vboxnetctl
&prompt.root; chmod 0660 /dev/vboxnetctlTo make this permissions change permanent, add these
lines to /etc/devfs.conf:own vboxnetctl root:vboxusers
perm vboxnetctl 0660To launch &virtualbox;,
type from a &xorg; session:&prompt.user; VirtualBoxFor more information on configuring and using
&virtualbox;, refer to the
official
website. For &os;-specific information and
troubleshooting instructions, refer to the relevant
page in the &os; wiki.&virtualbox; USB Support&virtualbox; can be configured
to pass USB devices through to the guest
operating system. The host controller of the OSE version is
limited to emulating USB 1.1 devices until
the extension pack supporting USB 2.0 and 3.0
devices becomes available on &os;.For &virtualbox; to be
aware of USB devices attached to the
machine, the user needs to be a member of the operator group.&prompt.root; pw groupmod operator -m yourusernameRestart the login session and
&virtualbox; for these changes to take
effect, and create USB filters as
necessary.&virtualbox; Host
DVD/CD AccessAccess to the host
DVD/CD drives from
guests is achieved through the sharing of the physical drives.
Within &virtualbox;, this is set up from the Storage window in
the Settings of the virtual machine. If needed, create an
empty IDE
CD/DVD device first.
Then choose the Host Drive from the popup menu for the virtual
CD/DVD drive selection.
A checkbox labeled Passthrough will appear.
This allows the virtual machine to use the hardware directly.
For example, audio CDs or the burner will
only function if this option is selected.HAL needs to run for
&virtualbox;
DVD/CD functions to
work, so enable it in /etc/rc.conf and
start it if it is not already running:hald_enable="YES"&prompt.root; service hald startIn order for users to be able to use
&virtualbox;
DVD/CD functions, they
need access to /dev/xpt0,
/dev/cdN, and
/dev/passN.
This is usually achieved by making the user a member of
operator.
Permissions to these devices have to be corrected by adding
these lines to /etc/devfs.conf:perm cd* 0660
perm xpt0 0660
perm pass* 0660&prompt.root; service devfs restart&os; as a Host with
bhyveThe bhyve
BSD-licensed hypervisor became part of the
base system with &os; 10.0-RELEASE. This hypervisor supports a
number of guests, including &os;, OpenBSD, and many &linux;
distributions. By default, bhyve
provides access to serial console and does not emulate a
graphical console. Virtualization offload features of newer
CPUs are used to avoid the legacy methods of
translating instructions and manually managing memory
mappings.The bhyve design requires a
processor that supports &intel; Extended Page Tables
(EPT) or &amd; Rapid Virtualization Indexing
(RVI) or Nested Page Tables
(NPT). Hosting &linux; guests or &os; guests
with more than one vCPU requires
VMX unrestricted mode support
(UG). Most newer processors, specifically
the &intel; &core; i3/i5/i7 and &intel; &xeon;
E3/E5/E7, support these features. UG support
was introduced with Intel's Westmere micro-architecture. For a
complete list of &intel; processors that support
EPT, refer to .
RVI is found on the third generation and
later of the &amd.opteron; (Barcelona) processors. The easiest
way to tell if a processor supports
bhyve is to run
dmesg or look in
/var/run/dmesg.boot for the
POPCNT processor feature flag on the
Features2 line for &amd; processors or
EPT and UG on the
VT-x line for &intel; processors.Preparing the HostThe first step to creating a virtual machine in
bhyve is configuring the host
system. First, load the bhyve
kernel module:&prompt.root; kldload vmmThen, create a tap interface for the
network device in the virtual machine to attach to. In order
for the network device to participate in the network, also
create a bridge interface containing the
tap interface and the physical interface
as members. In this example, the physical interface is
igb0:&prompt.root; ifconfig tap0 create
&prompt.root; sysctl net.link.tap.up_on_open=1
net.link.tap.up_on_open: 0 -> 1
&prompt.root; ifconfig bridge0 create
&prompt.root; ifconfig bridge0 addm igb0 addm tap0
&prompt.root; ifconfig bridge0 upCreating a FreeBSD GuestCreate a file to use as the virtual disk for the guest
machine. Specify the size and name of the virtual
disk:&prompt.root; truncate -s 16Gguest.imgDownload an installation image of &os; to install:&prompt.root; fetch ftp://ftp.freebsd.org/pub/FreeBSD/releases/ISO-IMAGES/10.3/FreeBSD-10.3-RELEASE-amd64-bootonly.iso
FreeBSD-10.3-RELEASE-amd64-bootonly.iso 100% of 230 MB 570 kBps 06m17s&os; comes with an example script for running a virtual
machine in bhyve. The script will
start the virtual machine and run it in a loop, so it will
automatically restart if it crashes. The script takes a
number of options to control the configuration of the machine:
controls the number of virtual CPUs,
limits the amount of memory available to
the guest, defines which
tap device to use,
indicates which disk image to use, tells
bhyve to boot from the
CD image instead of the disk, and
defines which CD image
to use. The last parameter is the name of the virtual
machine, used to track the running machines. This example
starts the virtual machine in installation mode:&prompt.root; sh /usr/share/examples/bhyve/vmrun.sh -c 1 -m 1024M -t tap0 -d guest.img -i -I FreeBSD-10.3-RELEASE-amd64-bootonly.isoguestnameThe virtual machine will boot and start the installer.
After installing a system in the virtual machine, when the
system asks about dropping in to a shell at the end of the
installation, choose Yes. A small
change needs to be made to make the system start with a serial
console. Edit /etc/ttys and replace the
existing ttyu0 line with:ttyu0 "/usr/libexec/getty 3wire" xterm on secureBeginning with &os; 9.3-RELEASE and
10.1-RELEASE the console is configured
automatically.Reboot the virtual machine. While rebooting the virtual
machine causes bhyve to exit, the
vmrun.sh script runs
bhyve in a loop and will automatically
restart it. When this happens, choose the reboot option from
the boot loader menu in order to escape the loop. Now the
guest can be started from the virtual disk:&prompt.root; sh /usr/share/examples/bhyve/vmrun.sh -c 4 -m 1024M -t tap0 -d guest.imgguestnameCreating a &linux; GuestIn order to boot operating systems other than &os;, the
sysutils/grub2-bhyve port must be first
installed.Next, create a file to use as the virtual disk for the
guest machine:&prompt.root; truncate -s 16Glinux.imgStarting a virtual machine with
bhyve is a two step process. First
a kernel must be loaded, then the guest can be started. The
&linux; kernel is loaded with
sysutils/grub2-bhyve. Create a
device.map that
grub will use to map the virtual
devices to the files on the host system:(hd0) ./linux.img
(cd0) ./somelinux.isoUse sysutils/grub2-bhyve to load the
&linux; kernel from the ISO image:&prompt.root; grub-bhyve -m device.map -r cd0 -M 1024MlinuxguestThis will start grub. If the installation
CD contains a
grub.cfg, a menu will be displayed.
If not, the vmlinuz and
initrd files must be located and loaded
manually:grub> ls
(hd0) (cd0) (cd0,msdos1) (host)
grub> ls (cd0)/isolinux
boot.cat boot.msg grub.conf initrd.img isolinux.bin isolinux.cfg memtest
splash.jpg TRANS.TBL vesamenu.c32 vmlinuz
grub> linux (cd0)/isolinux/vmlinuz
grub> initrd (cd0)/isolinux/initrd.img
grub> bootNow that the &linux; kernel is loaded, the guest can be
started:&prompt.root; bhyve -A -H -P -s 0:0,hostbridge -s 1:0,lpc -s 2:0,virtio-net,tap0 -s 3:0,virtio-blk,./linux.img \
-s 4:0,ahci-cd,./somelinux.iso -l com1,stdio -c 4 -m 1024MlinuxguestThe system will boot and start the installer. After
installing a system in the virtual machine, reboot the virtual
machine. This will cause bhyve to
exit. The instance of the virtual machine needs to be
destroyed before it can be started again:&prompt.root; bhyvectl --destroy --vm=linuxguestNow the guest can be started directly from the virtual
disk. Load the kernel:&prompt.root; grub-bhyve -m device.map -r hd0,msdos1 -M 1024Mlinuxguest
grub> ls
(hd0) (hd0,msdos2) (hd0,msdos1) (cd0) (cd0,msdos1) (host)
(lvm/VolGroup-lv_swap) (lvm/VolGroup-lv_root)
grub> ls (hd0,msdos1)/
lost+found/ grub/ efi/ System.map-2.6.32-431.el6.x86_64 config-2.6.32-431.el6.x
86_64 symvers-2.6.32-431.el6.x86_64.gz vmlinuz-2.6.32-431.el6.x86_64
initramfs-2.6.32-431.el6.x86_64.img
grub> linux (hd0,msdos1)/vmlinuz-2.6.32-431.el6.x86_64 root=/dev/mapper/VolGroup-lv_root
grub> initrd (hd0,msdos1)/initramfs-2.6.32-431.el6.x86_64.img
grub> bootBoot the virtual machine:&prompt.root; bhyve -A -H -P -s 0:0,hostbridge -s 1:0,lpc -s 2:0,virtio-net,tap0 \
-s 3:0,virtio-blk,./linux.img -l com1,stdio -c 4 -m 1024Mlinuxguest&linux; will now boot in the virtual machine and
eventually present you with the login prompt. Login and use
the virtual machine. When you are finished, reboot the
virtual machine to exit bhyve.
Destroy the virtual machine instance:&prompt.root; bhyvectl --destroy --vm=linuxguestBooting bhyve Virtual Machines
with UEFI FirmwareIn addition to bhyveload and
grub-bhyve, the
bhyve hypervisor can also boot
virtual machines using the UEFI userspace
firmware. This option may support guest operating systems
that are not supported by the other loaders.In order to make use of the UEFI
support in bhyve, first obtain the
UEFI firmware images. This can be done
by installing sysutils/bhyve-firmware
port or package.With the firmware in place, add the flags
to your bhyve command line.
The actual bhyve command may look
like this:&prompt.root; bhyve -AHP -s 0:0,hostbridge -s 1:0,lpc \
-s 2:0,virtio-net,tap1 -s 3:0,virtio-blk,./disk.img \
-s 4:0,ahci-cd,./install.iso -c 4 -m 1024M \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
guestsysutils/bhyve-firmware also contains a
CSM-enabled firmware, to boot guests with no
UEFI support in legacy
BIOS mode:&prompt.root; bhyve -AHP -s 0:0,hostbridge -s 1:0,lpc \
-s 2:0,virtio-net,tap1 -s 3:0,virtio-blk,./disk.img \
-s 4:0,ahci-cd,./install.iso -c 4 -m 1024M \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI_CSM.fd \
guestGraphical UEFI Framebuffer for
bhyve GuestsThe UEFI firmware support is particularly
useful with predominantly graphical guest operating systems
such as Microsoft &windows;.Support for the UEFI-GOP framebuffer may also be enabled
with the
flags. The framebuffer resolution may be configured with
and
, and
bhyve can be instructed to wait for
a VNC connection before booting the guest
by adding . The framebuffer may be
accessed from the host or over the network via the
VNC protocol.The resulting bhyve command
would look like this:&prompt.root; bhyve -AHP -s 0:0,hostbridge -s 31:0,lpc \
-s 2:0,virtio-net,tap1 -s 3:0,virtio-blk,./disk.img \
-s 4:0,ahci-cd,./install.iso -c 4 -m 1024M \
-s 29,fbuf,tcp=0.0.0.0:5900,w=800,h=600,wait \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
guestNote, in BIOS emulation mode, the framebuffer will cease
receiving updates once control is passed from firmware to
guest operating system.Using ZFS with
bhyve GuestsIf ZFS is available on the host
machine, using ZFS volumes
instead of disk image files can provide significant
performance benefits for the guest VMs. A
ZFS volume can be created by:&prompt.root; zfs create -V16G -o volmode=dev zroot/linuxdisk0When starting the VM, specify the
ZFS volume as the disk drive:&prompt.root; bhyve -A -H -P -s 0:0,hostbridge -s 1:0,lpc -s 2:0,virtio-net,tap0 -s3:0,virtio-blk,/dev/zvol/zroot/linuxdisk0 \
-l com1,stdio -c 4 -m 1024MlinuxguestVirtual Machine ConsolesIt is advantageous to wrap the
bhyve console in a session
management tool such as sysutils/tmux or
sysutils/screen in order to detach and
reattach to the console. It is also possible to have the
console of bhyve be a null modem
device that can be accessed with cu. To do
this, load the nmdm kernel module and
replace with
. The
/dev/nmdm devices are created
automatically as needed, where each is a pair, corresponding
to the two ends of the null modem cable
(/dev/nmdm0A and
/dev/nmdm0B). See &man.nmdm.4; for more
information.&prompt.root; kldload nmdm
&prompt.root; bhyve -A -H -P -s 0:0,hostbridge -s 1:0,lpc -s 2:0,virtio-net,tap0 -s 3:0,virtio-blk,./linux.img \
-l com1,/dev/nmdm0A -c 4 -m 1024Mlinuxguest
&prompt.root; cu -l /dev/nmdm0B
Connected
Ubuntu 13.10 handbook ttyS0
handbook login:Managing Virtual MachinesA device node is created in /dev/vmm for each virtual
machine. This allows the administrator to easily see a list
of the running virtual machines:&prompt.root; ls -al /dev/vmm
total 1
dr-xr-xr-x 2 root wheel 512 Mar 17 12:19 ./
dr-xr-xr-x 14 root wheel 512 Mar 17 06:38 ../
crw------- 1 root wheel 0x1a2 Mar 17 12:20 guestname
crw------- 1 root wheel 0x19f Mar 17 12:19 linuxguest
crw------- 1 root wheel 0x1a1 Mar 17 12:19 otherguestA specified virtual machine can be destroyed using
bhyvectl:&prompt.root; bhyvectl --destroy --vm=guestnamePersistent ConfigurationIn order to configure the system to start
bhyve guests at boot time, the
following configurations must be made in the specified
files:/etc/sysctl.confnet.link.tap.up_on_open=1/boot/loader.confvmm_load="YES"
nmdm_load="YES"
if_bridge_load="YES"
if_tap_load="YES"/etc/rc.confcloned_interfaces="bridge0tap0"
ifconfig_bridge0="addm igb0 addm tap0"&os; as a &xen;-HostXen is a GPLv2-licensed type
1 hypervisor for &intel; and &arm; architectures. &os;
has included &i386; and &amd; 64-Bit DomU
and Amazon
EC2 unprivileged domain (virtual machine) support since
&os; 8.0 and includes Dom0 control domain (host) support in
&os; 11.0. Support for para-virtualized (PV) domains has
been removed from &os; 11 in favor of hardware virtualized
(HVM) domains, which provides better performance.&xen; is a bare-metal hypervisor, which means that it is the
first program loaded after the BIOS. A special privileged guest
called the Domain-0 (Dom0 for short) is then
started. The Dom0 uses its special privileges to directly
access the underlying physical hardware, making it a
high-performance solution. It is able to access the disk
controllers and network adapters directly. The &xen; management
tools to manage and control the &xen; hypervisor are also used
by the Dom0 to create, list, and destroy VMs. Dom0 provides
virtual disks and networking for unprivileged domains, often
called DomU. &xen; Dom0 can be compared to
the service console of other hypervisor solutions, while the
DomU is where individual guest VMs are run.&xen; can migrate VMs between different &xen; servers. When
the two xen hosts share the same underlying storage, the
migration can be done without having to shut the VM down first.
Instead, the migration is performed live while the DomU is
running and there is no need to restart it or plan a downtime.
This is useful in maintenance scenarios or upgrade windows to
ensure that the services provided by the DomU are still
provided. Many more features of &xen; are listed on the Xen
Wiki Overview page. Note that not all features are
supported on &os; yet.Hardware Requirements for &xen; Dom0To run the &xen; hypervisor on a host, certain hardware
functionality is required. Hardware virtualized domains
require Extended Page Table (EPT)
and Input/Output Memory Management Unit (IOMMU)
support in the host processor.&xen; Dom0 Control Domain SetupThe emulators/xen package works with
&os; 11 amd64 binary snapshots and equivalent systems
built from source. This example assumes VNC output for
unprivileged domains which is accessed from a another system
using a tool such as net/tightvnc.Install emulators/xen:&prompt.root; pkg install xenConfiguration files must be edited to prepare the host
for the Dom0 integration. An entry to
/etc/sysctl.conf disables the limit on
how many pages of memory are allowed to be wired. Otherwise,
DomU VMs with higher memory requirements will not run.&prompt.root; sysrc -f /etc/sysctl.conf vm.max_wired=-1Another memory-related setting involves changing
/etc/login.conf, setting the
memorylocked option to
unlimited. Otherwise, creating DomU
domains may fail with Cannot allocate
memory errors. After making the change to
/etc/login.conf, run
cap_mkdb to update the capability database.
See for
details.&prompt.root; sed -i '' -e 's/memorylocked=64K/memorylocked=unlimited/' /etc/login.conf
&prompt.root; cap_mkdb /etc/login.confAdd an entry for the &xen; console to
/etc/ttys:&prompt.root; echo 'xc0 "/usr/libexec/getty Pc" xterm on secure' >> /etc/ttysSelecting a &xen; kernel in
/boot/loader.conf activates the Dom0.
&xen; also requires resources like CPU and memory from the
host machine for itself and other DomU domains. How much CPU
and memory depends on the individual requirements and hardware
capabilities. In this example, 8 GB of memory and 4
virtual CPUs are made available for the Dom0. The serial
console is also activated and logging options are
defined.&prompt.root; sysrc -f /boot/loader.conf hw.pci.mcfg=0
&prompt.root; sysrc -f /boot/loader.conf xen_kernel="/boot/xen"
&prompt.root; sysrc -f /boot/loader.conf xen_cmdline="dom0_mem=8192M dom0_max_vcpus=4 dom0pvh=1 console=com1,vga com1=115200,8n1 guest_loglvl=all loglvl=all"Log files that &xen; creates for the Dom0 and DomU VMs
are stored in /var/log/xen. This
directory does not exist by default and must be
created.&prompt.root; mkdir -p /var/log/xen
&prompt.root; chmod 644 /var/log/xen&xen; provides a boot menu to activate and de-activate
the hypervisor on demand in
/boot/menu.rc.local:&prompt.root; echo "try-include /boot/xen.4th" >> /boot/menu.rc.localActivate the xencommons service during system
startup:&prompt.root; sysrc xencommons_enable=yesThese settings are enough to start a Dom0-enabled
system. However, it lacks network functionality for the
DomU machines. To fix that, define a bridged interface with
the main NIC of the system which the DomU VMs can use to
connect to the network. Replace
igb0 with the host network
interface name.&prompt.root; sysrc autobridge_interfaces=bridge0
&prompt.root; sysrc autobridge_bridge0=igb0
&prompt.root; sysrc ifconfig_bridge0=SYNCDHCPRestart the host to load the &xen; kernel and start the
Dom0.&prompt.root; rebootAfter successfully booting the &xen; kernel and logging
into the system again, the &xen; management tool
xl is used to show information about the
domains.&prompt.root; xl list
Name ID Mem VCPUs State Time(s)
Domain-0 0 8192 4 r----- 962.0The output confirms that the Dom0 (called
Domain-0) has the ID 0
and is running. It also has the memory and virtual CPUs
that were defined in /boot/loader.conf
earlier. More information can be found in the &xen;
Documentation. DomU guest VMs can now be
created.&xen; DomU Guest VM ConfigurationUnprivileged domains consist of a configuration file and
virtual or physical hard disks. Virtual disk storage for
the DomU can be files created by &man.truncate.1; or ZFS
volumes as described in .
In this example, a 20 GB volume is used. A VM is
created with the ZFS volume, a &os; ISO image, 1 GB of
RAM and two virtual CPUs. The ISO installation file is
retrieved with &man.fetch.1; and saved locally in a file
called freebsd.iso.&prompt.root; fetch ftp://ftp.freebsd.org/pub/FreeBSD/releases/ISO-IMAGES/10.3/FreeBSD-10.3-RELEASE-amd64-bootonly.iso -o freebsd.isoA ZFS volume of 20 GB called
xendisk0 is created to serve as the disk
space for the VM.&prompt.root; zfs create -V20G -o volmode=dev zroot/xendisk0The new DomU guest VM is defined in a file. Some specific
definitions like name, keymap, and VNC connection details are
also defined. The following freebsd.cfg
contains a minimum DomU configuration for this example:&prompt.root; cat freebsd.cfg
builder = "hvm"
name = "freebsd"
memory = 1024
vcpus = 2
vif = [ 'mac=00:16:3E:74:34:32,bridge=bridge0' ]
disk = [
'/dev/zvol/tank/xendisk0,raw,hda,rw',
'/root/freebsd.iso,raw,hdc:cdrom,r'
]
vnc = 1
vnclisten = "0.0.0.0"
serial = "pty"
usbdevice = "tablet"These lines are explained in more detail:This defines what kind of virtualization to use.
hvm refers to hardware-assisted
virtualization or hardware virtual machine. Guest
operating systems can run unmodified on CPUs with
virtualization extensions, providing nearly the same
performance as running on physical hardware.
generic is the default value and
creates a PV domain.Name of this virtual machine to distinguish it from
others running on the same Dom0. Required.Quantity of RAM in megabytes to make available to the
VM. This amount is subtracted from the hypervisor's total
available memory, not the memory of the Dom0.Number of virtual CPUs available to the guest VM. For
best performance, do not create guests with more virtual
CPUs than the number of physical CPUs on the host.Virtual network adapter. This is the bridge connected
to the network interface of the host. The
mac parameter is the MAC address set on
the virtual network interface. This parameter is
optional, if no MAC is provided &xen; will generate a
random one.Full path to the disk, file, or ZFS volume of the disk
storage for this VM. Options and multiple disk
definitions are separated by commas.Defines the Boot medium from which the initial
operating system is installed. In this example, it is the
ISO imaged downloaded earlier. Consult the &xen;
documentation for other kinds of devices and options to
set.Options controlling VNC connectivity to the serial
console of the DomU. In order, these are: active VNC
support, define IP address on which to listen, device node
for the serial console, and the input method for precise
positioning of the mouse and other input methods.
keymap defines which keymap to use, and
is english by default.After the file has been created with all the necessary
options, the DomU is created by passing it to xl
create as a parameter.&prompt.root; xl create freebsd.cfgEach time the Dom0 is restarted, the configuration file
must be passed to xl create again to
re-create the DomU. By default, only the Dom0 is created
after a reboot, not the individual VMs. The VMs can
continue where they left off as they stored the operating
system on the virtual disk. The virtual machine
configuration can change over time (for example, when adding
more memory). The virtual machine configuration files must
be properly backed up and kept available to be able to
re-create the guest VM when needed.The output of xl list confirms that the
DomU has been created.&prompt.root; xl list
Name ID Mem VCPUs State Time(s)
Domain-0 0 8192 4 r----- 1653.4
freebsd 1 1024 1 -b---- 663.9To begin the installation of the base operating system,
start the VNC client, directing it to the main network address
of the host or to the IP address defined on the
vnclisten line of
freebsd.cfg. After the operating system
has been installed, shut down the DomU and disconnect the VNC
viewer. Edit freebsd.cfg, removing the
line with the cdrom definition or
commenting it out by inserting a #
character at the beginning of the line. To load this new
configuration, it is necessary to remove the old DomU with
xl destroy, passing either the name or the
id as the parameter. Afterwards, recreate it using the
modified freebsd.cfg.&prompt.root; xl destroy freebsd
&prompt.root; xl create freebsd.cfgThe machine can then be accessed again using the VNC
viewer. This time, it will boot from the virtual disk where
the operating system has been installed and can be used as a
virtual machine.