FreeBSD 9 with ZFS bootdisk
A large part of this tutorial is based on the excellent tutorials by George Kontostanos.
First of all, create a USB memory stick with the FreeBSD installer, as explained in the FreeBSD handbook in the section on Prepare the Installation Media.
Run the FreeBSD installer and select "install" to start and installation procedure.
After setting the keyboard settings, hostname and selecting the components to install, you will reach the partitioning stage.
Create the ZFS partition
At the partitioning stage, select "shell". (Thus not "Guided" or "Manual".)
You first get a helpful message:
- Use this shell to set up partitions for the new system. When finished, mount the system at /mnt and place an fstab file for the new system at /tmp/bsdinstall_etc/fstab. Then type 'exit'. You can also enter the partition editor at any time by entering 'bsdinstall partedit'.
First, determine the names of your disks:
# camcontrol devlist <Corsair Force 3 SSD 5.02> at scbus0 target 0 lun 0 (ada0,pass0) <WDC WD30EZRX-00MMMB0 80.00A80> at scbus3 target 0 lun 0 (ada1,pass1) <Imation Atom PMAP> at scbus7 target 0 lun 0 (da0,pass2)
For more info, you can search dmesg:
# grep 'ada0' /var/run/dmesg.boot # egrep '[ac]?da[0-9]' /var/run/dmesg.boot
In this case, the USB key with the installer was located at /dev/da0, and the SSD which would contain the new system at /dev/ada0. The 3 TByte HDD at /dev/ada1 is not of importance yet (it will be configured later).
Once determined, use gpart to list the existing partitions (this list was engineered to show a few examples):
# gpart show => 0 7821312 da0 BSD (3.7G) 0 1095504 1 freebsd-ufs (534M) 1095504 6725808 - free - (3.2G) => 34 30932925 ada0 GPT (14G) 34 500 1 freebsd-boot (250k) 534 30932797 2 freebsd-zfs (14G) => 34 30932925 ada1 GPT (14G) 34 500 1 freebsd-boot (250k) 534 10485760 2 freebsd-ufs (5.0G) 10485922 16777216 3 freebsd-ufs (8.0G) 27263138 3667968 4 freebsd-swap (1.8G) 30931106 1853 - free - (926k)
If a disk has no known recognised partition table, it is not listed, or gpart gives an error:
# gpart show ada0 gpart: No such geom: ada0
If the disk already has partitions (and you no longer want the data on it), you delete all but the first partition. There is no need to delete the "freebsd-boot" partition, as you will need that anyway.
# gpart delete -i 4 ada0 # gpart delete -i 3 ada0 # gpart delete -i 2 ada0
If the disk does not yet have a partition table or boot partition, you should create that first:
# gpart create -s gpt ada0 # gpart add -b 34 -s 500 -t freebsd-boot ada0 # gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada0
(Previously, a freebsd-boot partition of 94 blocks was enough; this is no longer true for FreeBSD 9.2. The listed 500 blocks should be more than enough. Unfortunately, gpart does not seem to print an error in case the bootcode is too large for the partition.)
Next, create the ZFS partition and give it a label (in this case, I choose the name 'ssd0'). If you want the ZFS partition to use all disk space (which is a sane default), there is no need to specify the size parameter:
# gpart add -t freebsd-zfs -l ssd0 ada0
However, I wanted to leave the last 4 GByte of the disk for two ZFS Intend Logs (ZIL) to speed up ZFS on other disks. In addition, I decided to use a ZFS device for swapping instead of a zvol.
# gpart add -t freebsd-swap -l swap0 -s 4G ada0 # gpart add -t freebsd-zfs -l ssd0 -s 108842639 ada0 # gpart add -t freebsd-zfs -l zil0 -s 4194304 ada0 # gpart add -t freebsd-zfs -l zil1 -s 4194304 ada0
The result is:
# gpart show -p ada0 => 34 117231341 ada0 GPT (14G) 34 94 ada0p1 freebsd-boot (47k) 128 4194304 ada0p2 freebsd-swap (4.0G) 4194432 108842639 ada0p3 freebsd-zfs (51G) 108842767 4194304 ada0p4 freebsd-zfs (2.0G) 113037071 4194304 ada0p5 freebsd-zfs (2.0G)
Create the ZFS file system
Now it's time to create the ZFS file system. We will call it zroot, but you can pick any name you like. The device name is /dev/gpt/ followed by the name you gave to the freebsd-zfs partition above. Mount it at /mnt as requested by the FreeBSD Installer:
# zpool create -R /mnt zroot /dev/gpt/ssd0
If you want to create a mirror of two disks, the command is:
# zpool create -R /mnt zroot mirror /dev/gpt/ssd0 /dev/gpt/ssd1
This will mount the zroot partition at /mnt, as can be confirmed with mount:
% mount zroot on /mnt (zfs, local, nfsv4acls) % zpool list NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT zroot 51.5G 104K 51.5G 0% 1.00x ONLINE /mnt ~% zfs list NAME USED AVAIL REFER MOUNTPOINT zroot 104K 51.3G 31K /mnt
Note: please use the
-R /mnt option, not the
-m /mnt option. The
-R option temporarily sets the mount point of the zpool (the altroot) to /mnt, exactly what we want. The
-m option permanently sets the mount point of the root dataset to /mnt, which is not what we want. In case you accidentally used the -m option, you should correct with:
# zfs set mountpoint=/ zroot # zfs set cachefile=none zroot
Now proceed to configure the create zroot pool:
# zpool set bootfs=zroot zroot # zfs set checksum=fletcher4 zroot
Now create datasets. Datasets are similar to partitions in other file systems. Each dataset can have some attributes. For example, you can specify that /tmp may not exceed a certain size.
# zfs create zroot/var # zfs create -o compression=lzjb -o exec=off -o setuid=off zroot/var/log # zfs create -o setuid=off zroot/tmp # chmod 1777 /mnt/tmp
Optionally disable keeping track of the access time of file (this reduces the number of writes to a SSD):
zfs set atime=off zroot
If you have not create a swap device, you can create a swap dataset:
# zfs create -V8G zroot/swap # zfs set org.freebsd:swap=on zroot/swap # zfs set checksum=off zroot/swap
Otherwise, make sure to mount your swap device(s):
# cat << EOF > /tmp/bsdinstall_etc/fstab # Device Mountpoint FStype Options Dump Pass# /dev/gpt/swap0 none swap sw 0 0 /dev/gpt/swap1 none swap sw 0 0 EOF
You may create multiple mount points with ZFS if desired.
Installation of FreeBSD
If you're done, exit the shell to proceed with the regular installation:
Alternatively, you can do the installation manually:
# cd /usr/freebsd-dist/ # tar --unlink -xpJf base.txz -C /mnt # tar --unlink -xpJf kernel.txz -C /mnt # tar --unlink -xpJf ports.txz -C /mnt
and edit /mnt/etc/rc.conf to your liking.
Configuration of Boot Parameters
At the end of the installation procedure, you must first configure some boot parameters before you can reboot. Otherwise, booting will fail because the ZFS driver is not yet loaded.
You must make these changes from the command line, and have two changes of dropping in the shell after the installation process.
- You can select "Yes" when asked "The installation is now finished. Before exiting the installer, would you like to open a shell in the new system to make any final manual modifications?"
- Select "No" to the previous question, but select "Live CD" when asked "Installation of FreeBSD is complete! Would you like to reboot into the installed system now?" You can log in as root (with an empty password).
The difference between the two is that in the first case, you end up in chrooted jail, while you end up in the full system in the second case. Let's assume you choose the first option, than you need to edit the file
/boot/loader.conf and add these two lines (if the file does not exist, you should create it):
If you would have chosen the "Live CD" option, you should have edited the file
Side note: If you followed the above descriptions, there is no need to change anything settings of your ZFS file system anymore. If you nevertheless want to set some changes and get this error:
# zfs set mountpoint=/ zroot internal error: failed to initialize ZFS library
This may be an indication that you are running in a chrooted jail. If you want to make changes to ZFS, make sure you choose the "Live CD" option, and not the "Shell" option, as explained in the previous section.
After a reboot, your machine should come up with a running FreeBSD system. If there are some errors in your settings, you may end up at the mountroot prompt.
After reboot I end up at the mount prompt:
And you can not boot ZFS from here:
mountroot> zfs:zroot Trying to mount root from zfs:zroot ... Mounting from zfs:zroot failed with error 2: unknown file system
This likely means that the bootcode you installed (with the
gpart bootcode command earlier) worked fine, but it was not able to find the ZFS mounts.
The most likely cause is that your did forgot to add these lines to
In order to fix this, you should reboot from the installer CD, and choose the "Live CD" option. You first need to mount the zpool under /mnt. First list the available pools:
# zpool list no pools available # zpool import pool: zroot id: 128742958409357345692345 state: ONLINE action: The pool can be imported using its name of numeric identifier. config: zroot ONLINE ada1p2 ONLINE
Now import the zroot pool, and mount under
# zpool import -R /mnt zroot
You can now edit