FreeBSD kernel cross-compiling

This is a short tutorial, detailing how I compiled a FreeBSD kernel on a faster amd64 computer (the compile system), and installed it on a slower i386 computer (the target system).

Install Kernel Source Code

On the fast computer, ensure that all source are installed in /usr/src

# ls /usr/src
COPYRIGHT          README             etc/               libexec/           sys/
LOCKS              UPDATING           games/             release/           tests/
MAINTAINERS        bin/               gnu/               rescue/            tools/
Makefile           cddl/              include/           sbin/              usr.bin/
Makefile.inc1      contrib/           kerberos5/         secure/            usr.sbin/
ObsoleteFiles.inc  crypto/            lib/               share/

If not, install the source. Previously, sysinstall could be used. Perhaps it is still possible to use bsdconfig, but the recommended method is simply install it yourself:

# Replace CPU with i382, amd64, or ia64, depending on your compile system
# Replace VERSION with the latest release version
wget https://download.freebsd.org/ftp/releases/CPU/VERSION-RELEASE/src.txz
e.g.:
wget https://download.freebsd.org/ftp/releases/i386/10.2-RELEASE/src.txz

# Unpack the archive to /usr/src:
tar --unlink -xpJf ports.txz -C /

Note that it does not really matter which CPU you pick: src.txz contains the source code for each target CPU.

Update Kernel Source Code

In order to update the kernel source code, run on the compile machine:

freebsd-update fetch
freebsd-update install

This updates the base system, including the kernel source code, as long as it is located in /usr/src.

Create a Kernel configuration

The kernel configuration should go in /usr/src/sys/CPU/conf/, with CPU the CPU type of your target system. E.g. i386. However, since /usr/src/ is easily overwritten, it is recommended to create the kernel configuration in /root/kernels and make a symlink:

mkdir /root/kernels
touch /root/kernels/SOEKRIS_ZFS
ln -s /root/kernels/SOEKRIS_ZFS /usr/src/sys/i386/conf/

Now create the kernel configuration in /root/kernels/SOEKRIS_ZFS. E.g.:

   # Kernel for a i386-based Soekris machine with ZFS
   include GENERIC
   options CPU_SOEKRIS
   # Allow ZFS crash on i386 CPU, by adjusting KSTACK_PAGES
   # See /usr/src/UPDATING entry 20121223
   options KSTACK_PAGES=4
   # Allow more kernel space
   # options KVA_PAGES=320

Read /usr/src/UPDATING to get some important hints.

Read https://www.freebsd.org/cgi/man.cgi?build%287%29 (build (7) man page)

cd /usr/src

make kernel-toolchain TARGET=i386    2>&1 | tee /root/kernels/kerneltoolchain.i386.log
# or make buildworld TARGET=i386

make TARGET=i386 KERNCONF=SOEKRIS_ZFS buildkernel    \
        2>&1 | tee /root/kernels/buildkernel.i386.log

make TARGET=i386 KERNCONF=SOEKRIS_ZFS DESTDIR=/root/kernels/wolfje installkernel    \
        2>&1 | tee /root/kernels/installkernel.i386.log

Note: in case you don't want to compile the kernel, but also the rest of the system (ports, ) then follow the instructions at: https://www.freebsd.org/doc/handbook/makeworld.html

In FreeBSD, the term “world” includes the kernel, core system binaries, libraries, programming files, and built-in compiler.

   ZFS notes
       When upgrading the boot ZFS pool to a new version, always follow
       these two steps:

       1.) recompile and reinstall the ZFS boot loader and boot block
       (this is part of "make buildworld" and "make installworld")

       2.) update the ZFS boot block on your boot drive

       The following example updates the ZFS boot block on the first
       partition (freebsd-boot) of a GPT partitioned drive ad0:
       "gpart bootcode -p /boot/gptzfsboot -i 1 ad0"

   To build a kernel
       If you are updating from a prior version of FreeBSD (even one just
       a few days old), you should follow this procedure.  It is the most
       failsafe as it uses a /usr/obj tree with a fresh mini-buildworld,

       make kernel-toolchain
       make -DALWAYS_CHECK_MAKE buildkernel KERNCONF=YOUR_KERNEL_HERE
       make -DALWAYS_CHECK_MAKE installkernel KERNCONF=YOUR_KERNEL_HERE

Install kernel

target# mv /boot/kernel /boot/kernel.old target# scp -r buildsystem.example.org:/root/kernels/wolfje/boot/kernel /boot/ target# reboot

Cross fingers

Update programs and libraries

On target machine:

freebsd-update fetch freebsd-update install

pkg update pkg upgrade

portsnap fetch portsnap install

reboot

Problems

Userland out out date

freebsd-update says it's upt-to-date

1. cp -r kernel kernel_10.2_stable

[root@wolfje] /boot# uname -r 10.2-RELEASE-p5

[root@wolfje] /boot# dmesg| head ACPI BIOS Error (bug): A valid RSDP was not found (20150515/tbxfroot-258) Copyright (c) 1992-2015 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994

       The Regents of the University of California. All rights reserved.

FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 10.2-RELEASE-p5 #0: Mon Oct 19 16:17:46 CEST 2015

   root@pakhuis:/root/kernels/obj.i386/i386.i386/usr/src/sys/SOEKRIS_ZFS i386

Yet: [root@wolfje] ~# file /bin/chmod /bin/chmod: ELF 32-bit LSB executable, Intel 80386, version 1 (FreeBSD), dynamically linked (uses shared libs), for FreeBSD 10.0 (1000510), stripped

-> kernel is 10.2, freebsd userland is 10.2, but in reality it is still on 10.0. OOPS!

Solution:

Seee dan.langille.org, 18 jan 2015, "Feilure with freebsd-update"

Solution:

env UNAME_r=10.0-RELEASE freebsd-update upgrade -r 10.2-RELEASE

This article is unfinished.