Unlike other operating systems, patches for the OpenBSD base system are distributed as source code patches. These patches are usually applied by compiling and installing them onto the target system. While that update procedure is well-documented, it is not always suitable for certain systems that do not have the OpenBSD compiler set installed for various reasons such as disk space constraints. To fill this gap, open source projects like binpatch were started to allow administrators to create binary patches using the BSD make system. This article proposes an alternative method to build binary patches using a chroot environment in an attempt to more closely mirror the instructions given in the OpenBSD patch files.

New OpenBSD administrators often find that OpenBSD is different from other operating systems in terms of its update process. Linux systems, for example, usually distribute all components, including the kernel and all userspace software, as packages; to update the packages, the administrator would use the distribution-specific update tool to update those packages (like apt-get for Debian systems and upgradepkg for Slackware).

OpenBSD uses a similar tool called pkg_add(1) to update third-party software (called "ports" in OpenBSD nomenclature). However, the OpenBSD base system has a different update process. The base system here refers to software that are part of stock OpenBSD, such as the tools in /bin and /usr/bin which are not third-party software. Patches to the base system are distributed as source code patches on http://www.openbsd.org/errata.html, which have to be hand-compiled and installed on the target system.

This means that the OpenBSD compiler distribution set (compXX.tgz) has to be installed on the target system in order to build and apply those patches. However, this is not always desirable for a few reasons:

• the compiler set tends to be very large (it is a 75MB compressed tarball on OpenBSD 4.0/i386), so it may not fit on a target system with disk space constraints;
• the target system may have very low memory that prohibits compilation;
• the target system may have a very slow CPU that would take a very long time to compile large programs;
• the target system may be a virtual machine with the above limitations.

In addition, an OpenBSD administrator might maintain a large number of such systems, making it inconvenient to compile and install patches by hand on each individual system.

Fortunately, excellent projects like binpatch (http://openbsdbinpatch.sourceforge.net/) exist to fill this void. Binpatch uses the BSD make system to build binary patches. It employs a custom Makefile in order to create binary patches based on the instructions given in the OpenBSD patch files from http://www.openbsd.org/errata.html.

A binary patch is simply a tarball (a .tar.gz or .tgz file) containing the pre-compiled files that are the result of applying the source code patch. To illustrate, suppose OpenBSD distributes a source code patch for the Apache webserver, which is part of the base system. The corresponding binary patch would be a tarball consisting of the new Apache programs, modules, manpages, and other files that are created as a result of patching and re-compiling Apache. The binary patch is created on a system with the full compiler set, and distributed to the target system where the compiler set is not available.

In this article, I will describe a technique that uses a chroot environment to create binary patches. This technique is in no way more superior compared to binpatch's BSD make system technique; it is merely an alternative method that attempts to more closely mirror the original instructions in the OpenBSD patch files by using a chroot environment. Please note that this proposed method is highly experimental and we do not recommend it for production use -- unless you really know what you are doing! I also assume you have some moderate experience and skills with OpenBSD administration.

You will first need access to the OpenBSD distribution files like baseXX.tgz, etcXX.tgz, and compXX.tgz, as well as the source code tarballs (src.tar.gz and sys.tar.gz). An easy way to have convenient access to these files is to order an OpenBSD CD from the OpenBSD store. To make this article easier to follow, let's suppose we are attempting to build a binary patch in OpenBSD 4.0 on the i386 platform. This process involves two separate machines: we will build the binary patch on a machine which we will call a build system, and we will deploy the binary patch on a target system.

First, we need some space to emulate a full OpenBSD system that we can use as a chroot environment. Let's create the following hierarchy in a new directory called /var/chbinpatch:

mkdir -p /var/chbinpatch/work/usr/src

Then, extract the OpenBSD distribution files ({base,etc,comp,man,misc}40.tgz) to /var/chbinpatch/work:

for i in base etc comp man misc; do tar zxpf /path/to/${i}40.tgz -C /var/chbinpatch/work; done

Note that the above command is meant for OpenBSD 4.0; for another version, such as OpenBSD 4.1, replace "40" with "41".

Next, extract the source code tarballs src.tar.gz and sys.tar.gz to /var/chbinpatch/work/usr/src:

tar zxpf /path/to/src.tar.gz -C /var/chbinpatch/work/usr/src
tar zxpf /path/to/sys.tar.gz -C /var/chbinpatch/work/usr/src

We now have an almost-complete OpenBSD "system" in /var/chbinpatch/work. However, one thing is missing -- to be able to compile in a chroot environment, gcc will need the necessary devices in /dev. However, directories like /var and /usr are mounted with the nodev option by default, thus it is not possible to use devices in /var/chbinpatch/work/dev. To work around this, we backup the existing /dev that we just extracted and create a temporary memory filesystem to act as /dev in the chroot environment.

mv /var/chbinpatch/work/dev /var/chbinpatch/work/dev-orig
mkdir /var/chbinpatch/work/dev
mount_mfs -o nosuid -s 8192 /dev/wd0b /var/chbinpatch/work/dev

If your swap device is not /dev/wd0b, change the mount_mfs(8) command accordingly.

gcc needs the standard devices to be present in /dev, so let's copy over the MAKEDEV script from the original /dev and execute it with the "std" option:

cd /var/chbinpatch/work/dev
cp -p ../dev-orig/MAKEDEV .
./MAKEDEV std

We now have a complete OpenBSD "system." Let's proceed by applying a patch from the OpenBSD errata page.

Suppose we would like to apply OpenBSD 4.0 errata #1 from http://www.openbsd.org/errata40.html. Download the patch file to /var/chbinpatch/work/usr/src:

cd /var/chbinpatch/work/usr/src
ftp ftp://ftp.openbsd.org/pub/OpenBSD/patches/4.0/common/001_httpd.patch

Let's enter the chroot environment:

/usr/sbin/chroot /var/chbinpatch/work /bin/ksh

For those unfamiliar with chroot(8), the above command will treat /var/chbinpatch/work as the root directory (/), and run /bin/ksh as the shell.

Now, to build a binary patch, we must figure out the files that will change after applying the patch, and isolate those files to be part of a tarball that can be distributed to a target system. This is where the chroot environment truly shines. In an actual non-chroot OpenBSD system, it would be difficult (though not impossible) to isolate those files because the system is actually running, and background processes might make changes to the filesystem. In contrast, inside a chroot environment, the system is totally "clean"; no processes are running that will make any change to the chrooted filesystem.

To actually be able to track file changes after applying a patch, we will need to create two "cookie" files, which are really just dummy marker files that are meant to facilitate the find(1) command to, well, find the files that have changed after the compile/install process. One cookie file is created before the build process (let's call it the "pre-installation cookie"), and the other after the build process (the "post-installation cookie"). The idea is to use find(1) to generate a list of files with timestamps later than the pre-installation cookie but earlier than the post-installation cookie. This will become clearer later in the article. Readers familiar with OpenBSD ports will notice that this cookie technique is borrowed from the make system in the OpenBSD ports tree.

Let's create the pre-installation cookie first:

touch /001_httpd.pre

Remember, we are in a chroot environment, so / is actually /var/binpatch/work. I would not recommend creating these dummy files if this is the regular / directory on an actual system! Also, take note to name your cookie according to the patch file (in this case, 001_httpd) so that you will be able to keep track of which cookies are used for which patch.

Now, proceed to build the patch according to the patch instructions. In this case, we are following the instructions in the OpenBSD 4.0 001_httpd.patch file:

cd /usr/src
patch -p0 < 001_httpd.patch
cd usr.sbin/httpd
make -f Makefile.bsd-wrapper obj
make -f Makefile.bsd-wrapper cleandir
make -f Makefile.bsd-wrapper depend
make -f Makefile.bsd-wrapper
make -f Makefile.bsd-wrapper install

After installation, create the post-installation cookie.

touch /001_httpd.post

The pre-installation and post-installation cookies are very important -- if they are not present, you will not be able to generate the binary patch accurately. So, always remember to create them!

Now, use the find(1) command to generate a list of files in /usr, /sbin, and /bin that were created by the "make install" process:

export P="001_httpd"
find /usr /sbin /bin -path /usr/src -prune -or -path /usr/obj -prune -or \
  -newer /${P}.pre -a ! -newer /${P}.post -type f | \
  grep -v '/usr/src' | grep -v '/usr/obj' | sed 's/^/./' >/${P}.list

Recall that the find(1) command is used to find files later than the pre-installation cookie but earlier than the post-installation cookie. Also, note that the above command intentionally excludes the /usr/src and /usr/obj directories, and pipes its output into a file called /001_httpd.list.

Now you can use that 001_httpd.list to create a tarball of the relevant files that are part of the patch:

cd /
tar cvzpf errata-4.0-001-i386.tar.gz $(cat 001_httpd.list)

Your binary patch, errata-4.0-001-i386.tar.gz, is now complete! Exit the chroot environment:

exit

To apply the binary patch, you will first need to use scp(1) to transfer it to the target OpenBSD system:

scp -p /var/chbinpatch/work/errata-4.0-001-i386.tar.gz user@target:

After the transfer, log in to the target system and extract it using tar(1):

ssh user@target
su -
cd /
tar zxvpf ~user/errata-4.0-001-i386.tar.gz

Your binary patch is now applied on the target system. Remember to follow any further instructions from the patch file if needed. For example, for this patch file you would have to restart Apache using the apachectl(8) command.

To summarize, when you are ready to build your next binary patch, use the following steps:

1. If you have unmounted the fake /var/chbinpatch/work/dev directory, remember to remount it and re-run "MAKEDEV std" to make the standard devices.
2. Download the patch file to /var/chbinpatch/work/usr/src
3. chroot into the environment.
4. Create the pre-installation cookie.
5. Apply the patch instructions in the patch file, then build and install the patch.
6. Create the post-installation cookie.
7. Use the find(1) command to generate a list of files that have changed as a result of the patch.
8. Create a binary patch using that list.

Remember to always build your binary patches in the order presented on the errata page.

This chroot method allows you to create binary patches using instructions that conform more closely to the original instructions from the patch files. In the development labs at Calyptix Security, we are currently developing a packaging mechanism to automate the creation of these binary patches. We are also experimenting with using this technique to apply binary patches using the pkg_add(1) command. Once complete, we hope to open source the resultant packaging mechanism if there is any interest.

I hope this article has been beneficial to you, and would be useful as you update your OpenBSD systems -- though remember that it is still considered experimental! I invite comments, corrections, and feedback to .

Thanks to J.C. Roberts for pointing out my incorrect use of the word "upgrade" in the original article; this has been corrected. Thanks to Todd T. Fries for suggesting a safer method to create the errata tarballs.

Lawrence Teo is VP Development and co-founder of Calyptix Security, which makes the OpenBSD-based AccessEnforcer all-in-one Internet security appliance for small to medium-sized businesses. He has been using OpenBSD for both work and play since version 2.5 in 1999.