Home » Random Goodies » Creating encrypted UNIX jails

Creating encrypted UNIX jails

As most of us lean towards security and stability, I’m a big fan of FreeBSD and it’s many capabilities.

The following article will cover creating a Geometry Eli encrypted UNIX jail in FreeBSD 8.1 using a predefined partition, and it assumes you have already created your slices and partitions as per your needs. This procedure is quite useful if you are running an enterprise server and wish to rest assured that if the drive is pulled out of the rack, your information will be safe from prying eyes. Jailed environments, or sub-systems, are also useful for your development needs as the jail is secluded from your host system and the environment can crash without effecting the entire system.

To begin, we first need to edit /etc/fstab and comment out our intended encrypted jail partition(s) and /tmp.

Depending on the size of our /tmp filesystem, the tmpsize in /etc/rc.conf will vary and the contents should read as follows, replacing aa.aa.aa.aa with the primary IP of the server:

Edit /etc/rc.conf:
# vi /etc/rc.conf
sendmail_submit_flags="-L sm-mta -bd -q30m -ODaemonPortOptions=Addr=localhost"
sendmail_outbound_flags="-L sm-queue -q30m"
syslogd_flags="-ss -b aa.aa.aa.aa"

We now need to make sure SSH only listens on the host system’s IP address as we do not want to allow any SSH access to the encrypted jail(s). For added security we restrict root from directly logging in.

Edit /etc/ssh/sshd_config:
# vi /etc/ssh/sshd_config
ListenAddress aa.aa.aa.aa:22
PermitRootLogin no

Now, we now need to set the system administrator mail alias.

Edit /etc/mail/aliases:
# vi /etc/mail/aliases
root: [email protected]

Now, we have to make the new aliases take effect.
# newaliases

We now need to grab the latest binaries for the FreeBSD Host System and install the ports collection.

# freebsd-update fetch install
# portsnap fetch extract

Once the binaries are done updating and the ports collection is installed we need to install portaudit and portmaster for easy management of security vulnerabilities and the ports collection itself.

# cd /usr/ports/ports-mgmt/portaudit
# make all install clean
# cd /usr/ports/ports-mgmt/portmaster
# make install clean
# rehash

Now run portaudit for the first time:

# portaudit -Fda

Usually, it will state that you have 0 security vulnerabilities if it shows otherwise we MUST update using portmaster:

# portmaster -a

Congratulations! The Host System has now been prepped. We can now begin to prepare the userlands for our jails and further configure the Host System for jail encryption and security.

Create /etc/standard-supfile in order to create our userland binaries. Please note, the release will vary depending on the version of FreeBSD you currently have installed and the following assumes FreeBSD 8.1 is currently being used:

# vi /etc/standard-supfile
*default host=ftp.uk.freebsd.org
*default base=/var/db
*default prefix=/usr
*default release=cvs tag=RELENG_8_1
*default delete use-rel-suffix
*default compress

Now, we must gather our source code, and begin buildworld for our userlands:

# csup /etc/standard-supfile
# cd /usr/src
# make buildworld
# mergemaster -p
# make installworld
# mergemaster -iU

This step will take a while depending on your connection and processing power, so sit back and relax for a good 30 minutes or longer. Once the compiling is done, we must initiate the encryption module for our jail(s) and tmp to mount properly:

Edit /boot/loader.conf
# vi /boot/loader.conf

Then load the module:

# kldload /boot/kernel/geom_eli.ko

To verify it has loaded successfully:

# kldstat
Id Refs Address Size Name
1 6 0xc0400000 992c30 kernel
2 1 0xc0d93000 111b8 geom_eli.ko -- note geom_eli module
3 2 0xc0da5000 25ff8 crypto.ko
4 2 0xc0dcb000 ab40 zlib.ko
5 1 0xc0dd6000 6a2c4 acpi.ko

We must now encrypt our swap and jail partition(s).

Edit /etc/fstab:
# vi /etc/fstab
Add .eli to our partition, the following example will show you the swap line before encryption and after you apply your change:

Before: /dev/da0s1b none swap sw 0 0
After: /dev/da0s1b.eli none swap sw 0 0

In the end, your fstab should look something like this, where “jail” is the selected jail partition:

# cat /etc/fstab
# Device        Mountpoint    FStype    Options        Dump    Pass#
/dev/da0s1b.eli        none        swap    sw        0    0
/dev/da0s1a        /        ufs    rw        1    1
#/dev/da0s1e        /jail/j1        ufs    rw        2    2
#/dev/da0s1d        /tmp        ufs    rw        2    2
/dev/acd0        /cdrom        cd9660    ro,noauto    0    0

Now a reboot is required to be assured everything loads properly:

# shutdown -r now

Once the system reboots successfully, we need to create the encryption key(s) so we can later mount the jail partition(s):

# dd if=/dev/random of=/home/bsdadmin/key1 bs=64 count=1

And now bind our keyfile(s) with the related jail(s) and set the passphrase:
# geli init -s 4096 -K /home/bsdadmin/key1 /dev/da0s1e

We must now assign our encryption key(s) to the respective partition(s):
# geli attach -k /home/bsdadmin/key1 /dev/da0s1e

At this point, we have successfully created an encryption key and assigned it to a respective partition, and by using ‘geli attach’ we have loaded the encrypted partition, which in this case is /dev/da0s1e.eli

Please note, for increased security, it is not suggested to keep the key file on the system.

Now, as the partition(s) are encrypted we must create new filesystem(s):
# newfs /dev/da0s1e.eli

We can now mount our fully encrypted partition(s), and verify everything was proper:
# mount /dev/da0s1e.eli /jail/j1
# df -h

We must now add our custom jail security and tcp socket tweaks to the host system control.

Edit /etc/sysctl.conf
# jail regarded setting, non default
# jail regarded setting, default
# blocking portscans
# optimize tcp
# socket killer

We must now make our adjustments take effect:

# sysctl -w security.jail.set_hostname_allowed=0
# sysctl -w security.jail.allow_raw_sockets=1
# sysctl -w security.jail.socket_unixiproute_only=1
# sysctl -w security.jail.sysvipc_allowed=0
# sysctl -w security.jail.enforce_statfs=2
# sysctl -w security.jail.chflags_allowed=0
# sysctl -w net.inet.tcp.blackhole=2
# sysctl -w net.inet.udp.blackhole=1
# sysctl -w net.inet.tcp.sendspace=78840
# sysctl -w net.inet.tcp.recvspace=78840
# sysctl -w net.inet.tcp.sendbuf_inc=16384
# sysctl -w net.inet.tcp.recvbuf_inc=16384
# sysctl -w net.inet.tcp.sendbuf_max=16777216
# sysctl -w net.inet.tcp.recvbuf_max=16777216
# sysctl -w net.inet.tcp.inflight.min=6144
# sysctl -w net.inet.tcp.mssdflt=1460
# sysctl -w net.inet.tcp.keepinit=1500
# sysctl -w net.inet.tcp.keepidle=10000
# sysctl -w net.inet.tcp.keepintvl=6000
# sysctl -w net.inet.tcp.always_keepalive=1
# sysctl -w net.inet.tcp.hostcache.expire=1
# sysctl -w net.inet.tcp.delayed_ack=1
# sysctl -w net.inet.tcp.delacktime=100

At this point, our Host System has been prepared and, we can now begin building the jails!
To begin, we need to create a /dev directory that the jails can use, without this
networked services will not run.

# mkdir /jail/j1/dev

And proceed to build the jail(s):

# cd /usr/src
# make installworld DESTDIR=/jail/j1
# make distribution DESTDIR=/jail/j1
# mount -t devfs devfs /jail/j1/dev

And now for some jail configuring, where “bb.bb.bb.bb” is the system IP which you wish the jail to use:

# touch /jail/j1/etc/fstab
# vi /jail/j1/etc/rc.conf

*NOTE* You can also use the primary IP of the server and a separate IP is not required.

We can now start the jail(s) using the jail command with the following command:

# jail "/path/to/jail" "fqdn" "jail_IP" /bin/sh /etc/rc

So to start our jail(s) we would use:

# jail /jail/j1 myjail.domain.tld bb.bb.bb.bb /bin/sh /etc/rc

In order to verify your jail(s) are mounted and to find the JID for each jail, which you will need to access them, you need to use the jls command:

# jls
# jls -v

Should output something like this:

JID  IP Address      Hostname                      Path
1  bb.bb.bb.bb  myjail.domain.tld        /jail/j1

We can now enter a jail using the jexec command and the jails JID.
Enter jail #1:

# jexec 1 /bin/sh
We must now setup our jail environment. We begin by changing the root password for our jail for security reasons.
Change root password:
# passwd

Install ports collection:
# portsnap fetch extract

Install portaudit and portmaster:
# cd /usr/ports/ports-mgmt/portaudit
# make install clean
# /usr/local/sbin/portaudit -Fda
# cd /usr/ports/ports-mgmt/portmaster
# make install clean

Install bash:
# cd /usr/ports/shells/bash
# make install clean
# ln -s /usr/local/bin/bash /bin/bash

Install bash-completion:
# cd /usr/ports/shells/bash-completion
# make install clean

After bash is installed, we can then enter the jail using the bash environment by using the following command, where 1 is the jail ID:

# jexec 1 /bin/bash

Congratulations! At this point, you have successfully completed an encrypted UNIX jail setup.

In order to “shutdown” and unmount the respective jail and it’s partition, the following procedure would have to be followed:

# jail -r 1
# geli detach /dev/da0s1e
# umount /jail/j1/dev
# umount /jail/j1