As most of us lean towards security and stability, I’m a big fan of FreeBSD and it’s many capabilities.
The following article will cover creating a Geometry Eli encrypted UNIX jail in FreeBSD 8.1 using a predefined partition, and it assumes you have already created your slices and partitions as per your needs. This procedure is quite useful if you are running an enterprise server and wish to rest assured that if the drive is pulled out of the rack, your information will be safe from prying eyes. Jailed environments, or sub-systems, are also useful for your development needs as the jail is secluded from your host system and the environment can crash without effecting the entire system.
To begin, we first need to edit /etc/fstab and comment out our intended encrypted jail partition(s) and /tmp.
Depending on the size of our /tmp filesystem, the tmpsize in /etc/rc.conf will vary and the contents should read as follows, replacing aa.aa.aa.aa with the primary IP of the server:
Edit /etc/rc.conf:
# vi /etc/rc.conf
sendmail_enable="NO"
sendmail_submit_enable="YES"
sendmail_submit_flags="-L sm-mta -bd -q30m -ODaemonPortOptions=Addr=localhost"
sendmail_outbound_enable="YES"
sendmail_outbound_flags="-L sm-queue -q30m"
sendmail_msp_queue_enable="YES"
syslogd_flags="-ss -b aa.aa.aa.aa"
tmpmfs="YES"
tmpsize="1024m"
tmpmfs_flags="-S"
We now need to make sure SSH only listens on the host system’s IP address as we do not want to allow any SSH access to the encrypted jail(s). For added security we restrict root from directly logging in.
Edit /etc/ssh/sshd_config:
# vi /etc/ssh/sshd_config
ListenAddress aa.aa.aa.aa:22
PermitRootLogin no
Now, we now need to set the system administrator mail alias.
Edit /etc/mail/aliases:
# vi /etc/mail/aliases
root: [email protected]
Now, we have to make the new aliases take effect.
# newaliases
We now need to grab the latest binaries for the FreeBSD Host System and install the ports collection.
# freebsd-update fetch install
# portsnap fetch extract
Once the binaries are done updating and the ports collection is installed we need to install portaudit and portmaster for easy management of security vulnerabilities and the ports collection itself.
# cd /usr/ports/ports-mgmt/portaudit
# make all install clean
# cd /usr/ports/ports-mgmt/portmaster
# make install clean
# rehash
Now run portaudit for the first time:
# portaudit -Fda
Usually, it will state that you have 0 security vulnerabilities if it shows otherwise we MUST update using portmaster:
# portmaster -a
Congratulations! The Host System has now been prepped. We can now begin to prepare the userlands for our jails and further configure the Host System for jail encryption and security.
Create /etc/standard-supfile in order to create our userland binaries. Please note, the release will vary depending on the version of FreeBSD you currently have installed and the following assumes FreeBSD 8.1 is currently being used:
# vi /etc/standard-supfile
*default host=ftp.uk.freebsd.org
*default base=/var/db
*default prefix=/usr
*default release=cvs tag=RELENG_8_1
*default delete use-rel-suffix
*default compress
src-all
Now, we must gather our source code, and begin buildworld for our userlands:
# csup /etc/standard-supfile
# cd /usr/src
# make buildworld
# mergemaster -p
# make installworld
# mergemaster -iU
This step will take a while depending on your connection and processing power, so sit back and relax for a good 30 minutes or longer. Once the compiling is done, we must initiate the encryption module for our jail(s) and tmp to mount properly:
Edit /boot/loader.conf
# vi /boot/loader.conf
geom_eli_load=”YES”
Then load the module:
# kldload /boot/kernel/geom_eli.ko
To verify it has loaded successfully:
# kldstat
Id Refs Address Size Name
1 6 0xc0400000 992c30 kernel
2 1 0xc0d93000 111b8 geom_eli.ko -- note geom_eli module
3 2 0xc0da5000 25ff8 crypto.ko
4 2 0xc0dcb000 ab40 zlib.ko
5 1 0xc0dd6000 6a2c4 acpi.ko
We must now encrypt our swap and jail partition(s).
Edit /etc/fstab:
# vi /etc/fstab
Add .eli to our partition, the following example will show you the swap line before encryption and after you apply your change:
Before: /dev/da0s1b none swap sw 0 0
After: /dev/da0s1b.eli none swap sw 0 0
In the end, your fstab should look something like this, where “jail” is the selected jail partition:
# cat /etc/fstab
# Device Mountpoint FStype Options Dump Pass#
/dev/da0s1b.eli none swap sw 0 0
/dev/da0s1a / ufs rw 1 1
#/dev/da0s1e /jail/j1 ufs rw 2 2
#/dev/da0s1d /tmp ufs rw 2 2
/dev/acd0 /cdrom cd9660 ro,noauto 0 0
#
Now a reboot is required to be assured everything loads properly:
# shutdown -r now
Once the system reboots successfully, we need to create the encryption key(s) so we can later mount the jail partition(s):
# dd if=/dev/random of=/home/bsdadmin/key1 bs=64 count=1
And now bind our keyfile(s) with the related jail(s) and set the passphrase:
# geli init -s 4096 -K /home/bsdadmin/key1 /dev/da0s1e
We must now assign our encryption key(s) to the respective partition(s):
# geli attach -k /home/bsdadmin/key1 /dev/da0s1e
At this point, we have successfully created an encryption key and assigned it to a respective partition, and by using ‘geli attach’ we have loaded the encrypted partition, which in this case is /dev/da0s1e.eli
Please note, for increased security, it is not suggested to keep the key file on the system.
Now, as the partition(s) are encrypted we must create new filesystem(s):
# newfs /dev/da0s1e.eli
We can now mount our fully encrypted partition(s), and verify everything was proper:
# mount /dev/da0s1e.eli /jail/j1
# df -h
We must now add our custom jail security and tcp socket tweaks to the host system control.
Edit /etc/sysctl.conf
# jail regarded setting, non default
security.jail.set_hostname_allowed=0
security.jail.allow_raw_sockets=1
# jail regarded setting, default
security.jail.socket_unixiproute_only=1
security.jail.sysvipc_allowed=0
security.jail.enforce_statfs=2
security.jail.chflags_allowed=0
# blocking portscans
net.inet.tcp.blackhole=2
net.inet.udp.blackhole=1
# optimize tcp
net.inet.tcp.sendspace=78840
net.inet.tcp.recvspace=78840
net.inet.tcp.sendbuf_inc=16384
net.inet.tcp.recvbuf_inc=16384
net.inet.tcp.sendbuf_max=16777216
net.inet.tcp.recvbuf_max=16777216
net.inet.tcp.inflight.min=6144
net.inet.tcp.mssdflt=1460
# socket killer
net.inet.tcp.keepinit=1500
net.inet.tcp.keepidle=10000
net.inet.tcp.keepintvl=6000
net.inet.tcp.always_keepalive=1
net.inet.tcp.hostcache.expire=1
net.inet.tcp.delayed_ack=1
net.inet.tcp.delacktime=100
We must now make our adjustments take effect:
# sysctl -w security.jail.set_hostname_allowed=0
# sysctl -w security.jail.allow_raw_sockets=1
# sysctl -w security.jail.socket_unixiproute_only=1
# sysctl -w security.jail.sysvipc_allowed=0
# sysctl -w security.jail.enforce_statfs=2
# sysctl -w security.jail.chflags_allowed=0
# sysctl -w net.inet.tcp.blackhole=2
# sysctl -w net.inet.udp.blackhole=1
# sysctl -w net.inet.tcp.sendspace=78840
# sysctl -w net.inet.tcp.recvspace=78840
# sysctl -w net.inet.tcp.sendbuf_inc=16384
# sysctl -w net.inet.tcp.recvbuf_inc=16384
# sysctl -w net.inet.tcp.sendbuf_max=16777216
# sysctl -w net.inet.tcp.recvbuf_max=16777216
# sysctl -w net.inet.tcp.inflight.min=6144
# sysctl -w net.inet.tcp.mssdflt=1460
# sysctl -w net.inet.tcp.keepinit=1500
# sysctl -w net.inet.tcp.keepidle=10000
# sysctl -w net.inet.tcp.keepintvl=6000
# sysctl -w net.inet.tcp.always_keepalive=1
# sysctl -w net.inet.tcp.hostcache.expire=1
# sysctl -w net.inet.tcp.delayed_ack=1
# sysctl -w net.inet.tcp.delacktime=100
At this point, our Host System has been prepared and, we can now begin building the jails!
To begin, we need to create a /dev directory that the jails can use, without this
networked services will not run.
# mkdir /jail/j1/dev
And proceed to build the jail(s):
# cd /usr/src
# make installworld DESTDIR=/jail/j1
# make distribution DESTDIR=/jail/j1
# mount -t devfs devfs /jail/j1/dev
And now for some jail configuring, where “bb.bb.bb.bb” is the system IP which you wish the jail to use:
# touch /jail/j1/etc/fstab
# vi /jail/j1/etc/rc.conf
network_interfaces=""
defaultrouter="bb.bb.bb.bb"
sshd_enable="NO"
nfs_server_enable="NO"
rpcbind_enable="NO"
mountd_enable="NO"
sendmail_enable="NONE"
syslogd_flags="-ss"
*NOTE* You can also use the primary IP of the server and a separate IP is not required.
We can now start the jail(s) using the jail command with the following command:
# jail "/path/to/jail" "fqdn" "jail_IP" /bin/sh /etc/rc
So to start our jail(s) we would use:
# jail /jail/j1 myjail.domain.tld bb.bb.bb.bb /bin/sh /etc/rc
In order to verify your jail(s) are mounted and to find the JID for each jail, which you will need to access them, you need to use the jls command:
# jls
or
# jls -v
Should output something like this:
JID IP Address Hostname Path
1 bb.bb.bb.bb myjail.domain.tld /jail/j1
We can now enter a jail using the jexec command and the jails JID.
Enter jail #1:
# jexec 1 /bin/sh
We must now setup our jail environment. We begin by changing the root password for our jail for security reasons.
Change root password:
# passwd
Install ports collection:
# portsnap fetch extract
Install portaudit and portmaster:
# cd /usr/ports/ports-mgmt/portaudit
# make install clean
# /usr/local/sbin/portaudit -Fda
# cd /usr/ports/ports-mgmt/portmaster
# make install clean
Install bash:
# cd /usr/ports/shells/bash
# make install clean
# ln -s /usr/local/bin/bash /bin/bash
Install bash-completion:
# cd /usr/ports/shells/bash-completion
# make install clean
After bash is installed, we can then enter the jail using the bash environment by using the following command, where 1 is the jail ID:
# jexec 1 /bin/bash
Congratulations! At this point, you have successfully completed an encrypted UNIX jail setup.
In order to “shutdown” and unmount the respective jail and it’s partition, the following procedure would have to be followed:
# jail -r 1
# geli detach /dev/da0s1e
# umount /jail/j1/dev
# umount /jail/j1
