Ultimate File Server: OpenSolaris and ZFS

When picking out a solution for your central file server, there is of course many options. None of them quite stack up to the ease of administration and redundancy, out of the box, as OpenSolaris and ZFS provide. With ZFS you can build cheap storage arrays with disks of varying size and different levels of redundancy. For this setup I'm going to go with a basic raidz configuration using 4x 1TB SATA drives.

A quick note on ZFS

For those of you aware of ZFS, you can probably go ahead and skip this section, but for those of you that don't I'll give a quick description of how things work in the ZFS world. Instead of partitions and volumes, ZFS gives you pools and datasets. The zpool command gives you control of your pools and zfs command gives you control over your datasets. There is almost no limit to the amount of ZFS datasets you can create and you shouldn't be scared to create new ones for almost everything. Each dataset created allows you to perform individual snapshots, set different share options (iscsi, nfs, etc), apply quotas, compression, and even level of redundancy for those extra important files. These datasets work in a hierarchical format, meaning that any subsets will inherit from their master. Take a look at the below as an example.

$ pfexec zfs create tank/shares
$ pfexec zfs set sharenfs=on tank/shares
$ pfexec zfs create tank/shares/music

The dataset, tank/shares/music, will inherit the sharenfs=on option applied to tank/shares. So it makes sense to create different levels of datasets. Put like datasets inside the same master dataset to easily control options for them all.

Get your ZFS raidz on

Depending on the value of your data you may wish to consider raidz2 but I'm going with the basic single parity raidz which allows for better write performance and is similar to a RAID-5 configuration. For more information on raidz you may wish to check out these RAIDZ configuration requirements and recommendations.

$ pfexec zpool create raidz tank c7d1 c8d0 c8d1 c10d1
$ zpool status tank
  pool: tank
 state: ONLINE
status: The pool is formatted using an older on-disk format.  The pool can
        still be used, but some features are unavailable.
action: Upgrade the pool using 'zpool upgrade'.  Once this is done, the
        pool will no longer be accessible on older software versions.
 scrub: none requested
config:

        NAME        STATE     READ WRITE CKSUM
        tank        ONLINE       0     0     0
          raidz1-0  ONLINE       0     0     0
            c7d1    ONLINE       0     0     0
            c8d0    ONLINE       0     0     0
            c8d1    ONLINE       0     0     0
            c10d1   ONLINE       0     0     0

errors: No known data errors

Pro tip: iostat -En will give you a list of all your disks.

The first line creates the raidz pool called "tank" using the four disks (c7d1, c8d0, c8d1, c10d1). The second one gives you a nice overview of health for your new pool. It will also tell you if you should upgrade your zpool to a newer version, which as you can see, needs to be done on my storage pool. I created it awhile ago and haven't had the time to upgrade just yet. The man pages for zpool and zfs provide a wealth of information and are good way to learn the ins and outs of maintaining a ZFS system.

NFS on ZFS

This is ridiculously easy to setup. There's no need for touching the exports file, especially since Solaris does not use /etc/exports as other Linux and BSD derivatives do. The following will set you up with a basic export to the 192.168.1.0/24 network and provide read/write access.

$ pfexec zfs create -p tank/share/movies
$ pfexec zfs set sharenfs='rw=192.168.1.0/24' tank/share
$ zfs get -r sharenfs tank/share
NAME               PROPERTY  VALUE              SOURCE
tank/share         sharenfs  rw=192.168.1.0/24  local
tank/share/movies  sharenfs  rw=192.168.1.0/24  inherited from tank/share

For those of you wondering, -r means recursive and can be applied in many different zfs sub-commands. If you need to disable root squashing (not recommended) you can do it per individual IP as follows:

$ pfexec zfs set sharenfs='rw=192.168.1.0/24,root=192.168.1.30/32' tank/share

This will only disable root squashing on 192.168.1.30. One final example, what if you need read only to a different network?

$ pfexec zfs set sharenfs='rw=192.168.1.0/24,root=192.168.1.30/32,ro=10.0.0.0/8' tank/share

Final notes

The real beauty of ZFS is that there is so much control over your file system. At first it may come off as confusing, but stick with it and you will appreciate an OpenSolaris file server over any Linux, *BSD, or Windows solution. To take this a step further, check out serving up iSCSI targets from ZFS, and with the proper network configuration, you can use these targets for KVM virtualization.

Tags: