Open Source Storage Area Network–Part 2

In my previous article, I discussed various options for the Operating System for my Storage Area Network. In this article, I will discuss my choice in more detail. Below you can find links to other parts of this guide.

Operating System

Just to recap from my previous article, I chose to build my Operating System from the distribution up. For me, this provided the most flexibility of all the options discussed. I chose to stick with Ubuntu, but any Linux distribution that has the Kernel 3.5 should contain all the code built in to function as a SAN (More on that later). I will be installing a few other packages on top of this Operating System, but for the most part, it will remain a vanilla install. Here are some other distributions with the required kernel:

  • OpenSuSE
  • Fedora

It is important to note that although the use of a graphical interface can be used, my guide will not require any UI other than a terminal and therefore function from a vanilla install of a server edition of Ubuntu.

There are many reasons that I prefer Ubuntu over other flavors of Linux. For me Ubuntu has been easy from both a desktop and a server standpoint. It has also worked well for me on hypervisor products like ESXi and Hyper-V making it great for testing. Security wise I like how Ubuntu defaults all users to a non-root level terminal. It helps make fewer mistakes when typing commands on the terminal. It also means that logging in is a little more secure as the root account never gets used directly. Ubuntu is also very good at staying up to date with regular releases and packages.

Downsides to using Ubuntu is the use of Debian packages instead of RPMs. This can make some tools more difficult to get running outside of the packages mentioned below. There might also be some syntax differences or path differences between packages. Each flavor of Linux prefers some folders over others.


Once my Operating System is installed and functioning, I needed several software packages to provide functions of our SAN. Some of these packages can be optional and there may be other packages you want to add to this list such as RAID management software. I will not cover these as different products have different tools.


LIO is already technically installed and stands for Linux IO. Linux IO was developed by a company called Datera (formerly Rising Tides). Because this package is in the kernel, I do not need to install it, but you need to understand it. LIO handles communication for all the major storage protocols for Linux. This includes:

  • Fibre Channel
  • Fibre Channel over Ethernet
  • iSCSI

LIO in itself cannot be managed without calling the APIs for configuration. It also does not handle features like replication or snapshots. This is where some of the other packages come into play. LIO does not include functionality for NFS and CIFS. Both of these are handled by other products.


TargetCLI is a free product also developed by Datera for LIO SAN management. They offer a feature rich Operating System that offers much more functionality, but has a yearly maintenance subscription. If you are looking to build this SAN for a small or medium business, I highly recommend looking at the route of the subscription. Because RTS OS requires a subscription, I will use their free tool for basic functionality.

OpenSSH Server

Because I are using this Operating System as a bare metal SAN, I need some way to manage the product remotely. You could connect a crash cart up and configure it as needed, but the choice is up to you. SSH gives me the ability to secure administer the SAN with a remote terminal window. There are free tools for various Operating Systems that will let you connect to an SSH server. In this guide, I will mostly be using PuTTY.


Because I am connecting using Fibre-Channel and iSCSI, I will not go into detail on setting up NFS. If you are following along and using Ubuntu, I recommend checking out this article for more information on setting up the NFS server. NFS can work outside LIO or beside LIO depending on your configuration. You can use LIO to connect a remote iSCSI system up and then use NFS to export paths should you choose.


I am also not going into detail on CIFS or SMB as well, but installing Samba will allow you to make your SAN appear on the network as a normal file server for Windows machines. Just like NFS, Samba can share folders based on configurations of LIO.


In this article I discussed the software aspect of our SAN. This included the Operating System and the supporting software to turn the system into a SAN. In our next article, I will be discussing the hardware I am using for this SAN.

Important Links

Finally, a good friend of mine has an online book on how to use Linux as well as tips for setting up various services such as NFS and Samba:

Open Source Storage Area Network–Part 1

In this multi-part blog post, I will be going over setting up a custom built Storage Area Network (SAN). In the commercial world, there are many different storage options from companies like NetApp, EMC, and Dell. Each of these companies has great strength when it comes to a SAN, but all have a downside when it comes to a small business or an IT professional wishing to learn more about the technology. That is that they cost a lot of money to acquire, set up, and support. There are some situations where spending a large amount of money may not be possible. That is where this guide comes in.

Like many IT professionals, I want to play around with various technologies at home, but do not have the budget to buy the latest or the greatest all the time. A SAN is one of these technologies in particular. I started looking into options to get as close to a professional SAN as possible. I discovered that there are many options out there that provide many of the features that commercial SAN solutions provide. In the discovery process, I found that some of the options would also work for a small business or test lab for a company.

One of the most important things to building a SAN is the hardware, but in this part of the guide, I will go over the Operating System. The Operating System determines the functionality that will be available such as iSCSI, CIFS, SMB, Fibre Channel, etc… I will only go over a few in this article.

Operating Systems

The few free and open source Operating Systems I have come across are listed below. Each one has some strengths and weaknesses. Please note that for this project, I am comparing these options for the purposes of a SAN and not a NAS. The difference between the two is outside the scope of this guide. As such, there are some features of these systems that are not covered under my review.

I have not installed or used most of these systems, but base all my reviews off the feature list, screenshots, and views of others on the internet. There are also many non-free operating systems that are much better, but the purpose of this guide is to find an open source solution.


OpenFiler is a very feature rich SAN Operating System. Unfortunately, there are plugins that enhance functionality that are not free. OpenFiler is a great product for an experienced IT professional who is just getting started with SANs and doesn’t know a lot about Linux, but wants full featured capabilities from their SAN including high availability.

I give OpenFiler a difficulty of intermediate because it still requires knowledge about iSCSI configurations and use of the CLI to configure settings for Fibre Channel. Throughout the web interface there are references to Linux terminology. I give OpenFiler a rating of 2 because it is a very straightforward product, but it is based on a dated Linux distribution called rPath and it seems to have stagnated in development because rPath is no longer maintained.


  • iSCSI
  • Fibre Channel
  • CIFS
  • NFS
  • Snapshots
  • Web Management Interface
  • High Availability


  • Community Supported or Paid Support
  • Fiber Channel requires CLI configuration or purchase of plugin
  • Linux 2.6 Kernel (Latest for Linux is 3.5+)
  • rPath Linux is no longer distributed or supported


FreeNAS is a fantastic Operating System for network storage. There are great features such as snapshots and thin provisioning. The interface is very intuitive and easy to use and FreeNAS is constantly being worked on and updated. For the average user, everything should be very manageable from the web interface.

FreeNAS is also given a difficulty of intermediate because you still end up needing to know iSCSI and other various Unix style terminology when setting up new disks and shares. I give FreeNAS a rating of 3 because it has a fantastic interface, but it is based on FreeBSD and that means a little extra work for someone who has never used FreeBSD. This is only because some of the commands and programs for management are slightly different from Linux distributions.


  • Thin Provisioning
  • NFS
  • iSCSI
  • Support for 10GbE Network Cards


  • Uses istgt for iSCSI (Explained in part 2)
  • Based on FreeBSD
  • Community Supported or Paid Support


OpenMediaVault is a feature rich NAS Operating System. There is a plugin for iSCSI targets that allows it to be a SAN, but it takes some work to get it installed. It would be great as a NAS box or using NFS for VMWare, but other users warn that the iSCSI implementation could be a lot better or that they have issues with iSCSI and VMWare.

I give OpenMediaVault a difficulty of Intermediate because the GUI has a vast number of settings and uses Linux terminology. I give it a rating of 2 because of their limited support for being a SAN as opposed to a NAS. iSCSI is limited and requires more advanced configurations to install compared to other plugins.


  • Granular ACLs on Shares
  • NFS
  • iSCSI


  • iSCSI Requires Complex Setup 
  • Community Supported

Ubuntu Server 12.10

Using Ubuntu Server, I can customize the packages that I would like. This allows me to have complete control over the features that I need. It also means that I can reduce the footprint and optimize the speed of operations. Although I chose Ubuntu, there are other options like OpenSuSE and Fedora. The only requirement for this option (using this guide) is Linux with kernel version 3.5 or newer.

This option is the most advanced option because we are installing all the required features after installing the operating system. There is no management interface other than the command line. Theoretically I can install a GUI and do some options, but for the most part, this system will be managed with SSH. I give this option a rating of 4 because it is completely tailored to my specific needs and allows me to choose the best packages for the features I need. The downside is that I require the most knowledge of all the options to manage the system.


  • Purpose Customized Operating System
  • iSCSI Targets
  • Fibre Channel over Ethernet
  • Fibre Channel


  • Requires Time to Set Up 
  • Community Supported (Per Package)


For my purposes, I decided to go with building out a SAN based off of Ubuntu. I could have picked any flavor of Linux as long as it runs at least the 3.5 kernel. I chose Ubuntu in specific because that is the flavor of Linux I have the most experience with.

Following my guide, you should expect to have a fully functioning SAN with features like iSCSI and Fibre Channel that you can use to run virtual machines or host mass quantities of storage. From there you can then connect to VMWare or Hyper-V among other systems.

In my next article, I will go over the requirements for building out the OS in more detail along with the packages I am using. I encourage everyone reading this article to look at all the various options before settling on a specific option.