I can honestly say I've been itching for this day to come, I've been like a kid peeling off the doors for the advent calendar on the run up to Christmas. I've been in the industry for close to 20 years and throughout that time I have had a heavy involvement with Fibre Channel SAN's, so the promise of putting some old skills to use on what I believe is a truly outstanding storage architecture (which to date has been solely ethernet based) has had me anticipating the Nimble Fibre Channel launch for a long time. So before we get into the details I thought I would answer some basic questions:
Why Fibre Channel and why now?
Since Nimble shipped its first product in 2010, we have seen a dramatic uptake by customers. Our growth has been relentless. Just last quarter we were running close to 4000 customers, adding over 650 in the last 3 months alone! Typically, customers will purchase the platform for a multitude of reasons (Performance/Capacity in a super small footprint, dramatically reducing latency and speeding up applications, reimagining traditional processes like backup, data protection or development ops, our truly world-class support). iSCSI is a great protocol and Nimble performance over Ethernet is unreal, however sometimes I will have a discussion with a prospect that won't entertain iSCSI or IP storage and the answer simply has to Fibre Channel! This can be for a variety of reasons but usually it boils down to three:
- Having an existing infrastructure that is based on Fibre Channel (HBA's and Switches) and the risk, cost and complexity involved in switching to Ethernet makes it unviable.
- Having a skills investment in Fibre Channel.
- Office Politics: The SAN sits with the storage team and we don't wish to have the Network team *splitters* own the SAN or in other words... 'I want to be in control of my own destiny!'.
Each one is a insurmountable argument, no matter how good your technology is, the answer will always be 'Come back when you have Fibre Channel!'... well here we are, Nimble Fibre Channel is here and it's good... real good !
What does Nimble Fibre Channel look like and when is it available ?
The good news here is really Fibre Channel is just a protocol! In the same way I can pick up the phone and dial a number via a PBX system, in largely the same manner as if I dialled a number on my mobile phone and routed that over a GSM network. So all the fundamentals of CASL and Nimble still apply, the management is identical and the Nimble features and implementation are untouched. Of course the physical presentation is different. Where we have 1GbE/10bE adaptors, you now have the ability to specify 16GbE Fibre Channel Target Ports (the number of ports range between the CS300, CS500 and CS700). It should also be noted that Fibre Channel is only supported on the newer CS300, CS500 and CS700 and not the older generation CS200 and CS400 platforms. From an implementation perspective we have implemented the Nimble Array to be based on the ALUA standard. Asymmetric Logical Unit Access. ALUA is a standard and in simple terms allows a storage device to communicate to an operating system which paths are preferred, on both a port by port basis and a volume by volume basis. This communication is an ongoing conversation to allow the most optimal paths to form a connection and for data to flow. Nimble OS 2.2.2 adds support for Fibre Channel and is currently Release Candidate designation.
I really have to hand kudos to Nimble Engineering team as not only is the implementation been executed flawlessly but they've actually made it really simple to deploy Fibre Channel, in many ways I think it's even easier than iSCSI! Hopefully you will see this over the coming days by reading this blog.
Setup and Install
Setting up the array is effortless. First rack and stack your array, put power and network to it and turn it on. After that you will need to download the Nimble Windows Toolkit or Discover the Array via Safari (OSX users). Once discovered you'll be faced with the End User License Agreement (EULA), 'read it' and accept it and then we are ready to start the install.
The first screen will show you the array Serial Number, the type of the array and the Nimble OS version (as mentioned above 2.2.2 is the first version of Nimble OS that supports FC). Then you want to select whether you are joining a Group or Setting up a new group:
Next you will be asked for the Array's Name and the Group's name; In my example below, my array is the Fibre Channel Array for the UK System Engineering Team, you'll also be asked for the Management IP and Network details and a password (no different to a conventional install so far!):
The next screen asks us for our subnet designation; In 2.2.2, we are forced to create a secondary subnet (here I am creating it for a replication network). Note: This requirement is something that is likely to be disappear in 2.2.3.
Next we see out first glimpse of Fibre Channel, we are now asked to assign the subnets in the previous step to the specific adaptors. At this point we can now see there are 8 individual Fibre Channel targets / ports in the array. I have set my management interface to eth1, my replication traffic to eth2 (currently unplugged which is why that port is red). I can then see that all my FC Target Adapters are plugged in and have negotiated to 8Gb (my switch was only a 8Gb switch so the 16Gb ports have auto-negotiated down to 8Gb).
Finally I am asked to set the Diagnostic IP address of each controller.
The cabling best practices to the fabrics should be followed and these are largely the same as the Ethernet standards.
In a dual fabric architecture, cable the same sibling ports on each controller to go to the same fabric. Such as:
fc1 (on Controller A and B) > Fabric A
fc2 (on Controller A and B) > Fabric B
fc3 (on Controller A and B) > Fabric A
fc4 (on Controller A and B) > Fabric B
If you know your switch's architecture well you may choose to balance the target ports across ASIC's or different Edge switches (if there is some benefit of locality of access).
In a single fabric again you may look at the distribution across ASIC's and Core/Edge designs but remember the Switch (and more importantly the Zoning!) will be a single point of failure in a single fabric topology.
Finally complete the install by undertaking the usual tasks of setting DNS, Timezone and Email Alerting/AutoSupport:
As you can see the setup of the Fibre Channel array is in someway much easier than the iSCSI array, as no explicit Data IP addresses or networking needs to be considered, it's just purely setting the Management IP Address and supporting monitoring framework. So going from a box in the datacenter, racked and stacked and configured is likely to take literally minutes!
How does FC look in the Nimble GUI?
As one would expect, the Nimble OS GUI is largely unchanged. There are some slight differences where Fibre Channel is concerned.
Clicking on the Manage > Array will show you an overview of the array and you will see the status of each of the FC Target Ports installed in the controller. As with ethernet, hovering over them will show their state, their physical location on the back of the array and their associated WWPN (World Wide Port Name).
Clicking on Administration > Network Configuration (and then Clicking the Active Settings) will show you a list of all the Fibre Channel Targets installed, which controller owns which adaptors, the link status, World Wide Node Names and World Wide Port Names (useful for when you plan to configure zoning and fabric aliases).
Clicking Edit here will allow you to Offline a port for any reason.
Note: Hovering over the link status will give you the switch name and the port that the target port is plugged into. A really nice touch!
Clicking on Manage > Initiator Groups
Allows a user to define Initiator Groups, these allow us to define which hosts can access which volumes (basic LUN masking). Clearly rather than seeing IQN's here (iSCSI identifiers), you will see World Wide Port Name's. There is a really nice feature here which I will show you tomorrow!
Finally, clicking on Monitor > Interfaces will show you the status and performance of each of the Ethernet and Fibre Channel ports.
For those that prefer Command Line there are a number of Fibre Channel specific CLI's - just type fc --help and you will get a full paged list of commands and options.
The following is a video of the installation process of a Nimble Fibre Channel array:
If you have any questions then please feel free to post below. Tomorrow we will look at basic LUN provisioning within a Windows environment.