Experiences upgrading from NimbleOS 1.4.9 to 2.2.5 with VMware NCM

Blog Post created by joebrug on Mar 26, 2015

Just wanted to share my experiences/results with the community. Huge thanks to Nick Dyer for the guidance during our first Nimble OS upgrade!


Setup: CS260 with 1GB iSCSI links, 3 VMware esxi 5.5 hosts.


We've been on OS 1.4.9 for a long time. When the system was configured, our SE used some scripts to set up the multi-pathing to work with VMware using round robin, etc.. this seemed to be working fine, didn't know any better.. but once we upgraded to NOS 2.x using VMware NCM, our IOAnalyzer tests were drastically different!


First, the upgrade: Downloaded 2.1.8 and hit Update, it took probably about 30 minutes or so, but it worked flawlessly. No downtime noticed by anyone, except for me, who had some constant pings set up for the controllers to see when they rebooted. I was about to upgrade immediately to 2.2.5, but noticed that the Hot Spare started reconstructing itself. Found out that this is due to the Raid-3P introduced in Nimble OS 2.x. Was told I could probably go ahead and upgrade to the 2.2.5 OS, but decided to wait until the reconstruct was finished. When that was done (roughly 24 hours), I went ahead and did the upgrade to 2.2.5 during business hours. Again, no one noticed a thing, and it went very smoothly and quick (Much faster than 1.x to 2.x).


After upgrading, I downloaded I/O Analyzer – VMware Labs

I picked a couple of the tests, fairly randomly and compared the results. The tests only ran for 120s (default). The first two tests were from the current setup: OS 2.2.5 without VMware NCM/Automatic Connection mode/Re-balancing. The last two tests were WITH NCM/Automatic/Re-balancing installed/configured/enabled. You'll note quite the difference!


Screenshot of Nimble GUI during Read test without NCM (still using "scripts" from 1.4.x):


Screenshot of Nimble GUI during Read test WITH NCM+Automatic Connection Method+Re-balancing installed/enabled/configured:



The Nimble/VMware setup seems much happier using the NCM with OS 2.x. Obviously, something wasn't configured properly, or things have just gotten that much better with 2.x+NCM. Also, the 4 Ethernet interfaces of the nimble went from maxing out at around 50MB/s to about 120MB/s! Evenly distributed across all 4 interfaces. Much more throughput, and way more IOPS!