by Nick Dyer
For many Nimble customers, one of the most exciting enhancements in Nimble OS 2.3 is the addition of all-flash service levels, also known as volume pinning. Implementing this feature is straightforward; let’s take a quick look at how it’s done.
First, there are one or two calculations you should note. In particular, the all-flash service levels capability is only available if your Flash-To-Disk ratio (FDR) is greater than 4% - that is your usable flash capacity within the system is greater than 4% of your total usable disk capacity. If your FDR is less than 4%, you will need to add more flash to your system in order to have available space for all flash volumes to reside.
It’s possible to set volumes to the “All Flash Service Level” within the Web UI, in the CLI, or even via our new REST API command set. It is also possible to set this characteristic according to when volumes were created, or retrospectively on volumes that already reside on the Nimble array.
In the above screenshot you can see that there is a new tab called “Performance” on the volume creation screen. By default, the “normal” radio button is selected – which is the Auto Flash Service Level that CASL uses by default and is what delivers us a ~97% average cache hit. However, we are now able to effectively pin full volumes within the flash space by clicking the “Pinned” radio button.
Pinning a volume within the flash cache will copy every block of data written to the volume into flash This could be deemed wasteful for flash capacity, as we’re now copying cold or stale blocks into the finite resource of flash. This is however very useful for guaranteeing an All Flash Service Level for specific volumes which must always have sub-millisecond read experience, and thus are willing to have every block reside in flash. Note: the process of pinning a volume into or out of All Flash mode is completely dynamic and non-disruptive.
In the example below, I have created a 100GB volume on the array called ESX-AllFlash, and you can see that it will reserve a full 100GB of space within the available pinnable flash capacity (ie space not being used for CASL's current flash usage) to allocate to this volume, because the default “Volume Quota” is 100%. If I were to alter the quota to 50%, you can see it has dropped to 50GB reserved in flash. Whilst we're still reserving space within the flash, the reservation will only be enforced when needed, meaning that in 99% of the time we are still over provisioning the usage of flash even when using All Flash Service Levels.
One important caveat: using quotas restricts the use of flash allocation, but may have repercussions on the capacity growth of the volumes, so it’s important to adjust your warning level of the volume to take that into account!
Most Nimble users will find it works best to use the standard Auto Flash Service Level by default for 99% of their volumes, and only use pinning when needing to define a full All Flash experience for specific workloads. Otherwise, there’s a risk of needing more flash for wasted or unused space.
It’s important to note that Nimble has not changed the way that CASL writes and organises data. Incoming IO is still always coalesced through NVRAM and DRAM, then sequentialised on NL-SAS drives. However, if you select to pin a volume, the array will inline copy all incoming data to flash, as well as storing the authoritative copy of the data on disk. You still do not need to RAID-protect or worry about flash erosion or failure, as you’re not keeping the primary copy of data on this finite resource.
You can also retrospectively pin volumes in flash to take them from a standard caching volume to an All Flash volume whenever you like. This can be done within the UI or the CLI, but not with the vCenter plugin (for what I hope are obvious reasons – giving the power for all flash selection to a non-storage user is very dangerous!).
In the UI you do this by editing the volume, and selecting the “Pinning” radio button, just like on the New Volume wizard:
We can also see if a volume is pinned or not by looking at its credentials in the Volume Details screen:
If you’ve specified to retrospectively pin a volume into flash, the array will inline copy all originally written data up to flash as part of a backend workflow. It will also copy any new writes to flash, regardless of whether they’re cache-worthy or not (or even whether they’re random or sequentially written).
Once a volume has been set to “pinned” and CASL has copied all the necessary blocks to the flash, we will see a notification also within the Monitor > Performance tab for that volume that the volume now fully resides with flash:
One really cool feature when considering All Flash Service Levels are our Zero-Copy Clones functionality. Nimble’s "thin" cloning technology allows us to create multiple, full read/write volumes that reference the original blocks of parent volumes using metadata points, without the need to duplicate the data on the backend array. This means we can create many clones of a single volume that is fully pinned into flash, and deliver an all-flash experience to multiple volumes while at the same time not wasting flash capacity with duplicated data! A great example of this could be VDI images, or even Windows Server images – we could create ‘sysprepped’ gold images of VMs that reside within a master datastore which has been defined with an All Flash Service Level, and then clone out multiple datastores to reference those images – which would all point back to the original blocks which reside within flash. Of course any differential changes we make to those clones will not be pinned by default and have the Auto Flash experience, but for most base images, this is completely acceptable!
Finally, here's a nice little change we've made in the Performance Report within Infosight - we now show you how much flash capacity is currently being used for pinned and unpinned volumes and how much is needed as you scale:
To summarise, the new All Flash Service Levels within Nimble OS 2.3 allows us to deliver a full all-flash experience to specified, individual workloads within the context of a mixed-IO working set on our enterprise arrays. This means it’s now possible to consolidate highly consuming IO applications as well as highly consuming capacity workloads without compromising on performance, availability, scalability or management. All this is under under a single user interface, allowing Nimble to deliver a unified, consolidated enterprise storage experience.
Below is a video on how simple and easy it is to set All Flash Service Levels when provisioning a volume and adding or removing it retrospectively (note: no sound on the video).