by Jason Monger


Microsoft ODX, or Offloaded Data Transfer, is a feature available in Windows 2012 and 2012R2 that offloads the process of copying data to a storage array. Nimble OS 2.3 adds support and compatibility with this feature.


Why is ODX important to our customers? When a file copy is initiated in Microsoft Windows, the host must read the data from the source disk. This traverses the IP network (iSCSI) or fabric (FC), and then is written to the destination disk from the host, again traversing the network or fabric. This process therefore uses network / fabric bandwidth and host CPU cycles to complete. With ODX compatible Microsoft operating systems and storage systems that support ODX, this process can be entirely offloaded to the storage array, thus freeing network / fabric and host CPU resources.


Under the covers the ODX process uses a token system; the Windows OS checks whether the storage is compatible and if it is instructs it to copy the data from source to destination location and provides the storage array with a token. The Windows OS uses this token to check with the storage array how the data copy is progressing until it is complete.

 

Think of the scenario where you wish to copy large quantities of data from disk A to disk B (or a copy/clone of a Virtual Machine in a Hyper-V environment). If the infrastructure has bottlenecks, either network, fabric, or host, this large data copy might have a detrimental effect on applications running on the host where they compete for the shared resources. In simple terms this might add latency to the environment and reduce application performance and therefore the user experience. With ODX these shared resources aren’t affected as the large data copy is done at the array level.

There are no special tools required from a client/application perspective to leverage ODX; only a compatible Microsoft OS and Nimble Storage array. If for any reason ODX doesn’t work, the copy will simply revert to the normal copy method via the host. 

Let’s take a look at this in action.

The test infrastructure is a host server, running Windows 2012R2, which is FC (fibre channel) attached to a Nimble Storage array. I created two disks on the Nimble array to copy the data between and brought them online, initialized, and created a simple volume on each. Note copying data inside a single disk won’t use ODX as Microsoft won’t actually copy data, it will simply update NTFS references with the second copy of data.

I then disabled ODX on the host using the following PowerShell:

Set-ItemProperty hklm:\system\currentcontrolset\control\filesystem –Name “FilterSupportedFEaturesMode” 1

*Set this to 1 to disable and 0 to enable ODX on the host server. The default is enabled.


Then I copied ~16GB of data from the source disk (J:\) to the destination disk (K:\) using Windows Explorer:

 

Whilst doing this I monitored the source disk reads, destination disk writes, and host CPU in Windows Performance Monitor and also monitored performance in the array UI:

Array performance:

Disk Read:                                                                           Disk Write:

Host CPU:

So the copy started at 12:58 and ended just over a minute later. The maximum read throughput was 493MB/s, write throughput 443MB/s, and host CPU hit 22% utilized (nothing else was happening on the host).

 

OK that’s my baseline, next I re-enabled ODX on the host again using PowerShell:

 

Note that enabling on the host doesn’t always take effect immediately. In some tests if I started the copy straight away it didn’t use ODX however in normal operations you’re not going to change this from the default value of 0 (enabled). So if you’re testing this yourself just wait a minute before kicking off the copy after enabling ODX again.

 

And here is the same set of monitoring captures with ODX enabled:

Array Performance:

Disk Reads:                                                                      Disk Writes:                                            

Host CPU:

 

This time, with ODX enabled, there is no data being read or written across the fabric and through the host. Instead there is minimal management traffic (token checking) between the host and array. Maximum read of 11MB/s, write of 20MB/s, and CPU utilization generally around 4%.

In my test above I copied between two disks attached to the same physical host. But what if you want to copy between two Windows servers which both support ODX? Well that works too! There is one caveat I would point out here and that is if the servers are virtualized on a hypervisor. If for example you were running the servers on VMware ESX copying data between the servers over a SMB share, would that work? Yes if the disks (Nimble volumes) are connected directly to the guest via the Microsoft initiator, but no if they are connected as RDMs. In the case of RDMs it doesn’t work because you are using the VMware initiator to connect to the volumes and not the Microsoft initiator.

In summary, using ODX will remove contention for host and fabric resources, and reduce CPU utilization as well. Your production workloads therefore will not have to contend with a large data copy across the network or between disks.

My experience of using ODX with Nimble Storage? Seamless, it just works!


More information can be found in the Windows Integration Guide for 2.3 here:

https://infosight.nimblestorage.com/InfoSight/cgi-bin/downloadFile?ID=documents/pubs_windows_integration_guide_2_3.pdf

If you have any questions or comments please leave them below.