8 Replies Latest reply: Aug 18, 2014 7:07 PM by Steve Galbincea RSS

    Performance on Large-Quantity, Small-Size Files

    wilbert Newbie

      Hi to all,


      I am just wondering how Nimble would perform on small-size, but numerous files.

      Let's say, I'll be using Nimble for file servers.

      Clients connecting to the server create and modify files of small sizes (less than 100KB), but there are millions of it.


      What can be the best performance policy to apply?

      Or what approach can be done to achieve optimized performance?



        • Re: Performance on Large-Quantity, Small-Size Files
          Nick Dyer Navigator

          I think ultimately the success or failure of this requirement would be down to the decision on what to use for host-side SMB/NFS shares - ie Linux shares or something like Windows 2012 R2. The latter is pretty darn good these days as file servers, which I never thought i'd say!


          The ultimate answer here is to test it out - i'm sure your local Nimble SE would only be happy to help!

            • Re: Performance on Large-Quantity, Small-Size Files
              wilbert Newbie

              Thanks Nick.


              Actually, I was testing Nimble and another storage (3-letter word..) for the comparison.

              Say I run a script that creates certain thousands files, 1KB size each for each file.

              Nimble took 5 mins to complete while the other was around 3 mins only.

              I tried to configure Nimble with different settings but it wouldn't give an equal or better result.


              The other storage was old so I was expecting that I would be left behind on this testing.

              By the way, both storage were connected on ESX host with Windows 2008 R2 guest OS VM.


              So I am really wondering where the "wrong" is.

            • Re: Performance on Large-Quantity, Small-Size Files
              wilbert Newbie

              It's CS220, version The throughput performance was hitting average of 1.4MB/s random write and 0.04MB/s seq write. IOPS performance was on 285 random write and 0.26 seq write. I only gave the write ones because Nimble seems very good on read side.


              For the setup, I believe NCM is available on Nimble Ver.2, no NCM. Connections for Nimble-to-ESX is 10Gb. By the way, I forgot to mention that we have test clients that we use to simulate similar environment on our company. Currently, we don't have AV installed yet.


              Clients<--100Mb Hub--1Gb Switch-->VM server(Windows File Share)<--10Gb-->Nimble Storage.

              This structure is same with the other storage brand setup, except that it connects with 4Gb FC protocol.


              We also performed benchmarking on the server side where Nimble volume is directly presented.

              Results showed that Nimble is way better on large block sizes files, but not on small one (4K size).


              I tried using Nimble volumes as ESX datastore, iSCSI target volume, and RDM.

              On those approaches, I applied different performance policies just to figure out which combinations will give results comparable to the other storage.


              I have been seeing a lot of impressive reviews about Nimble, and believing it.

              However, it is my bosses who I am wanting to make believe that Nimble is better than the current storage we have.

              This is why I need to figure out where the problem is.


              (Quite long, have patience reading this.)

                • Re: Performance on Large-Quantity, Small-Size Files
                  Nick Dyer Navigator

                  Good morning,


                  Thanks for the detailed environmental analysis. If you have VMware Enterprise licensing or above its worthwhile installing our Nimble Connection Manager for VMware PSP which will enhance multipathing & path management. It can be found on Infosight. Also if this is a fresh, new install it's worthwhile looking at NOS 2.1 as it provides enhancements over 1.4.11.


                  Are you presenting the storage volumes to the file share via in-guest iSCSI, or via VMware's VMFS?


                  Do you have a support case open with Nimble to track this issue? If so, please post the case number. If not, please do so!


                  Another test would be to load IOMeter and emulate the same test in that tool to see if you can recreate the problem, which may give us some more clues as to where to look next.

                • Re: Performance on Large-Quantity, Small-Size Files
                  Todd Moore Adventurer

                  In regards to the Vmware configuration... did you have multiple NIC's in the host, one to one vmkernel port to vmnic relationship, and if not using the NCM, did you change MPIO from the default?

                  • Re: Performance on Large-Quantity, Small-Size Files
                    Steve Galbincea Wayfarer

                    If what I am reading is correct, you don't have NCM - which is a critical piece of the puzzle when it comes to Nimble performance with VMware. You will see an order of magnitude difference using it vs. not using it. Even RR does not come close to the NIMBLE_PSP_ROUTED path selection policy.