13 Replies Latest reply: Feb 5, 2015 9:13 AM by Steve Wysocki RSS

    Recommended Switches for CS210, CS215, CS300

    Joe Eberhardt Wayfarer

      I was reaching out to see what everyone has been using for switches in their environments and trying to find one that performed well and was cost effective. We have been reviewing our hardware builds for a SAN install and thought that we should revisit the switch side and make sure that we're putting in a model that is known to work well in a production environment. I was reading through the forums and found some references to different Cisco and NetXtreme switches, but was looking for something that was known to work well with Nimble. We have put in HP Procurve 2920al switches on a few installs, which seem to work well and they're not overly expensive, but we're willing to make a change for something better. Thoughts?

        • Re: Recommended Switches for CS210, CS215, CS300
          Daniel Duffy Adventurer

          Using HP 5406zl with 10G to Nimble and ESX hosts here without any issues.  Works great, competitively priced, and lifetime support makes TCO very compelling.

          • Re: Recommended Switches for CS210, CS215, CS300
            Jason Herrera Wayfarer

            Joe - There are many considerations to evaluate when choosing an iSCSI switch.  I have listed those considerations below.  If the switch you are considering meets the items on the list then it should be good as an iSCSI switch.  The main consideration is the buffer space per port, if that is low then you will have performance issues.

             

            iSCSI Switch Features and Considerations:

            • Level 2 or preferably level 3 managed switch

            • 512K per port dedicated port buffer space

            • Supports flow control. It should be turned on

            • Supports Jumbo frame - needs to be configured on everything in the data path, use at customers discretion.  

            • Spanning tree. It should be turned off

            • Multicast filtering. It should be turned off

            • IP precedence.  It should be turned off.

            • Disable unicast storm control on each switch handling the iSCSI traffic. However, the use of broadcast and multicast storm control is recommended.

            • Re: Recommended Switches for CS210, CS215, CS300
              Chris Haynes Wayfarer

              We've had good success with Cisco 3850s and HP 3800s, using 10G ports.

               

              - Chris

                • Re: Recommended Switches for CS210, CS215, CS300
                  Ben Loveday Wayfarer

                  Hey Chris,

                   

                  How have the 3850s been for you? Are you running the latest 2.1.4 firmware with them? We have had some strange flow control issues when using the 3850s (with other vendor arrays) so keen to hear your experiences.

                   

                  Cheers,

                  Ben

                    • Re: Recommended Switches for CS210, CS215, CS300
                      Chris Haynes Wayfarer

                      Been working well for us.  We've only used Nimble storage for VMware ESXi hosts so far. We enable Flow Control on the storage ports and follow all the other Nimble recommendations that Jason mentioned above.  We haven't had any issues so far. We are a Nimble Partner so we have several different customers running different versions of IOS. The last Cisco 3850 we setup is running IOS version 03.02.03.SE RELEASE SOFTWARE (fc2). We also use HP 3800 switches on occasion and have not had any issues with those either.  I'd recommend either one.

                       

                      - Chris

                        • Re: Recommended Switches for CS210, CS215, CS300
                          Ben Loveday Wayfarer

                          Hey Chris,

                           

                          Thanks mate, good to know. So you haven't seen any issues where the switch reports flow control as being off? We had some strange issues with HP Storevirtual and ESX where they were all set to flow control autoneg and the switches wouldnt negotiate flow control on. When we switched back to the old switches flow control auto negotiated on no probs. We have tried 3.06 (i believe), and had heard of a known flow control bug but thats about the extent of my knowledge unfortunately.

                           

                          Thanks again!

                          Ben

                      • Re: Recommended Switches for CS210, CS215, CS300
                        Joe Eberhardt Wayfarer

                        Have you ever made adjustments on the number of queues to increase the buffer size for a set of ports? I was reading through one of the data sheets for an HP 3800 and found this.

                         

                        Selectable Queue Configurations:

                        Allow you to increase performance by selecting the number of queues and associated memory buffering that best meets the requirements of your network applications

                         

                        After reading some more on the HP 3800 series and also the Cisco 3850 series, it looks like each comes out of the box with 8 queues configured. There were some other references on the HP forums where people were saying that you could drop the number of queues down to 2 (cannot set it to 1) to increase performance. Just seeing if anyone has heard of changing this setting and if it actually made a difference in an iSCSI network by increasing the port buffer size.

                      • Re: Recommended Switches for CS210, CS215, CS300
                        Nick Furnell Wayfarer

                        Hi

                        We have just refreshed our two Data Centers and used two pairs of Brocade VDX 6730 - very nice SFP switches. We had some fun connecting them, mainly due to the new v4 firmware, which changed the command set from HP style to Cisco style (and it wiped everything off too). Fortunately the firmware update was at the point of installation, it would not have been much fun otherwise. Each pair is connected via 4 x 10Gb TwinAx, giving more than enough bandwidth and  allows for someone stupid to pull out quite a few before things start slowing down. Interestingly, Brocade have just released a "Design & Build" validation Test based on this range of switches Data Center Solution, Storage-Validation Test: Brocade VCS Fabric and Nimble CS220G-X2 Flash Storage Array - Brocade Com… so close to my design, it's untrue (we are running HP Gen8's as well).


                        My two Nimble CS220G's are firmware 2.0.8 & 2.1.4 - unfortunately the 2.1.4 is reporting duplicate Management IP's on the LAN (we are not sure why), it could be completely unrelated. I haven't updated the second array yet, that will be the test I guess.

                         

                        So far all good though, edge switches are Brocade ICX 6450's - these are so different from the CLI, that they could be from a different manufacturer!

                        • Re: Recommended Switches for CS210, CS215, CS300
                          Andy Bohne Newbie

                          We are using a pair of Arista 7050s for both our iSCSI traffic as well as our server LAN traffic.  We've been running this for about 9 months and I have no complaints.

                          • Re: Recommended Switches for CS210, CS215, CS300
                            pokermunkee Newbie

                            Any issues using a ProCurve 2920-24G switch for iSCSI and vMotion traffic for a CS300?

                             

                            Servers and CS300 are 1GbE.

                             

                            Saw someone post in another forum site that Nimble said not to use the 2920-24G due to buffer size.  But the specs I see on this switch looks good to me?

                              • Re: Recommended Switches for CS210, CS215, CS300
                                Jason Liu Adventurer

                                I'm using a Procurve 2910al-24G at our DR site with a Nimble CS220. I found the switch is not good enough probably due to the buffer size. If I do a big file copy on the nimble volume it slows down the whole switch to the point that I lose ping packets to the other computers on that switch. I'm thinking of replacing it with a 3600 series.

                              • Re: Recommended Switches for CS210, CS215, CS300
                                Jim Jones Wayfarer

                                I know this has already been marked as answered but we're using a pair of Cisco Nexus 3064T to power our datacenter these days and I can't be happier. These give me 10G copper out of 48 ports and then we are cross connecting with 2 bonded 20G twinax, with another 2 10G SFP for uplink to the distribution layer. If you can find your way around a Cisco switch already the leap to NX-OS isn't bad, and for the specific networking best practices from Nimble I've outlined their configuration here: Configuring Networking for Nimble-vSphere iSCSI - Koolaid.info

                                • Re: Recommended Switches for CS210, CS215, CS300
                                  Steve Wysocki Wayfarer

                                  I don't know how large you are trying to go here, but I just have two 48 port 28xx HP Procurve 1Gig switches for both my iSCSI fabric and controller failover, and I couldn't be happier with the performance.  In my shop, 10GB would be serious overkill.