32 Replies Latest reply: Mar 1, 2016 1:37 AM by Chris Aylott RSS

    So whats everyone using for switching and servers?

    John Wagner Wayfarer

      I'm curious what folks are using for their ISCSI switching and servers? If you dont want to say that's fine. Just trying to get a feel for what others are using and maybe what you like and don't like about them.

      We are looking at a few different server and switch vendors and haven't settled on any particular vendor yet.

       

      Thanks in advance.

       

      John

        • Re: So whats everyone using for switching and servers?
          Ryan Homan Newbie

          Using Cisco 3750x and HP A5500 for switching. I like the gui on the HP switches better than the cisco as you can complete most functions without needing to know command line.

           

          Using HP DL380 servers for VMware. The new gen8 series have tons of sensors and run a lot cooler than the older series.

          • Re: So whats everyone using for switching and servers?
            Jonathan Harris Wayfarer

            For switching we use Cisco's Catalyst 4510r with WS-X4748-UPOE+E linecards for GigE and WS-X4712-SFP+E linecards for 10GigE.  We have always used Cisco gear in our data center, but the upgrade to the chassis-based 4510r switch has been great; very robust and versatile. We use IBM x3550 servers in out VMware environment, which is primarily what our Nimble array is dedicated to.  All of our other servers are IBM xSeries from different generations.

            • Re: So whats everyone using for switching and servers?
              Jason Liu Adventurer

              We are a HP shop. We use ProCurve line switches. For server we use Proliant DL360 G7. HP servers are the best in my opinion and their support is excellent. We also have a IBM BladeCenter which gives us a lot of trouble and IBM support really sucks.

              • Re: So whats everyone using for switching and servers?
                David Jay Wayfarer

                Hi all!

                 

                We're using HP DL360's and DL380's Gen 8 servers. On the switching side, we were a bit on a budget, so we went over pretty much every 10GB switch available... we ended up with Netgear's 10GB switches, which after som initial issues and quirks are actually running fine. The reason we chose them is that we could get four switches instead of two, so now we have two switches in each DC fully redundant. I also have a fantastic technician who is networking agnostic. He'll make anything work as we want it to.

                 

                The biggest caveats with Netgear is getting to 3rd line support. 1st line are useless and just loop you around. Getting 3rd line and the issues are usually solved like with most others like Juniper, HP etc...

                 

                /Dave

                  • Re: So whats everyone using for switching and servers?
                    Mike Tedross Newbie

                    We are looking at upgrading our Dell power connect 1G iSCSI switches to an affordable 10G-BaseT switch.

                    We were leaning towards the Netgears. What issues did you have? were you able to fix them through Nimble? or Netgear's support?

                    I am have a CS300. do you happen to have the same Nimble SAN model?

                      • Re: So whats everyone using for switching and servers?
                        Paul Bell Wayfarer

                        I would stay away from NetGear... just not worth the perceived value!

                         

                        For a reliable, solid performing, "value" switch, I've used teh Brocade ICX 6610 with good results for small SANs

                        Brocade ICX 6610 Switch - Brocade

                        • Re: So whats everyone using for switching and servers?
                          David Jay Wayfarer

                          Hi!

                          Update on the Netgears... they were very very buggy when trying to add L3 support to them... and really don't use the stacking feature it just is not worth it.

                          If you run then as L2 switches and have them really dumbed down then they might be ok. Their support is a non-support, overall pretty much non-existent.

                          After having to many issues with Netgear we switched over to Arista 10GB, which are totally the opposite of Netgear, they are great, their support is great.

                           

                          We now took the Netgear back in to production for a different set of servers, we run them redundant and non stacked, dumbed down. The only thing we set in them is some VLAN's and Jumbo frames.

                          Things we know have been very problematic previously are L3, Stacking, NTP, and the web interface.

                          L3 was just really broken...

                          All this might be fixed in later updates of the fiwmware, I don't know.

                           

                          So to finalize, I would not get these for mission critical systems. There are other vendors in a similar price range that are much better. If you can afford then go with Arista they are the ****, and really really stable!

                           

                          /Dave

                      • Re: So whats everyone using for switching and servers?
                        Nelson Penn Newbie

                        For the network that connects our SANs and VMware Servers we are using Dell PowerConnect 5524 switches. For our core network we are using HP ProCurve 2800 series switches.

                         

                        For our servers we currently use Dell PE R610.  This year, though, we have switched to a company named ByteSpeed for computers and our next round of servers will most likely come from them too.  They provide 5 year warranties on desktops and servers as a standard and use name brand components (mostly Intel).

                         

                        We haven't had any issues that I am aware of. 

                        • Re: So whats everyone using for switching and servers?
                          Roddy Pimentel Newbie

                          We are using Dell PowerEdge Servers R720's, Cisco 3750X for iSCSI and Cisco 4506 Blade for Server Side Switching.

                           

                          We almost went all Dell, but I don't know.  Just a fondness for Cisco and being very familiar with the CLI.

                          • Re: So whats everyone using for switching and servers?
                            Andrew Geiger Wayfarer

                            We are using Dell M620 blades and Cisco B200 M2/3s.  The Dell servers access our arrays through a pair of Force10 MXL switches uplinked to a pair of Cisco Nexus 5548s.  The UCS servers connect up through the FI to the Nexus 5548 pairs.

                             

                            Both have setups have worked pretty well.  The Nexus and Force10 switches have their own peculiarities with the molded twinax cables though.

                            • Re: So whats everyone using for switching and servers?
                              Kevin Kumpf Wayfarer

                              We're using mostly the Nexus line from Cisco for switching and a variety of R"XXX" servers from Dell.  From a compatibility standpoint, I can't think of a single issue we've had related to Nimble/Cisco/Dell not playing together nicely.

                              • Re: So whats everyone using for switching and servers?
                                gsherart Newbie

                                We are using HP ProCurve 5400zl switches with 10Gb modules.  Servers are HP BL460c G8 blades (256G RAM, Dual 8 Core).  We boot off the Nimble so there is no local storage.

                                • Re: So whats everyone using for switching and servers?
                                  Jacob Wolf Newbie

                                  We're running four HP Gen8 BL460's on VMware with 192 GB RAM each at our primary datacenter. Switching there is through two 10GB HP Procurve 5820 series switches. Storage is Nimble CS240G and an old Drobo device for archive storage. We have around 50 VM's running in that environment (SQL and Citrix XenApp are the biggest hitters there).

                                   

                                  We're replicating at the storage level, a SQL cluster, and through DFS down to our DR and development site which is a single HP Gen8 DL380 running VMware. I just have a small Procurve 2910 there at the moment with a Nimble CS240 as the only storage on site.

                                   

                                  Both setups work extremely well, and my only gripe with the switches is the 5820's are really running the old 3com OS. The commands are different between the two Procurve products because of that which is frustrating. In a pinch, I also like to use the web interfaces on the switches as you can do most everyday tasks through there.

                                   

                                  We're booting all our servers into VMware from onboard SD cards.

                                  • Re: So whats everyone using for switching and servers?
                                    Jason Benway Adventurer

                                    We have Dell R7X0 server connected with 10GB twinax cables to cisco nexus 5548. I agree the twinax cables are VERY touchy. I think we've had to replace about 6 of them. We are using Qlogic CNA cards in the servers.

                                    • Re: So whats everyone using for switching and servers?
                                      John Wagner Wayfarer

                                      So we finally purchased and got in place our core. We ended up going with Dell Poweredge R720s and 4 Cisco 4900ms with a Nimble CS260 and CS220.

                                      If I had to do it over I would go with a different switch vendor.

                                      Otherwise all is running well and we are very happy with the setup and functionality so far.

                                      • Re: So whats everyone using for switching and servers?
                                        Todd Moore Adventurer

                                        For those that engage with a Nimble SE in the purchase process of an array, ask for that SE's feedback on switches specifically used for iSCSI. We see the good, the bad and the ugly.

                                        • Re: So whats everyone using for switching and servers?
                                          Jacob Wilde Wayfarer

                                          We're running Nexus 5548's and all servers are Cisco UCS Blades, things seem to running well, we might consider going to Jumbo Frames as we're all 10GbE and we've seen some latency issues with SQL when doing backups and maintenance jobs that hit the databases hard. VMware seems to fly along, only thing I could wish for is an additional pair of 10GbE ports per controller so I could direct connect the Nimble Arrays to our two UCS domains via Appliance Ports.

                                          • Re: So whats everyone using for switching and servers?
                                            Dave Cramp Newbie

                                            I'll chime into this too.

                                             

                                            DC1:

                                             

                                            Cisco Nexus 5548 switches & Cisco UCS Blades.  We run jumbo frames for iSCSI and NFS traffic VLANs.

                                             

                                            DC2

                                             

                                            Cisco Nexus 5547 switches & HP C7000 / BL460C G6/7/8 blades.  HP FlexConnect 10gbit blade switches with Jumbo frames for NFS and iSCSI traffic.

                                            • Re: So whats everyone using for switching and servers?
                                              dave hicks Newbie

                                              HP DL360 Gen 9

                                              HP 5406 10GBE

                                              HP5612 10GBE

                                              CS240

                                              • Re: So whats everyone using for switching and servers?
                                                Marty Booth Newbie

                                                Cisco UCS B200 M4 Blades

                                                Cisco C240s

                                                Cisco 6296 FIs

                                                Cisco Nexus 7k

                                                2x CS300s (DR site)

                                                2x CS500s (Prod site with All Flash Shelves)

                                                 

                                                We run the Nimble via appliance ports directly into the FIs, jumbo frames. I wish I hadn't done jumbo frames now, overcomplication of the environment for no real performance gain.

                                                 

                                                Melbourne VMUG UserCon 2015 - vSphere Storage Best Practices - YouTube

                                                 

                                                We replaced VMAXs and CLARiiONs...

                                                • Re: So whats everyone using for switching and servers?
                                                  Phillip Duncan Newbie

                                                  3 x Cisco UCS C220M4

                                                  2 x Meraki MS420

                                                  1 x Nimble CS300

                                                    • Re: So whats everyone using for switching and servers?
                                                      Kelli Sterley Newbie

                                                      Phillip -

                                                      We are looking at implementing a close setup.  Cisco UCS C220M3 and a Nimble CS215.

                                                       

                                                      Do you like your setup?  How long have you been running this setup?  Did you compare it to anything else?  If I may, what applications are you running?  Exchange?  SQL?

                                                       

                                                      We are still in the discovery phase so any help would be great!

                                                      Kelli

                                                        • Re: So whats everyone using for switching and servers?
                                                          Phillip Duncan Newbie

                                                          Hi Kelli,

                                                           

                                                          We are still in the implementation phase of our new environment but so far so good with 1 exception. We've had small issues with how the Meraki switch environment is performing and their support when we think there is a problem. We have been up and running for about 1 month now. The Nimble has been great, and the Cisco UCS have been performing well. Actually, now I remember that there are small issues with the Cisco UCS environment as well, when working with a VMWare hypervisor. Nothing that was too hard to solve but definitely wished it worked out of the box without any hiccups.

                                                           

                                                          We are a Public Safety agency so we had to get multiple vendors and platforms. We mainly compared it to a complete Dell solution (Dell switches, Dell servers, Dell SAN) but decided to go with Cisco and Nimble.

                                                           

                                                          We run a Windows AD environment, with multiple SQL VMs, multiple Exchange 2010 VMs, and other custom app servers. PM me if you want pricing or any other details.


                                                          We plan on purchasing another 3-2-1 solution for our secondary 911 center site and will probably go with Nimble/Cisco.

                                                           

                                                          Thanks,

                                                           

                                                          Phil

                                                      • Re: So whats everyone using for switching and servers?
                                                        ron.fisher@craneae.com Wayfarer

                                                        We have a variety depending on datacenter. Remote datacenters have DL360G8's with 1G or 10G uplinks to either HP A5920 or A5500Hi switches in IRF stacks with CS300's and moldy oldy Equallogic and CS120's.. Wish we would have gone with 380's for more PCI expansion. Main data center we have HPc7000 enclosures with virtual connect (converged FCoE and 10GE) running 10G meshed to HP A5920 IRF stack with CS500 iSCSI and VNX fibre channel connected.. smoking fast

                                                        • Re: So whats everyone using for switching and servers?
                                                          Kevin Hsieh Wayfarer

                                                          My servers are connected to Cisco 4948 or 4948e switches. Anything less than the 4500 series from Cisco really isn't recommended as they don't have the port buffers required to handle microbursts, and you will see lots of dropped frames. I was losing over 1-1.6% with EqualLogic on 3650E switches. My CS300 is connected to some 10G ports on my 6509. All my switches are used, which saves me a bundle. Currently running on R710, which I plan to start replacing in 2016 with newer gear and 10G networking on the host.

                                                          • Re: So whats everyone using for switching and servers?
                                                            Ian Noble Wayfarer

                                                            Cisco 9372TX in NX-OS mode, 48 ports of 10gb base-t & 6 x SFP 40gb uplinks, good value as it was the first Nexus kit in our environment.

                                                             

                                                            I was originally looking at the trident II based switches, the 9372 is trident II too but it also has a Cisco custom asic chip - I liked the extra buffers & micro burst protection that this added.


                                                            Could have went for the SFP based model but the extra up front and increased smartnet expense wasn't worth it for our environment, the Nimble supports SFP or 10g Base-T.



                                                            • Re: So whats everyone using for switching and servers?
                                                              jayst Newbie

                                                              i'm looking at Netgear 10Gbit switches as well. Specs seem good enough regarding buffers and latency, all at a very reasonable price.

                                                              M7100 | Fully Managed Switches | Switches | Business | NETGEAR

                                                              I would only use them for L2 iSCSI traffic. Would not put them in core or let them provide any L3 services.

                                                               

                                                              I'm very happy with Dell R630 and R730 servers ath the moment.

                                                              • Re: So whats everyone using for switching and servers?
                                                                Chris Lindloff Adventurer

                                                                Cisco 5548's for LAN and iSCSI

                                                                 

                                                                Dell R630's for VMware 6.x hosts.  Each host has 4 - 10gig ports (Intel X520-2?) 2 for LAN, 2 for iSCSI.  We run Jumbo Frames on the hosts/vmware and switches (preferred send/receive).  Nimble CS300 10gig, Jumbo on.

                                                                • Re: So whats everyone using for switching and servers?
                                                                  Chris Aylott Adventurer

                                                                  It really doesn't matter what switches they are providing they adhere to one rule - they have sufficient buffer cache to handle the throughput. You will know when it's an issue as a performance test will result in a saw blade graph where the cache becomes saturated. Re-transmits and high pause frame count are also indications of this saturation.

                                                                   

                                                                  Aim for =>512kb per port, most vendors however share one pool across all the ports! It is for this reason that stacking switches is bad news as any inter switch link (ISL) traffic will consume this cache also, this highlights the importance of ensuring you isolate your paths via different subnets or event & odd or bisect.

                                                                   

                                                                  All switch vendors publish these stats, if they don't - don't buy the switch.