1 Reply Latest reply: Jun 30, 2016 12:52 PM by RSS

    SAN Booting Issue with Cisco UCS (ESXi 6.0)

    Nigel Tan Newbie
    Visibility: Open to anyone



      I am attempting to install ESXi on Cisco UCS B200M4 blade server with SAN booting on Nimble storage (Cisco MDS 9148S as SAN fabric)

      Assuming all configurations are properly setup, i can't seem to get it function on ESXi 6.0.

      1. If put in 2 paths to Nimble volume, reboot server will have core dump error and cannot find system log folder.
      2. If terminate 1 path (Using Nimble GUI which remove 1 HBA from initiator group). Reboot server with no issue.

      In summary, SAN boot is successful if there is single path configured in HBA bios. But single path doesn't offer resiliency, and this will be a major concern.

      I have no problem booting up ESXi 5.5 with 2 configured paths (Primary& Standby).


      Has anyone encounter the similar issue with ESXi 6 SAN boot?

      We have logged several cases with VMware and Nimble and apparently, it wasn't really fruitful.


      • Cisco UCS Manager and Firmware Version: 3.11e
      • Boot-loader version: v3.6.0
      • Kernel version: 5.0(3)N2(3.11e)
      • System Version: 5.0(3)N2(3.11e)
      • Nimble OS Version:
      • ESXi NCM Version: nimble-ncm-for-esx6-2.3.1-600006
      • ESXi Version: Vmware-ESXi-6.0.0-3620759-Custom-Cisco-
      • vCenter Version: VMware-vCenter-Server-Appliance-
        • Re: SAN Booting Issue with Cisco UCS (ESXi 6.0)

          The config firmware / OS versions mentioned are known to work well.  Interestingly, it appears your issue was only experienced with ESXi 6.0 and not with ESXi 5.5.  (In general this makes me think that this problem is not with your config, but that particular OS release).

          However, Specifically you state that you are running Cisco custom image of ESXi 6.0 U2.  This version has been tested by Cisco and Nimble for dual path connectivity.  


          Let me ask a few clarifying questions:


          1) Any chance you actually installed ESXi 6.0_U1?   I ask as the U1 version has a known problem with some later model B200 M4s.  Reference Cisco's HCL  http://www.cisco.com/web/techdoc/ucs/interoperability/matrix/matrix.html)


          2) Out of curiosity, did you initially install the ESXi host with a single path or with multiple paths? 


          3) Are you experiencing this issue on a single host or is it more systemic?   


          4) The UCS to MDS connectivity should be verified.  FI-A and FI-B should be separate fabrics (each with it's own VSAN).  If you have multiple uplinks with port channels, verify the MDS upstream switch config matches.


          5)  Verify the UCS boot policy.  The primary connection should be pointing to the same interface on both Nimble Storage controllers (FC1.1 for example)  The secondary connection should be pointing to the same interface on both Nimble Storage controllers (FC 2.1 for example).    It should look something like this:  http://www.cisco.com/c/dam/en/us/td/docs/unified_computing/ucs/UCS_CVDs/smartstack_cs700.docx/_jcr_content/renditions/sm…


          If nothing obvious comes from these questions, then I would suggest engaging Nimble Storage support to do a through review of your environment.