In your case with both Cisco Blades and non-Cisco servers, I would recommend using a pair of upstream switches for the 10g connectivity.
According to the UCS Interoperability matrix (http://www.cisco.com/c/dam/en/us/td/docs/unified_computing/ucs/interoperability/matrix/r_hcl_B_rel2-22.pdf) only the Cisco UCS Servers are support for connectivity to the FIs with certain Cisco firmware.
Hope this helps.
you will always want to use switches as a connectivity medium to service the HA function within Nimble. As for appliance ports, you would leverage this setting when directly connecting to the FI's versus a distribution switch fabric. I have provided a link doc to the deployment guide that I believe will help answer your questions and provide setup clarity. I hope this helps!