This isn't a supported configuration - virtual IP floats across controllers so in your case not all vmkernel ports are going to see the VIP - I've had this same issue trying to direct connect hosts using HPE Storevirtual VSA.
When using iSCSI you will need at least a single switch. Fibre channel is the only way I've ever been able to direct attach storage to hosts but this was a different storage system with active/active controllers.
To be honest if you are investing a pair of DL380s and a CS220 at a remote site a pair of low-cost GbE switches would be a minor cost :-). Alternatively you could pull the CS220 back to a larger site or datacentre and go with a hyper-converged architecture that could help to reduce the remote site footprint.
Thanks for the reply Ben. We were re-using some equipment from one small DC to another I had it in my mind that it should work just fine. I've always placed stacked switches between hosts and Nimbles in the past, just didn't know it was a hard requirement. I just glad it is working as is until we can figure out how to get some switches up there.