I could really use some help in designing a solid architecture for a Zimbra email system, hosted by Ubuntu Linux (64bit) running on vSphere 5.5, that will allow rapid recovery from failures, and can later be expanded to allow HA. The plan from the beginning is to enable Vmotion for guests and storage, and to use Nimble snapshots for DR.
Our Nimble system is a CS460G-X2 with an ES1-H65 expansion shelf, and we're not using the Nimble exclusively for our email. We're using it for many different application, database, and file server purposes as well. The unit is dual-connected into a pair of 10Gb switches using iSCSI. We have roughly1400 users on our Zimbra system, and once this migration/upgrade project has completed, we will have a 3 VM guest email system, where each guest will be providing ldap, mta, and mailbox services.
Each of the VM guests will be set up with 3 VMFS datastores, each on its own Nimble volume. Those 3 datastores will be used for:
a) "Root & boot" linux volume - a small ext2 /boot partition, and on an LVM partition, the swap device and an ext4 root file system
b) VMware will create 3 "hard drives" on this Nimble volume, and these 3 drives will make up a striped LVM, ext4 file system. This file system will host the Zimbra email system
c) A "backup" volume, an ext4 file system which will hold 1 week of Zimbra email system backups
Do we need to set up the VMware hard disks as independant? IIRC, independent disks don't participate in VMware shapshots. If we do make the disks independant, should we just avoid VMware snapshots, instead using Nimble snapstots on all the Nimble volumes to allow us to roll changes back (like from a failed patch attempt?) Or should we just leave it as default, use VMware snapshots, and delete the snaps once the change proves successful, or roll-back to the earlier snap? In our testing, we were not able to revert to an earlier Nimble volume snapshot if there were any VMware snapshots on the guest.
For DR purposes, we're planning on using the Nimble snapshot capability and will take a snapshot of the "root & boot" and "backup" Nimble volumes once a week, but the Zimbra volume will have hourly snapshots. (The Zimbra backups are also archived to a different storage system, where they are held for 3 weeks, but that is outside the scope of this question.)
Unfortunately, the VM guests can only have a maximum of 4 VMware Paravirtual scsi adapters (I was really wishing for 5). So here is one part of my dilema:
Do I use 3 SCSI adapters, 1 for the "root & boot," 1 for the LVM Zimbra system (all three VMware hard disks on a single HBA,) and 1 for the backup system,
Do I use 4 SCSI adapters, 1 for the "root & boot," as well as the backup file system, and the other 3 for each of the 3 VM "hard disks" I'll be creating on the data store for the Zimbra system. The reason for keeping the Zimbra system and backup file systems on different HBAs is that otherwise, backups will take forever.
or is there a better way altogether?
I know that Linux creates a separate queue for each physical block device (Is this still true if the disk i/o scheduler is controlled by VMware vs Linux, as per setting the elevator setting to "noop?") Are there any advantages or disadvantages to having a single disk device on its own channel in a striped LVM environment, as opposed to having all 3 drives on a single disk channel when you're working in a virtual environment? From past experience, there were great performance advantages in having multiple channels for the disk i/o in hardware server settings.
As I'm something of a noob when it comes to vSphere, and a total newbie when it comes to the Nimble storage, I'm hoping that someone might be able to give me some helpful hints here, or say that I'm just plain wrong (and if so, please make a suggestion! :-) .) Please feel free to ask any questions if I wasn't clear on anything.
Finally, is there a good reference or references in coming up to speed in working with vSphere and SANs, in particular, with the Nimble. I just downloaded the "VMware on Nimble Storage-BPG." I've got a long history with working on networks and servers (Windows, Linux, & other *ix), and I've been in IT since 1989. Most of my experience with VMware over the years has been with VMware Workstation.
Thanks in advance,