12 Replies Latest reply: Feb 6, 2014 9:30 AM by Justin Rich RSS

    Scripting help - Mount replicated snapshot for backup

    julez Adventurer
    Visibility: Open to anyone

      So we're currently migrating to our new Nimble CS260s and we're looking for a way to do something similar to what we did previously with our old equipment.

       

      So what we're trying to do is take a snapshot at our co-location facility.
      Replicate that snapshot to our HQ.
      We would like to then script the most recent snapshot at HQ to be mounted to our backup server.
      The backup server could then call another script to tell NetBackup to backup that new drive that was mounted.
      After the backup was over that script would then dismount the snapshot.

       

      We are looking for a solution to do this with our new arrays.  Any help would be great!

       

      Justin Rich

        • Re: Scripting help - Mount replicated snapshot for backup
          julez Adventurer

          This could give a little bit better insight as to what we've done in the past and what we'd like to do.  Or if there's a better way someone can come up with it would be great too.  Unfortunately these are just scripts that have been in place since before a lot of us were here.

           

          WARNING PREPARE YOUR FACEPALM MEMES:
          So this is the batch file we had set to run to start everything off.

          ---

          @echo off

           

          rem Check for end of month and run if not

          "D:\Scheduled Netbackup Related Tasks\NetApp\is_last_day_of_month.exe"

          if "%ERRORLEVEL%"=="1" goto DONE

           

          rem Mount Vols

          call "D:\Scheduled Netbackup Related Tasks\NetApp\mount_exchange_sql_snaps_weekly_monthly.bat"

           

          rem Run Backup Jobs

          "D:\Program Files\VERITAS\NetBackup\bin\bpbackup" -i -p LTO4_NetApp_Snap -s Weekly -w 

           

          rem Unmount Vols

          call "D:\Scheduled Netbackup Related Tasks\NetApp\umount_exchange_sql_snaps_weekly_monthly.bat"

           

          :DONE

          ---

           

          Here's a snippet from the first batch file called.
          ---

          d "C:\Program Files\NetApp\SnapDrive"

           

          echo %date% > c:\Temp\sd_connect.log

           

          REM Exchange 2007 Data 01

           

          sdcli disk connect -p fas3040a:/vol/Exchange_2007_Data_01/.snapshot/exchsnap_mail_recent/data01.lun -d D:\snapmnt\mail\D\Exchange_2007_Data_01\data01 -I  backupserver <<MAC>> -dtype dedicated

          sdcli disk connect -p fas3040a:/vol/Exchange_2007_Data_01/.snapshot/exchsnap_mail_recent/data02.lun -d D:\snapmnt\mail\D\Exchange_2007_Data_01\data02 -I  backupserver <<MAC>> -dtype dedicated

          sdcli disk connect -p <<IP>>:/vol/Exchange_2007_Data_01/.snapshot/exchsnap_mail_recent/data03.lun -d D:\snapmnt\mail\D\Exchange_2007_Data_01\data03 -I  backupserver <<MAC>> -dtype dedicated

          sdcli disk connect -p <<IP>>:/vol/Exchange_2007_Data_01/.snapshot/exchsnap_mail_recent/data04.lun -d D:\snapmnt\mail\D\Exchange_2007_Data_01\data04 -I  backupserver <<MAC>> -dtype dedicated

          sdcli disk connect -p fas3040a:/vol/Exchange_2007_PubFolder_Data/.snapshot/exchsnap_mail_recent/exchange_2007_pubfolder_data.lun -d D:\snapmnt\mail\D\Exchange_2007_Data_01\PublicFolderData -I  backupserver <<MAC>> -dtype dedicated

          ---

           

          And the last batch file.

          ---

          cd "C:\Program Files\NetApp\SnapDrive"

           

          REM Exchange 2007 Data 01

           

          sdcli disk disconnect -d D:\snapmnt\mail05\D\Exchange_2007_Data_01\data01

          sdcli disk disconnect -d D:\snapmnt\mail05\D\Exchange_2007_Data_01\data02

          sdcli disk disconnect -d D:\snapmnt\mail05\D\Exchange_2007_Data_01\data03

          sdcli disk disconnect -d D:\snapmnt\mail05\D\Exchange_2007_Data_01\data04

          sdcli disk disconnect -d D:\snapmnt\mail05\D\Exchange_2007_Data_01\PublicFolderData

            • Re: Scripting help - Mount replicated snapshot for backup
              julez Adventurer

              Again ultimately the goal is to automate the mounting and unmounting of a replicated snapshot for backup purposes.
              The snapped volume would be from a Windows iSCSI volume.

               

              And of course why we want to backup the snapshot is because we just don't have the bandwidth to pull a crash consistent backup across the wire.

                • Re: Scripting help - Mount replicated snapshot for backup
                  Justin Rich Adventurer

                  So here are the steps as i see them.

                   

                  1. setup replication from production unit to secondary unit
                  2. snap/clone replica to a mountable volume (i havent played with replication, not sure how much needs to be one here)
                  3. Mount lun on a windows box
                  4. backup volume

                   

                  That sound about right? what parts do you have done?

                    • Re: Scripting help - Mount replicated snapshot for backup
                      julez Adventurer

                      Well we've just started to research this. 
                      We've got replication setup so the snapshots are available at HQ.

                      We can continue to use the same scripts we used for actually backing up the volume.

                       

                      So really we just need to figure out 2 and 3.

                       

                      How do we take a snap and mount it to our Windows 2008 R2 backup server as a mount point.
                      Then how do we dismount it.  All via command-line/powershell.

                       

                      We really haven't seen that sort of functionality yet.

                        • Re: Scripting help - Mount replicated snapshot for backup
                          Justin Rich Adventurer

                          Ok so then a good chunk of this can be done with my module.

                          If you have a snapshot you can use my module.

                           

                          you'll first use the New-NSClone which will clone that snapshot to a volume. once you've then got the cloned volume you can use the Add-NSInitiatorGroupToVolume to assign access to it so that the windows host can see the volume (physical machine?)

                           

                          after that you need to do some Iscsi stuff to mount the volume. i've got this code some place, but i'll have to dig for it.

                           

                          also, what OS is the windows backup box? 2012 would make the iscsi stuff a lot easier

                            • Re: Scripting help - Mount replicated snapshot for backup
                              julez Adventurer

                              We really appreciate your help Justin.  Unfortunately we're on 2008 R2 for the server running NetBackup.  Hopefully in the next year or so we'll be moving it to 2012 R2.  Unfortunately storage migration is followed by Exchange migration, then possibly a new backup solution.

                                • Re: Re: Scripting help - Mount replicated snapshot for backup
                                  David Tan Adventurer

                                  Good to see you have moved from NetApp to Nimble

                                   

                                  I am also a Nimble Storage user that is unfortunately stuck with Netbackup as the chosen backup product. I am trying to convince the powers that be to get Commvault as it integrates beautifully with Nimble Storage, so you don't need to bother trying to script up solutions like you are trying to do now. Also mostly Windows 2008R2 so all my scripts are powershell 2.0 based.

                                   

                                  I'm not sure where you run your scripts from but I use a designated server where I have setup putty.exe and plink.exe to talk to Nimble arrays using a private/public key pair setup (using sshkey array commands) when logged in as the service account that runs the scripts. This way no need to authenticate or store credentials in scripts. I can then use command or powershell to run any command like I am using ssh

                                   

                                  plink -batch admin@$($Target) $Command

                                   

                                  See if the following powershell code works. I've just taken snippets from my scripts so you'll probably need to play with it a bit...

                                   

                                  # set variables - array name, volume name and igroup name of the server to mount

                                  [string] $Target = "putnimblearraynamehere"

                                  [string] $volume = "putnimblevolnamehere"

                                  [string] $ComputerName = "putcomputertomountsnapshere"

                                   

                                  [string] $PartType = "LETTER"  # or "MOUNT"

                                  [string] $MountPath = "N:" # drive if letter, path if mount

                                  [string] $Igroup = "$ComputerName"  # I always name my igroup the same as the server

                                   

                                  # get the latest snapshot name

                                  [string] $Command = "snap --list --vol $volume `| grep Daily `|sed -n 1p `| awk `'`{print `$2`}`'"

                                  [string] $Snapname = plink -batch admin@$($Target) $Command

                                   

                                  #clone the snapshot and allow the new server to see the volume

                                  [string] $Clonename = "$($volume)-nbclone"

                                  $Command = "vol --clone $volume --snapname $Snapname --clonename $Clonename"

                                  plink -batch admin@$($Target) $Command

                                  $Command = "vol --online $Clonename"

                                  plink -batch admin@$($Target) $Command

                                  $Command = "vol --addacl $volume --apply_acl_to volume --initiatorgrp $Igroup"

                                  plink -batch admin@$($Target) $Command

                                   

                                  # on the server, make the iscsi connection - recommend using this powershell script from Adam Herbert

                                  # https://connect.nimblestorage.com/docs/DOC-1081#comment-1422

                                  # you can modify it so it will only attempt to connect volumes matching the name - eg "-nbclone"

                                  invoke-command -filepath <path>\NimbleMPIO.ps1 -computername $ComputerName

                                   

                                  # assign a drive letter or mountpoint to the disk

                                  [string] $DiskNumber = (Get-WmiObject -ComputerName $ComputerName -Namespace ROOT\WMI -Class MSiSCSIInitiator_SessionClass | Select -Unique TargetName, @{Label="LegacyName"; Expression={$_.Devices[0].LegacyName}} | where {$_.TargetName -like "*:$($VolumeName)-*"})[0].LegacyName.replace("\\.\PhysicalDrive","")

                                  [string] $DiskType = (Get-WmiObject -ComputerName $ComputerName  -query "Select Type from Win32_DiskPartition WHERE DiskIndex = $DiskNumber AND Index = 0").Type

                                  If ($DiskType -like "GPT*") { [string] $PartitionNumber = "2" } Else { [string] $PartitionNumber = "1" }

                                  $Result = Invoke-Command -ComputerName $ComputerName -ScriptBlock {param($pDiskNumber, $pPartitionNumber);"rescan","select disk $pDiskNumber","online disk", "attrib disk clear readonly", "select PARTITION $pPartitionNumber","attrib volume clear hidden","attrib volume clear readonly","assign $PartType=$MountPath" | diskpart} -ArgumentList @($DiskNumber,$PartitionNumber)

                                   

                                  # offline and delete the snapshot (this will drop off the iscsi connections)

                                  $Command = "vol --offline $Clonename --force"

                                  plink -batch admin@$($Target) $Command

                                  $Command = "vol --delete $Clonename"

                                  plink -batch admin@$($Target) $Command