6 Replies Latest reply: Oct 21, 2014 1:21 PM by Jason Liu RSS

    Automatically Mount Most Recent Snapshot and Connect to Target

    Chris Aylott Adventurer

      HI All,

       

      Hope it's useful to someone. Uses batch scripting, it's simple, working and tested.

       

      Applications: Clone and mount most recent snapshot...

      1. for development environment, scheduled it to mount the most recent snap at a DR environment so that devs would be working of the most recent data.
      2. for backup
      3. TODO - for backup and disable cache to prevent cache churn.
      4. TODO - clone all recent snapshots for a volume collection and mount on ESX to simulate DR test without affecting replication.

      or simply use it to quickly clone the most recent snapshot for a volume

       

      Behaviour:,

      BATCH - Finds most recent snapshot for the volume you specify, clones it (sets it online), fetches the new iqn which can then be passed into a powershell script to either connect the cloned snapshot volume to ESX or a Windows box. Uses text files to filter the results.

      POWERSHELL - Connects only the new volume target IQN and sets any offline disk online. From testing it will retain any drive letter previously associated. !! Make sure you limit your volumes by intiator groups for security, as is best practice. !!

      > can be run without executing anything to ensure you return the expected results.

       

      Environment:

      It's a batch script which executes a powershell script. Tested on MS2012 (will work on 2008 however the the ps1 will have to be changed) with array versions 1.4.x and 2.0.x. Run this script from the server you want to mount the volume onto.

       

      Useful references:

      For ps scripts using the iscsi cmdlets Windows PowerShell  PS >_: New iSCSI cmdlets in PowerShell V3 Part - 1

       

      Disclaimer:

      Test it yourself use at your own peril ... etc etc. Read the comments.

       

      Usage:

      1. Create a password-less connection to the array by generating a SSH key between the server and array.
        1. Install plink, puttyGEN and putty into c:\folder - PuTTY Download Page
        2. Run puttyGen selecting 1024 bits, see screenshot
        3. Save the key, selecting no to any passwords.
        4. Copy the key exactly as shown below.

      generate-ssh.jpg

           5. Connect to the array via putty using the login admin and your GUI password.

           6. sshkey --add <name> --type rsa --key <paste the ssh key from previous step>

           7. Open putty again prefix the array ip address with admin@ set the auth to SSH and point it at the key you saved in step 3, save the session with the name of your array.

      > You can now run commands on the array via plink directly from the machine.

      putty-auth.jpg

      2. Open powershell as administrator run Set-ExecutionPolicy Unrestricted > Read what this means to you! Using the Set-ExecutionPolicy Cmdlet

      3. Save the code below into c:\folder and rename i.e. recent-snapshot.bat save the powershell script as connect-target.ps1

      4. Change the modifications in the script, add the session name of putty from step 1.7 above to the array-name=XXXXX

      5. Execute - on first run comment out the sections where the snapshot is actually cloned to ensure you get the expected results.

          name.bat VOLUME-NAME

          i.e. C:\folder\recent-snapshot AA-SQL01-DB01

       

      Batch file

      :: This script will return the most recent snapshot for the volume and clone it, which sets it online.
      :: Run the bat file followed by the name of the volume i.e. recent-snapshot.bat AA-GOLD-EXCH01-DB01
      :: Once the volume has been created you can discover it on the host and connect to it.
      :: This can be done automatically however if scheduled the volume target iqn changes, increases by 1 every time.
      
      
      :: NUMBER OF MODIFICATIONS 5:
      :: MOD 1. Having setup a SSH key with your array define the putty session name (array name) below
      :: MOD 2. If you are mounting a snapshot for a backup and do not want it to churn your cache set to true NOT IN USE
      @ echo off
      set array-name=array-01
      set full-backup=false
      :: paramater taken from the command line - do not change
      set volume=%1
      
      
        echo Using the session %array-name%
        echo .
        echo .
      
      
      :: MOD 3. If this is not a scheduled task you will want to comment the lines below to set the clone volume offline and delete it.
      :: NOTE: there is a 20 second wait command, the ping, to allow the command to complete successfully
      :: This is where you would invoke a powershell command if it is mounted in ESX | MS, on testing this is not neccessary for MS.
      echo Removing the existing snapshot clone-%volume%
        echo .
        echo .
      plink %array-name% vol --offline clone-%volume% --force
      ping 192.0.2.2 -n 1 -w 20000 > nul
      plink %array-name% vol --delete clone-%volume%
      ping 192.0.2.2 -n 1 -w 20000 > nul
      
      
      plink %array-name% snap --list > snap-list.txt
      :: export the snapshot list
      findstr /B /I %volume% c:snap-list.txt > snap.txt
      :: find the snapshots related to the volume
      
      
      for /f "tokens=2 delims= " %%a in (snap.txt) do (
      :: retrieve the most recent volume collection name
        set mostrecent=%%a
        goto :break
        )
        :break
        echo Mounting the snapshot %mostrecent% for volume %volume% it will be called clone-%volume% it will take a minute or so to appear within the GUI.
        echo .
        plink %array-name% vol --clone %volume% --snapname %mostrecent% --clonename clone-%volume%
      
      
      :: MOD 4. If you don't need the IQN to pass onto Powershell to automatically mount the volume then comment out the section below.
      ping 192.0.2.2 -n 1 -w 60000 > nul
      :: wait for the volume to be cloned
      plink %array-name% vol --info clone-%volume% > vol-info.txt
      :: get the new volume information
      findstr /B /I iSCSI  c:vol-info.txt > iscsi.txt
      :: find the right line
      for /f "tokens=3 delims= " %%a in (iscsi.txt) do (
      :: get the IQN of the new volume created this can be passed into a powershell script for esx or MS.
        echo %%a
        set iqn=%%a
        )
      :: MOD 5. from here you pass the parameter into a Vmware or MS powershell script. Script below will set the new volume online on MS 2012 only!.
      :: The first time you mount the volume assign a drive letter and it will use the same letter every time, or you can add the command to the PS script.
      :: NOTE: On testing we found you need to restart the SQL service, add it to the PS script.
      
      
      powershell.exe -file c:\chris\connect-target.ps1 "%iqn%"
      
      exit /b
      
      
      
      
      

       

      Powershell to connect to target

      param(
      [string]$a
      )
      Write-Host $a
      Get-IscsiSession | Register-IscsiSession
      Connect-IscsiTarget -NodeAddress "$a"
      Get-Disk | ?{$_.operationalstatus -eq "offline"} | Set-Disk -IsOffline 0
      Get-Disk | ?{$_.operationalstatus -eq "online"}
      
      
      
      
      

       

      NOTE: Initiator groups and ACL's are not replicated, if run at a DR site these will have to be defined.

        • Re: Automatically mount most recent snapshot and connect to target.
          Alex Zagajewski Newbie

          For those of us who are still using Windows 2008 R2 (probably most of us) and don't have PowerShell V4 which has the iSCSI commandlets, you can use iscsicli.exe to connect the LUNs and then diskpart.exe to put the new volumes online.

           

          First, you need one more variable to account for the IP of the array, use the iSCSI discovery portal IP from the GUI:

           

          set array-ip=10.0.1.50

           

          Next, you have to attach and log in the new volume to the server using iscsicli.exe:

           

          iscsicli.exe qaddtarget %iqn% %array-ip%

          iscsicli.exe qlogintarget %iqn%

           

          Then I like to wait 20 seconds for the O/S to recognize it and output all disks using diskpart.exe so that you can find the new volume which will be offline and, in turn, put it online (this won't work if you have multiple volumes offline, of course):

           

          Create a new text file called listdisk.txt and save it in the same directory as all of your other batch files, executables, etc.  All it needs to contain is this line:

           

          list disk

           

          Back to the batch file:


          timeout /t 20 (much neater than pinging, btw)

          diskpart.exe /s listdisk.txt > disks.txt

           

          Create another new text file called offlinedisk.txt and save it in the same directory as all of your other batch files, executables, etc.  All it needs to contain is these three lines:

           

          select disk %disk%

          online disk

          attributes disk clear readonly

           

          Continuing in the batch file, you need to parse the disks.txt for the offline disk and put it online:

           

          findstr Offline disks.txt > offline.txt

          for /f "tokens=2" %%b in (offline.txt) do set disk=%%b

          diskpart.exe /s offlinedisk.txt

           

          Hope this helps!

            • Re: Re: Automatically mount most recent snapshot and connect to target.
              Alex Zagajewski Newbie

              Two more items:  First, a heartfelt thank you to Chris Aylott for putting together above code!

               

              Second, I forgot to add a couple of statements that would remove the "Unrestricted" access to the new volume and then add the initiator group of the server to which you're connecting the new volume:

               

              plink %array-name% vol --removeacl clone-%volume%

              plink %array-name% vol --addacl clone-%volume% --initiatorgrp initiatorgroup

                • Re: Automatically mount most recent snapshot and connect to target.
                  Alex Zagajewski Newbie

                  OK, more updates:

                   

                  I have found that because the serial of the cloned volume changes, Windows sees it as a new volume and does NOT always assign it the same drive letter or one at all.  My backup software looks to backup specific drive letters on the backup media server therefore having the same one is critical.  So I've added some more code that will take the new offline volume, find it and then assign it a drive letter.  This also means that you have to pass another variable upon execution of the batch file to determine the drive letter.  The new command will now be: C:\nimble\recent-snapshot.bat [Volume-Name] [Drive_Letter]

                   

                  You also have to add a new line towards the beginning of the batch file to grab this drive letter:

                   

                  :: drive letter to assign

                  set letter=%2

                   

                  So, the new stuff needed to assign a drive letter to the new drive:

                   

                  Add the following lines before the first iscsicli.exe line:

                  :: turn off automounting of basic volumes which will help later with drive letter assignment

                  diskpart.exe /s automountoff.txt

                   

                  :: assign a drive letter to this new volume, make sure it's not already utilized but you have to get the volume from the previously offline disk first

                  diskpart.exe /s findnolettervol.txt > diskdetail.txt

                  :: find the no letter volume from the new disk and assign a letter to it

                  findstr Volume diskdetail.txt > nolettervol.txt

                  :: get the volume number

                  for /f "tokens=2" %%c in (nolettervol.txt) do set vol=%%c

                  :: assign a drive letter to this volume using the passed parameter when batch was launched

                  diskpart.exe /s assignlettervol.txt

                  :: turn back on automounting of volumes which helps with removable storage, etc

                  diskpart.exe /s automounton.txt

                   

                  New text files to create, and their content:


                  automountoff.txt:

                       automount disable

                  automounton.txt:

                       automount enable

                  findnolettervol.txt:

                       select disk %disk%

                       detail disk

                  assignlettervol.txt:

                       select volume %vol%

                       attributes volume clear readonly

                       attributes volume clear hidden

                       assign letter %letter%

                   

                  I've noticed that the volumes would come up as read only which is a problem because Windows needs to write a signature.  Hence the added lines in the last text file.

              • Re: Automatically mount most recent snapshot and connect to target.
                Chris Aylott Adventurer

                Unable to edit the original post --- continues from point 2.

                 

                2. Open powershell as administrator run Set-ExecutionPolicy Unrestricted > Read what this means to you! Using the Set-ExecutionPolicy Cmdlet

                3. Save the code snippets below into c:\folder and rename accordingly i.e. recent-snapshot.bat.

                4. Run the create-diskpart-scripts.bat once only.

                5. Change the modifications in the script recent-snapshot.bat, add the session name of putty from step 1.7 above to the array-name=XXXXX

                6. Execute - ensure the drive letters you want to use are free, the script will not disconnect any targets on the first run, I would recommend not having any connected so you can see it in action.

                     recent-snapshot VOLUME DRIVE-LETTER

                    i.e. C:\folder\recent-snapshot AA-SQL01-DB01 F

                7. SQL-E-F-drive.bat is an example of usage with SQL

                 

                Batch file - create-diskpart-scripts.bat

                echo list disk >> listdisk.txt
                echo automount disable >> automountoff.txt
                echo select disk %%disk%% >> findnolettervol.txt
                echo detail disk >> findnolettervol.txt
                echo select volume %%vol%% >> assignlettervol.txt
                echo attributes volume clear readonly >> assignlettervol.txt
                echo attributes volume clear hidden >> assignlettervol.txt
                echo assign letter %%letter%% >> assignlettervol.txt
                echo select disk %%disk%% >> offlinedisk.txt
                echo online disk >> offlinedisk.txt
                echo attributes disk clear readonly >> offlinedisk.txt
                

                Batch file - recent-snapshot.bat

                :: This script will:
                :: 1. logout the existing iSCSI session
                :: 2. delete any existing snapshot previously created by this script
                :: 3. Create a clone copy of the most recent snapshot
                :: 4. change the ACL
                :: 5. connect the iSCSI target and associate it with a drive letter
                
                :: Usage
                :: Run the bat file followed by the name of the volume and the drive letter i.e. recent-snapshot.bat AA-GOLD-EXCH01-DB01 F
                
                :: NUMBER OF MODIFICATIONS 3:
                :: MOD 1. Having setup a SSH key with your array define the putty session name (array name) below
                :: MOD 2. Array discovery IP, used for connecting the target
                @ echo off
                set array-name=array-connect
                set discovery-ip=10.101.1.50
                
                :: MOD 3. If you do not want the cloned snapshot to have the default permission of full access (0) define the init group.
                set initator-grp=0
                
                :: paramater taken from the command line - do not change
                set volume=%1
                set letter=%2
                
                  echo Using the session %array-name%
                  echo .
                  echo .
                
                echo Disconnecting the iSCSI session for the existing mounted snpashot
                  echo .
                  echo .
                
                :: Logout of the old target
                :: Find the session ID of the existing connection - script has to be run once to get these values.
                findstr "Session Id" %volume%login.txt > %volume%sessionID.txt
                :: remove the gyberish leaving us with the SessionID
                for /f "tokens=4 delims= " %%a in (%volume%sessionID.txt) do (
                  set sessionid=echo %%a
                  goto :break
                  )
                  :break
                
                :: Logout the specified session ID
                iscsicli logouttarget %sessionid%
                
                echo Removing the existing snapshot clone-%volume%
                  echo .
                  echo .
                plink %array-name% vol --offline Clone-%volume% --force
                ping 192.0.2.2 -n 1 -w 20000 > nul
                plink %array-name% vol --delete Clone-%volume%
                ping 192.0.2.2 -n 1 -w 20000 > nul
                
                :: Find the most recent snapshot
                
                plink %array-name% snap --list > snap-list.txt
                :: export the snapshot list
                findstr /B /I %volume% c:snap-list.txt > snap.txt
                :: find the snapshots related to the volume
                
                for /f "tokens=2 delims= " %%a in (snap.txt) do (
                :: retrieve the most recent volume collection name
                  set mostrecent=%%a
                  goto :break
                  )
                  :break
                  echo Mounting the snapshot %mostrecent% for volume %volume% it will be called Clone-%volume% it will take a minute or so to appear within the GUI.
                  echo .
                plink %array-name% vol --clone %volume% --snapname %mostrecent% --clonename Clone-%volume%
                
                :: wait
                ping 192.0.2.2 -n 1 -w 20000 > nul
                
                :: Set the ACL on the created clone
                if %initiator-grp%==0 (GOTO :BREAK)
                  else (
                plink %array-name% vol --removeacl Clone-%volume%
                plink %array-name% vol --addacl Clone-%volume% --initiatorgrp %initiator-grp%)
                :BREAK
                
                :: Fetch the IQN
                plink %array-name% vol --info Clone-%volume% > vol-info.txt
                :: get the new volume information
                findstr /B /I iSCSI  c:vol-info.txt > iscsi.txt
                :: find the right line
                for /f "tokens=3 delims= " %%a in (iscsi.txt) do (
                :: get the IQN of the new volume created this can be passed into a powershell script for esx or MS.
                  echo %%a > %volume%iqn.txt
                  set iqn=%%a
                  )
                
                :: You now have the IQN - you can pass this into powershell via powershell.exe -file c:\putty\connect-target.ps1 "%iqn%"
                
                :: Prevent the volume from being automatically added when the disk is onlined
                diskpart.exe /s automountoff.txt
                
                :: Connect the new target
                iscsicli.exe qaddtarget %iqn% %discovery-ip%
                :: login to the target and save the output to extract the sessionID required when dismounting the volume on next run
                iscsicli.exe qlogintarget %iqn% > %volume%login.txt
                ping 192.0.2.2 -n 1 -w 20000 > nul
                
                :: fetch the disk
                diskpart.exe /s listdisk.txt > disks.txt
                
                :: online and clear read only attributes
                findstr Offline disks.txt > offline.txt
                for /f "tokens=2" %%b in (offline.txt) do set disk=%%b
                diskpart.exe /s offlinedisk.txt
                
                :: assign a drive letter to this new volume
                diskpart.exe /s findnolettervol.txt > diskdetail.txt
                :: find the no letter volume from the new disk and assign a letter to it
                findstr Volume diskdetail.txt > nolettervol.txt
                :: get the volume number
                for /f "tokens=2" %%c in (nolettervol.txt) do set vol=%%c
                :: online the volume with assigned drive letter
                diskpart.exe /s assignlettervol.txt
                
                  exit /b
                

                SQL server example - call this with a scheduled task

                :: Stop SQL Service
                net stop mssqlserver
                
                call recent-snapshot.bat CA-SQL01-DB-F F
                call recent-snapshot.bat CA-SQL01-LOG-E E
                
                :: Start SQL Service
                net start mssqlserver
                
                EXIT
                
                  • Re: Automatically Mount Most Recent Snapshot and Connect to Target
                    Jason Liu Adventurer

                    Hi Chris, I tried running your script but I'm getting error "GOTO was unexpected at this time" after "Mounting the snapshot %mostrecent% for volume %volume% it will be called Clone-%volume% it will take a minute or so to appear within the GUI. "

                     

                    Any idea where could be wrong? Thanks.

                     

                    P.S. I bypassed the ACL section now I'm having problem with this section:

                    1. for /f "tokens=3 delims= " %%a in (iscsi.txt) do
                    2. :: get the IQN of the new volume created this can be passed into a powershell script for esx or MS. 
                    3.   echo %%a > %volume%iqn.txt 
                    4.   set iqn=%%a 
                    5.   ) 

                    The echo didn't give me the right iqn, it's just empty. The iscsi.txt does have the right info like this:

                    iSCSI target: iqn.2007-11.com.nimblestorage:clone-fileserver-v665c089a794a082d.000000a6.1f1600b7