11 Replies Latest reply: Apr 24, 2013 1:31 PM by Ajay Singh RSS

    Snapshot Retention

    Wayfarer

      In a previous post, we touched on how Nimble’s efficient snapshots allow you to effectively protect your critical data, and we looked at recovery point objectives (RPO) specifically for vSphere across the customer base.

       

      In this post, we’ll take a look at snapshots from a slightly different angle.  The number of snapshots that an array can handle is one metric you can look at to determine whether an array has efficient snapshot technology. However, that by itself is not enough - the other important metric to look at is retention.  Snapshots capture a point in time of a data set, such as a database. As data changes, each snapshot uses up some capacity to capture the changes since the previous snapshot.

       

      Efficient snapshot technologies (such as Nimble’s) can use significantly less capacity than some older implementations in the industry (as much as 30x less capacity compared to some industry leaders). Such improvements in snapshot efficiency can allow for snapshots to be retained for sufficient time to form the basis of a reliable and cost effective data protection strategy.

       

      Let’s take a look at the maximum length of time that snapshots are retained across the Nimble customer base (taken on April 4th 2013).

       

      snapshot_retention_per_volume.JPG.jpg

      In this distribution, we can see that the vast majority of Nimble Storage customers retain their snapshots for several weeks or longer.  In fact, the distribution is fairly even once it gets over a period of 1 day.  Over 15% of Nimble volumes have a retention period of over 3 months, which means that these data sets can roll back to old data at multiple points in time, from over 3 months ago should the need arise.  This is significantly higher than typical retentions of snapshots on other primary storage architectures in the industry (which rarely exceed a week).

       

      The numbers are even more significant because the Nimble install base is growing very rapidly (and a substantial fraction of the volumes in existence are relatively ‘young’). This real world deployment data validates that snapshots can be used as an efficient and cost effective way to protect data for weeks and months at a time.

       

      Are you using snapshots as part of your data protection strategy?   If so, how long are you keeping your snapshots?

        • Re: Snapshot Retention
          Newbie

          I am using snapshots as part of our protection plan. Snapshots cover our hourly and daily "backups" and this data is replicated offsite. our long term backup of data is still done on tape. After using this for a while I am a little confused on how to best manage snapshots. There are many controls e.g. Reserve, Quota and Warnings but what is the most efficient and safest way to manage the snapshots?

           

          thanks

          -k

          • Re: Snapshot Retention
            Nelson Penn Newbie

            We have protections schedules setup for snapshots on an hourly, daily, weekly, monthly and quarterly basis.  Our snapshots are replicated to our second array at our Disaster Recovery Site.  After having used tape backup systems for over 20 years in different companies, I was very happy to finally get rid of them when we purchased our Nimble SANs.

             

            • Hourly snapshots are done during the business hours and expire the next day.
            • Daily snapshots are done after business hours and expire after one week.
            • Weekly snapshots are done once a week and expire after 5 weeks.
            • Monthly snapshots are done every 4 weeks and expire after 4 months.
            • Quarterly snapshots are done every 12 weeks and expire after 4 years.
              • Re: Snapshot Retention
                Wayfarer

                Great info Nelson!  Thanks for sharing how you're using snapshots for data protection and disaster recovery with the rest of the community.

                 

                - Sheldon

                  • Re: Snapshot Retention
                    Newbie

                    That sounds great, I have not been able to dump tape yet (10 year retention rules) but I have limited tape to monthly and yearly. I currently use snapshot for hourly and daily and replicate those to our DR site. I think the part I have not wrapped my head around is the snapshot quota, reserve, and warning and how those all work. I recently had a warning that my snapshots were at 72% approaching the quota and apparently at the quota it takes the volume offline? Anyway I had to manually go in and delete snapshots to clear it up. How should I set the quotas and retention so it manages the snapshots automatically?

                     

                    thanks

                    -k

                      • Re: Snapshot Retention
                        Nelson Penn Newbie

                        Ken, are you asking if the SAN can automatically delete the oldest snapshot(s) to stay within your defined quota? I am not sure they have that capability right now, however if you look at my snapshot schedule above you will see that it is setup to "manages the snapshots automatically".

                         

                        My situation may be completely different from yours as far as what I am attempting to retain with the snapshots, however here is an example of the volume collection setup that I have.

                         

                        Hourly snapshots are "replaced" by a daily snapshot because I am not attempting to maintain an hourly change history, I am just trying to minimize potential data loss.  So I only need to retain hourly snapshots until I have taken a daily snapshot. For example, snapshots can be taken every hour from 8 am to 8 pm (inclusive, so total snapshots = 13).  The number of snapshots I would retain would be 13 as it would cover the entire day.  The next day when a snapshot is taken at 8 am the previous 8 am snapshot is deleted.

                         

                        I have a similar setup for the daily, weekly, monthly, and quarterly snapshots.  The weekly replaces daily, the monthly replaces the weekly, and the quarterly replaces the monthly.

                         

                        Thus, the volume collection snapshot schedule "manages the snapshots automatically".

                          • Re: Snapshot Retention
                            Newbie

                            Hi Nelson,

                             

                            Thanks for all your feedback. So to keep your daily snapshots at 13 are you controlling that with the "number of snapshots to retain locally" option in the volume collection? If so then you don't even really need to use the snapshot quota....

                             

                            thanks

                            -k

                              • Re: Snapshot Retention
                                Nelson Penn Newbie

                                Yes that is correct.  The snapshot quota lets you set a limit on how much storage space the snapshots for a specific volume can use, but in my case I do not currently use a quota (unlimited).

                                  • Re: Snapshot Retention
                                    Ajay Singh Wayfarer

                                    thanks for sharing your approach Nelson. It is indeed simpler to manage snapshot usage by tying it to a business goal (e.g. number of days of retention desired for restore purposes), rather than a capacity based quota. The UI and InfoSight do give you visibility into space usage if you also want to monitor capacity.