Managing EOL

From SEPsesam
Jump to: navigation, search
Docs latest icon.png Welcome to the latest SEP sesam documentation version 4.4.3 Tigon. For previous documentation version(s), check Documentation archive.

Copyright © SEP AG 1999-2017. All rights reserved.

Any form of reproduction of the contents or parts of this manual is allowed only with the express written permission from SEP AG. When compiling and designing user documentation SEP AG uses great diligence and attempts to deliver accurate and correct information. However, SEP AG cannot issue a guarantee for the contents of this manual.

Overview

When configuring SEP sesam environment, you set up media pools and define the retention time. Media pool retention time is specified in days and defines how long the backed up data on media remains protected after the data is written to the medium. The retention time period starts with the date a save set is written to the medium and lasts for the period defined by media pool's EOL. When the protection expires, SEP sesam can re-use the media for backups again. This is the basic principle and the simplest scenario.

However, to ensure restorability of the complete backup chain and to protect from data loss, SEP sesam provides dependency-based retention strategy performed by automated EOL adjustment.

What is dependency-based retention

For example, INCR backups require all previous save sets (FULL, DIFF and INCR) to be available for a successful restore. If the retention time is viewed only from the perspective of an individual backup, it can ensure restorability of data for this particular backup only. But to enable the complete restoration of data that was backed up in the backup chain, all dependent backup save sets must be tracked and their retention time must be managed according to their dependencies. For example, INCR backup that was taken after the third INCR after the FULL, requires the FULL, the first, the second, and the third INCR to provide complete restore capability. If some save set in the backup chain is missing, you will not be able to recover your data to a specific point in time. For this reason, SEP sesam maintains control over dependencies among the individual backup save sets and provides dependency-based automated retention.

SEP sesam also allows you to manually adjust EOL for individual save set (saveset EOL) that is stored on data store and for all backup-related save sets (backup EOL). For tape media, extending the backup EOL might result in extending the media EOL of a tape. For the same reason as with the dependency-based automated retention, SEP sesam will automatically adjust EOL of backup chain save sets if required. For details, see Manual EOL adjustment.

Retention behavior and different EOL parameters

Typically, you specify a media pool retention time when creating a media pool. This retention time serves as a basis to determine EOL for backed up data. The EOL property can be managed for four object types:

Media pool EOL

Media pool EOL specifies the retention time (in days) for save sets written to the respective pool. The saveset EOL is calculated when the data is written to the medium: it starts with the date a save set is created and lasts for the period defined by media pool's retention time. For example, a media pool retention time is 30 days, and the data is backed up to the medium on the 1st of January, therefore the saveset EOL is 31st of January.

Saveset EOL

This is the expiry date for each save set. If a save set is a part of a backup chain, its EOL follows the rules of dependency-based retention; EOL of a previous save set in the chain must be the same or longer to enable the complete restoration of data. For example, you specify a media pool retention parameter to 30 days and run a FULL backup. This FULL save set will initially be kept for 30 days, for example, to the 31st of January. If any following INCR or DIFF save set in the chain has longer EOL, for example, its expiry date is the 3rd of February, the EOL of all preceding save sets, including the FULL, will be adjusted to the longer expiry date. For details on dependency-based automated retention, see automated EOL adjustment. For details on manually adjusting EOL, see manual EOL adjustment.

Backup EOL

This is the expiry date for all data that belongs to the same backup. Backup EOL is determined based on the longest EOL of all save sets that belong to the same backup, including migrated and replicated save sets. For example, adjusting backup EOL of a particular save set from the 3rd of February to the to 3rd of March results in changed EOL for all related backup data, i.e., original backup, migrated backup, replicated backup, as well as for all backups in a backup chain, if a save set with adjusted backup EOL is a part of it. For details on dependency-based automated retention, see automated EOL adjustment. For details on manually adjusting EOL, see manual EOL adjustment.

Tape media EOL

When a save set is stored on tape, every stored save set has its own saveset EOL, but this does not represent the actual expiration date of the tape. Its expiration date corresponds to the maximum retention time (the longest EOL) identified on tape. Only when all of the save sets on tape have their retention time expired and the tape is not locked (write-protected), the entire tape is eligible for re-use. For details on how manually extending EOL affects EOL of the tape media, see Manually extending EOL.

Note that EOL refers only to backups. SEP sesam logs, readability check logs, calendar sheet entries and restore tasks have separate retention parameters. For details, see retention periods.

What happens when EOL is reached

Once a save set's end of life is reached, its protection expires. The storage space of an expired save set is not immediately used; SEP sesam uses the GET_OLDEST policy to preserve the data on the media for the longest possible time. The expired save set can be re-used when the following conditions are met:

  • Typically, there must be no other save sets that depend on this save set. For details, see How SEP sesam deals with EOL-related backup chain dependencies. You can override this condition by explicitly allowing the EOL for the whole backup chain to be set to expire, thus deleting backup data on all related save sets.
  • If a save set is stored on tape, the EOL of all stored save sets must be expired.
  • Media must not be write-protected (locked).
  • SEP sesam Server automatically assigns the medium with the oldest EOL for re-use. The oldest medium is a medium with the oldest locked until (is backup day+ EOL) date in the media pool.
Information sign.png Note
If the save set resides on tape media, the tape will not be re-used until all the save sets on it are expired. Tape media EOL always corresponds to the maximum retention time (the longest EOL) identified on tape. More precisely, tape media EOL is max EOL of all save sets stored on the tape or max EOL of DIFF/INCR save sets referring to FULL/DIFF/INCR save set that is stored on other media or even data store. Only when the retention time of all save sets on tape has expired and the tape is no longer locked (write-protected), the entire tape is eligible for re-use.

Automated EOL adjustment

In some cases, SEP sesam automatically adjusts EOL to retain the consistency of backed up data and ensures successful restore.

Managing EOL-related backup chain dependencies

When a new INCR or DIFF backup is run or an INCR or DIFF backup is migrated, SEP sesam automatically adjusts EOL of all related save sets in order to retain the backup data and keep the backup chain readily available for restore. In some special cases, SEP sesam also automatically increases the EOL of the whole FDI backup chain, thus preventing the backup chains from being orphaned. See below sections for details.

Increased EOL of a DIFF or INCR save set

If the EOL parameter of a DIFF or INCR save set is increased, SEP sesam will increase EOL of all dependent backups (FULL and other DIFF and INCR). This way SEP sesam ensures that EOL for the FULL backup and other related DIFF and INCR is not shorter than the potentially modified DIFF or INCR save set's EOL.

Decreased EOL of a DIFF or INCR save set

If EOL of a DIFF or INCR save set is decreased to a point in time in the past, SEP sesam will issue a warning message, prompting you to confirm your decision to set the whole backup chain to already passed time. Setting the expired EOL for DIFF or INCR save sets results in purging and overwriting the complete backup chain!

Too short EOL of DIFF/INC save sets

If DIFF/INCR backup detects that a save set belonging to a FDI chain has too short EOL or is missing in the target pool, then any consecutive DIFF/INCR backup that is running on a pool with longer EOL will increase the EOL of the save set from the respective pool.

Information sign.png Note
If EOL is already expired, it cannot be automatically extended. In such a case, you can set a longer EOL manually. For details, see section Manual EOL adjustement.
Example
The backup chain has the following retention specified: FULL on pool MONTH (EOL:32), DIFF on pool WEEK (EOL:15) and INCR on pool DAY (EOL:7). EOL of such FDI chain is sufficient, therefore EOL is not modified.

Missing DIFF/INC save set in the target pool

If DIFF/INCR backup detects that a save set belonging to a FDI chain is missing in the target pool, SEP sesam increases the EOL of the FDI chain save sets with maximum EOL on other pool.

If DIFF/INCR migration detects that a save set belonging to a FDI chain is missing in the target pool, SEP sesam increases the EOL of the FDI chain save sets only in the target migration pool.

Last successful backup or migration is automatically retained

SEP sesam may automatically retain the last successful backup or migration save set when the next backup/migration fails. If this behavior is enabled, SEP sesam extends the EOL of the previous successful backup/migration, thus ensuring that at least one successful backup is retained.

Information sign.png Note
Whether the last successful backup or migration is retained, depends on SEP sesam version:

This behavior can be changed by using parameters eol_adjust_failed_backup and eol_adjust_failed_migration and setting the value to 1 (save sets are retained) or 0 (not retained) (see below commands).


As of SEP sesam Tigon V2, the following commands can be used to set the desired behavior:

  • To activate automatic retention of the last successful backup or migration save set (set by default in 4.4.3.47 Tigon V2)
    • Insert the following entries if they do not yet exist in the table defaults:
    sql "INSERT INTO defaults (key,user_name,value) VALUES ('eol_adjust_failed_backup','sesam','1')"
    sql "INSERT INTO defaults (key,user_name,value) VALUES ('eol_adjust_failed_migration','sesam','1')"
    
    • or update:
    sql "UPDATE defaults SET value='1' WHERE key='eol_adjust_failed_backup' AND user_name='sesam'"
    sql "UPDATE defaults SET value='1' WHERE key='eol_adjust_failed_migration' AND user_name='sesam'"
    
  • To deactivate automatic retention of the last successful backup or migration save set (deactivated by default in 4.4.3.42 Tigon)
    • Insert the following entries if they do not yet exist in the table defaults:
    sql "INSERT INTO defaults (key,user_name,value) VALUES ('eol_adjust_failed_backup','sesam','0')"
    sql "INSERT INTO defaults (key,user_name,value) VALUES ('eol_adjust_failed_migration','sesam','0')"
    
    • or update:
    sql "UPDATE defaults SET value='0' WHERE key='eol_adjust_failed_backup' AND user_name='sesam'"
    sql "UPDATE defaults SET value='0' WHERE key='eol_adjust_failed_migration' AND user_name='sesam'"
    

COPY backup fails

If a COPY backup fails, the EOL of the last successful or with warnings COPY backup is increased to the EOL of the failed backup.

Example
COPY backup in pool YEAR (EOL: 375) fails. SEP sesam checks for previous successful COPY backup in the same pool YEAR and increases its EOL. Note that SEP sesam does not check for COPY save sets in other pools, e.g., in pool DAY (EOL:5).

FULL backup fails

If a FULL backup fails, the EOL of the last successful or with warnings FULL/DIFF/INCR backup is increased to the EOL of the failed FULL backup.

Example 1
FULL backup in pool YEAR (EOL: 375) fails. SEP sesam checks for previous successful FDI backup chain in the same pool YEAR, and increases the EOL of the whole chain (FULL/DIFF/INCR savesets). Note that SEP sesam does not check for FDI save sets on other pools, e.g., in pool DAY (EOL:5).
Example 2
FULL backup in pool MONTH (EOL: 32) fails, but another FULL in pool YEAR (EOL:375) already exists. In this case, EOL is not modified.

DIFF/INCR backup fails

If a DIFF/INCR backup fails, the EOL of the last successful or with warnings FULL/DIFF/INCR backup is increased to the EOL of the failed DIFF/INCR backup.

Manual EOL adjustment

It is not recommended to manually adjust EOL. This will override the EOL that was defined by the retention time (in days) in media pool configuration and started on date a save set is written to the media. The following options should be used for special cases and exceptions, for example, to allow premature deletion of an individual save set, or to increase the retention time of a particular backup chain to be stored longer than specified by current EOL.

  • You can modify saveset EOL for each individual save set, stored on a data store. The saveset EOL parameter is available under several properties views, for example, as Locked until option in the backup task properties, or as Saveset EOL (e.g., Components -> Data Store -> Properties -> tab Savesets -> Saveset EOL). You can extend or reduce the save set's retention time by setting the exact expiry date in the calendar. If the adjusted save set is a part of a backup chain, the whole chain might be affected.
  • Additionally, there is also the backup EOL parameter. This is the expiry date for all data that belong to the same backup, including migrated and replicated save sets. You can check and modify the backup EOL parameter by setting the exact expiry date in the calendar in GUI, by using the Savesets properties (e.g., Main Selection -> Components -> Data Store -> Properties -> tab Savesets -> Backup EOL). If the adjusted save set is a part of a backup chain, the whole chain is affected.

For details, see section How SEP sesam deals with EOL-related backup chain dependencies.

Manually reducing EOL

  • If you are reducing the saveset EOL (expiration date) to a point in time in the past, the save sets with expired EOL will be deleted. If one of the save sets is part of a FDI backup chain, this may result in potential data loss due to impossibility to restore your backups.
  • If you are reducing the backup EOL (expiration date) to a point in time in the past, the EOL is adjusted for all save sets based on the same backup, including migrated and replicated save sets. Backups with expired EOL will be deleted. If one of the backups is part of a FDI backup chain, this may result in potential data loss due to impossibility to restore your backups.

Manually extending EOL

Extending EOL can be used for special cases, such as increasing the retention time of a particular backup data that has also been migrated and is stored on different media pools.

  • If you are extending the saveset EOL (expiration date) and one of the save sets is part of of a FDI backup chain, then the EOL of the previous save sets in the chain will also be increased.
  • If you are extending the backup EOL (expiration date), the EOL is adjusted for all save sets based on the same backup, including migrated and replicated save sets. This may lead to extremely extended EOL for save sets that originally have a very short EOL. If one of the backups is part of a FDI backup chain, then the EOL of the previous save sets in the chain will also be increased.
Information sign.png Note
  • Extending backup EOL of save sets stored on tape media may extend the EOL of the tape media! For save sets stored on tape media, a specific retention time that would only apply to one of the stored save sets cannot be set. Every save set that is stored on tape has its own EOL, but this does not represent the actual expiration date of the tape. Tape media EOL is max EOL of all save sets stored on the tape or max EOL of DIFF/INCR save sets referring to FULL/DIFF/INCR save set that is stored on other media or even data store.
  • To reduce or increase the tape media EOL (shown as Locked until in the tape properties), the media EOL (identified by tape label) can be adjusted. Manually adjusted EOL is valid for all save sets on tape.
  • If the tape media EOL date has been reached, but the tape should not be re-used, you can also lock the tape (by using write protection). This option overrides media EOL.

Checking backup chain dependencies

You can use the save set tree view in GUI to determine dependencies and EOL of a FDI backup chain. Use this overview prior to manually changing EOL parameter to avoid breaking a backup chain.

SEP Tip.png Tip
Checking the save set tree summary will provide instant information about location and status of available save sets for restore. By checking the summary, e.g., availability 5, you can search for the save sets that are not readily available, then migrate them to allow mount and enable selective restore.

The save set tree displays a save set related details together with potential dependent save sets belonging to the same backup chain. The save set details are read-only. By providing an overview of the backup chain, you get insight on restorability of backups.

You can open the save set tree view by double-clicking the selected backup in the backup list:

  1. From Main Selection -> Job State -> Backups or from Main Selection -> Monitoring -> Last Backup State, double-click the selected backup.
  2. In the backup task properties window, open the tab Savesets.

Save set tree.png
The save set tree displays all save sets belonging to the same backup chain with the following details:

saveset
SEP sesam unique identification assigned to a save set.
starttime
The time when the backup was started.
level
The backup level type used for save set: F (FULL), D (DIFF), I (INCR) or C (COPY).

More detailed information displayed for each save set:

pool
The media pool to which the save set belongs.
EOL
The time when the save set's protection expires. For details, see section EOL-related backup chain dependencies.
avail
The priority number, based on the location of the save sets. It is useful for identifying save sets that are readily available for restore. For example, a save set in media pool DAY (data store) is migrated to another pool DeDup and then migrated to tape. The tape will have the lowest avail/priority because it is not readily available for restore. Check also the Availability in the Status at the end of the tree view, which is calculated from the avail of all displayed save sets. See below Availability for details.
reason
Explains the above avail – availability of individual save sets for restore. The following information shows the relation between the location and avail/prio. Priority is assigned numerically, with 1 being the lowest priority and 6 (or 7 if called with a specific pool) being the highest.
TAPE_SINGLE = 0
TAPE_NOT_IN_LOADER = 1
DISK_OFFLINE = 2
TAPE_ONLINE = 3
DISK_HARD = 4
DISK_STORE_CLONE = 5 
DISK_STORE = 6 
REQUESTED_POOL = 7 (this will appear if called with a specific pool, e.g., all save sets on the pool DAY and the saveset_tree was called with target pool DAY)
drivegroup
Displays the name of the drive group related to the save set's media pool.
drives
The number of the drive that was used for backup.
labels
Displays an internal identification of a save set (a media pool name and a 5-digit number), a potential barcode, prio (numerical representation of the availability, see above item), and comment.
status
Displays the summary of save sets availability – status, message on availability and a numerical representation. For example, as soon as 1 save set is migrated to another pool and deleted from the original pool, the availability will be lowered.

Save set tree status.png

See also

Save sets protection: Retention time and EOL