5 0 0:Automatic Retention (EOL) Management
Data retention is the amount of time a backup is kept by SEP sesam. When you create a media pool, you set the retention time for the pool. Running respective backup tasks creates a backup chain on your backup storage consisting of a full backup, followed by differential and incremental backups, a backup metadata file, and may also include other dependent backups, such as migrated and replicated backups. Some INCR backups use CBT (changed block tracking).
What is retention time and EOL (End of Lifetime)
Retention time specifies the time period for which backup data is protected after it is written to the media, so that the savesets are preserved and available for restore. Data retention is important to ensure that all regulations and retention schedules are met. How long you should retain data depends on the nature of your business, as well as regulatory, legal, and other requirements.
- Retention time is set up at a media pool level and is specified in days.
- The retention period starts from the date a saveset is written to the media (at the end time of the first backup) and thus defines the expiration date of the saveset – EOL (End of Lifetime). For example, the retention time of a media pool is 30 days and the data is backed up to the media on January 1, therefore the saveset EOL is January 31. There are three different EOL types associated with object types and also depending on the storage media used; for details, see the section EOL (retention) types.
- When the protection (EOL) expires, SEP sesam can use the media for backups again. For more details, see the section What happens when retention expires.
SEP sesam provides automatic EOL (retention) management to ensure recoverability of the entire backup chain and protect against data loss, based on backup chain dependencies. You can also manually adjust the EOL of your data, as described in Changing Retention (EOL).
|EOL (retention time) refers only to backups and related migrated and replicated savesets. SEP sesam logs, readability check logs, calendar sheet entries, and restore tasks have separate retention parameters. For details, see retention periods.|
What are backup chain dependencies
A backup chain consists of full, differential and incremental backups. Running respective backup tasks creates a backup chain on your backup storage consisting of a full backup, followed by differential and incremental backups, and, additionally, a backup metadata file. Some INCR backups use Changed Block Tracking (CBT), an incremental backup technology for virtual machines that creates faster and smaller backups.
An FDI backup chain can contain any number of backups that each depend on other backups in the chain and can also depend on another backup in the number of backups. A backup chain can contain the first (primary) backup and one or more dependent backups, such as migrated and replicated backups.
- A primary backup can be a backup that does not require any other backups for a successful data restore. Thus, a primary backup can be a complete backup, but a dependent backup may require additional backups for a successful data restore as it may depend on the additional backups.
- For example, for INCR backups all previous savesets (FULL, DIFF and INCR) must be present for a successful restore.
- If retention time is considered only from the perspective of an individual backup, it can ensure restorability of data only for that particular backup.
- To enable full recovery of the data backed up in the backup chain, all dependent backup savesets are tracked and their retention time is managed according to their dependencies.
Dependent backups are also classified by the respective depth of the dependent backups in the backup chain. The respective depth can be a measure of how many backups are required to complete the restore of a system, e.g. a VM, to a predetermined state.
SEP sesam provides the saveset tree view to determine dependencies and EOL of an FDI backup chain. This view shows a data structure that relates savesets to their dependencies. You should use it before manually changing the EOL parameter to avoid breaking the backup chain. For details, see Backup Chain Dependencies.
Example of a 14-day retention scenario
In a typical 14-day retention scenario, the first execution of the backup job creates a full backup. This is followed by differential and incremental backups. Once the 14-day retention (EOL) is reached, the savesets are marked as EOL-free and can be purged. If the backup data is stored on tape, its protection expires when the tape media EOL expires. For example, the backup chain has the following retention: FULL on pool MONTH (retention time:32), DIFF on pool WEEK (retention time:15), and INCR on pool DAY (retention time:7). The EOL of such an FDI chain is sufficient and no retention adjustment is required.
Automatic EOL adjustment
However, in some cases SEP sesam automatically adjusts the EOL to retain consistency of the backed up data and keep the backup chain readily available for restore. For example, it may happen that the expiry time of some savesets has already expired, but they have not been deleted due to one or more rules of the backup chain dependencies.
There are six main rules that lead to an adjusted EOL.
Rules for the automatically adjusted retention time
If the retention time is considered only from the perspective of an individual backup, it can ensure restorability of data only for that particular backup. To enable complete restore of data backed up in the backup chain, SEP sesam tracks all dependent backup savesets and manages their retention time according to their dependencies.
For example, INCR backups require all previous savesets (FULL, DIFF and INCR) to be available for a successful restore: INCR backup taken as the third INCR after the FULL requires the FULL, the first, the second, and the third INCR to provide complete restore capability. If a saveset is missing from the backup chain, data recovery to a specific point in time is not possible. For this reason, SEP sesam maintains control over the dependencies among the individual backup savesets and provides six rules for dependency-based automatic retention.
Rule #1: Full backups do not expire as long as dependent DIFF/INCR exist
For example, you set a media pool retention parameter to 30 days and run a FULL backup. This FULL saveset is initially kept for 30 days, e.g., until January 31. If a subsequent INCR or DIFF saveset in the chain has a longer EOL, e.g. an expiration date of February 3, the EOL of all previous savesets, including the FULL, is adjusted to the longer expiration date.
Rule #2: An increased EOL of a DIFF/INCR saveset results in an increased EOL of all dependent savesets
If the EOL parameter of a DIFF or INCR saveset is increased, SEP sesam also increases the EOL of all dependent savesets (FULL and other DIFF and INCR). In this way, SEP sesam ensures that the EOL of the FULL saveset and other dependent DIFF and INCR is not shorter than the potentially modified EOL of the DIFF or INCR saveset.
Rule #3: A decreased EOL of a DIFF/INCR saveset leads to reduced EOL of all dependent savesets
If the EOL of a DIFF or INCR saveset is decreased, SEP sesam decreases the EOL of all dependent savesets (FULL and other DIFF and INCR).
|If you use the Expire function to delete unneeded saveset(s) or backup set(s), SEP sesam issues a warning message, asking you to confirm your decision to expire the entire backup chain. If you allow the DIFF or INCR saveset(s) to expire, the entire backup chain will be deleted and overwritten.|
Rule #4: A too short EOL of DIFF/INC savesets leads to an increased EOL
If the DIFF/INCR backup detects that a saveset belonging to an FDI chain has an EOL that is too short, then any subsequent DIFF/INCR backup that runs on a pool with a longer retention time will increase the EOL of the saveset from that particular pool.
|If the EOL of a saveset belonging to an FDI chain has already expired, it will not be extended. In this case, the next DIFF/INCR backup will be executed as FULL backup.|
Rule #5: A new or migrated DIFF/INC backup results in an adjusted EOL for dependent savesets
When a new INCR or DIFF backup is run or an INCR or DIFF backup is migrated, SEP sesam automatically adjusts the EOL of all related savesets to preserve the backup data.
Rule #6: The last successful backup or migration is automatically retained
SEP sesam automatically retains the last successful backup or migration saveset if the next backup/migration fails. By extending the EOL of the last successful backup/migration, SEP sesam ensures that at least one successful backup is retained. This behavior is enabled by default and can be changed by setting the values of the corresponding keys, as described in Customizing Global Retention Policy.
|SEP sesam also allows you to manually adjust EOL if the default retention does not meet the requirements, but you should be careful with this option. Manually adjusted EOL overrides the EOL defined by the retention time in the media pool configuration and should only be used for special cases and exceptions. Some special rules apply to the tape media EOL, see section EOL (retention) types.|
What happens when the saveset expires
The protection of a saveset expires when its EOL is reached. The storage space of an expired saveset is not used immediately; SEP sesam uses the GET_OLDEST policy to preserve the data on the media for as long as possible. The expired saveset can be used again if the following conditions are met:
- As a rule, there must not be any other savesets that depend on this saveset. An expired saveset is not deleted until all sets in the backup chain that depend on it have expired and been deleted. You can override this condition by explicitly allowing the expiry date (EOL) of the entire backup chain to expire, which deletes the backup data of all related savesets.
- If a saveset is stored on tape, the EOL of all stored savesets must have expired.
- The media must not be write-protected (locked).
- SEP sesam Server automatically allocates the media with the oldest EOL for re-use. The oldest medium is the medium with the oldest locked until (is backup day+ retention time) date in the media pool.
Tracking the adjusted EOL
In the Media view in the GUI, you can check the column Media EOL changed by that shows EOL-changes made via the automatic EOL adjustment by Backup ID/Saveset ID, as well as the EOL that the user changed manually. The modified EOL is also recorded in the main log and can be generated for audit trail purposes, see Audit Logging.