5 1 0:Automatic Retention (EOL) Management

Overview

Data retention is the amount of time a backup is kept by SEP sesam. When you create a media pool, you set the retention time for the pool. Running respective backup tasks creates a backup chain on your backup storage consisting of a full backup, followed by differential and incremental backups, a backup metadata file, and may also include other dependent backups, such as migrated and replicated backups. Some INCR backups use CBT (changed block tracking).

What is retention time and EOL (End of Lifetime)

Retention time specifies the time period for which backup data is protected after it is written to the media, so that the savesets are preserved and available for restore. Data retention is important to ensure that all regulations and retention schedules are met. How long you should retain data depends on the nature of your business, as well as regulatory, legal, and other requirements.

• Retention time is set up at a media pool level and is specified in days.
• The retention period starts from the date a saveset is written to the media (at the end time of the first backup) and thus defines the expiration date of the saveset – EOL (End of Lifetime). For example, the retention time of a media pool is 30 days and the data is backed up to the media on January 1, therefore the saveset EOL is January 31. There are three different EOL types associated with object types and also depending on the storage media used; for details, see the section EOL (retention) types.
• When the protection (EOL) expires, SEP sesam can use the media for backups again. For more details, see the section What happens when retention expires.

SEP sesam provides automatic EOL (retention) management to ensure recoverability of the entire backup chain and protect against data loss, based on backup chain dependencies. You can also manually adjust the EOL of your data, as described in Changing Retention (EOL).

	Note
	EOL (retention time) refers only to backups and related migrated and replicated savesets. SEP sesam logs, readability check logs, calendar sheet entries, and restore tasks have separate retention parameters. For details, see retention periods.

What are backup chain dependencies

A backup chain consists of full, differential and incremental backups. Running respective backup tasks creates a backup chain on your backup storage consisting of a full backup, followed by differential and incremental backups, and, additionally, a backup metadata file. Some INCR backups use Changed Block Tracking (CBT), an incremental backup technology for virtual machines that creates faster and smaller backups.

An FDI backup chain can contain any number of backups that each depend on other backups in the chain and can also depend on another backup in the number of backups. A backup chain can contain the first (primary) backup and one or more dependent backups, such as migrated and replicated backups.

• A primary backup can be a backup that does not require any other backups for a successful data restore. Thus, a primary backup can be a complete backup, but a dependent backup may require additional backups for a successful data restore as it may depend on the additional backups.
• For example, for INCR backups all previous savesets (FULL, DIFF and INCR) must be present for a successful restore.
• If retention time is considered only from the perspective of an individual backup, it can ensure restorability of data only for that particular backup.
• To enable full recovery of the data backed up in the backup chain, all dependent backup savesets are tracked and their retention time is managed according to their dependencies.

Dependent backups are also classified by the respective depth of the dependent backups in the backup chain. The respective depth can be a measure of how many backups are required to complete the restore of a system, e.g. a VM, to a predetermined state.
SEP sesam provides the saveset tree view to determine dependencies and EOL of an FDI backup chain. This view shows a data structure that relates savesets to their dependencies. You should use it before manually changing the EOL parameter to avoid breaking the backup chain. For details, see Backup Chain Dependencies.

Example of a 14-day retention scenario
In a typical 14-day retention scenario, the first execution of the backup job creates a full backup. This is followed by differential and incremental backups. Once the 14-day retention (EOL) is reached, the savesets are marked as EOL-free and can be purged. If the backup data is stored on tape, its protection expires when the tape media EOL expires. For example, the backup chain has the following retention: FULL on pool MONTH (retention time:32), DIFF on pool WEEK (retention time:15), and INCR on pool DAY (retention time:7). The EOL of such an FDI chain is sufficient and no retention adjustment is required.

Automatic EOL adjustment
However, in some cases SEP sesam automatically adjusts the EOL to retain consistency of the backed up data and keep the backup chain readily available for restore. For example, it may happen that the expiry time of some savesets has already expired, but they have not been deleted due to one or more rules of the backup chain dependencies.

There are six main rules that lead to an adjusted EOL.

Rules for the automatically adjusted retention time

If the retention time is considered only from the perspective of an individual backup, it can ensure restorability of data only for that particular backup. To enable complete restore of data backed up in the backup chain, SEP sesam tracks all dependent backup savesets and manages their retention time according to their dependencies.

For example, INCR backups require all previous savesets (FULL, DIFF and INCR) to be available for a successful restore: INCR backup taken as the third INCR after the FULL requires the FULL, the first, the second, and the third INCR to provide complete restore capability. If a saveset is missing from the backup chain, data recovery to a specific point in time is not possible. For this reason, SEP sesam maintains control over the dependencies among the individual backup savesets and provides six rules for dependency-based automatic retention.

Rule #1: Full backups do not expire as long as dependent DIFF/INCR exist

For example, you set a media pool retention parameter to 30 days and run a FULL backup. This FULL saveset is initially kept for 30 days, e.g., until January 31. If a subsequent INCR or DIFF saveset in the chain has a longer EOL, e.g. an expiration date of February 3, the EOL of all previous savesets, including the FULL, is adjusted to the longer expiration date.

Rule #2: An increased EOL of a DIFF/INCR saveset results in an increased EOL of all dependent savesets

If the EOL parameter of a DIFF or INCR saveset is increased, SEP sesam also increases the EOL of all dependent savesets (FULL and other DIFF and INCR). In this way, SEP sesam ensures that the EOL of the FULL saveset and other dependent DIFF and INCR is not shorter than the potentially modified EOL of the DIFF or INCR saveset.

Rule #3: A decreased EOL of a DIFF/INCR saveset leads to reduced EOL of all dependent savesets

If the EOL of a DIFF or INCR saveset is decreased, SEP sesam decreases the EOL of all dependent savesets (FULL and other DIFF and INCR).

	Warning
	If you use the Expire function to delete unneeded saveset(s) or backup set(s), SEP sesam issues a warning message, asking you to confirm your decision to expire the entire backup chain. If you allow the DIFF or INCR saveset(s) to expire, the entire backup chain will be deleted and overwritten.

Rule #4: A too short EOL of DIFF/INC savesets leads to an increased EOL

If the DIFF/INCR backup detects that a saveset belonging to an FDI chain has an EOL that is too short, then any subsequent DIFF/INCR backup that runs on a pool with a longer retention time will increase the EOL of the saveset from that particular pool.

	Note
	If the EOL of a saveset belonging to an FDI chain has already expired, it will not be extended. In this case, the next DIFF/INCR backup will be executed as FULL backup.

Rule #5: A new or migrated DIFF/INC backup results in an adjusted EOL for dependent savesets

When a new INCR or DIFF backup is run or an INCR or DIFF backup is migrated, SEP sesam automatically adjusts the EOL of all related savesets to preserve the backup data.

Rule #6: The last successful backup or migration is automatically retained

SEP sesam automatically retains the last successful backup or migration saveset if the next backup/migration fails. By extending the EOL of the last successful backup/migration, SEP sesam ensures that at least one successful backup is retained. This behavior is enabled by default and can be changed by setting the values of the corresponding keys, as described in Customizing Global Retention Policy.

	Note
	SEP sesam also allows you to manually adjust EOL if the default retention does not meet the requirements, but you should be careful with this option. Manually adjusted EOL overrides the EOL defined by the retention time in the media pool configuration and should only be used for special cases and exceptions. Some special rules apply to the tape media EOL, see section EOL (retention) types.

What happens when the saveset expires

The protection of a saveset expires when its EOL is reached. The storage space of an expired saveset is not used immediately; SEP sesam uses the GET_OLDEST policy to preserve the data on the media for as long as possible. The expired saveset can be used again if the following conditions are met:

• As a rule, there must not be any other savesets that depend on this saveset. An expired saveset is not deleted until all sets in the backup chain that depend on it have expired and been deleted. You can override this condition by explicitly allowing the expiry date (EOL) of the entire backup chain to expire, which deletes the backup data of all related savesets.

• If a saveset is stored on tape, the EOL of all stored savesets must have expired.

• The media must not be write-protected (locked).

• SEP sesam Server automatically allocates the media with the oldest EOL for re-use. The oldest medium is the medium with the oldest locked until (is backup day+ retention time) date in the media pool.

Tracking the adjusted EOL

In the Media view in the GUI, you can check the column Media EOL changed by that shows EOL-changes made via the automatic EOL adjustment by Backup ID/Saveset ID, as well as the EOL that the user changed manually. The modified EOL is also recorded in the main log and can be generated for audit trail purposes, see Audit Logging.

Note

SEP sesam automatically adjusts the EOL to retain consistency of the backed-up data in the backup chain. A backup chain consists of full, differential and incremental backups, a backup metadata file, and may also include other dependent backups, such as migrated and replicated backups. Adjusting the Backup EOL of savesets stored on tape media may affect the Media EOL of the tape. The latter may also depend on the EOL of FULL/DIFF/INCR savesets stored on other media or even in data stores.

The EOL property can be managed for three object types:

Saveset EOL

The expiry date for a single saveset:
The date the data was written to the media + the retention time of the media pool
If a saveset is part of a backup chain, its EOL follows the rules of dependency-based retention; the EOL of a previous saveset in the chain must be the same or longer to enable complete restore of the data.

Example

1. Media pool retention = 30 days.
2. A FULL backup is run -> FULL saveset is kept for 30 days (e.g., January 31 <YYYY>).
3. If a subsequent INCR or DIFF saveset in the chain has a longer EOL (e.g., February 3 <YYYY>), the EOL of the FULL saveset (and of all preceding savesets) is adjusted to match the longer EOL (e.g., February 3).

Extend retention

If you manually extend the Saveset EOL and one of the savesets is part of an FDI backup chain, the EOL of the previous savesets in the chain is automatically increased. Extending the EOL of savesets stored on tape media may extend the EOL of the tape media! See below Tape media EOL.

Shorten retention

Shortening the Saveset EOL results in a new expiry date that applies only to the selected individual saveset.

Delete a saveset

(see also: Deleting savesets, backups or tape media)

Right-clicking the saveset -> Saveset -> Delete Saveset deletes the selected saveset(s), unless the saveset(s) is/are part of a backup chain; in the latter case, the entire backup chain is affected. Deletion can be executed immediately or delayed until the next purge.

Backup EOL

The expiry date for all data belonging to the same backup, including migrated and replicated savesets.

Note

How SEP sesam manages failed backups depends on its version. In v. ≥ 4.4.3 Beefalo V2, SEP sesam retains the failed backup according to the media pool retention time together with the last successful backup or migration saveset. This is the default backup retention behavior and can be changed by modifying the EOL-related keys, as described in Customize the default retention behavior for backups and migration. These keys may not be supported in earlier versions, where failed backups were automatically deleted after 3 days.

Example

1. The EOL of a saveset belonging to the backup chain is extended from February 3 <YYYY> to March 3 <YYYY>.
2. All related backup data, i.e., original, migrated and replicated backups, as well as all backups in a backup chain, now have a Backup EOL set to March 3.

Extend retention

If you extend the Backup EOL, SEP sesam automatically increases the EOL of all dependent savesets (FULL and other DIFF and INCR).

Shorten retention

Shortening the Backup EOL only adjusts the EOL of the savesets that have a longer EOL than the newly set date, while the savesets with a shorter EOL are not affected (their EOL remains unchanged).

Delete a backup

(see also: Deleting savesets, backups or tape media)

Right-clicking the saveset -> Backup -> Delete Backup deletes all data belonging to the same backup (the entire backup chain) that will be deleted immediately or with the next purge.

Tape media EOL

The time until which backed up data on tape remains protected. It refers to tape media and is based on the longest EOL of all the different savesets stored on tape:
The expiry date of the tape = maximum EOL on the tape
A specific retention time that would apply to only one of the savesets stored on the tape cannot be set.

The tape media EOL may also depend on dependent FULL/DIFF/INCR savesets stored on other media or even data stores.
Only when all savesets on the tape have expired and the tape is not locked (write-protected) can the tape be used again.

Example

1. Tape media EOL = February 3 <YYYY>
2. The Backup EOL of a saveset on tape is extended to March 3 <YYYY>. If this is the longest retention period of all savesets on the tape, the Media EOL (Locked until date in tape properties) is automatically extended to March 3; if not, the Media EOL of the tape remains unchanged.

Extend retention

Extending the Media EOL results in a new expiry date that applies to all savesets on the tape.

Shorten retention

Shortening the tape Media EOL applies to all savesets on the tape.

Expire retention

(see also: Deleting savesets, backups or tape media)

If the tape is loaded in the drive or library, you can erase the tape. Right-click the Expire function in the Media view or display tape properties and click Expire Media. This action affects the entire tape. The metadata of the tape is removed and the tape is reinitialized. Deletion can be executed immediately or delayed until the next purge.

Where in GUI

You can use the GUI to modify the EOL parameter in several different ways:

Backup & Saveset EOL

• Job State -> Backups -> double-click a backup task -> task properties – Info 1 -> in the Storage location table: Saveset EOL
• Components -> Tapes -> Media -> select the media and open properties -> Savesets tab -> columns Backup EOL and Saveset EOL
• Components -> Media Pools -> select the media pool and expand it to open media -> double-click to open the media properties -> Media properties – Properties 1 -> tab Savesets: columns Backup EOL and Saveset EOL
• Components -> Data Stores -> double-click the selected data store to open the properties -> tab Savesets: columns Backup EOL and Saveset EOL

(Tape) Media EOL

• Components -> Tapes -> Media -> select one or more tapes -> right-click and select Change Media EOL
• Components -> Tapes -> Media -> select the tape to open its properties -> Locked until

Write protection for tape media

This special option allows you to set additional software protection for tape media. You can write-protect selected tape media by setting this option ON manually: Main Selection window -> Components -> Tapes -> Media -> column Write Protection (On/Off).

This option overrides the media pool retention time and any individually adjusted EOL values and sets a permanent protection for savesets on tape. Write-protected media cannot be used until write protection is deactivated (the media are locked). It can be turned off manually at any time. When the media are no longer write-protected, the retention time of the media pool applies.

Locking a backup

You can also lock your backups in the GUI and SEP sesam Web UI to prevent them from being deleted after the retention time has expired.

Note

How SEP sesam handles locked backups that are part of the backup chain depends on the version (see Automatic Retention (EOL) Management). In the current 5.0.0 Jaglion version, the lock status is automatically applied to all savesets of a backup (backup chain, migrated and replicated savesets). All backups belonging to the same chain are protected from deletion until one of them is unlocked. In v. < 4.4.3 Beefalo V2, the lock status was only applied to the selected backup. If you want to lock all backups of a backup chain as well as migrated and replicated backups while using some of the older SEP sesam versions, you have to lock each backup manually.

In GUI

There are several ways how to lock a backup in the GUI:

• Job State -> Backups -> double-click a backup task -> task properties – Info 1 -> above the Storage location table on the right select the check box Lock state
• Components -> Tapes -> Media -> select the media and open properties -> Savesets tab -> search for the column Locked and click the Off property, then select On
• Components -> Media Pools -> select a media pool and expand it to open the media -> double-click to open the Media properties -> Media properties – Properties 1 -> tab Savesets: search for the column Locked and click the Off property, then select On
• Components -> Data Stores -> double-click the selected data store to open the properties -> Savesets tab -> look for the column Locked and click the Off property, then select On

In Web UI

You can access the Web UI in one of the following ways:

• via the GUI: by clicking the Dashboard icon in the toolbar or
• under menu bar -> Activities -> Dashboard or via Main Selection -> Monitoring -> Dashboard
• or from Activities -> Restore Assistant
• or by entering the following address in the browser bar: http://[sesamserver]:11401/sep/ui/restore/.

Note

Make sure the web UI is in advanced mode. (You can change the UI mode in the side menu under the View option at the bottom left). The operations and options available after login may vary depending on the user type. Other display restrictions of Web UI may depend on the custom roles with specific permissions and UI mode. For details, see About Authentication and Authorization and User Roles and Permissions.

1. In Web UI, select Monitoring from the side menu to open a submenu, then go to Last backup state or Backups and click the link of the backup task you want to lock.
2. A new view with Details is displayed. Scroll down and select the red button Lock this backup next to Lock state.

See also

Changing Retention (EOL) – Backup Chain Dependencies – Backup Strategy Best Practices – Tape Management

Copyright © SEP AG 1999-2023. All rights reserved.

Any form of reproduction of the contents or parts of this manual is allowed only with the express written permission from SEP AG. When compiling and designing user documentation SEP AG uses great diligence and attempts to deliver accurate and correct information. However, SEP AG cannot issue a guarantee for the contents of this manual.