5 1 0:Automatic Retention (EOL) Management
Overview
Retention time defines a specific duration (in days) for which a backup or data set is protected against deletion or overwriting. It ensures that backups are retained for a predetermined period to meet operational, regulatory, or other requirements. Once the retention time for a backup expires, it may be deleted or overwritten based on the backup retention policies in place.
To set the retention time for backup data, SEP sesam uses the EOL attribute (End-of-Lifetime). The EOL is the expiration date of a backup or a saveset and defines the date when a backup or a saveset reaches the end of its defined retention time and can be removed or overwritten.
Retention time and EOL determine when backups can be safely removed from storage to free up space and resources. This ensures that data remains accessible for a specific period of time, and enables efficient management of storage resources by allowing the removal of backups that are no longer needed for recovery or compliance purposes.
Retention time is defined on a media pool level. When a saveset is written to a medium in a specific media pool, the expiration date is calculated and set, and retention begins. For example, a media pool has retention time of 30 days. If a saveset is created and written to this pool on January 1st, its EOL is set to January 31st. To maintain consistency of the backed-up data and keep the backup chain available for restore, SEP sesam automatically manages retention and adjusts the EOL of backups and media.
You can also manually adjust the EOL of your data, as described in Changing Retention (EOL).
Note | |
Retention time and EOL refer only to backups and related migrated and replicated savesets. SEP sesam logs, readability check logs, calendar sheet entries, and restore tasks have separate retention parameters. For details, see retention periods. |
Backup chain and EOL
For a backup chain, SEP sesam manages the EOL values of backups according to the rules of dependency-based retention, synchronizing the EOL of interdependent savesets in the chain. This ensures that no backup is prematurely removed from the system and prevents interruption of the backup chain. Interdependent backups remain available throughout their retention time to ensure the recoverability of the entire backup chain.
For more information refer to Backup Chain Dependencies.
Storage management
When the EOL is reached, the protection for a saveset expires. However, the storage space occupied by an expired saveset isn't immediately freed. When space is required, the savesets with the oldest EOL are deleted first. SEP sesam uses the GET_OLDEST policy to preserve data on the media for as long as possible.
To overwrite a saveset and reuse the storage space, there should be no other savesets dependent on a particular saveset. Automatic retention management tracks dependencies and synchronizes the expiration dates to prevent deletion of savesets that might be still required for restore. This condition can be bypassed by modifying the expiration date (EOL) of the backup chain manually.
Media can be reused when the EOL of all stored savesets expires and is not write-protected (locked).
SEP sesam automatically allocates the media in a media pool with the oldest expired EOL for reuse. The media that has the longest time since EOL expired is reused first.
EOL types
SEP sesam manages the EOL property for three main object types: savesets, backups, and tape media.
- Saveset EOL
- The expiration date of a single saveset. It is initially calculated by adding the backup day (the date when the data is written to the medium) and the retention time specific to the media pool.
- For example, in a media pool with 30 days retention time, a full backup is run on January 1st. EOL of this backup is set to January 31st. Five days later a differential backup is run, with EOL set to February 5th. Because differential backup depends on the preceding full backup, the EOL of the full backup is adjusted to match the EOL of the differential backup and is also set to February 5th.
- Backup EOL
- The expiration date of all savesets belonging to the same backup, including migrated and replicated savesets.
- For example, you extend the backup EOL for a saveset. The EOL is adjusted for all savesets belonging to this backup (original, migrated and replicated savesets) to match the new EOL.
Note | |
By default, failed backups are also retained for the configured media pool retention time. This backup retention behavior can be changed by modifying the EOL-related keys, as described in Customize the default retention behavior for backups and migration. These keys are not supported in versions prior to 4.4.3 Beefalo V2, where failed backups were automatically deleted after 3 days. |
- Media EOL
- The expiration date of a tape medium, determined by the longest saveset EOL on that medium. In case an interdependent saveset exists on another medium or datastore, the media EOL (Locked until attribute in tape properties) is adjusted accordingly. The tape can be reused after EOL of all savesets on tape expires (provided that the tape is not locked or write-protected). When another saveset is added to tape or EOL of an existing saveset on tape is changed, SEP sesam checks which EOL is the longest. If new EOL exceeds the current EOL of the tape, media EOL is automatically extended, otherwise it remains unchanged.
- For example, a media EOL is set to February 3rd. A saveset is added to the tape with EOL set to February 25th. The media EOL is extended to February 25th. Another saveset with EOL February 20th is added to the tape, and the media EOL remains set to February 25th.
Automatic adjustment of retention and EOL
If the retention time is considered from the perspective of an individual backup, it can ensure restorability of data only for that backup. To enable complete restore of data backed up in the backup chain, SEP sesam tracks all dependent backup savesets and manages their retention time according to their dependencies.
Automatic adjustment of EOL follows the rules of dependency-based retention and is based on the longest EOL of interdependent savesets in a backup chain. This ensures that a backup, on which other backups depend for restore, is not removed prematurely from the system, which could potentially disrupt the backup chain. If a saveset is missing from the backup chain, data recovery to a specific point in time is not possible. SEP sesam maintains control over the dependencies among individual backup savesets and provides rules for automatic retention management based on these dependencies.
INCR backups require all previous savesets (FULL, DIFF and INCR) in a chain to be available for a successful restore. INCR backup taken as the third INCR after the FULL requires the FULL, the first, the second, and the third INCR to provide complete restore capability.
Example
For example, the following backup strategy is employed:
- Monthly full backup every 4th Sunday, in media pool MONTHLY with retention time 31 days
- Weekly differential backup every Sunday, in media pool WEEKLY with retention time 14 days
- Daily incremental backup, in media pool DAILY with retention time 7 days
Backup chain starts with the full backup and ends with an incremental backup. Full backup in a chain is a standalone backup and doesn’t require any other backups to successfully restore data. Therefore, the EOL of the full backup doesn’t affect any other saveset EOL.
Differential backup depends on the full backup for restore. Therefore, if EOL of differential backup exceeds EOL of the full backup on which it depends, the EOL of the full backup is extended to match retention of the differential backup.
Incremental backup requires previous savesets of all backup levels in a chain for a successful restore. INCR backup taken as the third INCR after a DIFF requires the FULL, the last DIFF, and the last three INCRs to provide complete restore capability. The EOL of incremental backups is adjusted with every new dependent incremental backup. In addition, if EOL of incremental backup exceeds EOL of the differential and full backups on which it depends, the EOL of the differential and full backups is extended accordingly.
In case the entire backup chain is stored on tape, the media EOL matches with the longest EOL of saveset on the tape. If any of the subsequent savesets on the tape exceeds this EOL, the media EOL is adjusted accordingly.
The figure below illustrates the adjustment of saveset EOLs. Only the most significant backups are filled in to retain clarity for informational purposes and to avoid cluttering the image. Note that media EOL is initially set to match the first FULL backup, and is adjusted later with the second DIFF backup to match the longest saveset EOL on the tape.
Rules for the automatic EOL adjustment
Rule #1: Full backups do not expire as long as dependent DIFF/INCR exist
Full backups will not be allowed to expire or be deleted as long as there are dependent DIFF/INCR backups that rely on them. The EOL of a full backup is extended if the EOL of subsequent backups exceeds the EOL of the full backup they depend on. If the EOL of the dependent backups is shorter, the EOL of the full backup remains unchanged.
For example, A full backup has EOL set to February 1st. A subsequent DIFF backup has EOL set to January 23rd. Because the DIFF backup expires before the FULL backup (EOL of the FULL is longer), the backup chain is not broken and the EOL of the FULL is not adjusted. Another subsequent DIFF backup has EOL set to February 6th. Because the FULL backup expires before the dependent DIFF (EOL of the FULL is shorter), and the backup chain could become unrestorable, the EOL of the FULL is adjusted to February 6th.
Rule #2: An increased EOL of a DIFF/INCR saveset results in an increased EOL of all dependent savesets
If the EOL of a DIFF or INCR saveset is increased, SEP sesam also increases the EOL of all preceding savesets (FULL and other DIFF and INCR) on which it depends. SEP sesam ensures that EOL of the FULL saveset and other DIFF and INCR is not shorter than the potentially modified EOL of the dependent DIFF or INCR saveset.
Rule #3: A decreased EOL of a DIFF/INCR saveset leads to decreased EOL of all dependent savesets
If the EOL of a DIFF or INCR saveset is decreased, SEP sesam decreases the EOL of all dependent subsequent savesets (DIFF and INCR). Note that you cannot set the EOL in the past.
Rule #4: A too short EOL of DIFF/INC savesets leads to an increased EOL
If the DIFF/INCR backup detects that a saveset belonging to an FDI chain has an EOL that is too short, then any subsequent DIFF/INCR backup that runs on a pool with a longer retention time will increase the EOL of the saveset from that particular pool.
Note | |
If the EOL of a saveset belonging to an FDI chain has already expired, it will not be extended. In this case, the next DIFF/INCR backup will be executed as a FULL backup, with settings that were defined for that DIFF or INCR backup (most notably, the media pool). |
Rule #5: A new or migrated DIFF/INC backup leads to adjusted EOL for dependent savesets
When a new INCR or DIFF backup is run or an INCR or DIFF backup is migrated, SEP sesam automatically adjusts the EOL of all related savesets to match.
Rule #6: The last successful backup or migration is automatically retained
SEP sesam automatically retains the last successful backup or migration saveset if the next backup/migration fails. By extending the EOL of the last successful backup/migration, SEP sesam ensures that at least one successful backup is retained. This behavior is enabled by default and can be changed by setting the values of the corresponding keys, as described in Customizing Global Retention Policy.
Note | |
SEP sesam also allows you to manually adjust EOL if the default retention does not meet the requirements, but you should be careful with this option. Manually adjusted EOL overrides the EOL defined by the retention time in the media pool configuration. |
Rule #7: Media EOL cannot be shorter than the longest saveset EOL on that tape
For tape media, SEP sesam adjusts the media EOL to match the longest EOL of the savesets stored on the tape. The media EOL and saveset EOL are interdependent and changes affect one another.
If a saveset EOL on tape is increased or decreased, the media EOL is adjusted accordingly. Similarly, modifying media EOL impacts saveset EOL.
Viewing EOL
EOL for savesets and backups, and for tape media is displayed in GUI and Web UI in several views. In GUI you can also modify the EOL.
Where in Web UI
In Web UI the EOL is read-only and cannot be modified.
- Backup & Saveset EOL
- Monitoring -> Last Backup State -> click a backup task -> task properties – Storage Location -> in the storage locations table: Saveset EOL
- Monitoring -> Backups -> click a backup task -> task properties – Storage Location -> in the storage locations table: Saveset EOL
- Infrastructure -> Data Stores -> click the selected data store to open the properties -> in tab Properties click View savesets for this data store: columns Backup EOL and Saveset EOL
- Infrastructure -> Tapes -> click the selected tape to open the properties -> in tab Properties click View savesets for this tape: columns Backup EOL and Saveset EOL
- Tape media EOL
- Infrastructure -> Tapes, column Media EOL
Where in GUI
You can use the GUI to modify the EOL parameter in several different ways:
- Backup & Saveset EOL
- Job State -> Backups -> double-click a backup task -> task properties – Info 1 -> in the Storage location table: Saveset EOL
- Components -> Tapes -> Media -> select the media and open properties -> Savesets tab -> columns Backup EOL and Saveset EOL
- Components -> Media Pools -> select the media pool and expand it to open media -> double-click to open the media properties -> Media properties – Properties 1 -> tab Savesets: columns Backup EOL and Saveset EOL
- Components -> Data Stores -> double-click the selected data store to open the properties -> tab Savesets: columns Backup EOL and Saveset EOL
- Tape media EOL
- Components -> Tapes -> Media -> select one or more tapes -> right-click and select Change Media EOL
- Components -> Tapes -> Media -> select the tape to open its properties -> Locked until
Tracking the adjusted EOL
SEP sesam provides insight into how and when EOL values were modified. In Web UI, in the backup results window the Backup EOL displays EOL changes triggered manually or by automatic EOL adjustment process (in GUI this information can be found in the EOL changed by column in the Media view). Note that to display further details in the backup results window, you may have to switch to Advanced view in the Web UI.
When EOL adjustments occur automatically due to dependency-based retention or system-driven policies, this column displays the associated Backup ID or Saveset ID that initiated the adjustment, and helps you trace the specific backups or savesets responsible for the EOL change.
When a user manually alters the EOL, the EOL changed by column will show the username responsible for the modification.
The modified EOL is also recorded in the main log. You can generate these logs for audit and compliance purposes, providing a clear history of all EOL adjustments made (see also Audit Logging).
Note | |
Adjusting the Backup EOL of savesets stored on tape media may affect the Media EOL. The media EOL may also depend on the EOL of FULL/DIFF/INCR savesets stored on other media or even in datastores. |
See also
Changing Retention (EOL) – Backup Chain Dependencies – Backup Strategy Best Practices – Tape Management