Source:Data Store: Difference between revisions
(Minor update.) |
m (Removed info on DDLs (RS)) |
||
(22 intermediate revisions by the same user not shown) | |||
Line 7: | Line 7: | ||
<!--T:3--> | <!--T:3--> | ||
SEP sesam data store is a device type used for writing | The SEP sesam data store is a device type used for writing savesets directly to one or several configured storage locations – into the file system. SEP sesam uses a data store instead of a conventional media pool to define the '''storage repository'''. Data is still primarily backed up to a media pool, however, a data store is used underneath to save data to dynamically managed data areas, including disk backups. | ||
<!--T:4--> | <!--T:4--> | ||
SEP sesam can contain multiple data stores of different types and sizes | SEP sesam can contain multiple data stores of different types and sizes, depending on the type of data being backed up, the backup technique, and the storage location (a local disk, a virtualized storage device, storage appliances, etc.). The following data store types are supported:</translate> | ||
<ul><li><translate><!--T:5--> ''Path'': The default data store type, relevant | <ul><li><translate><!--T:5--> ''Path'': The default data store type, relevant to the configuration of most storage locations, unless one of the following data store types is required, e.g., for deduplication or backup to cloud storage.</translate></li> | ||
<li><translate><!--T:6--> ''SEP Si3 deduplication store'': Used for | <li><translate><!--T:6--> | ||
<li><translate><!--T:7--> ''NetApp Snap Store'': Used for [[Special:MyLanguage/ | ''SEP Si3 deduplication store'': Used for [[Special:MyLanguage/Deduplication#target|target-based (Si3T)]] and [[Special:MyLanguage/Deduplication#source|source-based deduplication (Si3S)]], [[Special:MyLanguage/Replication|replication]], [[Special:MyLanguage/5_0_0:Si3_NG_Direct_to_S3|backup directly to S3 cloud storage]] (Si3 NG store) and [[Special:MyLanguage/S3_Cloud_Storage_Backup|S3 cloud replication]] (Si3 store). | ||
<li><translate><!--T:8--> ''HPE StoreOnce'': Used for integration with [[Special:MyLanguage/ | |||
<li><translate><!--T:93--> ''HPE Cloud Bank Store'': Used for replicating the data to [[Special:MyLanguage/ | <!--T:104--> | ||
{{note|As of SEP sesam [[SEP sesam Release Versions|v. Jaglion]], two Si3 deduplication store types are available. The common name for both types is Si3 deduplication store, unless it is explicitly stated that the information refers exclusively to ''Si3'' (first generation) or ''Si3 NG'' (next generation). SEP sesam recommends using the new generation Si3 NG when creating a new data store. Si3 NG is advantageous over regular Si3 as it offers better performance, scaling, and resource savings. It allows you to back up your data directly to S3 cloud storage and restore the items you want directly from there.}} | |||
</translate></li> | |||
<li><translate><!--T:7--> ''NetApp Snap Store'': Used for [[Special:MyLanguage/NetApp_Storage_Snapshots_Backup|NetApp snapshots backup]].</translate></li> | |||
<li><translate><!--T:8--> ''HPE StoreOnce'': Used for integration with the [[Special:MyLanguage/SEP_sesam_HPE_StoreOnce_Configuration|HPE StoreOnce Catalyst storage system]]</translate>.</li> | |||
<li><translate><!--T:93--> ''HPE Cloud Bank Store'': Used for replication of data to [[Special:MyLanguage/HPE_StoreOnce_Replication|HPE Cloud Bank Storage]].</translate></li> | |||
<!--<li><translate><!--T:103--> ''HPE Cloud Volumes'' (in v. ≥ 5.0.0 ''Jaglion''): Used for backing up or replicating the data to [[Special:MyLanguage/5_0_0:HPE_Cloud_Volumes_Backup|HPE Cloud Volumes Storage]].</translate></li>--> | |||
</ul> | </ul> | ||
<translate> | <translate><!--T:10--> [[Image:Data_store_overview_Beefalo.jpg|800px|link=]]</translate> | ||
<!--T:10--> | |||
[[Image:Data_store_overview_Beefalo.jpg|800px|link=]]</translate> | |||
<br clear=all> | <br clear=all> | ||
Line 29: | Line 31: | ||
<!--T:91--> | <!--T:91--> | ||
SEP sesam provides additional ''View Mode'' button that enables you to switch between the table view and the tree view with grouped objects.</translate> | SEP sesam provides an additional ''View Mode'' button that enables you to switch between the table view and the tree view with grouped objects.</translate> | ||
*<translate><!--T:12--> '''''Tree view''''': SEP sesam default view shows the status of individual data stores grouped together in a hierarchical view.</translate> | *<translate><!--T:12--> '''''Tree view''''': SEP sesam default view shows the status of individual data stores grouped together in a hierarchical view.</translate> | ||
*<translate><!--T:13--> | *<translate><!--T:13--> | ||
Line 37: | Line 39: | ||
<!--T:15--> | <!--T:15--> | ||
SEP sesam data stores can be of different types depending on the type of data being backed up, backup technique, and used storage. The default and most commonly used data store type is ''Path''. The following information applies to | SEP sesam data stores can be of different types depending on the type of data being backed up, backup technique, and used storage. The default and most commonly used data store type is ''Path''. The following information applies generally to a data store, unless explicitly specified otherwise; data store types other than ''Path'' may have slightly different characteristics. For details, see the relevant articles: [[Special:MyLanguage/Configuring_an_Si3_Deduplication_Store|SEP Si3 deduplication store]], [[Special:MyLanguage/NetApp_Storage_Snapshots_Backup|NetApp Snap Store]], [[Special:MyLanguage/SEP_sesam_HPE_StoreOnce_Configuration|HPE StoreOnce]], and [[Special:MyLanguage/HPE_StoreOnce_Replication|HPE Cloud Bank Storage]]. | ||
<!--T:16--> | <!--T:16--> | ||
The difference between a conventional media pool, typically used for backing up directly to tapes, and a data store is in defining the storage space directory directly in the drive by using the operating system's partition functions. Therefore the data store space is managed at the partition level. You configure a data store by specifying its capacity and (optionally) setting the [[Special:MyLanguage/4_4_3_Beefalo:Data_Store#watermark|high watermark]] | The difference between a conventional media pool, typically used for backing up directly to tapes, and a data store is in defining the storage space directory directly in the drive by using the operating system's partition functions. Therefore the data store space is managed at the partition level. You configure a data store by specifying its '''capacity''' and (optionally) setting the [[Special:MyLanguage/4_4_3_Beefalo:Data_Store#watermark|high watermark]] (HWM) value. This is the upper value for the used disk space on the data store. Take into account that exceeding HWM and filling up the data store may cause backup issues. You should consider this when specifying the data store capacity. For details, see [[Special:MyLanguage/4_4_3_Beefalo:Data_Store#recommendations|Data store calculation recommendations]] below.</translate> | ||
{{<translate><!--T:17--> Note</translate>|<translate><!--T:18--> Only one data store should be used for each hard disk partition. Even though several data stores can be set up on one partition, you are advised against such configuration as each data store reads the values of the other partitions when checking partition allocation. Consequently, such coexisting data stores obstruct each other.</translate>}} | {{<translate><!--T:17--> Note</translate>|<translate><!--T:18--> Only one data store should be used for each hard disk partition. Even though several data stores can be set up on one partition, you are advised against such configuration as each data store reads the values of the other partitions when checking partition allocation. Consequently, such coexisting data stores obstruct each other.</translate>}} | ||
<translate><!--T: | <translate> | ||
As shown in the illustration below, a media pool still | <!--T:102--> | ||
As shown in the illustration below, a media pool still refers to a drive group. The drive group points to exactly one data store between the media pool and the drives. The connection between the data store and the related drive is static. | |||
<!--T:20--> | <!--T:20--> | ||
[[image: | [[image:Data_store_principle_graphic_Beefalo.png|750px|link=]]</translate> | ||
<br clear=all> | <br clear=all> | ||
Line 54: | Line 57: | ||
<!--T:22--> | <!--T:22--> | ||
Data store configuration consists of specifying the data store capacity and [[Special:MyLanguage/4_4_3_Beefalo:Data_Store#watermark|HWM]]. The data store capacity is space reserved for the SEP sesam data store and, optionally, non-SEP sesam data that might be stored on the same volume as the SEP sesam data store. If the data store is shared with non-SEP sesam data, you | Data store configuration consists of specifying the '''data store capacity''' and [[Special:MyLanguage/4_4_3_Beefalo:Data_Store#watermark|'''HWM''']]. The data store capacity is space reserved for the SEP sesam data store and, optionally, non-SEP sesam data that might be stored on the same volume as the SEP sesam data store. If the data store is shared with non-SEP sesam data, you have to obtain a special SEP sesam storage license. | ||
<!--T:105--> | |||
{{note|The Si3 deduplication store has specific hardware and sizing requirements. For more information, see [[Special:MyLanguage/Si3 Deduplication Hardware Requirements|Si3 Deduplication Hardware Requirements]] and [[Special:MyLanguage/Configuring_an_Si3_Deduplication_Store|Configuring Si3 Deduplication Store]].}} | |||
<!--T:23--> | <!--T:23--> | ||
Line 70: | Line 76: | ||
<!--T:28--> | <!--T:28--> | ||
A ''high watermark'' is a general parameter used | A ''high watermark'' is a general parameter that specifies the upper value for the used disk space on the data store. When this value is reached, the status of a data store changes from ''OK'' to ''Warning'', but backups continue to be performed. | ||
<!--T:106--> | |||
The ''Si3 repair area'' parameter is relevant only for SEP Si3 deduplication store. It specifies reserved space for unused Si3 files, as explained [[Special:MyLanguage/4_4_3_Beefalo:Data_Store#repair_area|below]].</translate> | |||
<translate><!--T: | <ul><li><translate><!--T:29--> | ||
'''''High watermark (HWM)''''': A parameter for managing disk space usage; it defines the upper value for the used disk space on the data store. This parameter can be specified manually for the ''Path'', ''Si3'' and ''NetApp Snap'' store, and is set automatically during the creation of a ''SEP Si3 Deduplication'' store. <br />When this value is reached, the status of a data store changes to ''Warning''. It serves as an indication for the administrator that the data store is running out of space, but does not prevent backups to be started. | |||
<!--T:107--> | |||
A data store purge process is running automatically for all ([[Special:MyLanguage/SEP_sesam_Glossary#EOL|EOL-free]]) savesets. The oldest free savesets are deleted first. Purge is done until all EOL-free savesets are deleted. For more details on EOL, see [[Special:MyLanguage/5 0 0:Automatic Retention (EOL) Management|Automatic Retention (EOL) Management]].</translate></li> | |||
<translate><!--T:33--> {{anchor|purge}}Events that trigger the data store purge are:</translate> | <translate><!--T:33--> {{anchor|purge}}Events that trigger the data store purge are:</translate> | ||
Line 88: | Line 94: | ||
<!--T:37--> | <!--T:37--> | ||
The manual execution of the data store purge process deletes the obsolete ([[Special:MyLanguage/SEP_sesam_Glossary#EOL|EOL-free]]) savesets. Another option for releasing the data store space is to clean up a data store, as described in the [[Special:MyLanguage/4_4_3_Beefalo:Data_Store#clean_up| | The manual execution of the data store purge process deletes the obsolete ([[Special:MyLanguage/SEP_sesam_Glossary#EOL|EOL-free]]) savesets. Another option for releasing the data store space is to clean up a data store by deleting orphaned savesets, as described in the [[Special:MyLanguage/4_4_3_Beefalo:Data_Store#clean_up|following section]].</translate> | ||
<translate><!--T:39--> {{anchor|repair_area}}</translate><li><translate><!--T:40--> '''''Si3 Repair Area''''': A parameter used by ''SEP Si3 Deduplication Store'' for managing disk space dedicated to Si3 files that were identified by a garbage collection job and are no longer used. These files are still kept in the repair area to enable possible repair of Si3 if there are any structural problems (may be caused by a file system error or by a crash of an operating system). The files in the repair area are removed automatically after the specified amount of time (SEP sesam default: 4 days) or when the disk usage threshold is reached. If the value is set to 0, the Si3 repair functionality is turned off.</translate></li> | |||
{{<translate><!--T:96--> note</translate>|<translate><!--T:97--> The ''Si3 repair area'' is available only in [[Special:MyLanguage/SEP_sesam_Glossary#UI_mode|''advanced'' UI mode]] (formerly ''expert'' GUI mode). To see the ''Si3 repair area'' field, make sure your UI mode is set to ''advanced''. For details, see [[Special:MyLanguage/4_4_3_Beefalo:SEP_sesam_GUI#UI_mode|Selecting UI mode]].</translate>}} | |||
<translate><!--T:39--> {{anchor|repair_area}}</translate><li><translate><!--T:40--> '''''Si3 Repair Area''''': A parameter used by ''SEP Si3 Deduplication Store'' for managing disk space dedicated to Si3 files | </ul> | ||
<translate>===={{anchor|clean_up}}Clean up orphaned savesets ==== <!--T:41--> | <translate>===={{anchor|clean_up}}Clean up orphaned savesets ==== <!--T:41--> | ||
Line 114: | Line 120: | ||
*<translate><!--T:50--> Data store volume sizing and capacity usage should be managed at the partition level. It is recommended that only SEP sesam data is stored on the respective volume.</translate> | *<translate><!--T:50--> Data store volume sizing and capacity usage should be managed at the partition level. It is recommended that only SEP sesam data is stored on the respective volume.</translate> | ||
*<translate><!--T:51--> The data store should be at least three times (3x) the maximum full backup size of the planned backup to allow the watermarks to work automatically and dynamically.</translate> | *<translate><!--T:51--> The data store should be at least three times (3x) the maximum full backup size of the planned backup to allow the watermarks to work automatically and dynamically.</translate> | ||
*<translate><!--T:52--> It may be necessary to scale up the data store to beyond 3x the maximum size when a longer hold-back time is stipulated or very big savesets are to be stored | *<translate><!--T:52--> It may be necessary to scale up the data store to beyond 3x the maximum size when a longer hold-back time is stipulated or very big savesets are to be stored.</translate> | ||
<!--*<translate><!--T:54--> | |||
*<translate><!--T:54--> | Because a backup is aborted rather than split up when a data store becomes full, the correlation between the size of the data store and the size of the biggest backup task must be determined carefully. Take the example of a data store with the capacity of 6 TB, where the biggest saveset is 2 TB and sum up of other saveset(s) is around 1 TB. With a retention time=3, a data store three times (3x) the maximum full backup size may be too small to allow the HWM to work properly. In such a case, it is recommended to scale up the data store size, e.g., by summing up all the savesets for 3 days and adding 15%.--> | ||
Because a backup is aborted rather than split up when a data store becomes full, the correlation between the size of the data store and the size of the biggest backup task must be determined carefully. Take the example of a data store with the capacity of 6 TB, where the biggest saveset is 2 TB and sum up of other saveset(s) is around 1 TB. With a retention time=3, a data store three times (3x) the maximum full backup size may be too small to allow the HWM to work properly. In such a case, it is recommended to scale up the data store size, e.g., by summing up all the savesets for 3 days and adding 15%. | |||
<!--T:108--> | |||
{{note|The Si3 deduplication store has specific hardware and sizing requirements. For more information, see [[Special:MyLanguage/Si3 Deduplication Hardware Requirements|Si3 Deduplication Hardware Requirements]] and [[Special:MyLanguage/Configuring_an_Si3_Deduplication_Store|Configuring Si3 Deduplication Store]].}} | |||
==== {{anchor|limitations}}Limitations ==== <!--T:55--></translate> | ==== {{anchor|limitations}}Limitations ==== <!--T:55--></translate> | ||
Line 123: | Line 131: | ||
*<translate><!--T:56--> When a media pool requires more than one data store, all data stores must be connected to the same SEP sesam Device Server (IP host). SEP sesam does not currently support network-distributed data stores being served by a single media pool.</translate> | *<translate><!--T:56--> When a media pool requires more than one data store, all data stores must be connected to the same SEP sesam Device Server (IP host). SEP sesam does not currently support network-distributed data stores being served by a single media pool.</translate> | ||
*<translate><!--T:57--> | *<translate><!--T:57--> | ||
When using more than one data store, only negative '''OR''' positive values can be used for specifying the capacity and HWM. SEP sesam does not support the use of negative and positive values at the same time. | When using more than one data store, only negative '''OR''' positive values can be used for specifying the capacity and HWM. SEP sesam does not support the use of negative and positive values at the same time. For more details on data store capacity and values, see [[Special:MyLanguage/4_4_3_Beefalo:FAQ#data_store_calculation|FAQ: How do I calculate the data store capacity]]. | ||
===={{anchor|properties}}Data store properties==== <!--T:58--> | ===={{anchor|properties}}Data store properties==== <!--T:58--> | ||
Line 136: | Line 144: | ||
<br clear=all> | <br clear=all> | ||
*<translate><!--T:61--> '''''Capacity''''': The size (in GiB) of the partition available for the backups. For examples on calculating a data store capacity, see [[Special:MyLanguage/FAQ#data_store_calculation|How do I calculate the data store capacity]].</translate> | *<translate><!--T:61--> '''''Capacity''''': The size (in GiB) of the partition available for the backups. For examples on calculating a data store capacity, see [[Special:MyLanguage/FAQ#data_store_calculation|How do I calculate the data store capacity]].</translate> | ||
*<translate><!--T:62--> '''''High watermark''''': The upper value (in GiB) for the used disk space on the data store. When this value is reached, a data store | *<translate><!--T:62--> '''''High watermark''''': The upper value (in GiB) for the used disk space on the data store. When this value is reached, the status of a data store changes from ''OK'' to ''Warning'', but backups continue to be performed. </translate> | ||
*<translate><!--T:63--> '''''Filled''''': The size (in GiB/TiB) of the occupied data store space by SEP sesam.</translate> | *<translate><!--T:63--> '''''Filled''''': The size (in GiB/TiB) of the occupied data store space by SEP sesam.</translate> | ||
*<translate><!--T:64--> '''''Used''''': Total used space (in GiB/TiB) on the partition (incl. SEP sesam external data).</translate> | *<translate><!--T:64--> '''''Used''''': Total used space (in GiB/TiB) on the partition (incl. SEP sesam external data).</translate> | ||
Line 144: | Line 152: | ||
<!--T:67--> | <!--T:67--> | ||
You can modify existing drive options and set additional by double-clicking the drive. In the ''Drive Properties'' window, you can select | {{anchor|access_mode}}You can modify existing drive options and set additional by double-clicking the drive. In the ''Drive Properties'' window, you can select a different drive group, browse the path for the data store, etc. Under ''Access mode'', you can also set the following permissions for the data store drive:</translate> | ||
*<translate><!--T:98--> '''''read/write''''' (default): Allows to perform the ''read'' operations (e.g., restore or using a drive as the source of a migration) and ''write'' operations (e.g., backup or using a drive as the target of a migration). As the ''write'' operations can occupy the drive for a while, consider using certain drives only for ''write'' operations and setting up the other drive(s) with ''read'' operations only.</translate> | |||
*<translate><!--T:99--> '''''read''''': Only ''read'' operations, such as restore or as a source of a migration, are allowed. It is recommended to set up additional drives in ''read'' mode to allow uninterrupted processing of tasks, such as restore.</translate> | |||
*<translate><!--T:100--> | |||
'''''write''''': Only ''write'' operations, such as backup or as the target of a migration, are allowed. The use of drives in ''write'' mode is recommended if these drives are used in combination with additional drives that are only used in ''read'' mode. | |||
<!--T:101--> | |||
The first drive in the list has an additional ''OS Access'' (previously ''Account'') tab where you can specify the credentials (user name and password) required to access the configured drive path.</translate><br /> | |||
<translate><!--T:68--> [[File:Drive_properties_Beefalo.jpg|500px|link=]]</translate> | <translate><!--T:68--> [[File:Drive_properties_Beefalo.jpg|500px|link=]]</translate> | ||
<br clear=all> | <br clear=all> | ||
Line 151: | Line 166: | ||
<!--T:70--> | <!--T:70--> | ||
Selecting a data store and clicking the tab '''Savesets''' opens a list of all savesets with their details. You can change EOL of ''individual saveset'', adjust ''backup-related EOL'', and ''lock or unlock'' individual saveset.</translate> | Selecting a data store and clicking the tab '''Savesets''' opens a list of all savesets with their details. You can change the EOL of the ''individual saveset'', adjust the ''backup-related EOL'', and ''lock or unlock'' the individual saveset. Note that in SEP sesam ≤ [[SEP_sesam_Release_Versions|4.4.3 ''Beefalo V2'']], the ''backup locked status'' is only applied to the selected backup, so you need to '''''manually lock all backups''''' in a backup chain as well as migrated and replicated backups. With v. [[SEP_sesam_Release_Versions|5.0.0 ''Jaglion'']], the ''locked status'' is '''''automatically applied to all savesets of a backup''''' (backup chain, migrated and replicated savesets). You can also lock your backups in [[Special:MyLanguage/SEP_sesam_Web_UI|SEP sesam Web UI]].</translate> | ||
{{<translate><!--T:94--> Tip</translate>|<translate><!--T:95--> The circles next to the EOL (Backup/Saveset) indicate the status of your saveset (''gray circle'' – EOL has expired, ''blue circle'' – saveset is protected; EOL has not yet been reached). For details, see [[Special:MyLanguage/ | {{<translate><!--T:94--> Tip</translate>|<translate><!--T:95--> The circles next to the EOL (Backup/Saveset) indicate the status of your saveset (''gray circle'' – EOL has expired, ''blue circle'' – saveset is protected; EOL has not yet been reached). For details, see [[Special:MyLanguage/SEP_sesam_Icons_Legend#bck_task|SEP sesam Icons Legend]].</translate>}} | ||
<translate><!--T:71--> [[image:Data_store_savesets_Beefalo.jpg|850px|link=]]</translate> | <translate><!--T:71--> [[image:Data_store_savesets_Beefalo.jpg|850px|link=]]</translate> | ||
Line 159: | Line 174: | ||
<ul><li><translate><!--T:72--> '''''Saveset EOL''''': The column ''Saveset EOL'' enables you to change EOL for each individual saveset, stored on the respective data store. You can extend or reduce its [[Special:MyLanguage/SEP_sesam_Glossary#retention_time|retention time]]. If the adjusted saveset is a part of a backup chain, the whole chain is affected as described below in [[Special:MyLanguage/4_4_3_Beefalo:Data_Store#backup_chain|EOL-related backup chain dependencies]].</translate></li> | <ul><li><translate><!--T:72--> '''''Saveset EOL''''': The column ''Saveset EOL'' enables you to change EOL for each individual saveset, stored on the respective data store. You can extend or reduce its [[Special:MyLanguage/SEP_sesam_Glossary#retention_time|retention time]]. If the adjusted saveset is a part of a backup chain, the whole chain is affected as described below in [[Special:MyLanguage/4_4_3_Beefalo:Data_Store#backup_chain|EOL-related backup chain dependencies]].</translate></li> | ||
<li><translate><!--T:73--> '''''Backup EOL''''': The column ''Backup EOL'' enables you to adjust EOL for all savesets containing the same data. This backup-related EOL is applied to all savesets with the same data, including migrated and replicated savesets.<br /> For example, adjusting EOL of a migrated saveset from 2.12.2017 to 12.12.2018 results in changed EOL for all related backup data, i.e., original backup, replicated backup, as well as for all backups in a backup chain, if a saveset with adjusted EOL is a part of it.</translate></li> | <li><translate><!--T:73--> '''''Backup EOL''''': The column ''Backup EOL'' enables you to adjust EOL for all savesets containing the same data. This backup-related EOL is applied to all savesets with the same data, including migrated and replicated savesets.<br /> For example, adjusting EOL of a migrated saveset from 2.12.2017 to 12.12.2018 results in changed EOL for all related backup data, i.e., original backup, replicated backup, as well as for all backups in a backup chain, if a saveset with adjusted EOL is a part of it.</translate></li> | ||
{{<translate> <!--T:74--> Tip</translate>|<translate><!--T:92--> How SEP sesam manages failed backups depends on its version. As of v. ≥ [[Special:MyLanguage/SEP_sesam_Release_Versions|4.4.3 ''Beefalo V2'']], SEP sesam keeps the failed backup according to the media pool retention time together with the last successful backup or migration saveset. This is the default backup retention behavior and can be changed by modifying retention-related keys, as described in [[Special:MyLanguage/ | {{<translate> <!--T:74--> Tip</translate>|<translate><!--T:92--> How SEP sesam manages failed backups depends on its version. As of v. ≥ [[Special:MyLanguage/SEP_sesam_Release_Versions|4.4.3 ''Beefalo V2'']], SEP sesam keeps the failed backup according to the media pool retention time together with the last successful backup or migration saveset. This is the default backup retention behavior and can be changed by modifying retention-related keys, as described in [[Special:MyLanguage/5_0_0:Changing_Retention_(EOL)#EOL_keys|Customize the default retention behavior for backups and migration]]. These keys may not be supported in previous versions, where failed backups were deleted automatically after 3 days.</translate>}} | ||
<li><translate><!--T:76--> {{anchor|backup_chain}}'''''EOL-related backup chain dependencies''''': You can extend or reduce the retention period for an individual saveset or backup-related saveset, as described above. Keep in mind that increasing EOL of a DIFF or INCR saveset will result in increased EOL of all dependent backups (FULL and other DIFF and INCR) | <li><translate><!--T:76--> {{anchor|backup_chain}}'''''EOL-related backup chain dependencies''''': You can extend or reduce the retention period for an individual saveset or backup-related saveset, as described above. Keep in mind that increasing EOL of a DIFF or INCR saveset will result in increased EOL of all dependent backups (FULL and other DIFF and INCR) to preserve the backup data. This keeps the backup chain readily available for restore. On the other hand, decreasing the EOL of a DIFF or INCR saveset leads to reduced EOL of all dependent savesets. If the EOL is changed to a date in the past, the savesets will be deleted during the next purge. How SEP sesam manages EOL depends on the version. For more details, see [[Special:MyLanguage/5_0_0:Automatic_Retention_(EOL)_Management|Automatic Retention (EOL) Management]] for v. [[SEP_sesam_Release_Versions|≥ 5.0.0 ''Jaglion '']] and [[Special:MyLanguage/4_4_3_Beefalo:Managing_EOL|Managing EOL]] for v. [[SEP_sesam_Release_Versions|4.3.3 ''Beefalo V2'']].</translate></li></ul> | ||
{{<translate><!--T:77--> note</translate>|<translate><!--T:78--> | {{<translate><!--T:77--> note</translate>|<translate><!--T:78--> | ||
Any saveset can be deleted if the following conditions are met: | |||
# Its EOL has expired. | # Its EOL has expired. | ||
# Typically, there must be no other savesets that depend on this saveset. You can override this condition by explicitly allowing the EOL for the whole backup chain to be set to expired, thus deleting backup data on all related savesets.</translate>}} | # Typically, there must be no other savesets that depend on this saveset. You can override this condition by explicitly allowing the EOL for the whole backup chain to be set to expired, thus deleting backup data on all related savesets.</translate>}} | ||
Line 185: | Line 200: | ||
<br clear=all> | <br clear=all> | ||
<translate>===={{anchor|Si3}}Si3 State, HPE Catalyst Store State and HPE Cloud Bank Store State tabs==== <!--T:85--> | <translate>===={{anchor|Si3}}Si3 State, HPE Catalyst Store State and HPE Cloud Bank Store State <!--and HPE Cloud Volumes tabs-->==== <!--T:85--> | ||
<!--T:86--> | <!--T:86--> | ||
The '''Si3 State''' tab is shown for Si3 data stores, '''HPE Catalyst Store State''' tab is shown for HPE Catalyst stores and '''HPE Cloud Bank Store State''' | The '''Si3 State''' tab is shown for Si3 data stores, '''HPE Catalyst Store State''' tab is shown for HPE Catalyst stores and '''HPE Cloud Bank Store State''' is shown for HPE Cloud Bank stores. <!--In v. ≥ 5.0.0 Jaglion, the HPE Cloud '''Volumes State''' is shown for HPE Cloud Volume stores. -->The tabs display the last deduplication message, the status of active tasks, encryption status, number of stored objects, data size before/after deduplication, dedup ratio, saved storage space, etc. You can also review the Si3 deduplication, HPE Catalyst store or HPE Cloud Bank store<!-- or HPE Cloud Volumes--> status in the media actions properties. | ||
<!--T:87--> | <!--T:87--> | ||
Line 197: | Line 212: | ||
<noinclude><div class="noprint"> | <noinclude><div class="noprint"> | ||
===See also=== <!--T:88--> | ===See also=== <!--T:88--> | ||
[[Special:MyLanguage/Configuring_a_Data_Store|Configuring a Data Store]] – [[Special:MyLanguage/ | [[Special:MyLanguage/Configuring_a_Data_Store|Configuring a Data Store]] – [[Special:MyLanguage/Configuring_Si3_Deduplication_Store|Configuring Si3 Deduplication Store]] – [[Special:MyLanguage/NetApp_Storage_Snapshots_Backup|NetApp Snap Store]] – [[Special:MyLanguage/SEP_sesam_HPE_StoreOnce_Configuration|HPE StoreOnce]] – [[Special:MyLanguage/HPE_StoreOnce_Replication|HPE Cloud Bank]] | ||
<!--T:89--> | <!--T:89--> | ||
{{Footer/en}}</div></noinclude></translate> | {{Footer/en}}</div></noinclude></translate> |
Revision as of 11:56, 14 June 2022
Data stores
The SEP sesam data store is a device type used for writing savesets directly to one or several configured storage locations – into the file system. SEP sesam uses a data store instead of a conventional media pool to define the storage repository. Data is still primarily backed up to a media pool, however, a data store is used underneath to save data to dynamically managed data areas, including disk backups.
SEP sesam can contain multiple data stores of different types and sizes, depending on the type of data being backed up, the backup technique, and the storage location (a local disk, a virtualized storage device, storage appliances, etc.). The following data store types are supported:
- Path: The default data store type, relevant to the configuration of most storage locations, unless one of the following data store types is required, e.g., for deduplication or backup to cloud storage.
- SEP Si3 deduplication store: Used for target-based (Si3T) and source-based deduplication (Si3S), replication, backup directly to S3 cloud storage (Si3 NG store) and S3 cloud replication (Si3 store).
Note As of SEP sesam v. Jaglion, two Si3 deduplication store types are available. The common name for both types is Si3 deduplication store, unless it is explicitly stated that the information refers exclusively to Si3 (first generation) or Si3 NG (next generation). SEP sesam recommends using the new generation Si3 NG when creating a new data store. Si3 NG is advantageous over regular Si3 as it offers better performance, scaling, and resource savings. It allows you to back up your data directly to S3 cloud storage and restore the items you want directly from there. - NetApp Snap Store: Used for NetApp snapshots backup.
- HPE StoreOnce: Used for integration with the HPE StoreOnce Catalyst storage system.
- HPE Cloud Bank Store: Used for replication of data to HPE Cloud Bank Storage.
Data store status iconsTemplate:Data Store Status Icons/enView Mode button
SEP sesam provides an additional View Mode button that enables you to switch between the table view and the tree view with grouped objects.
- Tree view: SEP sesam default view shows the status of individual data stores grouped together in a hierarchical view.
- Table view: A simple flat view that shows the status of individual data stores one after another.
SEP sesam data store concept
SEP sesam data stores can be of different types depending on the type of data being backed up, backup technique, and used storage. The default and most commonly used data store type is Path. The following information applies generally to a data store, unless explicitly specified otherwise; data store types other than Path may have slightly different characteristics. For details, see the relevant articles: SEP Si3 deduplication store, NetApp Snap Store, HPE StoreOnce, and HPE Cloud Bank Storage.
The difference between a conventional media pool, typically used for backing up directly to tapes, and a data store is in defining the storage space directory directly in the drive by using the operating system's partition functions. Therefore the data store space is managed at the partition level. You configure a data store by specifying its capacity and (optionally) setting the high watermark (HWM) value. This is the upper value for the used disk space on the data store. Take into account that exceeding HWM and filling up the data store may cause backup issues. You should consider this when specifying the data store capacity. For details, see Data store calculation recommendations below.
Note | |
Only one data store should be used for each hard disk partition. Even though several data stores can be set up on one partition, you are advised against such configuration as each data store reads the values of the other partitions when checking partition allocation. Consequently, such coexisting data stores obstruct each other. |
As shown in the illustration below, a media pool still refers to a drive group. The drive group points to exactly one data store between the media pool and the drives. The connection between the data store and the related drive is static.
Data store capacity
Data store configuration consists of specifying the data store capacity and HWM. The data store capacity is space reserved for the SEP sesam data store and, optionally, non-SEP sesam data that might be stored on the same volume as the SEP sesam data store. If the data store is shared with non-SEP sesam data, you have to obtain a special SEP sesam storage license.
Note | |
The Si3 deduplication store has specific hardware and sizing requirements. For more information, see Si3 Deduplication Hardware Requirements and Configuring Si3 Deduplication Store. |
When specifying the capacity value, a dedicated partition must have enough free space. The method for calculating the required disk space is:
space occupied by SEP sesam + free disk space = DS capacity
where DS capacity is the configured capacity value in SEP sesam's data store configuration. For examples on calculating a data store capacity, see How do I calculate the data store capacity.
More than one data store is required in a media pool only if the media pool uses data from several disk partitions, in which case all the drives of a media pool's data stores must be part of the same drive group. This ensures that the SEP sesam queue manager distributes the backups in this media pool to all data stores (balancing). For details on drive groups, see Drives.
HWM, purge and Si3 repair area
A high watermark is a general parameter that specifies the upper value for the used disk space on the data store. When this value is reached, the status of a data store changes from OK to Warning, but backups continue to be performed.
The Si3 repair area parameter is relevant only for SEP Si3 deduplication store. It specifies reserved space for unused Si3 files, as explained below.
- High watermark (HWM): A parameter for managing disk space usage; it defines the upper value for the used disk space on the data store. This parameter can be specified manually for the Path, Si3 and NetApp Snap store, and is set automatically during the creation of a SEP Si3 Deduplication store.
When this value is reached, the status of a data store changes to Warning. It serves as an indication for the administrator that the data store is running out of space, but does not prevent backups to be started. A data store purge process is running automatically for all (EOL-free) savesets. The oldest free savesets are deleted first. Purge is done until all EOL-free savesets are deleted. For more details on EOL, see Automatic Retention (EOL) Management.
Events that trigger the data store purge are:
- NEWDAY
- Sharing the data store drive after a backup
- Starting purge manually in GUI
- Si3 Repair Area: A parameter used by SEP Si3 Deduplication Store for managing disk space dedicated to Si3 files that were identified by a garbage collection job and are no longer used. These files are still kept in the repair area to enable possible repair of Si3 if there are any structural problems (may be caused by a file system error or by a crash of an operating system). The files in the repair area are removed automatically after the specified amount of time (SEP sesam default: 4 days) or when the disk usage threshold is reached. If the value is set to 0, the Si3 repair functionality is turned off.
Note | |
The Si3 repair area is available only in advanced UI mode (formerly expert GUI mode). To see the Si3 repair area field, make sure your UI mode is set to advanced. For details, see Selecting UI mode. |
Clean up orphaned savesets
You can manually remove orphaned savesets from the data stores by using the Clean up option in the Data Stores content pane, thus releasing the space that might be occupied by orphaned savesets. This is useful in cases when a data store seems to be inaccessible, its storage space is occupied, or SEP sesam space check shows non-sesam data.
Steps
- In the Main Selection -> Components, click Data stores to display the data store contents frame.
- In the content pane menu, click Clean up and select the data store (and the relevant drive number) for which you want to free up space by removing orphaned savesets.
Click Clean Up.
- You can check the status of the clean-up action in the data store properties under the Actions tab.
Data store calculation recommendations
- Data store volume sizing and capacity usage should be managed at the partition level. It is recommended that only SEP sesam data is stored on the respective volume.
- The data store should be at least three times (3x) the maximum full backup size of the planned backup to allow the watermarks to work automatically and dynamically.
- It may be necessary to scale up the data store to beyond 3x the maximum size when a longer hold-back time is stipulated or very big savesets are to be stored.
Note | |
The Si3 deduplication store has specific hardware and sizing requirements. For more information, see Si3 Deduplication Hardware Requirements and Configuring Si3 Deduplication Store. |
Limitations
- When a media pool requires more than one data store, all data stores must be connected to the same SEP sesam Device Server (IP host). SEP sesam does not currently support network-distributed data stores being served by a single media pool.
- When using more than one data store, only negative OR positive values can be used for specifying the capacity and HWM. SEP sesam does not support the use of negative and positive values at the same time. For more details on data store capacity and values, see FAQ: How do I calculate the data store capacity.
Data store properties
Double-clicking the data store displays all its details, e.g., the name of the data store, the store type, the message about the last executed action, the last action performed, its capacity, etc.
The buttons in the lower left corner allow you to create and delete drive, create new media pool, and delete a data store.
- Capacity: The size (in GiB) of the partition available for the backups. For examples on calculating a data store capacity, see How do I calculate the data store capacity.
- High watermark: The upper value (in GiB) for the used disk space on the data store. When this value is reached, the status of a data store changes from OK to Warning, but backups continue to be performed.
- Filled: The size (in GiB/TiB) of the occupied data store space by SEP sesam.
- Used: Total used space (in GiB/TiB) on the partition (incl. SEP sesam external data).
- Total: Maximum available space (in GiB/TiB) on the partition as reported by the operating system.
- Free: Available disk space (in GiB/TiB) for SEP sesam.
You can modify existing drive options and set additional by double-clicking the drive. In the Drive Properties window, you can select a different drive group, browse the path for the data store, etc. Under Access mode, you can also set the following permissions for the data store drive:
- read/write (default): Allows to perform the read operations (e.g., restore or using a drive as the source of a migration) and write operations (e.g., backup or using a drive as the target of a migration). As the write operations can occupy the drive for a while, consider using certain drives only for write operations and setting up the other drive(s) with read operations only.
- read: Only read operations, such as restore or as a source of a migration, are allowed. It is recommended to set up additional drives in read mode to allow uninterrupted processing of tasks, such as restore.
- write: Only write operations, such as backup or as the target of a migration, are allowed. The use of drives in write mode is recommended if these drives are used in combination with additional drives that are only used in read mode.
The first drive in the list has an additional OS Access (previously Account) tab where you can specify the credentials (user name and password) required to access the configured drive path.
Savesets tab
Selecting a data store and clicking the tab Savesets opens a list of all savesets with their details. You can change the EOL of the individual saveset, adjust the backup-related EOL, and lock or unlock the individual saveset. Note that in SEP sesam ≤ 4.4.3 Beefalo V2, the backup locked status is only applied to the selected backup, so you need to manually lock all backups in a backup chain as well as migrated and replicated backups. With v. 5.0.0 Jaglion, the locked status is automatically applied to all savesets of a backup (backup chain, migrated and replicated savesets). You can also lock your backups in SEP sesam Web UI.
Tip | |
The circles next to the EOL (Backup/Saveset) indicate the status of your saveset (gray circle – EOL has expired, blue circle – saveset is protected; EOL has not yet been reached). For details, see SEP sesam Icons Legend. |
- Saveset EOL: The column Saveset EOL enables you to change EOL for each individual saveset, stored on the respective data store. You can extend or reduce its retention time. If the adjusted saveset is a part of a backup chain, the whole chain is affected as described below in EOL-related backup chain dependencies.
- Backup EOL: The column Backup EOL enables you to adjust EOL for all savesets containing the same data. This backup-related EOL is applied to all savesets with the same data, including migrated and replicated savesets.
For example, adjusting EOL of a migrated saveset from 2.12.2017 to 12.12.2018 results in changed EOL for all related backup data, i.e., original backup, replicated backup, as well as for all backups in a backup chain, if a saveset with adjusted EOL is a part of it. - EOL-related backup chain dependencies: You can extend or reduce the retention period for an individual saveset or backup-related saveset, as described above. Keep in mind that increasing EOL of a DIFF or INCR saveset will result in increased EOL of all dependent backups (FULL and other DIFF and INCR) to preserve the backup data. This keeps the backup chain readily available for restore. On the other hand, decreasing the EOL of a DIFF or INCR saveset leads to reduced EOL of all dependent savesets. If the EOL is changed to a date in the past, the savesets will be deleted during the next purge. How SEP sesam manages EOL depends on the version. For more details, see Automatic Retention (EOL) Management for v. ≥ 5.0.0 Jaglion and Managing EOL for v. 4.3.3 Beefalo V2.
Tip | |
How SEP sesam manages failed backups depends on its version. As of v. ≥ 4.4.3 Beefalo V2, SEP sesam keeps the failed backup according to the media pool retention time together with the last successful backup or migration saveset. This is the default backup retention behavior and can be changed by modifying retention-related keys, as described in Customize the default retention behavior for backups and migration. These keys may not be supported in previous versions, where failed backups were deleted automatically after 3 days. |
Note | |
Any saveset can be deleted if the following conditions are met:
|
Media tab
The Media tab provides an overview of the configured media pools and media.
Actions tab
The Actions tab provides an overview of the media-related events. It displays the media status, the action type, its start/stop time, duration and message.
Si3 State, HPE Catalyst Store State and HPE Cloud Bank Store State
The Si3 State tab is shown for Si3 data stores, HPE Catalyst Store State tab is shown for HPE Catalyst stores and HPE Cloud Bank Store State is shown for HPE Cloud Bank stores. The tabs display the last deduplication message, the status of active tasks, encryption status, number of stored objects, data size before/after deduplication, dedup ratio, saved storage space, etc. You can also review the Si3 deduplication, HPE Catalyst store or HPE Cloud Bank store status in the media actions properties.