5 1 0:SEP sesam Server Disaster Recovery

Overview

A computer disaster refers to a system failure, disk drive failure, or other serious event that makes it impossible to access user data on the computer. To recover from such a disaster, data from SEP sesam's Server is used to restore the system to a working state.

Note that a standard backup is not suitable for disaster recovery. Disaster recovery (DR) includes a set of procedures that enable the recovery of critical technological infrastructures and systems after a computer disaster. During the process, server data is recovered from the saveset and restored to the system.

SEP sesam provides three types of disaster recovery solutions:

• Bare Metal Recovery Linux
• SEP sesam BSR Pro for Windows, and
• SEP sesam Server self-recovery.

This section describes the disaster recovery strategy, requirements and procedure for the SEP sesam Server self-recovery process.

How does the SEP sesam self-backup work?

To prepare for a possible breakdown of the SEP sesam Server, you need to perform a self-backup of the SEP sesam installation. This means that you have to configure at least one backup task named SESAM_BACKUP. This backs up the configuration files of SEP sesam, the var directory including all listings, the log files, the database and the INI-files. See Configuration Files.

To ensure consistent configuration after restoring from backup, this backup should be run daily, either in COPY or FULL mode. For details, see the section Preparing for Disaster Recovery.

Also, a disaster interface must be properly configured to support the disaster recovery process: sm_disaster (Linux) or sm_disaster.cmd/sm_disaster.ps1 (Windows). The disaster interface sends an email with a description of the disaster recovery process and an attachment that contains the SEP sesam bootstrap database with all the important data for disaster recovery. For details on how to activate this interface, see the section Preparing for Disaster Recovery.

The self-backup procedure consists of the following:

• When the SEP sesam self-backup starts (SESAM_BACKUP), the entire SEP sesam database is exported to the path <SESAM_ROOT>/var/<db>[_pg]/backup. The export files have names like sesam_db_20121223-20121224060003.sql.gz and are backed up to a predefined media pool. It is recommended that you configure a SEP sesam DR-dedicated media pool for storing all your SEP disaster recovery savesets.

• Each time SESAM_BACKUP is run, a bootstrap file of the SEP sesam database (for example, sesam_bootstrap_db_[datetime].sql.gz) is exported to the <SESAM_ROOT>/var/db[_pg]/backup directory. This file contains the SEP sesam configuration and the data history of the SEP sesam self-backups of the last 30 days.

• sm_disaster copies the content of input arguments (DISASTER or SESAM_BACKUP) from the SEP sesam Server to predefined locations on another computer (emails, copies of files, etc.). Information about the last disaster backup from SEP sesam is also stored.
• Finally, an email with a brief description of the recovery and the bootstrap file as an attachment is sent regularly to the email address you configured. The bootstrap export is used exclusively for SEP sesam disaster recovery, so you should save any version of this file in a safe place.

If you are using the Si3 NG deduplication store, you can back up your SEP sesam Server to S3 cloud storage. This ensures high resistance against attacks on backups and helps protect your SEP sesam Server. For more information refer to Configuring Si3 NG Deduplication Store or Configuring Si3 NG Deduplication Store with Object Lock.

Preparing for Disaster Recovery

For general details on configuring a backup and its prerequisites, see Standard Backup Procedure. Here only the information specific to disaster recovery is introduced.

To fully utilize the disaster recovery functionality and ensure that all disaster-relevant information is generated and sent, the following steps must be performed:

1. The backup task SESAM_BACKUP is normally configured after the installation of a SEP sesam Server. So it should already exist. If this is not the case, you have to configure it:
   Open the GUI and select in the Main selection -> Tasks -> By Clients -> your SEP sesam Server -> New Backup Task. The name of the backup task must be SESAM_BACKUP. It normally includes the directories <SESAM_ROOT>/var and <SESAM_ROOT>/bin/sesam and excludes the directories work and log. Click OK to save the task.
   



	Tip
	Disaster recovery is performed with log level 0. If required, you can set a higher log level for the restore in the properties of the backup task SESAM_BACKUP -> Options tab -> Restore options field. For details, see Setting log level on a per-task basis in GUI.

2. Once your disaster recovery task is configured, set up a backup schedule (GUI -> Scheduling -> Schedules). The SEP sesam backup should run at a time when no other activities are running in the SEP sesam environment. It is recommended to run this task every day. For details on scheduling, see Creating a Schedule.
3. Create an event to associate with the schedule. Select the backup level COPY. It is recommended that you select a disaster recovery dedicated media pool for storing all your disaster recovery savesets. For general information about creating a backup event, see Creating a Backup Event.
4. Activate the inteface sm_disaster: From the SEP sesam GUI menu, select Configuration -> Interfaces -> Disaster Interface. A window with the interface script is displayed.


5. Click Save to confirm the dialog. The file sm_disaster is read and stored in the directory <SESAM_ROOT>/bin/sesam.
   Tip: After installing SEP sesam, the file sm_disaster (Linux) or sm_disaster.cmd/sm_disaster.ps1 (Windows) is located in the directory <SESAM_ROOT>/skel/templates/. Alternatively, you can activate the disaster interface by copying sm_disaster from <SESAM_ROOT>/skel/templates to <SESAM_ROOT>/bin/sesam.

   Note

   The sm_disaster file has been completely redesigned for SEP sesam version 4.4.2. If your SEP sesam Server version is 4.4.2 or lower, you must replace the sm_disaster file after updating to SEP sesam Server 4.4.2: Windows: The MSI Installer will ask you if you want to overwrite the existing interface. Choose <yes> to overwrite the existing interface. Alternatively, copy <SESAM_BIN>/skel/templates/sm_disaster.ps1 to <SESAM_BIN>/bin/sesam/sm_disaster.ps1. Linux: Copy <SESAM_BIN>/skel/templates/sm_disaster to <SESAM_BIN>/bin/sesam/sm_disaster to get the latest version of the disaster interface.

6. Configure the SEP sesam email for the sesam account so that the interface sends messages after self-backup: From the GUI menu, select Configuration -> Email Settings -> New. An Email Account window is displayed.
   • In the Account field, type the name sesam (it must be lowercase).
   • Optionally, enter the name in the Customer field.
   • Enter the name of the sender (the name of the respective SEP sesam Server).
   • Enter the name or IP address of the outgoing mail server in the SMTP server field.
   • In the SMTP user field, enter the user name for the SMTP server. If the SMTP user name is not specified, the SMTP server probably does not require authorization.
   • If the mail server requires authentication, enter the password in the Password field.
   • Optionally, specify the SMTP port.
   • In the Recipient field, enter the address of the recipient of the DR notification emails. This is usually the address of the backup administrator.
   • Optionally, you can specify additional recipients in the CC and BCC fields.
   
   

Tip

To maximize the chances for successful disaster recovery, it is recommended to keep the SEP sesam bootstrap database, containing all the important data for disaster recovery, separate from your corporate environment. To achieve this, you can either forward disaster-related emails with the bootstrap database to a separate (e.g. cloud-based) email service or regularly back up those emails to a detached storage device, such as a USB stick. In addition, you should also perform periodic backups or migrations of the SESAM_BACKUP to a dedicated tape. By following these recommendations, you greatly improve your chances of having an unaffected SESAM_BACKUP and disaster email with the bootstrap file available in case of a disaster, such as a Trojan attack.

Recovering SEP sesam from disaster

Once configured, the disaster interface sends an email describing the disaster recovery procedure and an attachment containing the SEP sesam bootstrap database with all the important data for disaster recovery. Disaster recovery is done by checking all backups that contain DISASTER or SESAM from which you select and restore the last known good copy of the data.

Prerequisites

The following procedure describes the disaster recovery process for a SEP sesam Server. Before you reinstall the SEP sesam Server and start the actual recovery, make sure that the following prerequisites are met:

• On both Linux and Microsoft Windows, connect the backup storage hardware to the new SEP sesam Server. The components will be detected and pre-configured during the SEP sesam Server installation.
• Save the bootstrap file (received via email through the sm_distaster interface) to the SEP sesam Server you want to recover.
• You must carefully set the retention time (EOL) of backup savesets so that, in the event of a disaster recovery, important backup savesets are not deleted immediately after recovery. For details, see Managing EOL.

Restriction

You can only recover data from the same SEP sesam version that you are currently using. For example, you cannot use SEP sesam version 4.4.3.86 Beefalo V2 to restore version 4.4.3.70 Beefalo.

Overview of the disaster recovery procedure

	Warning
	It is absolutely necessary that the GUI is started as superuser Administrator and that the checkbox Disaster Recovery in the restore wizard is checked (see steps below), otherwise the recovery will fail!

SEP sesam disaster recovery procedure consists of the following steps:

1. Install the SEP sesam Server.
2. Optionally, if you are using Si3 NG, recover the Si3 NG deduplication store and S3 bucket storage.
3. Recover the SEP sesam Server.

You can also perform a disaster recovery by running restore from the command line. For details see section Alternative recovery procedure via command line.

Installing SEP sesam Server for disaster recovery

	Note
	You must download and install the same SEP sesam Server package that was installed on the old system. It is important to provide the same operating system environment as the original server, so set up the new server system in the same way as the previous one (same hostname, same IP address). For Linux, follow the installation procedure described in Installing Linux Server package.

Depending on your operating system, install a SEP sesam Server as follows:

On Windows

1. Double-click the SEP sesam Server installation file. Agree to the license agreement and select the Disaster recovery box when the first dialog appears.


2. Continue the installation until you get to the last step Ready to Install SEP sesam Server version <number>. Under the Import SEP sesam Server database option, click Browse and select the bootstrap file you have saved locally on the server, and then click Install.
3. If you are using the Si3 NG deduplication store, locate the <SESAM_ROOT>\var\ini\sm.ini file, change the entry for the Si3 NG service to sds=on and then restart SEP sesam service for changes to take effect:

 [DAEMON]
 sds=on 

On Linux

1. Download and install the SEP sesam Server package.

	Note
	The restore is performed with additional relocation logic for the database backup file which was created during SESAM_BACKUP. This file will be restored to gv_rw_db:restore.

2. If you are using the Si3 NG deduplication store, locate the <SESAM_ROOT>/var/ini/sm.ini file, change the entry for the Si3 NG service to sds=on and then restart SEP sesam service for changes to take effect:

 [DAEMON]
 sds=on 

3. Start the SEP sesam GUI as superuser Administrator and from the menu select File -> Import SEP sesam database. Select the bootstrap file that you have saved locally on the server and click OK to import the database.

	Note
	The GUI will lose the connection to the UI server.




4. After importing the bootstrap data, SEP sesam should have the basic configuration as before (e.g., all existing drives). However, you should check the following on your newly installed Linux server:
1. Check the system settings to ensure that the restored SEP sesam configuration matches to the current server environment (library and drive device names, disk partition device names and mount points, etc.). If not, the SEP sesam settings must be adjusted.
2. If recovery is being executed from physical media, open the GUI and perform a SEP sesam archive adjustment to update the SEP sesam media information with the physical tape library: Main selection -> Components -> Loaders -> select a loader -> click Archive Adjustment. For details, see Setting up Archive Adjustment.

Recovering the Si3 NG deduplication store

To recover the Si3 NG deduplication store and restore the data from the S3 bucket, the SI3 NG deduplication store must be reinitialized:

1. Determine the UUID of the required Si3 NG deduplication store using the following command:
   

   sm_db "select uuid from data_stores where name='Name_SI3-NG'"

2. Init the Si3 NG deduplication store with the queried UUID using the following command on the SEP sesam Server or the RTS/RDS where the SI3-NG was running before the failure:
   

   sm_dedup_interface -d {1st_drive_num SI3-NG} init -u  {UUID} -t {name_SI3-NG}

3. Run the Config_Drives (on SEP sesam Server only) using the following command:
   

   sm_config_drives {1st_drive_num SI3-NG}

The SI3 NG deduplication store will perform a re-index of the S3 bucket contents (depending on the size, this may take hours or even days).

When the directory structure is rebuild, you can continue with the recovery of the SEP sesam Server.

Recovering the SEP sesam Server

1. Start the SEP sesam GUI as superuser Administrator and under Main selection -> Job State -> Backups select the latest SESAM_BACKUP, right-click on it and select Restore. The restore wizard is displayed.

	Tip
	Disaster recovery can also be started from the toolbar via the Disaster recovery icon if you have activated SEP sesam disaster recovery key in the menu bar -> Configuration -> Defaults -> Settings tab by setting the value for disaster_recovery key to one (1).

2. In the window Select Task, check the Disaster restore checkbox and click Next. Only if Disaster restore option is set, the restore will be performed with the parameters that lead to a successful recovery.


3. In the Target Settings window, make sure that the default settings Restore to original target path and Overwrite existing items are selected.

	Note
	The restore is performed using additional relocation logic for the database backup file that was created during SESAM_BACKUP. This file is restored to gv_rw_db:restore.

4. Click Next and then Start to begin the restore.
5. When the restore is complete, the Complete the disaster restore dialog appears asking if you want to import the database automatically. Confirm with OK.

This loads the database export file located at gv_rw_db:restore. It also imports the result of the SESAM_BACKUP restore into the SEP sesam database.

	Note
	The GUI will lose the connection to the UI server.

After restarting the SEP sesam GUI, the system will be operational again with the data that was backed up in the last SEP sesam self-backup.

Alternative recovery procedure via command line

Normally, an email sent from the disaster interface should contain the SEP sesam bootstrap database file as an attachment. If there is no attachment, you can still perform a disaster recovery by running restore from the command line. You will find the required arguments in the disaster email, namely, the label of the media currently in use and the complete restore command that restores the data from the attached savesets bootstrap database file.

• Identify the location of the last disaster backup from SEP sesam and copy the restore command located at the very end of the disaster email. Make sure you set the profile first. For details, see What happens when I set a profile?

• If the sesam backup is located on a physical medium (e.g., a tape), load the appropriate medium into the drive.

After the recovery

Make sure the hardware is connected correctly, and then perform a test backup and restore on the devices.

Some backup results may appear incorrect in the job status views. This is because some time elapses between the last self-backup and the recovery time. Some scheduled jobs were not executed because the SEP sesam Server was not available at that time. The number of errors displayed depends on how long the SEP sesam Server was unavailable and on the number of scheduled backup jobs.

See also

Bare Metal Recovery Linux – SEP sesam BSR Pro for Windows – Configuration Files

Copyright © SEP AG 1999-2023. All rights reserved.

Any form of reproduction of the contents or parts of this manual is allowed only with the express written permission from SEP AG. When compiling and designing user documentation SEP AG uses great diligence and attempts to deliver accurate and correct information. However, SEP AG cannot issue a guarantee for the contents of this manual.