Source:SEP sesam Server Disaster Recovery: Difference between revisions

From SEPsesam
(fixed typos)
m (Fixed description for systemd)
Line 193: Line 193:
{{<translate><!--T:67-->
{{<translate><!--T:67-->
note</translate>|<translate><!--T:68-->
note</translate>|<translate><!--T:68-->
If you are using SEP sesam Server on SLES12 or on other Linux distribution using Systemd, use the following commands to restart the server:</translate>
If you are using SEP sesam Server on SLES12 or on other Linux distribution using systemd, use the following commands to restart the server:</translate>


  systemctl stop sesam.service  
  systemctl stop sepsesam.service  
  systemctl start sesam.service
  systemctl start sepsesam.service
}}
}}



Revision as of 20:21, 27 March 2020

Other languages:

Template:Copyright SEP AG en

Docs latest icon.png Welcome to the latest SEP sesam documentation version 4.4.3/4.4.3 Beefalo. For previous documentation version(s), check Documentation archive.


Overview

A computer disaster refers to a system failure, disk drive failure or other severe occurrence that renders user data on the computer inaccessible. To recover from such a disaster, data from SEP sesam's Server is used to restore the system to a working state.

Note that standard backup does not prepare for disaster recovery. Disaster recovery (DR) involves a set of procedures that enable the recovery of vital technology infrastructure and systems following a computer disaster. During the process, server data from save set is recovered and restored to the system.

SEP sesam provides three types of disaster recovery solutions:

This section describes disaster recovery strategy, requirements and procedure for the SEP sesam Server self-recovery process.

How the SEP sesam self-backup works?

To prepare for a possible breakdown of the SEP sesam Server, a self-backup of the SEP sesam installation must be performed. This means that you have to configure at least one backup task with the name SESAM_BACKUP. This will back up SEP sesam's var directory, including all listings, log files, database, and INI-files. This backup should be run daily, in either COPY or FULL mode. For details, see Preparation for Disaster Recovery below.

Additionally, a disaster interface must be properly configured to help carry out the disaster recovery process: sm_disaster (Linux) or sm_disaster.cmd/sm_disaster.ps1 (Windows). The disaster interface sends an email describing the recovery procedure in the event of disaster and an attachment containing the SEP sesam bootstrap database with all essential data for the disaster recovery. For details on how to activate this interface, see Preparation for Disaster Recovery below.

The self-backup procedure consists of the following:

  • When the SEP sesam self-backup starts (SESAM_BACKUP), the entire SEP sesam database is exported to the path <SESAM_ROOT>/var/<db>[_pg]/backup. Export files have names such as sesam_db_20121223-20121224060003.sql.gz and are backed up to a pre-defined media pool. It is recommended that you configure a SEP sesam DR-dedicated media pool for storing all your SEP disaster recovery save sets.
  • Every time SESAM_BACKUP is executed, a bootstrap file of the SEP sesam database (for example, sesam_bootstrap_db_[datetime].sql.gz) is exported to the <SESAM_ROOT>/var/db[_pg]/backup directory. This file contains the SEP sesam configuration and data history of SEP sesam self-backups for the last 30 days.
  • sm_disaster copies the contents of input arguments (DISASTER or SESAM_BACKUP) from the SEP sesam Server to predefined locations on another computer (emails, copies of files, etc.). Information about SEP sesam's last disaster backup is also stored.
  • Finally, an email with a short recovery description and the bootstrap file as an attachment are sent regularly to the e-mail address you have configured. The bootstrap export is used exclusively for SEP sesam system recovery in the event of a disaster, therefore you should save every version of this file to a safe location.

Preparing for Disaster Recovery

For general details on how to configure a backup and its prerequisites, see Standard Backup Procedure. Here only disaster recovery specific information is introduced.

To fully utilize disaster recovery's functionality and ensure that all disaster-related information is generated and sent, the following steps must be carried out:

  1. The backup task SESAM_BACKUP is ordinarily configured after the installation of a SEP sesam Server. Hence it should already be present. If it is not, you have to configure it:
    Open the GUI and from the Main selection -> Tasks -> By clients -> select your SEP sesam Server -> New backup task. The name of the backup task must be SESAM_BACKUP. It typically encompasses the <SESAM_ROOT>/var and <SESAM_ROOT>/bin/sesam directories, and excludes the work and log directories. Click OK to save the task.
    SEP sesam backup task.png
  2. Once your disaster recovery task is configured, set up a backup schedule (GUI -> Scheduling -> Schedules). The SEP sesam backup should run at a time when no other activities are running in the SEP sesam environment. It is recommended to run this task every day. For details on scheduling, see Creating a backup schedule.
  3. Create an event to be linked to the schedule. Select the backup level COPY. It is recommended that you choose a disaster recovery dedicated media pool for storing all your disaster recovery save sets. For general information on creating a backup event, see Creating a backup event.
  4. Activate the sm_disaster interface: In the SEP sesam GUI menu, select Configuration -> Interfaces -> Disaster interface. A window with the interface script is displayed.
    File:Activate disaster interface.png
  5. Click Save to confirm the dialog. The sm_disaster file is read and stored in the directory <SESAM_ROOT>/bin/sesam.
    Tip: After installing SEP sesam, the file sm_disaster (Linux) or sm_disaster.cmd/sm_disaster.ps1 (Windows) is located in the <SESAM_ROOT>/skel/templates/ directory. Alternatively, you can activate the disaster interface by copying sm_disaster from <SESAM_ROOT>/skel/templates to <SESAM_ROOT>/bin/sesam.
    Information sign.png Note
    The sm_disaster file has been completely redesigned for SEP sesam version 4.4.2. If your SEP sesam Server version is 4.4.2 or lower, you must replace the sm_disaster file after updating to SEP sesam Server 4.4.2:
    • Windows: the MSI Installer will ask you if the existing interface should be overwritten. Choose <yes> to overwrite the existing interface. Alternatively, copy <SESAM_BIN>/skel/templates/sm_disaster.ps1 to <SESAM_BIN>/bin/sesam/sm_disaster.ps1.
    • Linux: Copy <SESAM_BIN>/skel/templates/sm_disaster to <SESAM_BIN>/bin/sesam/sm_disaster to get the latest version of the disaster interface.
  6. Configure the SEP sesam email for the account sesam so that the interface sends messages after the self-backup: From the GUI menu, select Configuration -> E-Mail Settings -> Create new. An E-mail account window is displayed.
    • In the Account field, type the name sesam (it must be written in lowercase)
    • Optionally, enter the name in the Customer field.
    • Specify the name of the sender (the name of the respective SEP sesam Server)
    • In the SMTP server field, specify the name or the IP-address of the outgoing mail server.
    • In the SMTP user field, specify the user name for the SMTP server. If the SMTP user name is not specified, then the SMTP server presumably does not require authorization.
    • If the mail server requires authentication, enter the password in the Password field.
    • Optionally, specify the SMTP port.
    • In the Recipient field, enter the address of the recipient of the DR notification emails. Typically, this is the backup administrator's address.
    • Optionally, specify additional recipients in the CC and BCC field.
    Create e-mail account.png

Recovering SEP sesam from Disaster

Once configured, the disaster interface sends an email describing the recovery procedure in the event of disaster and an attachment containing the SEP sesam bootstrap database with all essential data for the disaster recovery. Disaster recovery is executed by inspecting all backups containing DISASTER or SESAM, from which you select the last known good copy of data and restore it.

Prerequisites

The procedure below describes the OS dependent disaster recovery process for a SEP sesam Server. Before reinstalling the SEP sesam Server and starting the actual recovery, perform the following step on both Linux and Microsoft Windows:

Connect the backup storage hardware to the new SEP sesam server. The components will be detected and pre-configured during the SEP sesam Server installation.
Information sign.png Note
As of SEP sesam version 4.4.3, the restore task option "Disaster Recovery" does not work on Linux systems. Instead, use the option "Complete restore" and perform additional steps as described below to recover your Linux server from disaster.

Recovery on Linux

  1. In the first step, you will download and install the same SEP sesam Server package that was installed on the old system. It is important to provide the same operating system environment as that of the original server, therefore set up the new server system in the same way as the previous one (same hostname, same IP address). Follow the installation procedure as described in Installing Linux Server package. Before starting the SEP sesam installation, make sure that you are logged in as root.
  2. After installing the SEP sesam Server, open the GUI and from the menu choose File -> Import sesam database. Select the bootstrap file that you have saved locally on the server and click OK to import the database.
  3. Import bootstrap file.png
  4. After importing the bootstrap data, SEP sesam should have the basic configuration as before (for example, all existing drives). However, you should verify the following on your newly installed Linux server:
    1. Check the system settings to ensure that the restored SEP sesam configuration conforms to the current server environment (library and drive device names, disk partition device names and mount points, etc.). If not, the SEP sesam settings must be adjusted.
    2. If recovery is being executed from physical media, open the GUI and perform a SEP sesam archive adjustment to update the SEP sesam media information with the physical tape library: Main selection -> Components -> Loaders -> loader -> Archive Adjustment.
  5. Start the whole SEP Server process daemon by running:
  6. /opt/sesam/bin/sesam/sm_main stop /opt/sesam/bin/sesam/sm_main start
    Information sign.png Note
    If you are using SEP sesam Server on SLES12 or on other Linux distribution using systemd, use the following commands to restart the server:
    systemctl stop sepsesam.service 
    systemctl start sepsesam.service
    
  7. From the GUI Main selection -> Job State -> Backups select the most recent SESAM_BACKUP, right-click it and select Restore. The restore wizard is displayed.
  8. In the Select task window, check the complete restore of selected task box and click Next.
  9. In the Save and Start window, under the Target path settings select the option New restore target and enter the path to some temporary directory, e.g., /tmp/sesam_restore. Click Start to start the restore.
  10. Set the profile to enable SEP sesam commands to be executed globally:
  11. source /var/opt/sesam/var/ini/sesam2000.profile
  12. Then run the following:
  13. sm_db import -n -f /opt/sesam/skel/db/postgres/sesam_db.sql
  14. Find out which backup is the most recent one:
  15. ls -lart /tmp/sesam_restore/var/opt/sesam/var/db_pg/backup/
  16. Run the following command by replacing the <sesam_db_file_name> with actual export filename, e.g., sesam_db_20171214-20171215030025.sql.gz:
  17. sm_db import -n -f /tmp/sesam_restore/var/opt/sesam/var/db_pg/backup/<sesam_db_file_name>.sql.gz
  18. Run another sm_db import
  19. sm_db import -n -f /opt/sesam/skel/db/postgres/sesam_insert_start.sql
  20. Restore the .lis files:
  21. cd /tmp/sesam_restore/var/opt/sesam/var/lis by moving them: mv * /var/opt/sesam/var/lis/
  22. Copy the license file:
  23. cp /tmp/sesam_restore/var/opt/sesam/var/ini/sm_lic.ini /var/opt/sesam/var/ini/ and confirm the overwrite: "/var/opt/sesam/var/ini/sm_lic. ini" overwrite? y
  24. Copy the sm_java.policy file:
  25. cp /tmp/sesam_restore/var/opt/sesam/var/ini/sm_java.policy /var/opt/sesam/var/ini/ and confirm the overwrite: cp:"/var/opt/sesam/var/ini/sm_java. policy" overwrite? y
  26. At the end of the recovery procedure the sesam service must be restarted. Use the command:
  27. sm_main stop followed by: sm_main start

After restarting SEP sesam, the system is again operational with the data that was backed up in the last SEP sesam self-backup.

Recovery on Windows

  1. In the first step, you will download and install the same SEP sesam Server package that was installed on the old system. It is important to provide the same operating system environment as that of the original server, therefore set up the new server system in the same way as the previous one.
    Double-click the SEP sesam Server installation file. Agree to the license agreement and check the Disaster recovery box when the first dialog appears.
  2. Sesam-Server-DR Win Start-Dialog en.jpg
  3. Continue with the installation until you get to the last step Ready to Install SEP sesam Server version <number>. Under the option Import SEP sesam Server database click Browse and select the bootstrap file that you have saved locally on the server, then click Install.
  4. After importing the bootstrap data, SEP sesam should have the basic configuration as before (for example, all existing drives). However, you should verify the following on your newly installed Windows server:
    1. Check the system settings to ensure that the restored SEP sesam configuration conforms to the current server environment (library and drive device names, disk partition device names and mount points, etc.). If not, the SEP sesam settings must be adjusted.
    2. If recovery is being executed from physical media, open the GUI and perform a SEP sesam archive adjustment to update the SEP sesam media information with the physical tape library: Main selection -> Components -> Loaders -> loader -> Archive Adjustment.
  5. Start the SEP sesam GUI and from the Main selection -> Job State -> Backups select the most recent SESAM_BACKUP, right-click it and select Restore. The restore wizard is displayed.
  6. In the Select task window, check the Disaster restore box and click Next.

    Sesam-Server-DR Lx RW-Select-Task en.jpg


  7. In the Save and Start window, select the following settings:
    • Set the target path to Restore to original target path
    • Set the execution options to overwrite existing files
  8. Click Start to start the restore.
  9. After the restore has been completed, the dialog Complete the disaster restore appears asking whether you want to import the database automatically. Confirm by clicking OK.
  10. At the end of the recovery procedure the sesam service must be restarted.

After restarting the SEP sesam GUI, the system is again operational with the data that was backed up in the last SEP sesam self-backup.

Alternative recovery procedure using command line

Typically, an email sent by disaster interface should have the SEP sesam bootstrap database file attached. If there is no attachment, you can still perform disaster recovery by running restore from the command line. You can find the required arguments in the disaster email, namely, the label of the media currently being used and the complete restore command, which restores the data from the save sets bootstrap database file attached.

  • Identify the location containing SEP sesam's last disaster backup and copy the restore command, located at the very end of the disaster email. Make sure to set the profile first. For details, see What happens when I set a profile?.
  • If the sesam backup is located on a medium (e.g., a tape), load the specific medium into the drive.

After the recovery

Make sure that the hardware is correctly connected then run a test backup and restore to the devices.

Some backup results may appear faulty in the job status views. This is because a certain amount of time passes between the last self-backup and the recovery time. A number of scheduled jobs were not run because the SEP sesam Server was not available at the time. The number of errors displayed depends on how long the SEP sesam Server was not available and of the number of backup jobs that were scheduled.