Archive:Disaster Recovery for Linux 3.0

From SEPsesam

Template:Copyright SEP AG en


Introduction

This document includes information about SEP Sesam BSR Linux with REAR integration. Relax-and-Recover (REAR) (http://relax-and-recover.org/) is a setup-and-forget Linux bare metal disaster recovery solution based on GPL licence. This solution has been integrated into a SEP sesam backup client installation.

Requirements

  • SEP Sesam client >= 4.2.2
  • SEP Sesam Server >= 4.2.2
  • Enough free space in /tmp/, /boot/ and /var/opt/sesam/ to keep the temporary ISO image (ca. 100 MB, depending on the system)!

Supported distributions

See the following list in our support matrix:

SEP REAR Support Matrix

Package requirements

There are some requirements which packages have to be installed, that a ReaR-based backup can be performed successfully.

  • RHEL 6
    • syslinux

Limitations

  • Backup of Sesam Server and Sesam RDS is currently not supported
  • Backup of Systems running on Citrix XEN Virtual Machines is currently not supported

Dissimilar Hardware

In case you want to provide dissimilar hardware scenarios in your environment, please make sure to test this behavior beforehand. As SEP Sesam for REAR is generally able to restore to dissimiliar hardware there can always be certain limitations. As SEP cannot guarantee the dissimiliar hardware restore to work on every platform we do not give general support for it. In case you want to restore to dissimilar hardware, please test the restore szenario carefully to see if there are any limitations.

Backup

The SEP Sesam Disaster Recovery allways includes a special Linux BSR backup task and a complete file system backup of the clients. This documentation explains what is happening during the backup and how the backup tasks should be configured.

Functionality

BSR-Linux3-Funtionality.jpg


During backup the following steps are performed:

  1. The backup task creates a bootable CDROM or DVD image from the running system, including the SEP Sesam client
  2. The backup task creates a boot option in the systems bootloader which can be booted in case that the original boot config fails
  3. The created iso image is transferred to the backup server and backed up by a regular file backup
  4. A regular file backup is performed with a second task

Creating the backup tasks

Disaster Recovery Task

First step is to create a backup task for the Linux BSR console, this happens by choosing the Linux BSR backup option while browsing the client. This backup task will create the bootable CD / DVD image and the appropriate new bootloader entries on the client. This backup task should always be performed before the regular file backup.


Rear backup 1.jpg


Rear backup 2.jpg

Special task options

The following special options can be passed to the backup job "additional save options":

-a nodelete : do not delete generated iso image after backup 
-v 3 : create REAR Logfile with debugging output (stored in: /var/opt/sesam/var/lib/rear/var/log/rear/)

Filesystem backup

The second backup task is needed to recover a system completely is a full backup of the clients filesystem (Backup type: "Path Backup" Source: "all"):


Rear backup 3.jpg


Creating the bootable restore disk image

After the Disaster Recovery Task has completed, a restore of it has to be performed to gain the bootable .iso file containing the recovery environment. Therefor, a restore of the BSR task (rear-test_BSR in the pictured example) is done using the standard options.

When the restore task has completed successfully, the iso file stored in /var/opt/sesam/var/lib/rear/var/lib/rear/output/rear-[hostname].iso on the client can now be used to create a bootable media for the restore.

Restore

Functionality

BSR-Linux3-Recovery.jpg


During recovery the following steps are performed:

  1. Restore the ISO image to a different Linux machine, mount it to the recovery host or create a CDROM and use it as boot medium
  2. The administrator boots the recovery console either via the created CDROM Image (2.1) or - if possible - via the systems booloader (2.2)
  3. The administrator recovers the systems partitions and filesystem configuration via the recovery console
  4. A restore of the complete filesystem backup of the client is performed to the newly created partition

After the steps have been performed successfully the system is rebooted

Performing a restore

Restoring the CDROM Image to a filesystem

In case the system is not bootable via the recovery console anymore (in case of a complete filesystem crash) the system must be booted via the recovery console CDROM. The first step is restoring the created ISO image to a random system and burning it to a media of your choice.

Booting the System from external media like USB/CDROM (or on-disk recovery console)

Bootmenu

After the media has been inserted the following boot menu will appear:


Rear restore 1.jpg


Networking

During bootup the system will provide you with a selection of network interfaces. Choose the right network interface which is needed to ensure connectivity with your backup server. The IP configuration of the original system will be used:


Rear restore 2.jpg


Login to the disaster recovery console

After the system has finished booting you can login to the disaster recovery console with the username "root" and no password. The disaster recovery console is also reachable via SSH, in case you have to login remotely. The recovery process can be started with the command "rear recover" just as shown in this screenshot:


Rear restore 3.jpg


Restoring disk information

The disk restore wizard will suggest a local disk to be partitioned, select the appropriate disk to be used:


Rear restore 4.jpg


Confirm the disk layout configuration:


Rear restore 5.jpg


Start the recovery and let the recovery console create the filesystems and partitions:


Rear restore 6.jpg


Rear restore 7.jpg


After the recovery check wether if all filesystem have been created, in the screenshot above you can see that there is an newly created mountpoint "/mnt/local". This will be the restore target for the next step: restoring the filesystem data.


Rear restore 8.jpg


Restoring the filesystem backup

After the partitions and filesystems have been created the next step is to restore the files of the system to the recovery console.


Rear restore2 1.jpg


Rear restore2 2.jpg


Restore target must be set to /mnt/local/

The restore target must be changed from the default value "all" to the new target "/mnt/local/" on the recovery console


Rear restore2 3.jpg


After the filesystem backup has been restored exit the disaster recovery console with the command "exit" and reboot the system.

Remote login via SSH

The recovery process can also be initiated remotely, means that the booted system image is reachable via SSH. Username for login is 'root' and the password is either empty or 'recover'.

Troubleshooting

Known Issues

REAR image hangs during bootup

In case the system hangs during bootup like shown in the following image:

Rear-hang.jpg

try booting the system with the ACPI=OFF option (this option can be specified on the command line in the boot menu prompt, after the options BACKUP=SESAM OUTPUT=ISO)

RHEL6/CENTOS6

Client won't start in recovery image

The Sesam client will not automatically start on RHEL6 based Recovery images as the file "/etc/init.d/functions" is missing within the recovery image. The client can be started manually via:

/opt/sesam/bin/sesam/sm_main start
Recovered system wont boot because of missing "libblkid.so.1"

If the recovered system won't boot, this is most probably caused by SELinux which is activated by default. Especially on RedHat Enterprise Linux 6 or CentOS 6 systems, follow these steps after rebooting from the REAR recover:

  • Press a key when prompted by the boot loader (GRUB):

Rhelcentos grub1.jpg


  • Select the appropriate boot loader entry:

Rhelcentos grub2.jpg


  • Press 'e' to edit the commands for the selected entry:

Rhelcentos grub3.jpg


  • Add 'selinux=0' to the commands:

Rhelcentos grub4.jpg


  • Press 'Enter' to confirm the changes and 'b' to boot up the machine with SELinux disabled.

When having access to the system, change the option SELINUX of /etc/selinux/config to the following:

SELINUX=permissive

Afterwards, reboot the system and feel free to set the SELINUX value back to 'enforcing' if needed.


RHEL7

backup fails with ERROR: The LSB package is not installed

If the backup fails with:

ERROR: The LSB package is not installed.

Please install the lsb package:

yum install redhat-lsb-core mkisofs syslinux
backup fails with: ERROR: Cannot find required programs: mingetty

See the followign documentation:

https://github.com/rear/rear/issues/348

Please edit:

/var/opt/sesam/var/lib/rear/usr/share/rear/conf/default.conf

in and in the line:

# required programs. Same as above, but if they are missing, we abort.
REQUIRED_PROGS=(
"$SCRIPT_FILE"

remove the line:

mingetty

and try the backup again.

Further documentation

  • For further information about Relax and Recover see here