5 1 0:NDMP Backup

From SEPsesam
Other languages:


Docs latest icon.png Welcome to the latest SEP sesam documentation version 5.1.0 Apollon. For previous documentation version(s), check documentation archive.


Overview


This article describes SEP sesam NDMP backup, introduced in version 4.4.3 Grolar. For details on backing up the NetApp NFS volumes via NFS, see NetApp NFS Backup.

SEP sesam enables you to protect and manage your storage file servers by providing support for Network Data Management Protocol (NDMP). NDMP is a common protocol for backing up and restoring data on storage devices that do not support installation of a backup agent. It ensures interoperability between NAS file servers and SEP sesam, and is used as an interface that enables SEP sesam to backup various NAS appliances and copy this data to a SEP sesam Server or Remote Device Server. SEP sesam supports version 4 of the NDMP protocol. For a list of supported NAS appliances, see the support matrix.

NDMP remote configuration

SEP sesam provides NDMP remote configuration, where the data from the NDMP host is backed up and transferred over a TCP/IP connection to SEP sesam Server's connected storage, e.g., disk storage or a virtual tape library that is used for data protection and archiving.

SEP sesam provides the SEP sesam Data Management Application (SDMA) as a session master that is responsible for initiating and managing backup and restore operations of NDMP compliant systems. Each such operation is called an NDMP session. The SDMA cannot directly access source data; it serves as a remote NDMP client. Access to the source data and the destination of the backed up data is provided by NDMP data services.

NDMP data service on NDMP host provides an interface to the source – the primary data storage, e.g., filers that store "live" data on an active file system. During backup, it converts the content of the file system into a data stream and sends it via the NDMP data connection to the SEP sesam Server or RDS. Note that the backup data format depends on the NDMP appliance and user-defined configuration of this format. During restore, it reads the data stream and recreates the file system.
Another NDMP data service is located on the SEP sesam NDMP host (SNDMP) and provides the interface to the SEP sesam storage. During backup, it writes a data stream to the SEP sesam Server or RDS. During restore, it reads content from data storage and converts it into a data stream in an NDMP data connection.

The SDMA establishes and manages connection and communication between the source and the destination NDMP data services. SDMA uses NDMP control connection to issue commands to the servers while the data stream flows via the NDMP data connection, thus ensuring fast and efficient communication between the remote services.

The following figure shows the SEP sesam NDMP remote configuration.
NDMP.png

Cluster Aware Backup (CAB) backup

SEP sesam supports the NDMP CAB (Cluster Aware backup) extension to perform backup and restore operations for cluster nodes running ONTAP. The NDMP v4 protocol (supported as of ONTAP 8.2 and later) is modified with a NetApp Cluster Aware Backup (CAB) extension which allows mutual communication between a backup software (SDMA) and the NDMP server.

Information sign.png Note
For now, the CAB extension is supported only in the SVM-scoped NDMP mode, which allows you to perform backups and restores at the Storage Virtual Machine (SVM) if the NDMP data service is enabled on the SVM. See NetApp-specific NDMP configuration for more configuration details on SVM-scoped mode.

You can back up and restore all volumes hosted across different nodes in an SVM of a cluster. The entire cluster is available for the backup, which means that all the volumes are exposed irrespective of where they are hosted. The SDMA notifies the NDMP server about the volume to be backed up (or restored) prior to establishing the data connection. CAB extension enables the NDMP server to identify the node on which the volume is hosted and to appropriately establish a data connection on the node that hosts a volume.

The CAB advantage is that it enables SEP sesam to perform a local backup or restore by connecting to the cluster management interface IP address and accessing all the volumes belonging to all storage virtual machines of the cluster over this single interface.

Information sign.png Note
If CAB is not supported, then backup/restore can be performed only over the IP address of the cluster physical node on which the backup source volume is located.

Backup level

NDMP supports only backup levels numbered in the range 0 to 9. The table shows SEP sesam backup levels and the corresponding NDMP backup levels:

SEP sesam backup level NDMP level
FULL, COPY 0
DIFF 1
INCR 2

NDMP supports full backups and incremental backups. The first NDMP backup must be a FULL backup (level 0). A backup level greater than 0 is an incremental backup of all objects that have changed since the last backup of a lower level, and should be performed next. For example, level 1 (DIFF) is a backup of all objects that have been changed since the FULL backup (level 0). Level 2 (INCR) is a backup of all objects that have been changed since the last level 1 (DIFF), and so on. For details, see Environment variables supported for dump.

Information sign.png Note
Unlike the SEP sesam incremental backup, which stores only data created or modified after the last backup (FULL, DIFF or INCR), NDMP incremental (level 2) stores all objects modified since the last differential backup. Such backups occupy a lot of your storage space, so you need to make sure that there is enough free space on the target storage system for your NDMP backups.

Key features

Supported backup formats

The NDMP protocol does not specify a backup data format, it depends on the used NDMP applliance. For a list of supported NAS appliances, see the support matrix. SDMA is implemented on all supported Linux and Windows systems.

The following backup formats are supported depending on appliance:

  • dump (default): Back up files and directories to tape. It is not deduplication- and compression aware and does not preserve deduplication when backing up data.
  • smtape: This format is only supported by NetApp. An image level backup that backs up blocks of data of the origin volume. It is deduplication aware and preserves dedup when backing up and restoring data. Does not support file history or direct access recovery (DAR), thus providing only recovery of a complete volume (including snapshots).
  • zfs: This format is only supported by Oracle Solaris ZFS. Share-based for both file systems and volumes. Does not support file history or direct access recovery (DAR), but may be faster for some datasets. It is only possible to restore the whole file system or volume (selective restore is not supported).

For example, with NetApp you can choose between the normal NDMP dump and smtape backup.

SEP sesam NDMP integration advantages

  • The NDMP protocol allows for backup and restore operations of heterogeneous filers and excludes the necessity of mapping a drive on an alternate server and then backing up.
  • SEP sesam provides centralized and automated backup and restore of data residing on remote NAS systems that do not have locally attached libraries.
  • NDMP backups can be stored to all SEP sesam supported storage devices. Si3 deduplication can also be used for NDMP data and can significantly reduce the size of NAS NDMP backups.
  • SEP sesam allows storage sharing between different NAS systems as well as combined NDMP and non-NDMP backup, thus optimizing storage space.
  • All backup levels (COPY, FULL, INCR, and DIFF) and selective restore from dump backups are supported.

NDMP restrictions

  • NDMP data can only be restored to the same vendor and NAS device running the same or higher OS version as the NAS device that was used for backup.
  • NAS deduplicated or compressed data is uncompressed before being sent across the network to a SEP sesam Server or RDS in uncompressed form.
  • NDMP backups might be time-consuming in large NAS configurations with large volumes.
SEP Tip.png Tip
Direct access recovery (DAR) is used during backup to enable restoring a single directory or file from a backup (selective restore). As of version 4.4.3 Grolar, DAR is enabled by default (no configuration is required). If required, it can be disabled. For details, see section dump backup.

Prerequisites

  • SEP sesam Server v. 4.4.3 Grolar or higher. Note that backup and restore procedures are version-dependant. For details on NDMP backup & restore in v. 4.4.3 and 4.4.3 Tigon, see NDMP Backup 4.4.3 and NDMP Restore 4.4.3.
  • NDMP must be configured on your NAS appliance. Refer to your NAS appliance documentation for configuration instructions. Some basic information are provided below in the section Configuring NDMP host.
  • You must ensure the bidirectional communication between the SDMA and NAS. The NAS appliance will attempt to connect to SDMA, which needs access to port 10000 on the NDMP data server (NAS) and on the SDMA host system. Once the connection is established, unlimited ports must be opened to allow the transfer of the information to SEP sesam. Therefore, there must be no firewall between NAS and SDMA.
Information sign.png Note
NDMP uses port 10000 by default, but the server may listen on a different port. To specify a different port for the NDMP server, enter the following option in the backup task properties -> tab Options -> Backup options (previously Save options) field:
-a port=<value>
  • It is recommended that 10 gigabit Ethernet (10GbE) is available for larger NDMP backups.
  • The target media pool for backing up your NAS device must have sufficient space to contain the NAS data, which is always backed up to the pool uncompressed.
  • An additional NDMP license is required for SEP sesam with NDMP. For details, see Licensing.

Installing SEP sesam NDMP backup package

SEP NDMP module is already included in the installation package. SEP sesam NDMP (sbc_ndmp) must exist on the system which serves as your data mover. For details, see What is a SEP sesam Data Mover.

SEP sesam with NDMP requires an additional NDMP license. For more information, contact SEP sesam support at support@sep.de.

Configuring NDMP host

NDMP configuration involves several steps, including authentication with the NDMP server. Typically, you should use the root password or non-root user with a system-generated, NDMP-specific password. Refer to your NAS appliance documentation for configuration instructions!

NDMP configuration steps for NetApp are described in NetApp-specific NDMP configuration.

Adding NDMP host as a SEP sesam client

If you want to protect your NDMP-ready system, you must configure your NDMP host as a SEP sesam client.

  1. From Main Selection -> Components -> Clients, select the location of your choice, e.g., Storage systems, and click the New Client button. The New Client window opens.
  2. Enter the client Name. From the Platform drop-down list, select NDMP. When selecting the NDMP platform, the operating system (NETAPP) and PROXY access mode are set automatically.
  3. Information sign.png Note
    If you want to configure the Dell EMC Unity NDMP client (supported from Beefalo V2), select the NDMP as platform and EMC VNX as the operating system. The PROXY access mode is set automatically.

    NDMP new client.jpg

  4. Click the newly displayed NDMP Access tab. Enter the NDMP user name and password. From the Data mover drop-down list, select the system where the SDMA is running.
  5. NDMP access.jpg
  6. Click OK to add your NDMP host as a client.

Configuring NDMP backup

For NAS backup, the SEP sesam Server acts as a backup session master responsible for managing media, scheduling and monitoring NDMP backup and restore. To configure the NDMP backup, you must create a backup task (select the data to be backed up), then set up a backup schedule (specify when you want to back up your data) and link a backup event to it (specify how to back up the data and where to back it up to).

For general details on how to configure a backup and its prerequisites, see Standard Backup Procedure. However, configuring an NDMP backup differs slightly from the standard procedure and requires some NDMP-specific steps, which are described below.

Depending on your appliance, you can configure a traditional NDMP dump backup or smtape backup (NetApp specific option). The backup task creation is similar for all NDMP backups, the main difference is that you must define additional arguments for smtape backup in the backup task properties -> tab Options -> Backup options (previously Save options) field.

Direct access recovery (DAR) index is mandatory to restore a directory or a single file from a backup (selective restore). As of version 4.4.3 Grolar, DAR is enabled by default (no configuration is required).

Information sign.png Note
  • You have to create a separate task for each volume you want to back up. With NDMP implementations which allow to set a directory as a backup source, you need a separate task for each directory that is selected as a source.
  • The NDMP backup is based on the NDMP backup levels, which are not the same as the SEP sesam backup levels. Be sure to familiarise yourself with the NDMP backup levels and the corresponding SEP sesam levels, as some restrictions may apply. For details, see the section Backup Levels.
  • When creating a Dell EMC Unity NDMP backup task and browsing for a source, NetApp is incorrectly shown in the Client File View. Make sure to select the appropriate NDMP source (not NetApp) for Dell EMC Unity NDMP backup.

Dump backup

  1. From Main Selection -> Tasks -> By Clients, select the NDMP host to be backed up and click New Backup Task. The New Backup Task window opens.
  2. Select the Source. Click the Browse button (big arrow) and select the volume you want to back up.
  3. NDMP bck source volume.jpg
    SEP Tip.png Tip
    Typically, NDMP implementations only allow to select/display volumes, therefore SEP sesam cannot expand the tree structure to display and select the underlying directories. However, some implementations allow to specify a directory as a backup source manually, while other allow to exclude files and folders from backup (see step 4). See examples below for details.

    In the Source field, you can manually enter the directory to be backed up for the following NDMP implementations:

    Clustered Data ONTAP (NetApp)
     <storage_virtual_machine_name>/<volume_name>:/<directory>
    7-Mode Data ONTAP
     /vol/<volume_name>:/<directory>
    EMC VNX
    EMC NDMP implementation does not support backing up individual files and folders. You can only exclude files and directories, as described in step 4.
  4. If you have browsed the source, the task type and task name are set automatically. If you have entered the source manually, enter the name of the task in the Task name field.
  5. Depending on the NDMP implementation, you can manually exclude files and directories from backup. Switch to the Options tab and under Additional call arguments in the Backup options (previously Save options) field specify the files and directories to be excluded by using the relevant syntax:
  6. Clustered Data ONTAP (NetApp) and 7-Mode Data ONTAP
    Specify the file names or patterns which you do not want to back up with the exact name. You must use integrated common parameter -x to work as exclude list, for example -x /dir1,/dir2.
    Note that a file or directory name must be preceded with a forward slash "/" and followed by a comma (without space!), if there are more items listed. The exclude list can contain up to 32 names. You can also use an asterisk (*) (a wildcard character) as the first or the last character of the string; each string can have up to two asterisks. For example, to exclude directories smallfiles1, smallfiles2, smallfiles3, smallfiles4 ... or 1data, 2data, 3data:
     -x /smallfiles* 
     -x /*data
    To exclude dir1, dir2 and dir3, use:
     -x /dir1,/dir2,/dir3
    If your backup source is a directory (and not the whole volume), e.g., /vol/cifs1:/Home/Base, and you want to exclude one of its subdirectories, do NOT specify the whole path for the exclude, but only the directory that you want to exclude. For example, if you have specified a backup source /vol/cifs1:/Home/Base and you want to exclude the subdirectory /vol/cifs1:/Home/Base/WIKI/temp_files, specify the exclude only with the subdirectory name and not with the whole path:
     -x /WIKI/temp_files
    EMC VNX
    You have to specify the following variables to sbc_ndmp: EMC_EDIRxx and EMC_EFILExx (where xx needs to be replaced with increasing number, for example EMC_EDIR01=/cifs_3gb/images, EMC_EDIR02=/cifs_3gb/music etc.).

    If there is more than one file or directory to exclude, they must be enumerated from number 01; a max of 50 variables can be specified (EMC_EDIR50 and EMC_EFILE50 are the highest possible variables). You can also use the wildcard characters to specify the file or folder patterns. Example:

     -a env=EMC_EDIR01:/cifs_3gb/folder1,env=EMC_EDIR02:/cifs_3gb/folder2 

    NDMP backup exclude VNX.jpg

  7. By default, SEP sesam uses direct access recovery (DAR) to enable selective restore. While DAR can greatly reduce the time it takes to restore individual files, it might be unnecessary in case you want to backup and consequently restore only the whole volume. In such case, you may disable DAR to improve backup performance by entering the following option in the Options tab -> Backup options (previously Save options) field:
  8.  -o list=none
  9. Save your task.
  10. Configure a schedule for your backup task, as explained in Creating a Schedule. Once you have configured a schedule, you must create a new backup event for it.
    SEP Tip.png Tip
    You can also add your backup task to an existing schedule by double-clicking the backup task, selecting the tab Schedules and adding it to one or more schedules.
  11. Go to Main Selection -> Scheduling and select the schedule you created for NDMP backup, then click New Backup Event. Select one of the backup levels (FULL, DIFF or INC) from the Backup level drop-down list. For more information on configuring a backup event and available options, see Creating a Backup Event.

smtape backup

smtape (SnapMirror-To-Tape) is only supported by NetApp. It is an image-based high-performance disaster recovery solution from Data ONTAP that backs up blocks of data of the origin volume. It is deduplication aware and preserves dedup when backing up and restoring data.

  1. Create the task as described in the above procedure. Before saving the task, click the tab Options. In the Backup options (previously Save options) field, enter:
  2.  -a ndmp_tape_format=smtape
  3. Save your task.
  4. Configure a schedule and create a new backup event for it: Select the schedule you created for NDMP backup, and then click New Backup Event.

You can view the status of your backup jobs by selecting Last Backup State in the Main Selection window. The backup status overview provides detailed information on the last run of backup jobs, including the task name, start and stop time of the last backup, backup level, data size, throughput, assigned media pool, etc.

Information sign.png Note
You can check the details of your backups online by using the Web UI. For details, see SEP sesam Web UI.

Known issues

If you have any problems with NDMP, check the Troubleshooting Guide.

What is next?

NDMP Restore


See also

NetApp-specific NDMP configurationNDMP RestoreStandard Backup Procedure

Copyright © SEP AG 1999-2024. All rights reserved.
Any form of reproduction of the contents or parts of this manual is allowed only with the express written permission from SEP AG. When compiling and designing user documentation SEP AG uses great diligence and attempts to deliver accurate and correct information. However, SEP AG cannot issue a guarantee for the contents of this manual.