4 4 3:Analyzing SEP sesam Log Files
Analyzing SEP sesam log files is very useful for detecting operations that caused errors or malfunctions, for example, in case of a failed backup.
SEP sesam creates two protocols or log files for each backup day: the status file and the day log. An error log is the subset of the entire day log, where only error messages are recorded. Log files can be printed or sent by email. The default location (main directory) for the log files is
SESAM_VAR/log. You can check backup logs (state, day or error) in the GUI (Main Selection -> Logging -> State/Day/Error Log).
Log files creation order during a backup
When a scheduled backup is performed, the log files are generated in a certain order. Note that when you are analyzing a problem and you see that the corresponding log files are missing from a certain point in the past, a cause of the problem is most likely positioned just before that point. An example of how a log file is generated is given below.
- The SEPuler creates the log file sm_sepul_event_xxx.log, for example sm_sepul_event_20181004.log.
- The queue manager writes the sm_qm_main_xxx.log, for example sm_qm_main_20180913.log.
- If the backup was able to start, a bck_*.log is created.
- When the backup starts, a backup .not log (notification) is created in the SEP sesam's
SESAM_VAR/lisdirectory, e.g., smhg00_all- 20181004_001_SF20131004090011986@YlyxvqJCsHm.not.
- In case of optional media init, a sm_init_X_20180915.log is created. The X stands for the SEP sesam drive number.
- The information for monitoring drives and performance data is written in the sms log (sm_sms_watch_X_20181004.log).
- The files which are backed up are first written to a *.lis file (list of the backed up files and directories) and to sgm file (segment-file of used segment markers on the used tapes) on the device server in the
SESAM_VAR/work/smslisdirectory. Once the backup is finished, these files are copied from the device server to the SEP sesam
SESAM_VAR/lisdirectory. This data is needed for a selective restore.
Course of action for log file analysis
The recommended course of action depends on the failure. For example, if a scheduled backup did not run or have failed, proceed as follows.
Check the backup log in the GUI/Web UI
From Main Selection -> Backups, double-click the relevant Failed backup and open the Main Log tab to check the backup log.
Check if a medium init error happened
If a medium init error occured (bck_*.log), it is the cause a failed backup.
Check if a backup does not have a log or have failed
If the backup does not have a log yet or have failed:
- Check the status and daily logs in the
SESAM_VAR/protdirectory for events and errors at that particular time.
- Check if a .not log exists in the
SESAM_VAR/lisdirectory. If not, then the possible causes for the error are as follows:
- Client is not accessible (DNS, ping).
- There is no media available.
- The backup did not start yet.
SESAM_VAR/log/lgcdirectory. The logs should be listed chronologically in the terminal, e.g., Linux ls -lart).
- The bck_*.log is created by the program sm_backup; name convention: bck_<job_name>_<save_set_ID>_<sesam_day>.log. Note that unlike most other logs, the bck_*.log must be read from the beginning to find the first error message that may reveal the cause of the failure.
- Check the license.
- Set time range: Check if the backup is within the starting time frame.
- Alive test: Check if the client is active and reachable.
- CHECK_MEDIUM: Check the availability of the media.
- iGET_PREPARED_MEDIA: Check the media pool. If msg=0 appears, there is no media available.
- GET_BACKUP_MEDIUM: The sm_sms_interface is doing something on the tape (getlabel, init, etc.). For tasks started simultaneously, search for the largest backup file which contains the log files of the executed media init.
- In case of tape media, the following Options may be set as described in Configuring a Media Pool. They help you control which media will be used for a backup. For example, for devices which load media in sequential order, or if you do not want an unattended backup to fail because the specified media are not available, you may choose to use empty media policy.
- empty: If this option is selected and there is no EOL-free media available in the requested media pool, SEP sesam uses any suitable media for backup – empty media and media that are unrecognized by SEP sesam.
- spare: If this option is selected and there is no EOL-free media available in the requested media pool, then media from the SPARE_ pool are used for backup.
- other: If this option is selected and there is no EOL-free media available in the requested media pool, then any EOL-free media from other media pools are used.
- Note: The EOL defines how long the backed up data on media remains protected after the data is written to the medium (see Managing EOL). When the protection expires, SEP sesam can re-use the media for backups again. You can check the life cycle of a tape in the daily log; for details on media initialization, checked the *.sms log files in the
- sm_notify is delivered.
- With the search pattern "Cmd= sbc" you jump directly to the command given in the log which calls up the backup.
- If the backup has been started successfully, then the .not log is created in the
- During an active backup all log information are appended to .not log.
- If there are any problems with the media init, the errors will be recorded in the sm_init_<drive>_<sesam_day>.log. If a log includes error: all media with eol restriction, then no media in the requested pool is available. There may be further attempts to get a backup media according to the media pool options you have specified (see above Options.
- If there are any problems with the subsequent tapes after writing backup data until the End Of Media (EOM) is reached, they will be written in the sm_sms_watch_<drive>_<sesam_day>.log. If this log includes error: all media with eol restriction, then no media in the requested pool is available.
- To check the communication between the client and the SEP sesam Server either over SMSSH (default, via port 11322) or over CTRL (via port 11301), look for sm_sshd_<sesam_day>.log or sm_ctrld_<PID>.lgc in the
SESAM_VAR>/log/lgcdirectory on the client. The logs on the client are generated when the SEP sesam Server performs a task on the client, for example:
- Execution of a backup.
- Execution of a command.
- Browsing the GUI file wizard (when creating a task).
- Whenever the SEPuler finds a task that has to be executed, the operation is recorded in (sm_sepul_event_<schedule_identifier>_<sesam_day>.log). With QUE_SUBMIT a job is put into a queue; sm_backup shows that the backup is being put into the queue.
- On a SEP sesam Server, search the log sm_sms_watch_<drive>_<sesam_day>.log from bottom upwards to check the drive information. This includes:
- Monitoring of a drive. This log is generated only on a SEP sesam Server (also for the drives on RDS, the corresponding watch logs are located on the SEP sesam Server).
- Data throughput of a drive.
- The search string "‘+++ EOM"‘ shows media changes related to end of media (tape is full). The sm_sms_interface init command:
- STATUS=SUCCESS: Successful media initialization.
- STATUS=IO-ERROR: There is a problem with media or a drive. If necessary, check the
SESAM_VAR>/log/messageson Linux or the event log on Windows for any hardware problems. To confirm that there is a hardware problem, check the sms log in the
- The log sm_sms_watch_0_<sesam_day>.log regularly displays the process status of the sm_main processes and the processes in the sm_qm_main queues, in addition to regular check of available space GET_FREE_SPACE_OF_DIR.