VSS Troubleshooting

From SEPsesam
Jump to: navigation, search
Other languages:
Deutsch • ‎English

Copyright © SEP AG 1999-2019. All rights reserved.

Any form of reproduction of the contents or parts of this manual is allowed only with the express written permission from SEP AG. When compiling and designing user documentation SEP AG uses great diligence and attempts to deliver accurate and correct information. However, SEP AG cannot issue a guarantee for the contents of this manual.

Docs latest icon.png Welcome to the latest SEP sesam documentation version 4.4.3/4.4.3 Beefalo. For previous documentation version(s), check documentation archive.


Volume Shadow Copy Service (VSS) issues

SEP sesam uses Microsoft’s Volume Shadow Copy Service (VSS) to perform backup for various task types. It uses only the VSS components, which are provided by Microsoft. VSS failures are typically caused by system configuration and not by SEP sesam. Note that VSS has some restrictions and limitations. For more detailed information on VSS, see Microsoft's article Volume Shadow Copy Service.

This section provides provides only limited instructions for most common VSS issues. In more specific and difficult cases, refer to Microsoft support (KB).

Common VSS problems

Follow the instructions below if you encounter a VSS error. Run a test after each step to make sure that the problem has been resolved.

1. Backup is failing with a 10054 error. The event viewer shows the error: Cryptographic Services failed while processing the OnIdentity() call in the System Writer Object.

Problem

  • The VSS backup on Windows 8.1, Windows 10 or 2012/R2 fails with error. The backup log shows:
  • Info: Backup finished. Status: ERROR Data sending failed. (10054) An existing connection was forcibly closed by the remote host.
    

    More specific information about the cause of the error is shown in the application event log:

    Cryptographic Services failed while processing the OnIdentity() call in the System Writer Object.
    Details:
    AddLegacyDriverFiles: Unable to back up image of binary Microsoft Link-Layer Discovery Protocol.
    System Error:
    Access is denied.
    

Cause

  • The MSLLDP driver's security permissions do not allow NETWORK_SERVICE to access the driver record. This may cause the backup process to hang for a long period of time or fail.

Solution

  1. Open an administrative command prompt (NOT PowerShell) and execute the following command as one long command without carriage returns (do not move the cursor to the beginning of the line):
  2. sc sdset MSLLDP D:(D;;CCDCLCSWRPWPDTLOCRSDRCWDWO;;;BG)(A;;CCDCLCSWRPWPDTLOCRSDRCWDWO;;;SY)(A;;CCDCLCSWRPDTLOCRSDRCWDWO;;;BA)(A;;CCDCLCSWRPWPDTLOCRSDRCWDWO;;;SO)(A;;LCRPWP;;;S-1-5-80-3141615172-2057878085-1754447212-2405740020-3916490453)(A;;CCLCSWLOCRRC;;;SU)S:(AU;FA;CCDCLCSWRPWPDTLOCRSDRCWDWO;;;WD)
    
  3. If you get a successful result, e.g., [SC] SetServiceObjectSecurity SUCCESS, the problem is solved. Otherwise, reboot your Windows system.

2. Windows reboot

Solution

  • Reboot your Windows. A Windows reboot often eliminates various VSS writer problems.

3. Disk space

Problem

  • If there is insufficient disk space, VSS snapshots may fails.

Solution

  • Make sure that sufficient disk space is available.

4. Multiple backup jobs with VSS

Problem

  • If multiple VSS jobs are started, the operating system reports the following VSS error message:
  • VSS_E_SNAPSHOT_SET_IN_PROGRESS
    

Possible causes

  • It is not possible to run multiple VSS snapshots at the same time. Windows can only start a VSS snapshot for each partition.

Solution

  • Run the backup jobs from a client sequentially, for example:
    • 22:00: Start System Recovery Backup
    • 23:00: Start Custom Path Backup.
  • If the temporal separation is not possible, try the following solution. These must be set in the sm.ini following settings on the client:
  • [SBC_OPTIONS]
    VSS_WAIT_FOR_ASYNC_OP=180
    VSS_DELAY=90
    VSS_MAX_RETRY=6
    

    With these settings, the VSS sesam client tries to execute operations sequentially and waits for an active VSS operation. The matching settings may vary depending on system and backup jobs.

5. Third-party backup software

Solution

  • Check if some other backup software from different vendor is running on the system, for example:
    • Windows Backup, NTbackup
    • Acronis, Backup Exec, etc.

Disable this backup software if necessary.

6. Remove old VSS snapshots

Solution

  • Start the CMD command prompt as administrator and type the following command:
  • vssadmin delete shadows /all
    

    The command cleans up all VSS snapshots of all volumes.

7. VSS writer status check

Solution

  • Start the CMD command prompt as administrator and type the following command:
  • vssadmin list writers
    
  • If a VSS writer error appears, restart the following services:
    • Volume Shadow Copy Service
    • VSS application writer (Exchange, MS-SQL, Hyper-V)
    • COM+ System Application Service
    • Distributed Transaction Coordinator Service
  • Run the command vssadmin list writers again and check if the issue has been resolved.

8. Windows event log check

Solution

  • Check the event logs of the Windows Event Viewer. Search for VolSnap errors corresponding to VSS at the time of the SEP sesam backup job.
  • It often helps to search the internet for the event ID of the fault to get an indication of the error cause.

9. Re-register VSS and COM+ components

Step 1:

  • The following scripts re-register all VSS and COM+ components.
  • Copy the commands in a new *.bat file. Start the batch script in the command line as administrator:
  • For 64-bit and 32-bit systems:

    cd /d %windir%\system32
    Net stop vss
    Net stop swprv
    regsvr32 ole32.dll
    regsvr32 vss_ps.dll
    Vssvc /Register
    regsvr32 /i swprv.dll
    regsvr32 /i eventcls.dll
    regsvr32 es.dll
    regsvr32 stdprov.dll
    regsvr32 vssui.dll
    regsvr32 msxml.dll
    regsvr32 msxml3.dll
    regsvr32 msxml4.dll
    regsvr32 Vssapi.dll
    regsvr32 Vssui.dll
    net start vss
    net start swprv
     

Step 2:

  • Copy the commands in a new *.bat file. Start the batch script in the command line as administrator:
  • For 64-bit systems:

    Net stop vss
    Net stop swprv
    regsvr32.exe /i %windir%\system32\eventcls.dll
    regsvr32.exe /i %windir%\system32\swprv.dll
    regsvr32.exe %windir%\system32\vssui.dll
    regsvr32.exe %windir%\SysWOW64\vss_ps.dll
    regsvr32.exe %windir%\SysWOW64\msxml.dll
    regsvr32.exe %windir%\SysWOW64\msxml2.dll
    regsvr32.exe %windir%\SysWOW64\msxml3.dll
    regsvr32.exe %windir%\SysWOW64\msxml4.dll
    regsvr32.exe %windir%\SysWOW64\ole32.dll
    regsvr32.exe %windir%\SysWOW64\oleaut32.dll
    regsvr32.exe %windir%\SysWOW64\es.dll
    regsvr32.exe %windir%\SysWOW64\comsvcs.dll
    vssvc /register
    net start swprv
    net start vss
    net stop winmgmt
    regsvr32 wmiutils.dll  
    net start winmgmt 
    

10. VM utility update

Solution

  • If the system is a virtual machine, uninstall the installed utility version and start the VM then reinstall the latest VM utility version:
    • VMware vSphere: VMware Tools
    • Microsoft Hyper-V: Hyper-V Integration Services
    • Citrix XenServer: XenServer Tools

11. Windows update

Solution

  • Install the latest Windows updates, service packs and hot fixes from Microsoft.
  • Run a test after the update.

VSS backup fails with VSS error

If you encounter a VSS error while backing up your Windows client, you need to know what is its cause; you should open the Windows Event Viewer and check related logs. These errors may be any of the following:

"VSS_ERROR_UNEXPECTED_ERRORCODE" 
"VSS_E_PROVIDER_VETO"
"VSS_E_VOLUME_NOT_SUPPORTED" 
Information sign.png Note
You may receive other similar error messages and/or aborted backups if you try to perform a VSS backup on volumes larger than 64 TB in size, even if the volumes are excluded from the backup. Note that writable snapshots or snapshots larger than 64 TB produce the same error.

Microsoft states the limitation of "64 TB" in the article Usability limit for Volume Shadow Copy Service (VSS) in Windows.

The above errors may be caused by any of the following:

Problem 1

  • When trying to perform a VSS backup with volumes larger than 64TB in Windows server, you get the following error:
"DB Module: [ [NInternal::CVssAsyncDecorator::Wait] - S_OK]"

and Windows event logs display a VSS event ID 12289

"VSS_ERROR_UNEXPECTED_ERRORCODE" 

Cause

  • Microsoft has a limitation of 64 TB as the maximum size for volumes when performing backups with VSS. It is not possible to utilize VSS for larger volumes, even if they are excluded from backup.

Solution

  • Enable the Volume Shadow Copy Service for all drives: Right-click on any of the listed Hard Disk Drives and click Configure Shadow Copies. In the Shadow Copies window, click Enable and then enable the shadow copies for all drives.

Problem 2

  • When trying to perform a VSS backup of a volume that is larger than 60 TB or when you create VSS backups of multiple volumes in Windows Server 2008/Windows Storage Server 2008, the VSS backup fails with the following message:
ERROR: [CVssServer::CreateSnapshot(): IVssBackupComponents::DoSnapshotSet: WaitForAsyncOperation] - VSS_E_PROVIDER_VETO 

Possible causes

  • As explained before (see above Note), Microsoft has a usability limit for volumes and snapshots with VSS.
  • This error can also occur if a VSS provider blocks the snapshot creation for the following reasons: Software has been installed with its own VSS provider or the existing VSS component is corrupted.
  • HP Device Access Manager (FLCDLOCK service; a part of the HP Protect Tools suite) is preventing VSS operations to properly access drives partitions.

Solution

  • Due to the limitation within volsnap.sys, do not use any volumes larger than 60 TB as the backup source or destination for Windows backup. Remove the volumes larger than 60 TB from the VSS snapshot action. Reconfigure all disks with a current size larger than 60 TB to 60 TB or less to prevent VSS from stopping when creating snapshot.
    Additionally, check and (if required) increase the size of shadow storage on the respective drives. Use the Vssadmin command-line tool from an elevated command prompt (where c: should be replaced with the correct drive):
vssadmin Resize ShadowStorage /For=C: /On=C: /Maxsize=5%
  • Fix VSS problems and then re-register the VSS components. Additionally, check and (if required) increase the size of shadow storage on the respective drives by using the Vssadmin command-line tool, as described above in the previous solution.
  • Uninstall HP Device Access Manager and restart your system. Note that an incomplete uninstallation of HP Device Access Manager may cause problems. You can use the third-party removal tool to correctly and completely uninstall Device Access Manager for HP ProtectTools. For details, see UninstallHelps.com article How to uninstall Device Access Manager for HP ProtectTools?. Note that this link takes you to the third-party site; SEP provides such links for information purposes only and is not responsible for any information, availability or accuracy of such outside resources.


Problem 3

  • VSS Path backup from a Windows Storage Spaces volume fails with error:
Error:   DB Module: [ [CVssServer::WaitForAsyncOperation: IVssAsync::QueryStatus] - VSS_E_PROVIDER_VETO]

Cause

  • VSS snapshots are not supported for Windows Storage Spaces.

Solution

  • To back up data from Windows Storage Spaces volumes, you have to disable the option Backup with VSS (enabled by default) in the backup task properties. For details, see Disabling/enabling VSS-based backup.

See also

SEP sesam Volume Shadow Copy Service (VSS) for Windows