Go Back   Tech Support Forum > Microsoft Support > Windows XP Support

User Tag List

Ntbackup and recent portable USB drives

This is a discussion on Ntbackup and recent portable USB drives within the Windows XP Support forums, part of the Tech Support Forum category. I have an old machine XPSP3 32bit Home running NTbackup when I plug in an external drive Z: it causes


Closed Thread
 
Thread Tools Search this Thread
Old 12-10-2018, 01:46 AM   #1
Registered Member
 
RaceQ's Avatar
 
Join Date: Aug 2017
Location: Perth, Western Australia
Posts: 43
OS: XP[HP]32SP[23] AOS 5.0.1 Past: W2KSSP4 W2KW W98SE Slackware Ubuntu SUSE DOS UNIX Z/OS PDP-11



I have an old machine XPSP3 32bit Home running NTbackup when I plug in an external drive Z: it causes lots of error messages when in the backup log. I a am backing up C: to H: drive. The drives C: and H: are two internal drives. If I remove Z: (a portable USB3 storage device plugged in to USB2 port) before the backup it runs with NO ERRORS. The dozens of errors are like this example:
WARNING: Portions of "\Documents and Settings\All Users\Application Data\AVAST Software\Avast\log\idpagent.log" cannot be read. The backed up data is corrupt or incomplete.
This file will not restore correctly.

(even though this says warning if you can not backup a file it should be an error status ?? )

All drives tested with chkdsk surface tests were fine. Tested after reboot with no major applications running. Tested restoring a few of the files that had the error message they are all fine. NTbackup is using the shadow copy service.

It has been like this for many months while I look for solutions - I only just found it was the presence of portable drive that triggers warnings.

Just curious anyone seen this before?

Any ideas why that may be occurring?
RaceQ is offline  
Sponsored Links
Advertisement
 
Old 12-10-2018, 02:48 AM   #2
Moderator TSF
Hardware Team Moderator
 
Stancestans's Avatar
 
Join Date: Apr 2009
Posts: 5,137
OS: Windows 10



Quote:
Originally Posted by RaceQ View Post
I have an old machine XPSP3 32bit Home running NTbackup when I plug in an external drive Z: it causes lots of error messages when in the backup log. I a am backing up C: to H: drive. The drives C: and H: are two internal drives. If I remove Z: (a portable USB3 storage device plugged in to USB2 port) before the backup it runs with NO ERRORS. The dozens of errors are like this example:
WARNING: Portions of "\Documents and Settings\All Users\Application Data\AVAST Software\Avast\log\idpagent.log" cannot be read. The backed up data is corrupt or incomplete.
This file will not restore correctly.

(even though this says warning if you can not backup a file it should be an error status ?? )
That is a question for the developers of NTbackup. FWIW, I too would log that as a warning, not an error, because it doesn't affect the backup process as a whole.


Quote:
All drives tested with chkdsk surface tests were fine. Tested after reboot with no major applications running. Tested restoring a few of the files that had the error message they are all fine. NTbackup is using the shadow copy service.

It has been like this for many months while I look for solutions - I only just found it was the presence of portable drive that triggers warnings.

Just curious anyone seen this before?

Any ideas why that may be occurring?
Sounds like NTbackup is attempting to read the contents of the log file. I find that suspicious because the backup program should not be doing that. It should only concern itself with copying the file, not accessing its contents. The log file is encrypted and its contents are accessible to Avast developers only. I don't see what the problem is because the file gets backed up and restores just fine despite the warnings (complaints from the backup program). Those files are most likely getting locked by Avast's real-time defence processes (background scans) that get proactive when the ext drive is plugged in.
Stancestans is offline  
Old 12-10-2018, 09:16 AM   #3
Moderator
Windows Tech Team
Hardware Tech Team
 
Join Date: Aug 2008
Location: INDIA
Posts: 2,976
OS: Windows 10 | CentOS | Manjaro



Any chance the external drive has it's own "Automated' back schedule, causing the files to get locked causing this error ?
__________________


tristar is offline  
Sponsored Links
Advertisement
 
Old 12-13-2018, 03:33 AM   #4
Registered Member
 
RaceQ's Avatar
 
Join Date: Aug 2017
Location: Perth, Western Australia
Posts: 43
OS: XP[HP]32SP[23] AOS 5.0.1 Past: W2KSSP4 W2KW W98SE Slackware Ubuntu SUSE DOS UNIX Z/OS PDP-11



Quote:
Originally Posted by Stancestans View Post
...Those files are most likely getting locked by Avast's real-time defence processes (background scans) that get proactive when the ext drive is plugged in.
One of the first tests I did months ago was testing if Avast was the issue by disabling it while doing a backup. 3 years ago I had tracked a problem to Avast with dot net assemblies. I found dotnet runtime generates a random named DLL for the assembly and compiles the assembly to the temp dir. Avast was scanning them and causing the app to fail to load the DLL, so it could be some low level hook into the OS. The only problem with this is I tested with Avast disabled.

Avast has a system tray option to disable shield control till system is rebooted. I had done past tests with the "shield control" disabled and it did not fix the issue. I only have two components of Avast normally active "File shield" and "behaviour shield". I am a software developer and even I find Avast shield settings are complex. As far as I can I know, I did testing with Avast off. I will try again to double check. A more dramatic test is uninstalling Avast which would be a PIA to setup again.

You said "Avast's real-time defence processes (background scans) that get proactive when the ext drive is plugged" - are you saying Avast behaves differently when external drive storage is plugged in? It is an interesting idea - is that a theory

I went back and reviewed the warning message from 46 logs over 10. I excluded those files in subfolders for Avast, Windows, Zonealarm and any that could plausibly be open by background processes. This does account for a lot of the warning messages. That still left files which I know were not open that still got warnings, so this subset is files ending with:
no extension
mbs - mail for opera (opera was not open)
csv - data file in doc folder
js, rdf, manifest - in a copy of Firefox profile folder from another PC
zip, jpg, png, exe, xml, txt, htm - in assorted folders

Not every file on disk of this type gets a warning. For example, I have many csv files and only one got a warning. Some were consistently reported every ntbackup run and other sometimes and new additions came and went.
RaceQ is offline  
Old 12-13-2018, 03:53 AM   #5
Registered Member
 
RaceQ's Avatar
 
Join Date: Aug 2017
Location: Perth, Western Australia
Posts: 43
OS: XP[HP]32SP[23] AOS 5.0.1 Past: W2KSSP4 W2KW W98SE Slackware Ubuntu SUSE DOS UNIX Z/OS PDP-11



Quote:
Originally Posted by tristar View Post
Any chance the external drive has it's own "Automated' back schedule, causing the files to get locked causing this error ?
It is a Western Digital disk and WDsmartware v1.2.08 (WDSW) is installed but the but the service processes "WDsmartwareBackgroundService.exe" / "WDDMService.exe" are disabled. I have never set anything up in that app and do not use it - could have been used under a different login. The only thing on Z: is my stuff - mainly backups. WDSW came with an another external drive which is no longer operational.
RaceQ is offline  
Old 12-13-2018, 06:02 AM   #6
Registered Member
 
RaceQ's Avatar
 
Join Date: Aug 2017
Location: Perth, Western Australia
Posts: 43
OS: XP[HP]32SP[23] AOS 5.0.1 Past: W2KSSP4 W2KW W98SE Slackware Ubuntu SUSE DOS UNIX Z/OS PDP-11



Quote:
Originally Posted by RaceQ View Post
As far as I know, I did testing with Avast off. ...

... 46 logs over 10 months. Although problem existed longer I only started keep logs last 10 months looking for some pattern. ...
2 minor edits to previous post.
RaceQ is offline  
Old 12-13-2018, 08:33 AM   #7
Moderator
Windows Tech Team
Hardware Tech Team
 
Join Date: Aug 2008
Location: INDIA
Posts: 2,976
OS: Windows 10 | CentOS | Manjaro



Why not exclude the Avast App data location from the Backup schedule ?
__________________


tristar is offline  
Old 12-14-2018, 12:59 AM   #8
Moderator TSF
Hardware Team Moderator
 
Stancestans's Avatar
 
Join Date: Apr 2009
Posts: 5,137
OS: Windows 10



Quote:
Originally Posted by RaceQ View Post
One of the first tests I did months ago was testing if Avast was the issue by disabling it while doing a backup. 3 years ago I had tracked a problem to Avast with dot net assemblies. I found dotnet runtime generates a random named DLL for the assembly and compiles the assembly to the temp dir. Avast was scanning them and causing the app to fail to load the DLL, so it could be some low level hook into the OS. The only problem with this is I tested with Avast disabled.

Avast has a system tray option to disable shield control till system is rebooted. I had done past tests with the "shield control" disabled and it did not fix the issue. I only have two components of Avast normally active "File shield" and "behaviour shield". I am a software developer and even I find Avast shield settings are complex. As far as I can I know, I did testing with Avast off. I will try again to double check. A more dramatic test is uninstalling Avast which would be a PIA to setup again.
Avast has provisions for exporting and importing its settings. That should help to quickly and easily restore it back to how you had it configured before uninstalling it. Sometimes simply disabling/pausing the real-time shields doesn't cut it. There are other components such as filesystem filters and/or drivers that remain active, including service(s) that are not halted simply by disabling shields.

Quote:
You said "Avast's real-time defence processes (background scans) that get proactive when the ext drive is plugged" - are you saying Avast behaves differently when external drive storage is plugged in? It is an interesting idea - is that a theory
Avast DOESN'T behave differently when a (removable) storage device is plugged in. It does what it's designed to do, i.e real-time monitoring, just like any other reputable AV does. When a storage device becomes present, a chain of events take place, starting with the mounting of any volumes that it contains. Several processes and activities are triggered. This is not a theory, it is a fact, one that you can quickly establish using Process Monitor. As long as a process is doing something with the disk, the AV's real-time defences will also be monitoring those processes, and as long as the AV is doing something, its protected data files (such as that encrypted log file that NTBackup is trying to access) will also be in use.

Quote:
I went back and reviewed the warning message from 46 logs over 10. I excluded those files in subfolders for Avast, Windows, Zonealarm and any that could plausibly be open by background processes. This does account for a lot of the warning messages. That still left files which I know were not open that still got warnings, so this subset is files ending with:
no extension
mbs - mail for opera (opera was not open)
csv - data file in doc folder
js, rdf, manifest - in a copy of Firefox profile folder from another PC
zip, jpg, png, exe, xml, txt, htm - in assorted folders

Not every file on disk of this type gets a warning. For example, I have many csv files and only one got a warning. Some were consistently reported every ntbackup run and other sometimes and new additions came and went.
Are ALL warnings about NTBackup being unable to access portions of said files? If you can't ignore those warnings, then investigate each of the affected files and the disks involved.
Stancestans is offline  
Old 12-15-2018, 06:23 PM   #9
Registered Member
 
RaceQ's Avatar
 
Join Date: Aug 2017
Location: Perth, Western Australia
Posts: 43
OS: XP[HP]32SP[23] AOS 5.0.1 Past: W2KSSP4 W2KW W98SE Slackware Ubuntu SUSE DOS UNIX Z/OS PDP-11



Quote:
Originally Posted by Stancestans View Post
...

Avast DOESN'T behave differently when a (removable) storage device is plugged in ...


Are ALL warnings about NTBackup being unable to access portions of said files? If you can't ignore those warnings, then investigate each of the affected files and the disks involved.
Yes agree pro-active scanning is running all the time by AV software.

Just checking you understand Drive Z: (removable storage) has been plugged in for days. There is no writing or access to disk Z: as backup is writing to H: (non-removable storage) I still get the problem. Unplug Drive Z: and no problem with backup.

Just to clarify all logs I am taking about refer to NTbackup output logs.
RaceQ is offline  
Old 12-15-2018, 06:54 PM   #10
Registered Member
 
RaceQ's Avatar
 
Join Date: Aug 2017
Location: Perth, Western Australia
Posts: 43
OS: XP[HP]32SP[23] AOS 5.0.1 Past: W2KSSP4 W2KW W98SE Slackware Ubuntu SUSE DOS UNIX Z/OS PDP-11



Quote:
Originally Posted by tristar View Post
Why not exclude the Avast App data location from the Backup schedule ?
Yes that would solve one occurrence but there are dozens of reports over many directories sometimes changing from run to run. Certainly I do want all my document files on backup so I would not exclude them. For example 1710 items were reported from last backup had problem and I would not want excluded at 100+ files. Going back a few months 1126 file reported and I would not want excluded at least 100+ files. No pattern explains all files chosen.
RaceQ is offline  
Old 12-15-2018, 07:09 PM   #11
Moderator TSF
Hardware Team Moderator
 
Stancestans's Avatar
 
Join Date: Apr 2009
Posts: 5,137
OS: Windows 10



You may want to use a more reliable backup solution than the obsolete NTbackup, which is a question I intended to ask from the beginning. Macrium Reflect 7 Free supports XP if I'm not mistaken. If that doesn't work, try version 6. If you decide to stick to deprecated solutions, I'm afraid that's where I wash my hands and desert this topic.
Stancestans is offline  
Old 12-15-2018, 07:13 PM   #12
Registered Member
 
RaceQ's Avatar
 
Join Date: Aug 2017
Location: Perth, Western Australia
Posts: 43
OS: XP[HP]32SP[23] AOS 5.0.1 Past: W2KSSP4 W2KW W98SE Slackware Ubuntu SUSE DOS UNIX Z/OS PDP-11



Quote:
Originally Posted by tristar View Post
Any chance the external drive has it's own "Automated' back schedule, causing the files to get locked causing this error ?
Well talking about other disk related software I do have Diskeeper
installed. The service task has been disabled for a few years - I do not use it. I did test it Today for some testing purposes, then deactivated it again. I see nothing indicating it is actively running something low level - it certainly would if I start the background service.
RaceQ is offline  
Old 12-16-2018, 04:57 PM   #13
Registered Member
 
RaceQ's Avatar
 
Join Date: Aug 2017
Location: Perth, Western Australia
Posts: 43
OS: XP[HP]32SP[23] AOS 5.0.1 Past: W2KSSP4 W2KW W98SE Slackware Ubuntu SUSE DOS UNIX Z/OS PDP-11



Quote:
Originally Posted by Stancestans View Post
You may want to use a more reliable backup solution than the obsolete NTbackup, which is a question I intended to ask from the beginning. Macrium Reflect 7 Free supports XP if I'm not mistaken. If that doesn't work, try version 6. If you decide to stick to deprecated solutions, I'm afraid that's where I wash my hands and desert this topic.
Yes there are alternatives. In my first post I explained if I stop Z: my backups run fine so that is the workaround. I have already automated stopping the drive and restarting after backup completes.

I am interested in an explanation of why the presence of drive Z: interferes with the ntbackup even though the I/O.

I started looking at their KB docs and found something curious see my post
RaceQ is offline  
Old 12-16-2018, 06:27 PM   #14
Registered Member
 
RaceQ's Avatar
 
Join Date: Aug 2017
Location: Perth, Western Australia
Posts: 43
OS: XP[HP]32SP[23] AOS 5.0.1 Past: W2KSSP4 W2KW W98SE Slackware Ubuntu SUSE DOS UNIX Z/OS PDP-11



I found this note about VSS (used by another backup product) https://knowledgebase.macrium.com/di...rd+sector+size

Quote:
The windows snapshot component VSS will fail if a disk with a non standard sector size is attached to your system. This error will occur regardless of whether the disk is a backup source or target. This can be an internal disk or one connected by USB or eSata. The issue is actually caused by USB enclosures that modify the disk drive geometry in the USB to SATA bridge.
I knew about the sector size issue already in the context of source or target in backup run, which is why I do not write backups to Z: directly. Drive Z: is 3TB, 4K sectors.

"The windows snapshot component VSS will fail if a disk with a non standard sector size is attached to your system" well that explains what I am seeing. Previous MS articles where unclear and always in reference to source or target during backup being a known problem. I have not seen an MS article explicitly saying presence was a problem. Well done Macrium support site and thanks to @Stancestans; for leading me to Macrium site.

Even though 4K physical sectors are not supported in context of VSS. I find the drive Z: works fine in terms of read/write files to Drive Z:. NTFS can have different cluster sizes even back in XP. Currently drive Z: is 93% full there are no problems with everyday read/write storage use on drive Z:.

"USB enclosures that modify the disk drive geometry in the USB to SATA bridge" - I don't understand why this breaks you read/writing ANY drive via VSS. Logically it should only impact 4K sector drive, but it matches what I see: insert drive = stops VSS read/writing everywhere in which case it breaks ALL backup software that depends on VSS, so that would be most of them including Macrium. The VSS error return codes do not include this in their problem determination and it still applies up to Win 7. I would have to get Win 8 or 10 for this to be a non issue.
RaceQ is offline  
Old 12-16-2018, 10:10 PM   #15
Registered Member
 
RaceQ's Avatar
 
Join Date: Aug 2017
Location: Perth, Western Australia
Posts: 43
OS: XP[HP]32SP[23] AOS 5.0.1 Past: W2KSSP4 W2KW W98SE Slackware Ubuntu SUSE DOS UNIX Z/OS PDP-11



Did some testing on 5 jpg files that had got "backed up data is corrupt or incomplete." error, when I restored the end of each file is corrupt and padded with the repeating 20 byte string "This is padding!! \x0D\x0A" 958, 569, 1387, 830, and 434 times respectively. Some old posts found via archive.org mention the issue occurs when reading bad source data (but my original files are ok). Using the ntbackup GUI the files are clearly marked with "!" icon in selection list indicating file is not a complete backup copy. It is reported in ntbackup log as "... was restored from an incomplete or corrupt backup".

I am mentioning this in case anyone every comes across files restored from ntbackup and finds text at the end of the file (padding added specifically by Microsoft ntbackup).

System restore uses VSS to make recovery points. I wonder it they have corrupt files in "System Volume Information" folder. This one is a little harder to test on XP Home.

As far as I can tell Windows defragmenter and diskeeper software do not use VSS.
RaceQ is offline  
Old 12-21-2018, 10:42 PM   #16
Registered Member
 
RaceQ's Avatar
 
Join Date: Aug 2017
Location: Perth, Western Australia
Posts: 43
OS: XP[HP]32SP[23] AOS 5.0.1 Past: W2KSSP4 W2KW W98SE Slackware Ubuntu SUSE DOS UNIX Z/OS PDP-11



Quote:
Originally Posted by RaceQ View Post
Some old posts found via archive.org
vbulletin breaks links manually change https to http after ...web/... part of the link
RaceQ is offline  
Closed Thread

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is on
Smilies are on
[IMG] code is on
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off


Post a Question


» Site Navigation
 > FAQ
  > 10.0.0.2
Powered by vBadvanced CMPS v3.2.3


All times are GMT -7. The time now is 07:58 AM.


Powered by vBulletin® Version 3.8.8
Copyright ©2000 - 2020, vBulletin Solutions, Inc.
vBulletin Security provided by vBSecurity v2.2.2 (Pro) - vBulletin Mods & Addons Copyright © 2020 DragonByte Technologies Ltd.
User Alert System provided by Advanced User Tagging v3.1.0 (Pro) - vBulletin Mods & Addons Copyright © 2020 DragonByte Technologies Ltd.
Copyright 2001 - 2018, Tech Support Forum

Windows 10 - Windows 7 - Windows XP - Windows Vista - Trojan Removal - Spyware Removal - Virus Removal - Networking - Security - Top Web Hosts