With regard to the PCMag review:
Neil is known to use commercial keyloggers in his detection testing. I know this because I read a recent review of Ad-Aware 2007 in the most recent issue of PCMag (Aug 24 2007 or something) and they said that Ad-Aware 2007 was also poor in detected the commercial keyloggers used for testing. With that said, Spyware Terminator currently does not detect commercial keyloggers as spyware, since they are legitimate software titles which are used by companies to monitor employees' activities or by parents intending to monitor their childrens' activities. So poor performance in Neil's keylogger test was not surprising.
With regard to the poor performance in the spyware test:
I was viewing a thread at SixFiles.com about to Spyware Terminator and I saw a user (who certainly appears to be positive towards Spyware Terminator) post email correspondence with Neil. Here is the quote from the thread:
Quote:
*********************************************************************************
Hello Neil,
Your review of Spyware Terminator (ST) has raised a number of questions among users of the software (including myself). While I felt the review went into considerable depth it actually would have been better if you had provided even more detail. For instance, how about a detailed list of all the malware you used during the detection tests? It would also be good to show specifically which malware were detected and which were not for each application. I understand that would be too much information to put into a pcmag.com review, but you could provide this information in a larger online review posted to a different site, or a newsgroup (such as alt.comp.freeware where there was a heated discussion about the review).
There is also a question about the difference between detecting malware that is already on the computer Vs blocking/preventing it from getting into the system to begin with. Folks who represent Crawler say that ST is much better at blocking malware with it's real time shield (RTS) then it is detecting it after the fact. It seems valid to me that the RTS of an application should be evaluated separately since it is providing a distinctly separate function.
Regarding the HIPS functionality of ST, you implied that it was less then the real deal, but it would have been more helpful if you had compared it to another HIPS application that you feel does the job properly so the reader could understand why HIPS as implemented in ST is deficient.
Last, you mentioned that ST failed to block Trojans and root kits. Some people are of the opinion that antivirus software should bear the responsibility for dealing with this kind of malware. Wouldn't it be fair to ask Crawler if ST is designed to detect Trojans and root kits? Do you personally feel that AS should be capable of blocking Trojans and root kits? On average do most AS applications have that capability?
Thanks for taking some time to read this. I would be very interested to hear what you have to say.
(the author's response follows)
**********************************************************************************
I can't release the sample names because I'm not able to change sample sets quickly enough. Picture if the list were widely known - then product X would work double-time to make sure they get *those*.
If you will read the review you will see that it clearly distinguishes the separate tests for removal and for blocking.
Host-based Intrusion Prevention System defines software that detects and prevents intrusions, typically attacks that exploit vulnerabilities in the operating system or in third-party software. Merely blocking every unknown program is not remotely the same thing.
A virus is a malicious program that "reproduces" itself. Trojans and rootkits are malicious programs that do not. One could easily argue that an antivirus is not responsible for protecting against Trojans and rootkits since by definition they are not viruses. In any case, all of the significant antispyware programs *do* attempt to protect against Trojans and rootkits - if one product says "Oh, that's not my area", customers will not be impressed.
-njr
|
His reply is not a very pleasing one. While he may be a malware collector, testing products against more recent malware, spyware, adware, etc. there is nothing stopping him from testing products against older samples. Because he is unwilling to list the samples/threats he uses, we will never know how old or new his database is.
I bring this up because Spyware Terminator is only 1.5 years old (v1.0 October 2005). Im sure you, as an advanced computer user, know that samples become very hard to locate once they become ineffective and part of internet history. While these samples are ineffective and will never harm your computer if installed today, there is nothing stopping Neil from testing these samples during his tests with antispyware products.
Is there room to improve with our definitions? Absolutely. We are working hard to improve our database, doing what is necessary to make it one of the best out there.
Testing malware/adware/spyware is hard because there is so much out there, with so many varients, installing files with wacky names and installing ridiculous amounts of registry keys. Test results for programs will vary from test to test, unfortunately.