30 Jan. 2015

Stop using Virustotal to measure how AV sucks!

By Zoltan Balazs, @zh4ckJuly 10, 2018|  Endpoint Protection Testing

We recently came across an article which again is a FUD regarding how AV sucks. VirusTotal has been writing  about this years ago. Although we agree that most AV is not as good as it is stated in the marketing materials, but that is not the point.

Let’s look at this particular exploit for a moment. Quoting from the article, “The attack, originating from traffichaus.com, was launched though an iFrame which was not detected by 52 anti-virus products, researchers said.” Uploading a malicious Flash file to Virustotal, looking at the 0/57 detection rate is just lame.

Here are just a few AV components which can block the infection:

  • URL blacklisting (not effective, but still)
  • URL reputation – high false positive rates but can be very effective
  • Analyses of HTML/Javascript code to detect the exploit kit (this looks pretty effective when implemented properly)
  • Analyses of the exploit code itself, whether it is Flash, Javascript, Silverlight, whatever. Exploit kits are very good at obfuscating this layer.
  • Blocking the malware download – URL blacklisting/reputation/static AV signatures/heuristics. These detections can be bypassed, but works most of the time.
  • Blocking the execution of the malware – some AV engines do have real exploit protection, which can detect that an unknown exploit tries to start malware on the machine – and block it. I know it, I have seen it with my very own eyes. Multiple times. I also saw this protection being bypassed. It is not 100% perfect.
  • Block the malware by reputation – this can be very effective, when previously unknown binaries are blocked. A few AV uses this technique.
  • Block malware based on how it interacts with the OS – I don’t consider this as a real protection, as it means the malware already started, did something (malicious), and after some time one of the actions are flagged as malicious by the AV. Although this is somehow late, this can still block the real risk, e.g. banking trojans stealing your money.
  • Scheduled scans: Very late detection, but still, it is better late than never.

Looking at this particular exploit, I can confirm some AV did detect and block this threat on day 0. Others did not. Angler exploit kit is especially dangerous because of in-memory-malware – which renders a lot of AV protection components useless.



And yes, all AV can be bypassed. Imagine an AV which provides 100% proactive protection, out of the box. If something like this could have happened, this will mean all the developers of the AV company should have been fired, as the AV does not need any maintenance/updates at all. Which means most of the update you get with your AV engine is about threats which bypassed the AV yesterday …

Only a real world protection test, testing all AV components can measure how effective today endpoint protections/internet security suites/AV engines really are.

We would like to thank Kafeine for sharing the sample with us.


Using Virustotal to compare AV protection is unprofessional, and lame. Don’t do that. Especially when it comes to exploit kits …

PS: using Virustotal to test Android AV is more realistic than Windows AV, or especially exploits. AV on Android is most of the time just plain static scanner.

Update (2015/01/30): the screenshot has been fixed