Website vulnerability scanners benchmark 2024
Discover which website vulnerability scanners deliver the most accurate findings and how you can use these insights for your security workflow.
This benchmark compares the best website scanners to help you understand their detection capabilities and limits and give you objective data for improving your tooling and workflows.
Overview of this website scanners benchmark
Necessary
Why compare the best web app scanners?
The transparent results in this benchmark provide clear data useful for validating anecdotal evidence and vendor-provided information, for lack of standardized testing.
The insights about the actual effectiveness of leading web app scanners from this analysis can help security specialists effectively evaluate and choose the right tools for their needs.
Transparent
What kind of findings does this benchmark include?
The results in this benchmark come from rigorously testing 6 of the most popular website vulnerability scanners against Broken Crystals and DVWA (Damn Vulnerable Web Application).
The key findings and full list of results include the types of vulnerabilities and vulnerable paths detected during testing.
Realistic and relevant
How does this benchmark evaluate the scanners?
The benchmark examines how the vulnerabilities each scanner reported compare to the target’s security posture, revealing their rate of true positive, false positive, and false negative results.
Transparent criteria ensure a fair, standardized comparison across open-source and commercial tools, reflecting what appsec engineers look for in a DAST (Dynamic Application Security Testing) tool.
Methodology for this website scanners benchmark
Testing period: February 2024
Detections: all scanners were updated with the latest detections as of February 2024.
Configuration: each scanner was manually configured for maximum crawl coverage and enabled to use all available vulnerability detections.
6
popular website scanners evaluated
2
independent testbeds - DVWA & Broken Crystals
3
criteria: true positives, false positives, false negatives
107
vulnerable paths tested
Criteria for evaluating website scanners in this benchmark
True positive rate: reported vulnerabilities that actually exist.
The true positive rate was calculated as
=* 100number of detected true positive vulnerabilities
total number of true positive vulnerabilities reported by all scanners
False positive rate: reported vulnerabilities that do not actually exist.
The false positive rate was calculated as
=* 100number of detected vulnerabilities which were false positive
total number of false positive vulnerabilities reported by all scanners
False negative rate: reported vulnerabilities that exist but the scanner did not report.
The false negative rate was calculated as
=* 100number of undetected true positive vulnerabilities
total number of true positive vulnerabilities reported by all scanners
Watch the benchmark breakdown
Watch Stefan Mihalache, Security Research Engineer, break down the results of this benchmark in just 4 minutes, and see how the best website vulnerability scanners did against each of the targets.
Download the benchmark![Comparing the BEST Website vulnerability scanners [2024]](/_vercel/image?url=%2Fimages%2Fstefan-mihalache-benchmark-video.webp&w=1536&q=100)
The results of this website scanners benchmark
Tool rankings
Overall detection
The evaluation of leading web application vulnerability scanners indicates that both commercial tools and the prominent open-source tool ZAP provide generally consistent vulnerability detection, with only minor differences observed.
Detection accuracy
While most web app scanners demonstrated effective accuracy with minimal false positives, ZAP had notable inaccuracies in DVWA scans. The Website Vulnerability Scanner on Pentest-Tools.com, however, consistently reported a lower rate of false positives across both targets.
Real world relevance
With web security vulnerabilities and technology continuously evolving, scanners must constantly refine their detection capabilities, making static benchmarks less relevant over time. This is why it’s crucial to recognize that a scanner's benchmark performance might not translate directly to real-world scenarios.
The most accurate website vulnerability scanners
When looking at their performance across both vulnerable targets and their roster of vulnerabilities, there is a similar level of detection availability among the evaluated tools. When looking at accuracy, though, one scanner in particular stands out through the number of false positives it produced.
The best website scanners against Broken Crystals
Broken Crystals was one of the main targets for testing the most popular website vulnerability scanners because it reflects modern web application environments.
This publicly available target incorporates a wide range of technologies and vulnerabilities, including a REST and a GraphQL API, XSS and SQL injection vulnerabilities, as well as security issues arising from flawed JWT and GraphQL implementations.
These components make Broken Crystals an impartial and relevant testbed that enables this benchmark to transparently and accurately compare each scanner's reported vulnerabilities against the actual security posture of the target.
- In the Broken Crystals assessment, Invicti’s Acunetix secured the leading position by a significant margin. The Pentest-Tools.com Website Vulnerability Scanner achieved second place, in a tie with Burp Suite from Port Swigger. Notably, ZAP exceeded the performance of both Qualys and Rapid7 InsightAppSec, earning a commendable fourth place.
- Regarding false positives, all scanners performed moderately, each producing some, but none standing out negatively. The slight variations among them didn't provide a decisive differentiation, suggesting a generally even match across the scanners.
The best website scanners against DVWA
DVWA (Damn Vulnerable Web Application) is an unmissable testbed for testing website vulnerability scanners against traditional web applications.
This well-known publicly available target helps assess the scanners' detection accuracy, providing a clear benchmark for how effective they are in identifying common vulnerabilities in a widely recognized and relevant testing environment.
- In tests against DVWA, Burp Suite leads the field, identifying 29 out of 39 vulnerabilities. Following closely, the Pentest-Tools.com Website Vulnerability Scanner detected the second highest number of vulnerabilities. Rapid7 InsightAppSec and Invicti’s Acunetix rank 3rd and 4th respectively, with Rapid7 identifying 19 vulnerabilities and Acunetix 18.
- When it comes to false positives, the scan results differ quite a lot from the Broken Crystals tests. ZAP is excluded from this comparison because it incorrectly reported 88 instances of SQL injection false positives, a significantly higher rate than the other scanners.
See the full benchmark results and all the data behind them
Download the benchmark
9 things you can do with this benchmark of website scanners
1. Evaluate web app scanner capabilities
Compare the effectiveness of popular website scanners by assessing their ability to crawl a wide range of application functionalities, identify a diverse roster of vulnerabilities, and accurately report findings. Save time and resources while researching tools to strengthen your attack surface mapping and web app vulnerability detection capabilities.
2. Benchmark internal tools
Compare the effectiveness of your in-house tools against commercial and open-source web app vulnerability scanners regarded as industry standards to highlight areas of strength and opportunities for improvement in your setup.
3. Enhance your CI/CD pipeline
Identify the best web app scanners to integrate into your CI/CD pipeline and detect vulnerabilities early in the development process, ensuring secure code before deployment.
Evaluate which tools help you avoid gaps in detection and which can effectively complement your security toolset and workflow.
4. Select tools for web app penetration testing
Choose website vulnerability scanners that can truly inform and complement manual testing, improving your team’s ability to detect critical security issues and speed up remediation - especially in time-sensitive contexts.
5. Improve stakeholder communication
Use benchmark data to communicate the effectiveness of web application security tools to stakeholders and decision-makers whose budget approval you need to pursue internal projects.
6. Inform negotiations with vendors
Leverage the findings from this website scanners benchmark in negotiations with vendors or use it to provide feedback for product improvements you find necessary for your needs.
7. Optimize resource allocation
Allocate resources more efficiently by understanding which web application scanners provide the best coverage and accuracy for your requirements.
8. Improve support for compliance efforts
Choose web app scanners that align with compliance requirements by evaluating their detection capabilities against industry standards.
9. Community sharing
Share the findings in this benchmark to improve collective knowledge within your team, organization, or community. (Your feedback on ways to improve this benchmark is more than welcome!)
See the full benchmark results and all the data behind them
Download the benchmark
Benchmark FAQs
In the benchmark testbeds, vulnerabilities are defined based on established standards, including OWASP Top 10 and the Common Weakness Enumeration (CWE). In addition, issues identified by scanners that serve as effective defense-in-depth measures, such as anomalies in the Content Security Policy (CSP) header, were also included.
While this approach introduces a degree of subjectivity, it aligns with the practical responsibilities of security engineers who evaluate scanner reports during real-world assessments.