How to perform authenticated website scans with Pentest-Tools.com

This article shows how to scan a web application that requires authentication using the Website Vulnerability Scanner tool.

Learn how to configure the authentication options of the scanner in a way that it automatically passes the login page and performs in-depth scanning.

Pro tip: To access the authentication options of the scanners you need to first log in to your Pentest-Tools.com account.

I. Why authenticated scans?

If authentication is not enabled, the scanner covers only a small set of application functionality, the one exposed before the user has to authenticate. However, there may be other uncovered vulnerabilities that you can identify after logging in to the application.

Another good reason for performing authenticated scans is that it mimics the activities of a malicious authenticated user, which has additional attack vectors. Authenticated users have access to more functionality than unauthenticated users.

II. Supported authentication methods

Our Website Vulnerability Scanner supports four methods you can use to perform authenticated scans:

1. Automatic – Form-based authentication

2. Recorded – Recording-based authentication

3. Cookies – Cookie-based authentication

4. Headers – Headers authentication

The following sections of the article show you how to configure the scanner and how to interpret the results when doing authenticated scans.

The Website Vulnerability Scanner provides two types of scans: Light and Full. Only the Full scan supports authentication.

Here is the web interface of the scanner that allows you to configure the authentication options:

To configure the authentication for a Full scan you have to:

1. Insert the URL of the website that you want to scan (the target)

2. Enable the “Authentication” option

3. Choose suitable method

4. Add required login information

5. *Check the authentication process

*You can use the Check authentication functionality or start a scan directly, however, we recommend you first check that the recording is producing a successful login. All 4 methods support this feature.

The process can take a minute to complete, therefore we will ask you to not leave or refresh the page while this is running. You will see a screenshot of the application after logging in, if you believe that it is correct, you can proceed with the scan.

1. Automatic – form-based authentication

The “Automatic” option allows the user to make an authenticated scan by having a valid pair of credentials in the target application. This method by default finds the login form in the URL provided by the user, sends the credentials, and screenshots the response

It now includes a beta feature powered by machine learning, designed to better detect and interact with complex login flows. This feature works as a fallback. If the classic authentication method fails, it will automatically step in to complete the login. This means you can test applications with tricky or custom login flows more effectively.

In the tool page start configuring by choosing the “Automatic” tab.

You will have to provide the following details:

• The login URL of the application(for example [https://pentest-ground.com:81/login]. This is usually different from the target URL and needs to include the login form.

• The correct username and password

Here is a sample configuration of the “Automatic” option:

When using Check Authentication functionality, here is an example of a successful login screenshot, displaying the landing page of the target application.

2. Recording-based authentication

Complex web applications require the option to load dynamic pages and components. For an automated scanner to authenticate on a specific target, it is necessary to fulfil these requirements. The Recorded method uses Recorder Panel in the Chrome Browser to record user events when logging into the account used for scanning. The client has to upload the recording of the authentication process that you get following the steps described next.

To learn more about the steps required to setup this authentication method, read our dedicated support article.

3. Cookie-based authentication

HTTP Cookies are pieces of data that a web browser receives from the server and are usually used to identify the web session of a user (they are also called session cookies). After receiving a session cookie, the browser sends it with each HTTP request that it makes to that server. It is helpful to know that the request is associated with that particular user.

Our Cookie-based authentication option mimics the behaviour of a web browser that already has a session cookie. The user has to insert a valid session cookie in the ‘Cookie header’ field. You can take the session cookie from an already established web session (you need to manually log into your web app and get the cookies from your browser).

The web interface for this type of authentication looks like this:

To get the cookies you need, please follow the steps from here.

4. Headers authentication

HTTP headers enable the communication between a client and the server carrying information about the client browser, the server, the accessed page, and more. Some applications use specific headers to create sessions for users (login). As a result, the headers’ authentication method is required for users who need to scan these types of websites/applications (authentication).

The cookie-based method uses cookies, which are a special type of headers, but not every application uses cookies to authenticate users. There are applications that rely on other special headers as an additional layer of security, such as token-based authentication (JSON web token)

In order to retrieve the headers needed, please follow the same steps described in the section above (cookie-based) and insert them in the format:

Header1: the value of Header1

Header2: Subheader1 = value of Subheader1; Subheader2 = value of Subheader2

More details for the setup and functionality of the Header method can be found in our dedicated support article.

III. How to get the session cookie

First, you have to manually authenticate in the target web application using your web browser. Second, you need to get the session cookie string from the browser.

For example, using Google Chrome, you perform the following actions:

• Enter Developer Tools – by Menu > More tools > Developer Tools (or Ctrl + Shift + I)

• Enter the ‘Network’ Tab

• Click on the ‘Name’ section

• Choose a URL that displays an additional ‘Cookies’ tab.

• Go to the ‘Headers’ Tab (for that URL)

• Scroll to Request Headers and see the Cookie header

• Copy the string from the Cookie header and insert it as in this example below: ‘PHPSESSID=a765feb13b4112f3d12f3dfa12e;aaid=ad4b654ad48f4d545a64d75ea’ (a list with name=value separated by “;”)

Here is the Developer tools interface:

IV. How to analyze the results

Authenticated scanning covers more application functionality and pages than unauthenticated scans.

You’ll know that the authentication was successful if you get an additional “Authentication complete” message in the final scan report. Furthermore, the Spider results should contain more crawled URLs than the unauthenticated scan.

V. Behind the scenes

The main steps phases of a web vulnerability scan are spidering and active scanning. In order to perform these as an authenticated user, the scanner asks for the user’s session cookie. You can use it for every request to uncover more information than an unauthenticated scan.

The Login authentication option performs an additional step in order to obtain the session cookie. First, the scanner detects the login form from the URL provided and tries authenticating with the credentials. Second, the scanner tries to obtain the session cookie from the authenticated user and to use it from this point in every request.

In the Cookie-based authentication, having the session cookie already provided by you, the spidering and active scanning start directly. If the cookie is correct, the scan will result in more URLs discovered and analyzed for vulnerabilities.

Please note that the scanner cannot know if the provided cookie is correct or not. It will use it as is and you decide to validate if the scan has reached the desired parts of the application.

VI. Compatibilities

We currently support the option to perform authentication testing on Single Page Applications (SPAs) as our Website Scanner has been upgraded so it can load, detect, and log in the user given the credentials. This method is available using the Form-based authentication method and the Recording-Based authentication Recording-based authentication.

Recording and form options now support LocalStorage-dependent applications, but only when authentication is done on the same domain, not through a third party.

With the Headers-based authentication implemented, our Website Vulnerability Scanner now provides more accurate assessments and enhanced security for the application based on tokens (JWT).

You can find a step-by-step guide on how to perform an authenticated website scan with JWT  in our Support center.

Test your web application to discover hidden vulns using authenticated scanning

Authenticated scanning provides more coverage within a web application, as it discovers more dynamic URLs. When you perform more in-depth scanning, there is a higher chance to find well-hidden vulnerabilities and render your web applications more secure.