> ## Documentation Index
> Fetch the complete documentation index at: https://pentest-tools.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# ML Classifier

> Machine-learning detection of soft 404 error pages for cleaner scan results

The ML Classifier uses machine learning to identify pages that return a 200 OK status code but actually represent error or non-existent content (soft 404s). This reduces false positives in scan results.

## What is a soft 404?

A "soft 404" occurs when a web server returns a successful HTTP status code (typically 200) for a page that doesn't actually exist or contains error content. Common examples include:

* Custom "Page not found" pages that return 200 instead of 404
* Error pages styled to match the site design
* Placeholder pages with generic content
* Pages that redirect to a homepage or parent directory

These soft 404 pages can pollute scan results with false positives, making it harder to identify real findings.

## How it works

The ML Classifier analyzes HTTP responses to distinguish between:

* **Legitimate pages**: Real content that should be reported as findings
* **Soft 404 pages**: Error pages disguised as valid responses

The classifier uses multiple detection techniques:

<AccordionGroup>
  <Accordion title="Response comparison">
    Compares page content against known 404 response patterns for the target site.
  </Accordion>

  <Accordion title="Content analysis">
    Analyzes page content for common error indicators and patterns.
  </Accordion>

  <Accordion title="Machine learning classification">
    Uses trained models to classify ambiguous responses.
  </Accordion>
</AccordionGroup>

## Usage in Website Scanner

The ML Classifier is integrated into the [Website Scanner](/tools/website-scanner).

### Enabled tests

The classifier runs automatically when the following tests are enabled in the **Initial Tests** section:

| Test                                  | Generated finding             |
| ------------------------------------- | ----------------------------- |
| **Find admin consoles**               | Administration consoles found |
| **Find sensitive files**              | Sensitive files found         |
| **Find interesting files**            | Interesting files found       |
| **Search for information disclosure** | Server information disclosure |
| **Software identification**           | Server software identified    |

<Note>
  The ML Classifier is enabled by default for these tests.
</Note>

### How it improves results

Without soft 404 detection, these tests might report hundreds of false positives, pages that appear to exist but are actually custom error pages. The ML Classifier filters these out, so you only see legitimate discoveries.

## Usage in URL Fuzzer

The ML Classifier is also integrated into the [URL Fuzzer](/tools/url-fuzzer).

### How it works

When fuzzing for hidden files and directories, the URL Fuzzer sends many requests that will return error pages. The ML Classifier:

1. Analyzes each response
2. Identifies soft 404 patterns
3. Filters out false positives from the results

<Note>
  The URL Fuzzer doesn't generate findings directly, but its results are cleaned by the ML Classifier to show only legitimate discoveries.
</Note>

## AI data handling

* **Proprietary models**: The ML Classifier uses our own self-hosted classification models
* **Secure infrastructure**: Data is processed within our isolated infrastructure
* **No external training**: Your data is not used to train any AI models
* **Predictive, not generative**: These models classify data rather than generate content, eliminating "hallucination" risks

For complete details on how we handle data in our AI features, see our [AI Data Policy](/account-billing/security/where-to-find-policies#ai-data-policy).

## Related topics

* [Website Scanner](/tools/website-scanner)
* [URL Fuzzer](/tools/url-fuzzer)
* [AI Data Policy](/account-billing/security/where-to-find-policies#ai-data-policy)