This article is a deep dive into how Nmap works, to understand its internal structure, and master its functionality.

Network administrators and penetration testers use port scanning to discover open communication channels on computer systems. For an attacker, this is the first step to get info about the target’s network and identify a potential way in, since services running on an open port could be vulnerable to attacks.

Multiple tools can produce good results, but some port scanners are better for a particular task than others. Our focus is on Nmap (Network Mapper), by far the most popular tool for network discovery and port scanning. Some of its features include Host Discovery, Port Scan, Service and OS fingerprinting, and Basic Vulnerability detection. There is also a graphical version known as Zenmap, which offers easy access to scanning options and network mapping diagrams.

In this article, we will describe how Nmap can help you to:

Discover live hosts on a network
Scan for open ports
Discover services
Test for vulnerabilities

Port scanning alternatives

Nmap is not the only port scanner available, and other tools in this category are suitable for particular needs. Some of the more popular are:

Unicornscan is useful for collecting network and OS information, and it comes with features like asynchronous TCP and UDP scanning, port scanning, and service and OS fingerprinting.
Angry IP Scanner is a GUI-based tool for high-speed scanning, allowing users to run ping sweeps of the network. Based on the live IPs detected, it can scan for ports and services, reveal MAC addresses, as well as resolve hostnames.
Masscan is widely known as the fastest port scanner. It has both a command line and a graphical interface, and the default transmission rate is 100 packets per second.
Onetwopunch is a powerful script that combines the features of unicornscan and Nmap tools for faster and more accurate results. It uses unicornscan to scan all 65535 ports, and then feeds the results to Nmap for service fingerprinting. This way, the user gets a complete list of open ports and the services running on them.

1. Nmap host discovery

By default, Nmap uses requests to identify a live IP. In the older version of the tool, the option for ping sweep was -sP; in the newer version, it is -sn. To discover available hosts, the following packets are sent (as seen in the below screen capture below from Wireshark packet analyzer):

ICMP echo request
A TCP SYN packet to port 443
A TCP ACK packet to port 80
An ICMP timestamp request

packets sent from from the Wireshark packet analyzer

If the target is unknown and large, the recommendation is to identify hosts first. Scanning the ports at this stage would generate too much traffic, take time and resources, and is likely to trigger security alerts.

Below are some methods to identify live IPs:

ARP scanning can be used to stealthily discover the hosts in the local LAN. Getting an ARP reply means that the hosts exist and since this ARP is needed for routing packets, a firewall won’t interfere in the exchange.

nmap -n -sn -PR --send-eth 192.168.100.1-20

command prompt interface

ARP scanning

Above, you can see an ARP request and reply captured by Wireshark.

ICMP scan can also identify live hosts by sending an ICMP Echo request. A live host will send back a reply, signalling its presence on the network.

nmap -sP -PE 192.168.100.1/24

Using the -PP option, Nmap will send ICMP timestamp requests (type 13), expecting ICMP timestamp replies (type 14) in return. If a type 14 ICMP packet is received, then Nmap assumes the host is alive.

nmap -sP -PP 172.26.1.4

The -PM option sends ICMP address mask (netmask) requests (type 17), expecting an ICMP address mask reply (type 18) in return. Once again, if a type 18 packet is received, the host is alive.

nmap -sP -PM 172.26.1.4

Keep in mind that ICMP messages may be blocked by some firewalls, so this technique may not always work.

TCP scans represent another way to discover hosts, using commands to send out TCP SYN or TCP ACK ping messages:

With a TCP SYN scan, Nmap sends an SYN packet to a given port on the target. If the machine replies with an SYN/ACK or RST packet for the specified port, Nmap knows the host is up. Lack of a response for a certain period leads to marking the host as down.

nmap -sP -PS 21 IP

During a TCP ACK scan, Nmap sends an empty TCP packet with the ACK flag set to port 80. If the host is up, it will answer with an RST packet since the connection doesn’t exist. If the host is down, there will be no response. The port can be defined by the user.

nmap -sP -PA IP

If a list of live IP addresses already exists, host discovery is not necessary and you can move to the next step, finding open ports.

nmap -Pn IP

2. Scan for open ports

Nmap identifies the status of ports based on the response it receives for an SYN request.

Open Port: Nmap receives “syn-ack” as the probe response
Closed Port: Nmap receives an “RST” as the probe response
Filtered: Nmap marks the port as open | filtered when it does not receive any response, which could be due to firewall filtering

scanning for open ports 2 interfaces captured

There are multiple techniques you can use for port scanning:

Stealth Scan, also known as SYN scan or half-open scan, is the default and most popular technique. Its stealth comes from not performing a 3-way handshake to complete the connection and the packet exchange is as follows:

The scanner sends an SYN packet.
If the port is open, the machine replies with SYN/ACK;
If the port is closed the machine sends RST;
If no response is received after several retries, the port is marked as filtered.
Once the scanner receives SYN/ACK from the machine, it sends the RST packet and marks it as an open port.

nmap -sT IP

stealth scan with nmap

The images below show the packet exchange during the scanning procedure, as captured by Wireshark:

Packet 526 sends an SYN packet from the source IP to 192.168.100.19 on TCP port 135
Packet 545 sends an SYN-ACK packet from IP 192.168.100.19 on TCP port 135
Packet 546 sends an RST packet from the source IP to 192.168.100.19 on TCP port 135 to close the connection. The same goes for port 445 and port 80

the packet exchange during the scanning process

TCP Connect scan completes the 3-way handshake with the target machine and makes for a good alternative to the stealth scan. The process is as follows:

The scanner sends an SYN packet.
If the port is open, the machine will send SYN/ACK;
If the port is closed, the machine will send RST;
If no response is received after several retries, the port is marked as filtered.
Once the scanner receives SYN/ACK, it sends the ACK packet to complete the connection.

nmap -sT IP

nmap connection started

For a peek behind the scenes, we have captured the traffic to better understand the packet exchange process. You can see that the connection starts with an SYN packet visible inline 121 and the handshake is complete when the ACK packet is delivered.

nmap connection starts with an SYN packet visible inline 121

Unlike the SYN scan, the results from TCP connections are slow and the completion of the connection may create a log entry that could reveal the intrusion attempt; it works when the source IP is whitelisted by firewalls, IDS or IPS security gear.

UDP Scans are slower than the TCP port scan and, because of this, are often ignored by security auditors.

Nmap runs the check by sending a UDP packet to the ports. For most of them, the packet is empty and for the common ports, the packet contains the protocol-related payload.

Getting an “ICMP port unreachable error (type 3, code 3)” message means that the port is closed, lack of response signifies that the port is open or filtered, which makes it slow and inaccurate; if the response contains any data, it means that the port is open.

nmap -sU IP

nmap connecting to server

In the traffic capture below, packets 78349 and 78350 contain the UDP response for the probe performed on port 2049. For many of the closed ports, the response is shown as “port unreachable.”

the traffic capture with packets containing the UDP response

3. Discover services

Nmap can identify services by listening to open communication ports for the welcome banner. Many common services (SSH, Telnet, FTP, SMTP) identify themselves this way.

If a banner is not advertised, Nmap sends a probe and waits for a reply. The data received using the service scan (-sV) command is compared to thousands of signatures Nmap keeps in its database file, specifically for service fingerprinting purposes.

nmap -sV IP

nmap using the service scan

4. Test for vulnerabilities

Nmap can find vulnerabilities in the network through the Nmap Script Engine (NSE) – a flexible feature activated with the -sC option that allows users to write scripts for task automation.

NSE comes with a rich collection of scripts that can help in the network discovery process, with vulnerability exploitation, and backdoor detection. The database is available at “/usr/share/nmap/scripts/” on Linux and “C:\Program Files (x86)\Nmap\scripts” on Windows.

a collection of scripts displayed

nmap -sC IP

Testing for a specific vulnerability on a remote target is possible via the –script command:

nmap --script=<nse script> -p <port> IP

You can use this command to check for anonymous login permission on an FTP server:

nmap --script= ftp-anon.nse -p 21 192.168.226.130

check for anonymous login permission on an FTP server

The cache of NSE scripts offers the possibility to check for specific vulnerabilities that have already been reported. For instance, there is a script that checks for a backdoor in the VSFTPD server:

nmap --script= ftp-anon.nse -p 21 192.168.226.130

script checking for a backdoor in the VSFTPD server

Learn how to use Nmap to discover open communication channels

Nmap is a powerful tool for penetration testers and network administrators alike. Each new release extends its capabilities way beyond the simple port scanner the project started as. It is a mature tool that can also identify critical vulnerabilities and perform some web application-level testing.

See the Nmap official website for detailed information on all the commands and features.

Inside Nmap, the world’s most famous port scanner