Netlas vs Shodan: Platforms Comparison

IoT search engines are essential tools for any red team specialist. With several solutions available on the market, each offering its own set of pros and cons. This article will review and compare two of them: Netlas and Shodan.

Bellow you’ll find a technical, in-depth analysis of the capabilities offered by both search engines. It evaluates them across three key parameters: capabilities, volume and relevance of data, and enterprise features. The goal is to assist specialists in making an informed choice while highlighting the strengths and weaknesses of each platform.

Note: This article was firstly published at Netlas.io blog.

Capabilities

Let’s begin with a brief overview of the capabilities of both search engines. It’s important to note that this article will not cover query syntax or other technical details; you can find that information in the documentation for Netlas and Shodan. Instead, we will focus on how the applications differ and the specific purposes for which they are best suited.

Search Engine

The primary function of both of these tools is to search for internet-connected devices. In Netlas, this is accomplished using the Responses Search Tool, while in Shodan, the Shodan Search Engine is employed.

The following images will illustrate these tools in action.

Using these tools, you can analyze the responses from hundreds of millions of servers to requests made by Netlas and Shodan crawlers. This capability allows users to gain insights into the configuration and vulnerabilities of various devices connected to the Internet.

External Attack Surface Management

In addition to searching for IoT elements, each of the search engines provides tools for implementing external attack surface management.

For Netlas, these tools include the Attack Surface Discovery Tool and the Private Scanner. While you can find more detailed information about these tools in our article “Complete Guide on Attack Surface Discovery”, I will provide a brief description here.

The Attack Surface Discovery Tool is designed to simplify the construction of an attack surface. It visually represents all objects and their connections on an intuitive graph, making it accessible for users with minimal knowledge of information security or computer networks. To add new nodes, you simply click on an existing one, which provides a list of possible searches; you can then select one to include related objects in the graph based on your search criteria.

Additionally, nodes can be grouped, allowing users to search for multiple objects of the same type simultaneously. You can also exclude nodes, which means they can be hidden without impacting the overall construction of the graph.

The following image illustrates an example of a graph created using the Attack Surface Discovery Tool.

The Private Scanner enables users to obtain the most up-to-date information on their chosen hosts by leveraging the capabilities of Netlas. Once a scan is initiated, the targets from the specified surface are scanned across more than 1,200 ports, and all collected data is organized into a separate private index. Users can search this index at any time, just as they would in the Public Data.

One of the key advantages of the Private Scanner is the ability to launch a scan directly from the Discovery Tool with a single click. This feature allows users to scan the surface immediately after completing its construction.

The image below showcases the interface of the Private Scanner.

In contrast, Shodan provides tools for attack surface management, including Scanner and Monitor. Shodan’s Scanner allows users to perform on-demand scans of IP addresses or networks, while Shodan Monitor enables continuous monitoring of specified IP ranges or domains, providing real-time alerts for any changes or new vulnerabilities detected.

The Shodan Scanner operates similarly to the Private Scanner in Netlas. Users can enter multiple IP addresses in CIDR format, after which the specified targets will be scanned. Unfortunately, the input is restricted to IPs and ranges, meaning that users cannot scan domain names directly.

Shodan Monitor, on the other hand, is a dashboard that provides an overview of key indicators related to your attack surface, such as open ports, potential vulnerabilities, and other relevant metrics. This tool allows users to keep track of their monitored assets and receive updates on any changes or new threats that may arise.

Similar to Netlas Private Scanner, Shodan Monitor allows users to search their own separate index, which consists of scan responses. This functionality enables users to easily access and analyze the results of their scans, facilitating better management of their attack surface and the identification of potential vulnerabilities.

CLI and API

Both search engines provide the capability to perform searches via the command line or API. To install the corresponding tools, you can use the following commands:

pip install netlas

pip install shodan

After entering the appropriate API keys, you can utilize the tools without needing to open a browser, as demonstrated in the following images.

You can also harness the power of these search engines within your Python scripts. For guidance on how to implement this, be sure to read the relevant documentation for Netlas andShodan.

Other Ways to Use

In addition to the web version and API, both search engines provide various usage methods. While there are many integrations available, two of the most valuable methods worth highlighting are:

1. Browser Plugins: Both engines offer plugins that can be installed in Google Chrome ( Netlas, Shodan) and Mozilla Firefox ( Netlas, Shodan). While their functionality is somewhat limited compared to the full capabilities of the search engines, these plugins allow you to quickly check whether the site you are visiting is safe while browsing.
2. Modules for Maltego: Maltego is a leading investigative tool that integrates data from hundreds of sources, including those discussed in this article. By connecting Shodan or Netlas to Maltego, you gain access to their data in the form of transformations, allowing for in-depth analysis and visualization of relationships and connections within your investigations.
3. Data Files and Streaming API: For users who prioritize complete anonymity, both search engines offer ways to access data in-house. In the case of Netlas, this is achieved through the Datastore and Streaming API, based on Redis Streaming. The Datastore option allows users to purchase datasets — CSV or JSON files containing specific types of data, such as the most popular databases or all scanned WHOIS records. A full list of available datasets is provided on the relevant page. Notably, all datasets are free for subscriptions above the Corporate level. The Streaming API, available as a paid feature for Enterprise subscriptions, allows users to receive live, real-time data for immediate use. Shodan offers similar features, including the Firehose API and Bulk Data Files (BDF). The Firehose API functions similarly to Netlas’ Streaming API, providing live data streams. The BDF feature provides data received daily. These features available to users with an Enterprise subscription.

Overall, you can observe that the capabilities of both search engines are quite similar. Next we will discuss some minor features.

Minor Capabilities

This short section will include features that were difficult to attribute to a specific area. These are small things that make it easier to use or make the user experience more enjoyable.

Netlas Tools

Let’s start with additional Netlas tools. In addition to the Responses Tool, Discovery Tool, and Private Scanner mentioned above, these include the following:

• IP/Domain Info. This tool allows the user to obtain brief information about a domain name or IP address, including open ports, reputation, and basic fields from the WHOIS protocol.

• DNS Search. The main task of this tool is to provide users with information about DNS records. Supports both Forward DNS Search and Reverse DNS.
• IP WHOIS Search. Provides search by WHOIS protocol fields. Netlas collects WHOIS data for every existing IP address, which can be very valuable in investigations or Surface builds.
• Domain WHOIS Search. A tool similar to the previous one. The main difference is that it contains WHOIS protocol data for domain names.
• Certificates Search. Using this tool, the user can search the fields of SSL certificates. Very often, this allows you to detect certain devices and services if they have specific certificates.

Netlas Team Access

Another interesting point in working with Netlas is the use of team access. After joining a team, users can see private scans and ASD graphs that the administrator shared with them. An approximate view of such access will be presented in the following image.

This setup significantly streamlines group work, as any team member can access the shared results. Without it, users would need to send surfaces as documents, which could slow down the workflow considerably.

Shodan Minor Features

Shodan also offers some small but useful features that may surprise you.

One such feature is the additional tools available under its umbrella. These tools are not directly related to the search engine itself but complement its functionality. Here is a short list of main tools:

• InternetDB API. A standalone tool, free for non-commercial use, that provides a brief summary of IP addresses, including open ports, vulnerabilities, and tags. It’s essentially a stripped-down version of the main search engine, offering only basic data without additional details.
• nrich. CLI tool, the functionality of which is similar to the previous tool.
• GeoNet tools. Another useful feature of Shodan allows users to evaluate the performance of networks on servers located in different regions around the world. This helps assess global connectivity and server responsiveness.

In addition, Shodan offers very tiny improvements, such as, for example, changing the search engine theme if you are tired of the standard light and dark ones.

Features Comparison Summary

In this section we have prepared an excellent table, however, unfortunately, Medium does not support this functionality. You can read it in the original article on the Netlas blog.

Technical Differences

Before comparing the volume and relevance of data, it’s important to highlight the different scanning methods used by Netlas and Shodan. In this section, we will examine their distinct approaches to scanning, data labeling, and the number of ports scanned by each search engine.

Scan Differences

Netlas first assumes the presence of a corresponding protocol for the scanned port and then verifies if the service matches the expected protocol. If it doesn’t, the service is recorded as RAW; otherwise, it is recorded as the initially detected protocol.

For Shodan, we were unable to find detailed information on the exact scanning process. Based on our understanding, we can assume that Shodan first detects the service and then scans it, assigning the RAW designation only when no known protocol is detected. Additionally, due to the lack of protocol filters in the search, we couldn’t accurately estimate the percentage of RAW data.

Shodan may show slightly more accurate results when searching by protocols. However, the RAW data in Netlas is minimal — only a fraction of a percent — so this difference is unlikely to have a significant impact except in exceptional cases.

Number of Scanning Ports

Next, it’s important to note that the number of ports scanned by both search engines is limited.

When scanning for Public Data, Netlas scanners examine no more than 146 ports per IP address, consisting of 141 TCP ports and 5 UDP ports. A detailed list can be found in therelevant section of the documentation. While this may appear limited, the issue can be partly resolved by using the Private Scanner, which increases the number of scanned ports to ~1300.

At the same time, Shodan continuously scans 1237 ports, and the full list can be accessed via the corresponding API request. However, it’s important to note that the frequency of port scanning varies. For instance, a commonly used port like 80 might be checked daily, while less popular ports, such as 9203, may only be scanned once a month. More details on this can be found in the next subsection.

Scanning Timing

As mentioned earlier, Shodan does not scan all ports with the same frequency. During our research, we observed cases where some ports on a single IP address were scanned just a day ago, while others were scanned several weeks ago. This is due to Shodan’s crawlers prioritizing more frequently used ports.

In contrast, Netlas crawlers treat all ports equally important. Therefore, the date of scanning an IP address is also the scan date for each of its open ports, regardless of their popularity.

These differences can potentially mislead users when comparing the relevance of data from both systems. If the focus is solely on the date of the last IP address scan, Shodan might appear to have an advantage. However, as demonstrated in the experiment described in the next section, the freshness of the data is generally comparable if more attention is paid to the ports themselves.

Volume and Relevance of Data, Engines Limitations, Enterprise Features

These sections consist almost entirely of tables, so we recommend reading it in the original article.

Summary

In summary, the choice between these search engines largely depends on the user’s goals. If the focus is on researching websites and web services, Netlas is undoubtedly the better option. However, for those interested in more obscure applications, Shodan may provide superior results.

One of the main drawbacks of Netlas is the current absence of a monitoring feature, while Shodan restricts many functions (such as searching by domain names or DNS records) to higher subscription tiers.

Overall, Netlas is likely the best choice for small companies, offering lower costs with a comparable amount of data, making it ideal for startups with limited budgets. However, for large companies, using both tools may be the most effective solution. While it may not be inexpensive, combining Netlas and Shodan ensures access to the most comprehensive information, meeting a wide range of needs.