So you have been tasked with performing a penetration test of the internet-facing systems of a customer. The security assessment is due soon, and you have to find a reasonable solution to gain information about the customer’s networks and systems as well as targets to attack — all of this in the nick of time.
The first step is always gathering as much useful info as you can, but this process can take up some time. Thankfully there are plenty of solutions if you want to avoid using outdated command-line methods.
For many years now, specialists have turned to OSINT based tools and services, which have proved extremely quick and reliable in penetration testing. In this article, we will give you an overview of what is OSINT and list some of the tools commonly used in penetration testing.
What is Open Source Intelligence?
The term open-source intelligence, or OSINT, was coined in the late 1980s by the US military. They argued that reform of intelligence was needed to cope with the dynamic nature of information, especially at a tactical level on battlefields. The concept of OSINT has traversed into different fields since then and is now commonly used in cybersecurity.
Open-source intelligence is defined as information collected from sources open to the public, mostly through the internet. The term itself does not mean internet information, as info from a public library book can also be viewed as OSINT (a library is a publically available source).
We will look over some of the OSINT tools security experts use daily.
Spyse is a recent development in the field of cybersecurity. This search engine scans the internet every couple of days to collect data using OSINT technology, mixed with custom-made algorithms. They store this data in the Spyse database and make it available instantly for users. This alleviates the need to use command-line methods for information gathering, which can be time-consuming.
Spyse provides information about:
-IPv4 (Open ports, banners, protocols, ISP, etc…)
-Domains and subdomains (the biggest subdomain database on the web)
-Digital certificates information
-Autonomous Systems (Number, IPv4/ IPv6 ranges, WHOIS data…)
Google Dorks have been around for a minute, with specialists using it as far back as 2002. This query-passed, open-source intelligence tool helps users effectively target index or search results.
The flexibility of Dorks makes it one of the most used tools in the field, and the process even has its own nickname — Google Hacking. It uses operators which make the search for information much faster. Here are some operators and indexing options provided by the service:
-Filetype: an operation primarily used to find file types or search for a specific string;
-Intext: an indexing option used for finding text on a particular page;
-Ext: used for searching for a specific extension in a file;
-Inurl: used to find a specific string or word in a URL;
-Intitle: Search for a title for words mentioned in the URL;
Specialists use the Harvester for gathering e-mail accounts, as well as names of subdomains, virtual hosts, open ports and banners, and employee names. All this information is collected from public sources like search engines and PGP key servers.
Another tool popular amongst pentesters is Recon-ng. This is another neat reconnaissance tool with a similar interface to Metasploit. You can run Recon-ng from the command line, which places you into a shell-like environment. Here you can edit options, perform reconnaissance, and output results to various report types. They have an interactive console that is loaded with neat features like command completion and contextual help.
For Linux and Windows users, we recommend SpiderFoot. This is another high-configuration open-source reconnaissance tool developed with Python. Easily integrable, interactive GUI and a powerful command-line interface makes SpiderFoot a go-to tool for pentesters.
The tool smartly queries over 100+ OSINT sources and gathers information on emails, names, IP addresses, domains names, and more. It can also find more extensive information on a single target such as netblocks, emails, web servers, etc. SpiderFoot also understands how data is related to each other, making workflow much easier for pen-testers.
This open-source intelligence tool collects information about geolocation by using social networking platforms and image hosting sites. Pretty creepy, isn’t it? The reports are presented on a map, as shown below, and you can filter data based on location and date. Reports can be downloaded in CSV or KML format for further studying.