Creating a Workspace
The workspace is an area that will help keep your reconnaissance organized. Each workspace has it’s own directory inside the hidden .recon-ng directory in the home directory.
First we will find an organization to recon and build our workspace around this company. We will use HackerOne to get our company.
This is how Wikipedia describes HackerOne:
“HackerOne is a vulnerability coordination and bug bounty platform that connects businesses with cybersecurity researchers (aka, hackers). It is one of the first companies to embrace and utilize crowd-sourced security and hackers as linchpins of its business model, and is the largest cybersecurity firm of its kind.”
Even though we are only performing reconnaissance in a non-intrusive manner, we will use a company from HackerOne’s Directory. Under the right conditions, this company has agreed to recon and scanning. We will only be using recon-ng. Figure 1 shows the company we will use in the tutorial but feel free to select a different company from HackerOne or use any one that you are authorized to test against.
Figures 2 and 3 show the scope that is authorized for testing including eligible submissions and domains.
workspaces -h shows us the different option we have a available to use (Figure 4).
Next we will add our workspace using the following command (Figure 5)
After this command you are automatically placed into your new workspace. workspaces list will show you the status of your workspaces.
Next, we will add our company and our domain. This will add information to the SQLite database. To add information into the database, we need to understand the schema, the layout of the tables. To look at the schema of the database run the following command (Figure 7)
There are thirteen different tables, we will view the schema of the tables we use in this tutorial.
Running the add companies command will make the other columns available. Press enter if you want to leave that column blank.
Add the domain using the following command (Figure 9).
To verify that the domain was added successfully run the command shown in Figure 10.
A simple way of thinking about adding to the tables is shown in the next Figure 11.
Now that we’ve added data to the database and know how to ensure that data was manually inserted correctly lets move on to importing and exporting data.
Importing Data into the Database
We will uses theHarvester to gather information about United.com and import this into recon-ng’s database.
From Edge Security http://www.edge-security.com/theharvester.php
“The objective of this program is to gather emails, subdomains, hosts, employee names, open ports and banners from different public sources like search engines, PGP key servers and SHODAN computer database.
This tool is intended to help Penetration Testers in the early stages of the penetration test in order to understand the customer footprint on the Internet. It is also useful for anyone that wants to know what an attacker can see about their organization.”
If theHarvester isn’t already installed, i.e. you aren’t using Kali Linux, you can clone it from here: https://github.com/laramies/theHarvester
We called theHarvester to gather data on domain united.com using all the data sources listed in the help screen. We directed the output to my recon-ng folder using the ‘>’ operator. The sample command we used follows:
./theHarvester.py -d united.com -b all > ~/recon-ng/harvester.txt
The file name is harvester.txt. This is an ugly file that well will parse through using a few linux utilities. Sample results are shown in the next figure.
The next step is to make this snippet and clean it up a bit with some Linux utilities. We will use grep and AWK to trim the tree.
Grep and AWK
grep is a command-line utility for searching plain-text data sets for lines matching a regular expression.
This is by no means the perfect way. This is just one of many to get the results you need. Using grep, we will create a list of email addresses from harvester.txt file. (Figure 14)
grep @united.com harvester.txt > united_emails.txt
If you are interested in the file contents use the cat command to view file the contents in the terminal
Next, we will create a list of hosts for import from theHarvester results. (Figure 16)
grep ":" harvester.txt
Grep will also help create the virtual host list. Also take note that since “united.com” is the only domain in scope, it becomes part of the command.
grep ":" harvester.txt | grep united.com
The pattern that we wanted to match was “=” and I didn’t want to count the lines after the pattern so I chose to use 200 as my line count after the pattern, as shown in Figure 17.
This command was a little harder to figure out. The pattern that we wanted to match was “=” and I didn’t want to count the lines after the pattern so I chose to use 200 as my line count after the pattern.
grep -A200 "=" harvester.txt | grep united.com > virtual_hosts.txt
It is time to import our information into recon-ng.
Using the show modules command, we get a list of modules broken down by categories. We will use import/list module from the Import category.
The “show info” command shows the options to use and the table and columns that will be needed for the import.
To find the column and table, we will use the “show schema” command. This will give use a list of the Tables and the different columns in each.
To import email addresses, we will the “contacts” table and the email column. Our file name will be the united_email.txt file we created using theHarvester. The “set” statement, sets the variables for the import. The “run” command executes the module.
set TABLE contacts set COLUMN email set FILENAME united_emails.txt run
The “show contacts” command show the data inside the “Contacts Table”. This is a second verification that the data imported correctly.