Secure Network Management

Creating a Workspace

The workspace is an area that will help keep your reconnaissance organized.  Each workspace has it’s own directory inside the hidden .recon-ng directory in the home directory.

First we will find an organization to recon and build our workspace around this company.  We will use HackerOne to get our company.

This is how Wikipedia describes HackerOne:

“HackerOne is a vulnerability coordination and bug bounty platform that connects businesses with cybersecurity researchers (aka, hackers). It is one of the first companies to embrace and utilize crowd-sourced security and hackers as linchpins of its business model, and is the largest cybersecurity firm of its kind.^[1]”

Even though we are only performing reconnaissance in a non-intrusive manner, we will use a company from HackerOne’s Directory.  Under the right conditions, this company has agreed to recon and scanning.  We will only be using recon-ng. Figure 1 shows the company we will use in the tutorial but feel free to select a different company from HackerOne or use any one that you are authorized to test against.

Figure 1: HackerOne Company

Figures 2 and 3 show the scope that is authorized for testing including eligible submissions and domains.

Figure 2: Eligible Items

Figure 3: Allowed Domains

workspaces -h shows us the different option we have a available to use (Figure 4).

Figure 4: Workspace -h

Next we will add our workspace using the following command (Figure 5)

workspaces add

Figure 5: Adding a Workspace

After this command you are automatically placed into your new workspace. workspaces list will show you the status of your workspaces.

Figure 6: List of workspaces

Next, we will add our company and our domain.  This will add information to the SQLite database. To add information into the database, we need to understand the schema, the layout of the tables. To look at the schema of the database run the following command (Figure 7)

show schema

Figure 7: Show Schema

There are thirteen different tables, we will view the schema of the tables we use in this tutorial.

 add companies

Running the add companies command will make the other columns available.   Press enter if you want to leave that column blank.

Figure 8: Add Company

Add the domain using the following command (Figure 9).

add domains

Figure 9: Add Domains

To verify that the domain was added successfully run the command shown in Figure 10.

show domains

Figure 10: List Domains

A simple way of thinking about adding to the tables is shown in the next Figure 11.

Figure 11: Table Visualization

Now that we’ve added data to the database and know how to ensure that data was manually inserted correctly lets move on to importing and exporting data.

Importing Data into the Database

We will uses theHarvester to gather information about United.com and import this into recon-ng’s database.

From Edge Security  http://www.edge-security.com/theharvester.php

“The objective of this program is to gather emails, subdomains, hosts, employee names, open ports and banners from different public sources like search engines, PGP key servers and SHODAN computer database.

This tool is intended to help Penetration Testers in the early stages of the penetration test in order to understand the customer footprint on the Internet. It is also useful for anyone that wants to know what an attacker can see about their organization.”

If theHarvester isn’t already installed, i.e. you aren’t using Kali Linux, you can clone it from here: https://github.com/laramies/theHarvester

Figure 12: theHarvester

We called theHarvester to gather data on domain united.com using all the data sources listed in the help screen.  We directed the output to my recon-ng folder using the ‘>’ operator. The sample command we used follows:

./theHarvester.py -d united.com -b all > ~/recon-ng/harvester.txt

The file name is harvester.txt. This is an ugly file that well will parse through using a few linux utilities. Sample results are shown in the next figure.

Figure 13: Sample Results

The next step is to make this snippet and clean it up a bit with some Linux utilities. We will use grep and AWK to trim the tree.

Grep and AWK

grep is a command-line utility for searching plain-text data sets for lines matching a regular expression.

This is by no means the perfect way.  This is just one of many to get the results you need. Using grep, we will create a list of email addresses from harvester.txt file. (Figure 14)

grep @united.com harvester.txt > united_emails.txt

Figure 14: grep command

If you are interested in the file contents use the cat command to view file the contents in the terminal

cat united_emails.txt

Figure 15: cat results

Next, we will create a list of hosts for import from theHarvester results. (Figure 16)

grep ":" harvester.txt

Figure 16: grep host

Grep will also help create the virtual host list.  Also take note that since “united.com” is the only domain in scope, it becomes part of the command.

grep ":" harvester.txt | grep united.com

The pattern that we wanted to match was “=” and I didn’t want to count the lines after the pattern so I chose to use 200 as my line count after the pattern, as shown in Figure 17.

Figure 17 grep for Virtual Hosts

This command was a little harder to figure out. The pattern that we wanted to match was “=” and I didn’t want to count the lines after the pattern so I chose to use 200 as my line count after the pattern.

grep -A200 "=" harvester.txt | grep united.com > virtual_hosts.txt

It is time to import our information into recon-ng.

Using the show modules command, we get a list of modules broken down by categories. We will use import/list module from the Import category.

show modules

Figure 18: Import Modules

The “show info” command shows the options to use and the table and columns that will be needed for the import.

show info

Figure 19: Show Info

To find the column and table, we will use the “show schema” command.  This will give use a list of the Tables and the different columns in each.

show schema

Figure 20: Show Schema

To import email addresses, we will the  “contacts” table and the email column. Our file name will be the united_email.txt file we created using theHarvester. The “set” statement, sets the variables for the import. The “run” command executes the module.

set TABLE contacts
set COLUMN email
set FILENAME united_emails.txt
run

Figure 21: Email Import

The “show contacts” command show the data inside the “Contacts Table”. This is a second verification that the data imported correctly.

Figure 22: Show Contacts

Part 3: Usage and Reporting