Using the Duplicate Bookmark Detection

The EMM OSINT Suite automatically detects duplicate bookmarks. This is a useful feature to:

Combining the search results from multiple search engines
Repeating a search and discover new search results

As an example we search for a prominent person and combine the search results into a single folder.

This tutorial describes an example scenario to demonstrate the feature.

Perform the following tasks:

Opening EMM OSINT Suite
Creating a Case Project
Perfoming an Internet search
Organizing Bookmarks

Performing an Internet Search

To perform a search do the following:

Double Click on Bing in the Search Tool View, the application opens a browser window pointing to http://www.bing.com
Search for Barack Obama:

$images_download\attachments\2588737\bing-search-barack-obama.png$

Select from the main menu Web > Extract Search Result Links to extract the result links of the search and to store them as bookmarks in the project:

$images_download\attachments\2588737\bing-search-extract-urls.png$

The system extracts the first 100 result links (you can change the maximum number of extracted links under Window > Preferences > OSINT > Link Extraction). The links are stored in a newly created folder ("Bing_<Time Stamp>") under the bookmarks folder of the case project.

Each result link is stored in a bookmark file which is a XML file containing the following data: URL, Title, creation time stamp, search query, search engine). If you open the Properties View which is located next to the Console View (under the browser window) and select a bookmark file you can see the data contained in the file.

$images_download\attachments\2588737\bing-search-results.png$

In this example the system has already detected a duplicate bookmark which is marked with a (d) in front of its title.

By default the system shows the title of the bookmark file and not the file name. You can change this behaviour if you click on the small triangle in the upper right corner of the workspace navigator view and select "Customize View". Then change to the Content tab and switch off "Bookmark Title", confirm with "OK".

Now we search again but we use Google as search engine. As the end result we have two sub folders in our bookmarks folder:

$images_download\attachments\2588737\bing-and-google-search-results.png$

The first folder contains the Bing search results, the second one contains the Google search results. Before we download the pages behind these search results, we want to eliminate duplicate bookmarks.

Duplicate Bookmark

A bookmark file is considered a duplicate of another bookmark, if it contains the same URL and has a newer creation time stamp.

Organizing Bookmarks

In order to organize the bookmark files and delete duplicate bookmarks, do the following:

Click on the Bookmarks folder in the Workspace Navigator
Right click and select Organize Bookmarks > Delete Duplicates

The deletion is only performed on the selected folder and its sub folders. It does not affect bookmarks stored above the selected folder.

The system now deletes all duplicate bookmarks from the Bookmarks folder and shows the results in the Console view. The markup of duplicate bookmarks dissapears from the bookmarks folder and its sub folders:

$images_download\attachments\2588737\deleted-duplicate-bookmarks.png$

The duplicate bookmark detection is also helpful if you want to repeat a search over time (for example each week) and see which search results are new. If you simply delete the duplicates only the new results remain.