Open Source Intelligence - Reusable Libraries

by abala

Open Source Intelligence has played a pivotal role in key events like tracing Covid-19 origins, MH17 downing, the Boston Marathon bombing, and the Myanmar refugee crisis. Approximately 500 million tweets are published every day, totaling over 200 billion posts in a year. Facebook users upload 350 million photos per day. YouTube users add nearly 720,000 hours of new video every day. Almost all devices are online today in the connected world.

While monitoring messages was exclusive to intelligence agencies, the tons of information available in the public realm today has made it possible for general and security enthusiasts to look for insights that might not have been possible earlier. The U.S. Department of State defines OSINT as "intelligence that is produced from publicly available information and is collected, exploited, and disseminated promptly to an appropriate audience to address a specific intelligence requirement."

Designed correctly, OSINT can reduce risk across a variety of common risks such as weather conditions, disease outbreaks, corporate risk management, data privacy, reputation management, in addition to higher-order tasks like national security and cybersecurity. Do not construe this as legal advice, promotion, or authorization to indulge in any activity whatsoever.

Use the open source, cloud APIs, or public libraries listed below in your application development based on your technology preferences, such as primary language. The below list also provides a view of the components' rating on different dimensions such as community support availability, security vulnerability, and overall quality, helping you make an informed choice for implementation and maintenance of your application. Please review the components carefully, having a no license alert or proprietary license, and use them appropriately in your applications. Please check the component page for the exact license of the component. You can also get information on the component's features, installation steps, top code snippets, and top community discussions on the component details page. The links to package managers are listed for download, where packages are readily available. Otherwise, build from the respective repositories for use in your application. You can also use the source code from the repositories in your applications based on the respective license types.

OSINT Framework

The OSINT framework enables gathering information from free tools or resources. The below open source libraries introduce and enable gathering information based on the OSINT Framework.
O

OSINT-Frameworkby lockfale

OSINT Framework

JavaScript Updated: 3 mo ago License: Permissive

Support
Quality
Security
License
Reuse
s

sherlockby sherlock-project

🔎 Hunt down social media accounts by username across social networks

Python Updated: 4 d ago License: Permissive

Support
Quality
Security
License
Reuse
O

OSINTby sinwindie

Collections of tools and methods created to aid in OSINT collection

Python Updated: 4 mo ago License: No License

Support
Quality
Security
License
Reuse

Target Reconnaissance

Recon-ng is a full-featured reconnaissance framework designed with the goal of providing a powerful environment to conduct open source web-based reconnaissance quickly and thoroughly.
r

recon-ngby lanmaster53

Open Source Intelligence gathering tool aimed at reducing the time spent harvesting information from open sources.

Python Updated: 4 mo ago License: Strong Copyleft

Support
Quality
Security
License
Reuse
r

recon-ng-marketplaceby lanmaster53

Official module repository for the Recon-ng Framework.

Python Updated: 7 mo ago License: Strong Copyleft

Support
Quality
Security
License
Reuse

Information Collection

theHarvester and similar tools gather emails, names, subdomains, IPs and URLs using multiple public data sources.
t

theHarvesterby laramies

E-mails, subdomains and names Harvester - OSINT

Python Updated: 2 d ago License: Strong Copyleft

Support
Quality
Security
License
Reuse
p

pwnedhubby lanmaster53

JavaScript Updated: 1 mo ago License: Proprietary

Support
Quality
Security
License
Reuse
m

metagoofilby laramies

Metadata harvester

Python Updated: 5 mo ago License: Strong Copyleft

Support
Quality
Security
License
Reuse

Track Online Assets

Shodan and Amass enable researchers to see the exposed assets.
s

shodan-pythonby achillean

The official Python library for Shodan

Python Updated: 6 mo ago License: Proprietary

Support
Quality
Security
License
Reuse
A

Amassby OWASP

In-depth Attack Surface Mapping and Asset Discovery

Go Updated: 4 mo ago License: Permissive

Support
Quality
Security
License
Reuse
M

My-Shodan-Scriptsby random-robbie

Collection of Scripts for shodan searching stuff.

Python Updated: 3 mo ago License: Permissive

Support
Quality
Security
License
Reuse

Google Search

Google dorks provides information through the usage of operators, which are otherwise difficult to extract using simple searches.
p

pagodoby opsdisk

pagodo (Passive Google Dork) - Automate Google Hacking Database scraping and searching

Python Updated: 3 mo ago License: Strong Copyleft

Support
Quality
Security
License
Reuse
B

BigBountyReconby Viralmaniar

BigBountyRecon tool utilises 58 different techniques using various Google dorks and open source tools to expedite the process of initial reconnaissance on the target organisation.

C# Updated: 3 mo ago License: Permissive

Support
Quality
Security
License
Reuse
F

Fast-Google-Dorks-Scanby IvanGlinkin

The OSINT project, the main idea of which is to collect all the possible Google dorks search combinations and to find the information about the specific web-site: common admin panels, the widespread file types and path traversal. The 100% automated.

Shell Updated: 6 mo ago License: No License

Support
Quality
Security
License
Reuse
Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from
over 430 million Knowledge Items

Find more libraries on