Open Source Intelligence Tools
by Ashok Balasubramanian Updated: Aug 23, 2022
Open source for Open source! From tracking Elon Musk’s flights to critical activities like analyzing war crimes, financial irregularities, and tracking refugee relocation, open source intelligence is used by global analysts, and enthusiasts to surface vital information. Open source intelligence (OSINT) uncovers, processes, and reports insights from data obtained from public sources for different intelligence purposes. While traditional reporting has always focused on uncovering intelligence with public information, the proliferation of digital across all facets of life and the subsequent digital trail has paved the way for larger-scale OSINT. The initial practice started with cyber security and multiple experts looking for the next threat, attack surfaces, breaches, and others. But today, it has grown across security, law enforcement, geopolitics, human rights, competitive intelligence, and many other areas. The core tenet of OSINT is not to deal with classified information. The main sources for OSINT are digital media, citizen media (social), public government data, academic research, and network scans. Techniques include scraping or harvesting data, transforming it, and then analyzing and reporting it for use. This is usually done in volunteer groups, given the vast amount of data. Open source libraries help you automate open source intelligence gathering. Here are popular open source libraries for popular OSINT tools like web scrapers, Maltego, Amass, TinEye, Sentinel Hub, Shodan, Crowdtangle, InVID and others.
The next web scraper. See through the <html> noise.
A Smart, Automatic, Fast and Lightweight Web Scraper for Python
Python 4987 Version:v1.1.14 License: Permissive (MIT)
The official Python library for Shodan
Python 2139 Version:1.28.0 License: Others (Non-SPDX)
Download and process satellite imagery in Python using Sentinel Hub services.
Python 685 Version:v3.8.4 License: Permissive (MIT)
Set of Maltego transforms to inferface with a MISP Threat Sharing instance, and also to explore the whole MITRE ATT&CK dataset.
Python 119 Version:v1.4.5 License: Strong Copyleft (AGPL-3.0)
TypeScript 50 Version:v0.2.87 License: Permissive (MIT)
Code of the InVID EU project plugin for video and image verification
Python library for TinEye Services (MatchEngine, MobileEngine, MulticolorEngine and WineEngine)
Python 10 Version:1.6.5 License: Permissive (MIT)
Node.js library for the TinEye API