Open Source Intelligence Tools

share link

by Ashok Balasubramanian dot icon Updated: Aug 23, 2022

technology logo
technology logo

Guide Kit Guide Kit  

Open source for Open source! From tracking Elon Musk’s flights to critical activities like analyzing war crimes, financial irregularities, and tracking refugee relocation, open source intelligence is used by global analysts, and enthusiasts to surface vital information. Open source intelligence (OSINT) uncovers, processes, and reports insights from data obtained from public sources for different intelligence purposes. While traditional reporting has always focused on uncovering intelligence with public information, the proliferation of digital across all facets of life and the subsequent digital trail has paved the way for larger-scale OSINT. The initial practice started with cyber security and multiple experts looking for the next threat, attack surfaces, breaches, and others. But today, it has grown across security, law enforcement, geopolitics, human rights, competitive intelligence, and many other areas. The core tenet of OSINT is not to deal with classified information. The main sources for OSINT are digital media, citizen media (social), public government data, academic research, and network scans. Techniques include scraping or harvesting data, transforming it, and then analyzing and reporting it for use. This is usually done in volunteer groups, given the vast amount of data. Open source libraries help you automate open source intelligence gathering. Here are popular open source libraries for popular OSINT tools like web scrapers, Maltego, Amass, TinEye, Sentinel Hub, Shodan, Crowdtangle, InVID and others.

Goutteby FriendsOfPHP

PHP doticonstar image 9229 doticonVersion:v4.0.3doticon
License: Permissive (MIT)

Goutte, a simple PHP Web Scraper

Support
    Quality
      Security
        License
          Reuse

            Goutteby FriendsOfPHP

            PHP doticon star image 9229 doticonVersion:v4.0.3doticon License: Permissive (MIT)

            Goutte, a simple PHP Web Scraper
            Support
              Quality
                Security
                  License
                    Reuse

                      Amassby OWASP

                      Go doticonstar image 9025 doticonVersion:v3.22.2doticon
                      License: Others (Non-SPDX)

                      In-depth Attack Surface Mapping and Asset Discovery

                      Support
                        Quality
                          Security
                            License
                              Reuse

                                Amassby OWASP

                                Go doticon star image 9025 doticonVersion:v3.22.2doticon License: Others (Non-SPDX)

                                In-depth Attack Surface Mapping and Asset Discovery
                                Support
                                  Quality
                                    Security
                                      License
                                        Reuse

                                          x-rayby matthewmueller

                                          JavaScript doticonstar image 5710 doticonVersion:2.3.4doticon
                                          License: Permissive (MIT)

                                          The next web scraper. See through the <html> noise.

                                          Support
                                            Quality
                                              Security
                                                License
                                                  Reuse

                                                    x-rayby matthewmueller

                                                    JavaScript doticon star image 5710 doticonVersion:2.3.4doticon License: Permissive (MIT)

                                                    The next web scraper. See through the noise.
                                                    Support
                                                      Quality
                                                        Security
                                                          License
                                                            Reuse

                                                              autoscraperby alirezamika

                                                              Python doticonstar image 5239 doticonVersion:v1.1.14doticon
                                                              License: Permissive (MIT)

                                                              A Smart, Automatic, Fast and Lightweight Web Scraper for Python

                                                              Support
                                                                Quality
                                                                  Security
                                                                    License
                                                                      Reuse

                                                                        autoscraperby alirezamika

                                                                        Python doticon star image 5239 doticonVersion:v1.1.14doticon License: Permissive (MIT)

                                                                        A Smart, Automatic, Fast and Lightweight Web Scraper for Python
                                                                        Support
                                                                          Quality
                                                                            Security
                                                                              License
                                                                                Reuse

                                                                                  shodan-pythonby achillean

                                                                                  Python doticonstar image 2186 doticonVersion:1.28.0doticon
                                                                                  License: Others (Non-SPDX)

                                                                                  The official Python library for Shodan

                                                                                  Support
                                                                                    Quality
                                                                                      Security
                                                                                        License
                                                                                          Reuse

                                                                                            shodan-pythonby achillean

                                                                                            Python doticon star image 2186 doticonVersion:1.28.0doticon License: Others (Non-SPDX)

                                                                                            The official Python library for Shodan
                                                                                            Support
                                                                                              Quality
                                                                                                Security
                                                                                                  License
                                                                                                    Reuse

                                                                                                      sentinelhub-pyby sentinel-hub

                                                                                                      Python doticonstar image 705 doticonVersion:v3.9.1doticon
                                                                                                      License: Permissive (MIT)

                                                                                                      Download and process satellite imagery in Python using Sentinel Hub services.

                                                                                                      Support
                                                                                                        Quality
                                                                                                          Security
                                                                                                            License
                                                                                                              Reuse

                                                                                                                sentinelhub-pyby sentinel-hub

                                                                                                                Python doticon star image 705 doticonVersion:v3.9.1doticon License: Permissive (MIT)

                                                                                                                Download and process satellite imagery in Python using Sentinel Hub services.
                                                                                                                Support
                                                                                                                  Quality
                                                                                                                    Security
                                                                                                                      License
                                                                                                                        Reuse
                                                                                                                          Python doticonstar image 119 doticonVersion:v1.4.5doticon
                                                                                                                          License: Strong Copyleft (AGPL-3.0)

                                                                                                                          Set of Maltego transforms to inferface with a MISP Threat Sharing instance, and also to explore the whole MITRE ATT&CK dataset.

                                                                                                                          Support
                                                                                                                            Quality
                                                                                                                              Security
                                                                                                                                License
                                                                                                                                  Reuse

                                                                                                                                    MISP-maltegoby MISP

                                                                                                                                    Python doticon star image 119 doticonVersion:v1.4.5doticon License: Strong Copyleft (AGPL-3.0)

                                                                                                                                    Set of Maltego transforms to inferface with a MISP Threat Sharing instance, and also to explore the whole MITRE ATT&CK dataset.
                                                                                                                                    Support
                                                                                                                                      Quality
                                                                                                                                        Security
                                                                                                                                          License
                                                                                                                                            Reuse

                                                                                                                                              APIby CrowdTangle

                                                                                                                                              JavaScript doticonstar image 104 doticonVersion:Currentdoticon
                                                                                                                                              License: Permissive (MIT)

                                                                                                                                              API Documentation

                                                                                                                                              Support
                                                                                                                                                Quality
                                                                                                                                                  Security
                                                                                                                                                    License
                                                                                                                                                      Reuse

                                                                                                                                                        APIby CrowdTangle

                                                                                                                                                        JavaScript doticon star image 104 doticonVersion:Currentdoticon License: Permissive (MIT)

                                                                                                                                                        API Documentation
                                                                                                                                                        Support
                                                                                                                                                          Quality
                                                                                                                                                            Security
                                                                                                                                                              License
                                                                                                                                                                Reuse

                                                                                                                                                                  sentinelhub-jsby sentinel-hub

                                                                                                                                                                  TypeScript doticonstar image 50 doticonVersion:v0.2.95doticon
                                                                                                                                                                  License: Permissive (MIT)

                                                                                                                                                                  Download and process satellite imagery in JavaScript or TypeScript using Sentinel Hub services.

                                                                                                                                                                  Support
                                                                                                                                                                    Quality
                                                                                                                                                                      Security
                                                                                                                                                                        License
                                                                                                                                                                          Reuse

                                                                                                                                                                            sentinelhub-jsby sentinel-hub

                                                                                                                                                                            TypeScript doticon star image 50 doticonVersion:v0.2.95doticon License: Permissive (MIT)

                                                                                                                                                                            Download and process satellite imagery in JavaScript or TypeScript using Sentinel Hub services.
                                                                                                                                                                            Support
                                                                                                                                                                              Quality
                                                                                                                                                                                Security
                                                                                                                                                                                  License
                                                                                                                                                                                    Reuse
                                                                                                                                                                                      JavaScript doticonstar image 16 doticonVersion:v073-sna-dev-beta-1doticon
                                                                                                                                                                                      License: Others (Non-SPDX)

                                                                                                                                                                                      Code of the InVID EU project plugin for video and image verification

                                                                                                                                                                                      Support
                                                                                                                                                                                        Quality
                                                                                                                                                                                          Security
                                                                                                                                                                                            License
                                                                                                                                                                                              Reuse

                                                                                                                                                                                                invid-verification-pluginby AFP-Medialab

                                                                                                                                                                                                JavaScript doticon star image 16 doticonVersion:v073-sna-dev-beta-1doticon License: Others (Non-SPDX)

                                                                                                                                                                                                Code of the InVID EU project plugin for video and image verification
                                                                                                                                                                                                Support
                                                                                                                                                                                                  Quality
                                                                                                                                                                                                    Security
                                                                                                                                                                                                      License
                                                                                                                                                                                                        Reuse
                                                                                                                                                                                                          JavaScript doticonstar image 15 doticonVersion:Currentdoticon
                                                                                                                                                                                                          License: Others (Non-SPDX)

                                                                                                                                                                                                          Support
                                                                                                                                                                                                            Quality
                                                                                                                                                                                                              Security
                                                                                                                                                                                                                License
                                                                                                                                                                                                                  Reuse

                                                                                                                                                                                                                    invid-verification-pluginby invideu

                                                                                                                                                                                                                    JavaScript doticon star image 15 doticonVersion:Currentdoticon License: Others (Non-SPDX)

                                                                                                                                                                                                                    Support
                                                                                                                                                                                                                      Quality
                                                                                                                                                                                                                        Security
                                                                                                                                                                                                                          License
                                                                                                                                                                                                                            Reuse
                                                                                                                                                                                                                              Python doticonstar image 10 doticonVersion:1.6.5doticon
                                                                                                                                                                                                                              License: Permissive (MIT)

                                                                                                                                                                                                                              Python library for TinEye Services (MatchEngine, MobileEngine, MulticolorEngine and WineEngine)

                                                                                                                                                                                                                              Support
                                                                                                                                                                                                                                Quality
                                                                                                                                                                                                                                  Security
                                                                                                                                                                                                                                    License
                                                                                                                                                                                                                                      Reuse

                                                                                                                                                                                                                                        tineyeservices_pythonby TinEye

                                                                                                                                                                                                                                        Python doticon star image 10 doticonVersion:1.6.5doticon License: Permissive (MIT)

                                                                                                                                                                                                                                        Python library for TinEye Services (MatchEngine, MobileEngine, MulticolorEngine and WineEngine)
                                                                                                                                                                                                                                        Support
                                                                                                                                                                                                                                          Quality
                                                                                                                                                                                                                                            Security
                                                                                                                                                                                                                                              License
                                                                                                                                                                                                                                                Reuse
                                                                                                                                                                                                                                                  JavaScript doticonstar image 10 doticonVersion:2.0.3doticon
                                                                                                                                                                                                                                                  License: Permissive (MIT)

                                                                                                                                                                                                                                                  Node.js library for the TinEye API

                                                                                                                                                                                                                                                  Support
                                                                                                                                                                                                                                                    Quality
                                                                                                                                                                                                                                                      Security
                                                                                                                                                                                                                                                        License
                                                                                                                                                                                                                                                          Reuse

                                                                                                                                                                                                                                                            tineye-api-nodeby TinEye

                                                                                                                                                                                                                                                            JavaScript doticon star image 10 doticonVersion:2.0.3doticon License: Permissive (MIT)

                                                                                                                                                                                                                                                            Node.js library for the TinEye API
                                                                                                                                                                                                                                                            Support
                                                                                                                                                                                                                                                              Quality
                                                                                                                                                                                                                                                                Security
                                                                                                                                                                                                                                                                  License
                                                                                                                                                                                                                                                                    Reuse
                                                                                                                                                                                                                                                                      PHP doticonstar image 1 doticonVersion:1.1.0doticon
                                                                                                                                                                                                                                                                      License: Permissive (MIT)

                                                                                                                                                                                                                                                                      PHP library for the TinEye API

                                                                                                                                                                                                                                                                      Support
                                                                                                                                                                                                                                                                        Quality
                                                                                                                                                                                                                                                                          Security
                                                                                                                                                                                                                                                                            License
                                                                                                                                                                                                                                                                              Reuse

                                                                                                                                                                                                                                                                                tineye-api-phpby TinEye

                                                                                                                                                                                                                                                                                PHP doticon star image 1 doticonVersion:1.1.0doticon License: Permissive (MIT)

                                                                                                                                                                                                                                                                                PHP library for the TinEye API
                                                                                                                                                                                                                                                                                Support
                                                                                                                                                                                                                                                                                  Quality
                                                                                                                                                                                                                                                                                    Security
                                                                                                                                                                                                                                                                                      License
                                                                                                                                                                                                                                                                                        Reuse