Scraper Libraries - Page 1

y
you-getby soimort

Python dotstar image 47551 dotVersion:Currentdot
License: Proprietary (Proprietary)

:arrow_double_down: Dumb downloader that scrapes the web

Support
    Quality
      Security
        License
          Reuse
            t
            twintby twintproject

            Python dotstar image 15023 dotVersion:Currentdot
            License: Permissive (MIT)

            An advanced Twitter scraping & OSINT tool written in Python that doesn't use Twitter's API, allowing you to scrape a user's followers, following, Tweets and more while evading most API limitations.

            Support
              Quality
                Security
                  License
                    Reuse
                      r

                      Python dotstar image 13156 dotVersion:Currentdot
                      License: Permissive (MIT)

                      Pythonic HTML Parsing for Humans™

                      Support
                        Quality
                          Security
                            License
                              Reuse
                                n
                                newspaperby codelucas

                                Python dotstar image 12865 dotVersion:Currentdot
                                License: Permissive (MIT)

                                News, full-text, and article metadata extraction in Python 3. Advanced docs:

                                Support
                                  Quality
                                    Security
                                      License
                                        Reuse
                                          G
                                          Goutteby FriendsOfPHP

                                          PHP dotstar image 9229 dotVersion:Currentdot
                                          License: Permissive (MIT)

                                          Goutte, a simple PHP Web Scraper

                                          Support
                                            Quality
                                              Security
                                                License
                                                  Reuse
                                                    p
                                                    portiaby scrapinghub

                                                    Python dotstar image 8890 dotVersion:Currentdot
                                                    License: Permissive (BSD-3-Clause)

                                                    Visual scraping for Scrapy

                                                    Support
                                                      Quality
                                                        Security
                                                          License
                                                            Reuse
                                                              i
                                                              instaloaderby instaloader

                                                              Python dotstar image 6040 dotVersion:Currentdot
                                                              License: Permissive (MIT)

                                                              Download pictures (or videos) along with their captions and other metadata from Instagram.

                                                              Support
                                                                Quality
                                                                  Security
                                                                    License
                                                                      Reuse
                                                                        i

                                                                        Python dotstar image 5727 dotVersion:Currentdot
                                                                        License: Permissive (Unlicense)

                                                                        Scrapes an instagram user's photos and videos

                                                                        Support
                                                                          Quality
                                                                            Security
                                                                              License
                                                                                Reuse
                                                                                  x
                                                                                  x-rayby matthewmueller

                                                                                  JavaScript dotstar image 5710 dotVersion:Currentdot
                                                                                  License: Permissive (MIT)

                                                                                  The next web scraper. See through the <html> noise.

                                                                                  Support
                                                                                    Quality
                                                                                      Security
                                                                                        License
                                                                                          Reuse
                                                                                            f
                                                                                            ferretby MontFerret

                                                                                            Go dotstar image 5345 dotVersion:Currentdot
                                                                                            License: Permissive (Apache-2.0)

                                                                                            Declarative web scraping

                                                                                            Support
                                                                                              Quality
                                                                                                Security
                                                                                                  License
                                                                                                    Reuse
                                                                                                      a
                                                                                                      autoscraperby alirezamika

                                                                                                      Python dotstar image 5239 dotVersion:Currentdot
                                                                                                      License: Permissive (MIT)

                                                                                                      A Smart, Automatic, Fast and Lightweight Web Scraper for Python

                                                                                                      Support
                                                                                                        Quality
                                                                                                          Security
                                                                                                            License
                                                                                                              Reuse
                                                                                                                p
                                                                                                                python-scrapingby REMitchell

                                                                                                                Jupyter Notebook dotstar image 3993 dotVersion:Currentdot
                                                                                                                no licences License: No License (No License)

                                                                                                                Code samples from the book Web Scraping with Python http://shop.oreilly.com/product/0636920034391.do

                                                                                                                Support
                                                                                                                  Quality
                                                                                                                    Security
                                                                                                                      License
                                                                                                                        Reuse
                                                                                                                          s
                                                                                                                          scrape-itby IonicaBizau

                                                                                                                          JavaScript dotstar image 3917 dotVersion:Currentdot
                                                                                                                          License: Permissive (MIT)

                                                                                                                          🔮 A Node.js scraper for humans.

                                                                                                                          Support
                                                                                                                            Quality
                                                                                                                              Security
                                                                                                                                License
                                                                                                                                  Reuse
                                                                                                                                    p
                                                                                                                                    python-gooseby grangier

                                                                                                                                    HTML dotstar image 3874 dotVersion:Currentdot
                                                                                                                                    License: Permissive (Apache-2.0)

                                                                                                                                    Html Content / Article Extractor, web scrapping lib in Python

                                                                                                                                    Support
                                                                                                                                      Quality
                                                                                                                                        Security
                                                                                                                                          License
                                                                                                                                            Reuse
                                                                                                                                              O
                                                                                                                                              OnlyFansby DIGITALCRIMINAL

                                                                                                                                              Python dotstar image 3419 dotVersion:Currentdot
                                                                                                                                              License: Strong Copyleft (GPL-3.0)

                                                                                                                                              Scrape all the media from an OnlyFans account - Updated regularly

                                                                                                                                              Support
                                                                                                                                                Quality
                                                                                                                                                  Security
                                                                                                                                                    License
                                                                                                                                                      Reuse
                                                                                                                                                        c

                                                                                                                                                        Python dotstar image 3074 dotVersion:Currentdot
                                                                                                                                                        License: Permissive (MIT)

                                                                                                                                                        A Python module to bypass Cloudflare's anti-bot page.

                                                                                                                                                        Support
                                                                                                                                                          Quality
                                                                                                                                                            Security
                                                                                                                                                              License
                                                                                                                                                                Reuse
                                                                                                                                                                  f
                                                                                                                                                                  fake-useragentby fake-useragent

                                                                                                                                                                  HTML dotstar image 3047 dotVersion:Currentdot
                                                                                                                                                                  License: Permissive (Apache-2.0)

                                                                                                                                                                  Up-to-date simple useragent faker with real world database

                                                                                                                                                                  Support
                                                                                                                                                                    Quality
                                                                                                                                                                      Security
                                                                                                                                                                        License
                                                                                                                                                                          Reuse
                                                                                                                                                                            s
                                                                                                                                                                            scrapy-splashby scrapy-plugins

                                                                                                                                                                            Python dotstar image 2900 dotVersion:Currentdot
                                                                                                                                                                            License: Permissive (BSD-3-Clause)

                                                                                                                                                                            Scrapy+Splash for JavaScript integration

                                                                                                                                                                            Support
                                                                                                                                                                              Quality
                                                                                                                                                                                Security
                                                                                                                                                                                  License
                                                                                                                                                                                    Reuse
                                                                                                                                                                                      E

                                                                                                                                                                                      C# dotstar image 2622 dotVersion:Currentdot
                                                                                                                                                                                      no licences License: No License (No License)

                                                                                                                                                                                      Emby/Jellyfin 的一个日本电影刮削器插件,可以从某些网站抓取影片信息。

                                                                                                                                                                                      Support
                                                                                                                                                                                        Quality
                                                                                                                                                                                          Security
                                                                                                                                                                                            License
                                                                                                                                                                                              Reuse
                                                                                                                                                                                                c
                                                                                                                                                                                                crawlergoby Qianlitp

                                                                                                                                                                                                Go dotstar image 2474 dotVersion:Currentdot
                                                                                                                                                                                                License: Strong Copyleft (GPL-3.0)

                                                                                                                                                                                                A powerful browser crawler for web vulnerability scanners

                                                                                                                                                                                                Support
                                                                                                                                                                                                  Quality
                                                                                                                                                                                                    Security
                                                                                                                                                                                                      License
                                                                                                                                                                                                        Reuse
                                                                                                                                                                                                          t
                                                                                                                                                                                                          thalby emadehsan

                                                                                                                                                                                                          JavaScript dotstar image 2362 dotVersion:Currentdot
                                                                                                                                                                                                          License: Permissive (MIT)

                                                                                                                                                                                                          Getting started with Puppeteer and Chrome Headless for Data Mining

                                                                                                                                                                                                          Support
                                                                                                                                                                                                            Quality
                                                                                                                                                                                                              Security
                                                                                                                                                                                                                License
                                                                                                                                                                                                                  Reuse
                                                                                                                                                                                                                    i
                                                                                                                                                                                                                    instagram-scraperby realsirjoe

                                                                                                                                                                                                                    Python dotstar image 2204 dotVersion:Currentdot
                                                                                                                                                                                                                    License: Permissive (MIT)

                                                                                                                                                                                                                    scrapes medias, likes, followers, tags and all metadata. Inspired by instagram-php-scraper,bot

                                                                                                                                                                                                                    Support
                                                                                                                                                                                                                      Quality
                                                                                                                                                                                                                        Security
                                                                                                                                                                                                                          License
                                                                                                                                                                                                                            Reuse
                                                                                                                                                                                                                              J
                                                                                                                                                                                                                              JSFinderby Threezh1

                                                                                                                                                                                                                              Python dotstar image 2091 dotVersion:Currentdot
                                                                                                                                                                                                                              no licences License: No License (No License)

                                                                                                                                                                                                                              JSFinder is a tool for quickly extracting URLs and subdomains from JS files on a website.

                                                                                                                                                                                                                              Support
                                                                                                                                                                                                                                Quality
                                                                                                                                                                                                                                  Security
                                                                                                                                                                                                                                    License
                                                                                                                                                                                                                                      Reuse
                                                                                                                                                                                                                                        m
                                                                                                                                                                                                                                        metascraperby microlinkhq

                                                                                                                                                                                                                                        HTML dotstar image 2049 dotVersion:Currentdot
                                                                                                                                                                                                                                        License: Permissive (MIT)

                                                                                                                                                                                                                                        Get unified metadata from websites using Open Graph, Microdata, RDFa, Twitter Cards, JSON-LD, HTML, and more.

                                                                                                                                                                                                                                        Support
                                                                                                                                                                                                                                          Quality
                                                                                                                                                                                                                                            Security
                                                                                                                                                                                                                                              License
                                                                                                                                                                                                                                                Reuse
                                                                                                                                                                                                                                                  g
                                                                                                                                                                                                                                                  google-play-scraperby facundoolano

                                                                                                                                                                                                                                                  JavaScript dotstar image 1979 dotVersion:Currentdot
                                                                                                                                                                                                                                                  License: Permissive (MIT)

                                                                                                                                                                                                                                                  Node.js scraper to get data from Google Play

                                                                                                                                                                                                                                                  Support
                                                                                                                                                                                                                                                    Quality
                                                                                                                                                                                                                                                      Security
                                                                                                                                                                                                                                                        License
                                                                                                                                                                                                                                                          Reuse
                                                                                                                                                                                                                                                            I
                                                                                                                                                                                                                                                            InstaLooterby althonos

                                                                                                                                                                                                                                                            Python dotstar image 1871 dotVersion:Currentdot
                                                                                                                                                                                                                                                            License: Strong Copyleft (GPL-3.0)

                                                                                                                                                                                                                                                            Another API-less Instagram pictures and videos downloader.

                                                                                                                                                                                                                                                            Support
                                                                                                                                                                                                                                                              Quality
                                                                                                                                                                                                                                                                Security
                                                                                                                                                                                                                                                                  License
                                                                                                                                                                                                                                                                    Reuse
                                                                                                                                                                                                                                                                      f

                                                                                                                                                                                                                                                                      Python dotstar image 1763 dotVersion:Currentdot
                                                                                                                                                                                                                                                                      License: Permissive (MIT)

                                                                                                                                                                                                                                                                      Scrape Facebook public pages without an API key

                                                                                                                                                                                                                                                                      Support
                                                                                                                                                                                                                                                                        Quality
                                                                                                                                                                                                                                                                          Security
                                                                                                                                                                                                                                                                            License
                                                                                                                                                                                                                                                                              Reuse
                                                                                                                                                                                                                                                                                b
                                                                                                                                                                                                                                                                                blackbirdby p1ngul1n0

                                                                                                                                                                                                                                                                                Python dotstar image 1697 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                no licences License: No License (No License)

                                                                                                                                                                                                                                                                                An OSINT tool to search for accounts by username in social networks.

                                                                                                                                                                                                                                                                                Support
                                                                                                                                                                                                                                                                                  Quality
                                                                                                                                                                                                                                                                                    Security
                                                                                                                                                                                                                                                                                      License
                                                                                                                                                                                                                                                                                        Reuse
                                                                                                                                                                                                                                                                                          J
                                                                                                                                                                                                                                                                                          JobFunnelby PaulMcInnis

                                                                                                                                                                                                                                                                                          Python dotstar image 1655 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                          License: Permissive (MIT)

                                                                                                                                                                                                                                                                                          Scrape job websites into a single spreadsheet with no duplicates.

                                                                                                                                                                                                                                                                                          Support
                                                                                                                                                                                                                                                                                            Quality
                                                                                                                                                                                                                                                                                              Security
                                                                                                                                                                                                                                                                                                License
                                                                                                                                                                                                                                                                                                  Reuse
                                                                                                                                                                                                                                                                                                    d
                                                                                                                                                                                                                                                                                                    dirbotby scrapy

                                                                                                                                                                                                                                                                                                    Python dotstar image 1627 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                    no licences License: No License (No License)

                                                                                                                                                                                                                                                                                                    Scrapy project to scrape public web directories (educational) [DEPRECATED]

                                                                                                                                                                                                                                                                                                    Support
                                                                                                                                                                                                                                                                                                      Quality
                                                                                                                                                                                                                                                                                                        Security
                                                                                                                                                                                                                                                                                                          License
                                                                                                                                                                                                                                                                                                            Reuse
                                                                                                                                                                                                                                                                                                              n
                                                                                                                                                                                                                                                                                                              news-pleaseby fhamborg

                                                                                                                                                                                                                                                                                                              Python dotstar image 1626 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                              License: Permissive (Apache-2.0)

                                                                                                                                                                                                                                                                                                              news-please - an integrated web crawler and information extractor for news that just works

                                                                                                                                                                                                                                                                                                              Support
                                                                                                                                                                                                                                                                                                                Quality
                                                                                                                                                                                                                                                                                                                  Security
                                                                                                                                                                                                                                                                                                                    License
                                                                                                                                                                                                                                                                                                                      Reuse
                                                                                                                                                                                                                                                                                                                        u
                                                                                                                                                                                                                                                                                                                        uptonby propublica

                                                                                                                                                                                                                                                                                                                        HTML dotstar image 1613 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                        License: Permissive (MIT)

                                                                                                                                                                                                                                                                                                                        A batteries-included framework for easy web-scraping. Just add CSS! (Or do more.)

                                                                                                                                                                                                                                                                                                                        Support
                                                                                                                                                                                                                                                                                                                          Quality
                                                                                                                                                                                                                                                                                                                            Security
                                                                                                                                                                                                                                                                                                                              License
                                                                                                                                                                                                                                                                                                                                Reuse
                                                                                                                                                                                                                                                                                                                                  e
                                                                                                                                                                                                                                                                                                                                  event-collectby fossasia

                                                                                                                                                                                                                                                                                                                                  Python dotstar image 1510 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                  no licences License: No License (No License)

                                                                                                                                                                                                                                                                                                                                  event website listing to Open Event format scraper and converter

                                                                                                                                                                                                                                                                                                                                  Support
                                                                                                                                                                                                                                                                                                                                    Quality
                                                                                                                                                                                                                                                                                                                                      Security
                                                                                                                                                                                                                                                                                                                                        License
                                                                                                                                                                                                                                                                                                                                          Reuse
                                                                                                                                                                                                                                                                                                                                            l

                                                                                                                                                                                                                                                                                                                                            JavaScript dotstar image 1476 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                            License: Weak Copyleft (LGPL-2.1)

                                                                                                                                                                                                                                                                                                                                            Scrapers for loklak in javascript

                                                                                                                                                                                                                                                                                                                                            Support
                                                                                                                                                                                                                                                                                                                                              Quality
                                                                                                                                                                                                                                                                                                                                                Security
                                                                                                                                                                                                                                                                                                                                                  License
                                                                                                                                                                                                                                                                                                                                                    Reuse
                                                                                                                                                                                                                                                                                                                                                      s
                                                                                                                                                                                                                                                                                                                                                      scraperby causal-agent

                                                                                                                                                                                                                                                                                                                                                      Rust dotstar image 1407 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                      License: Permissive (ISC)

                                                                                                                                                                                                                                                                                                                                                      HTML parsing and querying with CSS selectors

                                                                                                                                                                                                                                                                                                                                                      Support
                                                                                                                                                                                                                                                                                                                                                        Quality
                                                                                                                                                                                                                                                                                                                                                          Security
                                                                                                                                                                                                                                                                                                                                                            License
                                                                                                                                                                                                                                                                                                                                                              Reuse
                                                                                                                                                                                                                                                                                                                                                                r
                                                                                                                                                                                                                                                                                                                                                                rvestby tidyverse

                                                                                                                                                                                                                                                                                                                                                                R dotstar image 1400 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                License: Proprietary (Proprietary)

                                                                                                                                                                                                                                                                                                                                                                Simple web scraping for R

                                                                                                                                                                                                                                                                                                                                                                Support
                                                                                                                                                                                                                                                                                                                                                                  Quality
                                                                                                                                                                                                                                                                                                                                                                    Security
                                                                                                                                                                                                                                                                                                                                                                      License
                                                                                                                                                                                                                                                                                                                                                                        Reuse
                                                                                                                                                                                                                                                                                                                                                                          M
                                                                                                                                                                                                                                                                                                                                                                          Metamorphosisby killme2008

                                                                                                                                                                                                                                                                                                                                                                          Java dotstar image 1320 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                          License: Permissive (Apache-2.0)

                                                                                                                                                                                                                                                                                                                                                                          A high available,high performance distributed messaging system.

                                                                                                                                                                                                                                                                                                                                                                          Support
                                                                                                                                                                                                                                                                                                                                                                            Quality
                                                                                                                                                                                                                                                                                                                                                                              Security
                                                                                                                                                                                                                                                                                                                                                                                License
                                                                                                                                                                                                                                                                                                                                                                                  Reuse
                                                                                                                                                                                                                                                                                                                                                                                    w
                                                                                                                                                                                                                                                                                                                                                                                    wombatby felipecsl

                                                                                                                                                                                                                                                                                                                                                                                    Ruby dotstar image 1281 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                    License: Permissive (MIT)

                                                                                                                                                                                                                                                                                                                                                                                    Lightweight Ruby web crawler/scraper with an elegant DSL which extracts structured data from pages.

                                                                                                                                                                                                                                                                                                                                                                                    Support
                                                                                                                                                                                                                                                                                                                                                                                      Quality
                                                                                                                                                                                                                                                                                                                                                                                        Security
                                                                                                                                                                                                                                                                                                                                                                                          License
                                                                                                                                                                                                                                                                                                                                                                                            Reuse
                                                                                                                                                                                                                                                                                                                                                                                              O
                                                                                                                                                                                                                                                                                                                                                                                              OpenWPMby openwpm

                                                                                                                                                                                                                                                                                                                                                                                              Python dotstar image 1254 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                              License: Proprietary (Proprietary)

                                                                                                                                                                                                                                                                                                                                                                                              A web privacy measurement framework

                                                                                                                                                                                                                                                                                                                                                                                              Support
                                                                                                                                                                                                                                                                                                                                                                                                Quality
                                                                                                                                                                                                                                                                                                                                                                                                  Security
                                                                                                                                                                                                                                                                                                                                                                                                    License
                                                                                                                                                                                                                                                                                                                                                                                                      Reuse
                                                                                                                                                                                                                                                                                                                                                                                                        w

                                                                                                                                                                                                                                                                                                                                                                                                        JavaScript dotstar image 1212 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                                        License: Weak Copyleft (LGPL-3.0)

                                                                                                                                                                                                                                                                                                                                                                                                        Web data extraction tool implemented as chrome extension

                                                                                                                                                                                                                                                                                                                                                                                                        Support
                                                                                                                                                                                                                                                                                                                                                                                                          Quality
                                                                                                                                                                                                                                                                                                                                                                                                            Security
                                                                                                                                                                                                                                                                                                                                                                                                              License
                                                                                                                                                                                                                                                                                                                                                                                                                Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                  s
                                                                                                                                                                                                                                                                                                                                                                                                                  search-script-scrapeby stanfordjournalism

                                                                                                                                                                                                                                                                                                                                                                                                                  Python dotstar image 1206 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                                                  no licences License: No License (No License)

                                                                                                                                                                                                                                                                                                                                                                                                                  101 real world web scraping exercises in Python 3 for data journalists

                                                                                                                                                                                                                                                                                                                                                                                                                  Support
                                                                                                                                                                                                                                                                                                                                                                                                                    Quality
                                                                                                                                                                                                                                                                                                                                                                                                                      Security
                                                                                                                                                                                                                                                                                                                                                                                                                        License
                                                                                                                                                                                                                                                                                                                                                                                                                          Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                            q
                                                                                                                                                                                                                                                                                                                                                                                                                            quotesbotby scrapy

                                                                                                                                                                                                                                                                                                                                                                                                                            Python dotstar image 1191 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                                                            License: Permissive (MIT)

                                                                                                                                                                                                                                                                                                                                                                                                                            This is a sample Scrapy project for educational purposes

                                                                                                                                                                                                                                                                                                                                                                                                                            Support
                                                                                                                                                                                                                                                                                                                                                                                                                              Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                Security
                                                                                                                                                                                                                                                                                                                                                                                                                                  License
                                                                                                                                                                                                                                                                                                                                                                                                                                    Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                      l

                                                                                                                                                                                                                                                                                                                                                                                                                                      Python dotstar image 1158 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                                                                      License: Strong Copyleft (GPL-3.0)

                                                                                                                                                                                                                                                                                                                                                                                                                                      A library that scrapes Linkedin for user data

                                                                                                                                                                                                                                                                                                                                                                                                                                      Support
                                                                                                                                                                                                                                                                                                                                                                                                                                        Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                          Security
                                                                                                                                                                                                                                                                                                                                                                                                                                            License
                                                                                                                                                                                                                                                                                                                                                                                                                                              Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                t
                                                                                                                                                                                                                                                                                                                                                                                                                                                toutatisby megadose

                                                                                                                                                                                                                                                                                                                                                                                                                                                Python dotstar image 1157 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                                                                                License: Strong Copyleft (GPL-3.0)

                                                                                                                                                                                                                                                                                                                                                                                                                                                Toutatis is a tool that allows you to extract information from instagrams accounts such as e-mails, phone numbers and more

                                                                                                                                                                                                                                                                                                                                                                                                                                                Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                  Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                    Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                      License
                                                                                                                                                                                                                                                                                                                                                                                                                                                        Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                          t
                                                                                                                                                                                                                                                                                                                                                                                                                                                          trafilaturaby adbar

                                                                                                                                                                                                                                                                                                                                                                                                                                                          Python dotstar image 1105 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                                                                                          License: Strong Copyleft (GPL-3.0)

                                                                                                                                                                                                                                                                                                                                                                                                                                                          Python & command-line tool to gather text on the Web: web crawling/scraping, extraction of text, metadata, comments

                                                                                                                                                                                                                                                                                                                                                                                                                                                          Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                            Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                              Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                    a
                                                                                                                                                                                                                                                                                                                                                                                                                                                                    artooby medialab

                                                                                                                                                                                                                                                                                                                                                                                                                                                                    JavaScript dotstar image 1086 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                                                                                                    License: Permissive (MIT)

                                                                                                                                                                                                                                                                                                                                                                                                                                                                    artoo.js - the client-side scraping companion.

                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                          License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                              p
                                                                                                                                                                                                                                                                                                                                                                                                                                                                              pjscrapeby nrabinowitz

                                                                                                                                                                                                                                                                                                                                                                                                                                                                              JavaScript dotstar image 1006 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                                                                                                              License: Permissive (MIT)

                                                                                                                                                                                                                                                                                                                                                                                                                                                                              A web-scraping framework written in Javascript, using PhantomJS and jQuery

                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        m
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        metainspectorby metainspector

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Ruby dotstar image 990 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        License: Permissive (MIT)

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Ruby gem for web scraping purposes. It scrapes a given URL, and returns you its title, meta description, meta keywords, links, images...

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  o
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  osi.igby th3unkn0n

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Python dotstar image 986 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  no licences License: No License (No License)

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Information Gathering Instagram.

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            m
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            mlscraperby lorey

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Python dotstar image 975 dotVersion:Currentdot
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            no licences License: No License (No License)

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            🤖 Scrape data from HTML websites automatically by just providing examples

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      y
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      you-getby soimort
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      :arrow_double_down: Dumb downloader that scrapes the web
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Pythondotstar image 47551dotUpdated: 1 y agodot License: Proprietary (Proprietary)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                t
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                twintby twintproject
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                An advanced Twitter scraping & OSINT tool written in Python that doesn't use Twitter's API, allowing you to scrape a user's followers, following, Tweets and more while evading most API limitations.
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Pythondotstar image 15023dotUpdated: 1 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          r
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          requests-htmlby psf
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Pythonic HTML Parsing for Humans™
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Pythondotstar image 13156dotUpdated: 1 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    n
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    newspaperby codelucas
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    News, full-text, and article metadata extraction in Python 3. Advanced docs:
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Pythondotstar image 12865dotUpdated: 1 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              G
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Goutteby FriendsOfPHP
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Goutte, a simple PHP Web Scraper
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              PHPdotstar image 9229dotUpdated: 1 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        p
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        portiaby scrapinghub
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Visual scraping for Scrapy
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Pythondotstar image 8890dotUpdated: 1 y agodot License: Permissive (BSD-3-Clause)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  i
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  instaloaderby instaloader
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Download pictures (or videos) along with their captions and other metadata from Instagram.
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Pythondotstar image 6040dotUpdated: 1 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            i
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            instagram-scraperby arc298
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Scrapes an instagram user's photos and videos
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Pythondotstar image 5727dotUpdated: 2 y agodot License: Permissive (Unlicense)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      x
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      x-rayby matthewmueller
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      The next web scraper. See through the <html> noise.
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      JavaScriptdotstar image 5710dotUpdated: 1 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                f
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                ferretby MontFerret
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Declarative web scraping
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Godotstar image 5345dotUpdated: 1 y agodot License: Permissive (Apache-2.0)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          a
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          autoscraperby alirezamika
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          A Smart, Automatic, Fast and Lightweight Web Scraper for Python
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Pythondotstar image 5239dotUpdated: 1 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    p
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    python-scrapingby REMitchell
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Code samples from the book Web Scraping with Python http://shop.oreilly.com/product/0636920034391.do
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Jupyter Notebookdotstar image 3993dotUpdated: 1 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              s
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              scrape-itby IonicaBizau
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              🔮 A Node.js scraper for humans.
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              JavaScriptdotstar image 3917dotUpdated: 1 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        p
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        python-gooseby grangier
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Html Content / Article Extractor, web scrapping lib in Python
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        HTMLdotstar image 3874dotUpdated: 1 y agodot License: Permissive (Apache-2.0)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  O
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  OnlyFansby DIGITALCRIMINAL
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Scrape all the media from an OnlyFans account - Updated regularly
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Pythondotstar image 3419dotUpdated: 1 y agodot License: Strong Copyleft (GPL-3.0)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            c
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            cloudflare-scrapeby Anorov
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            A Python module to bypass Cloudflare's anti-bot page.
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Pythondotstar image 3074dotUpdated: 1 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      f
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      fake-useragentby fake-useragent
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Up-to-date simple useragent faker with real world database
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      HTMLdotstar image 3047dotUpdated: 1 y agodot License: Permissive (Apache-2.0)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                s
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                scrapy-splashby scrapy-plugins
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Scrapy+Splash for JavaScript integration
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Pythondotstar image 2900dotUpdated: 1 y agodot License: Permissive (BSD-3-Clause)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          E
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Emby.Plugins.JavScraperby JavScraper
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Emby/Jellyfin 的一个日本电影刮削器插件,可以从某些网站抓取影片信息。
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          C#dotstar image 2622dotUpdated: 1 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    c
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    crawlergoby Qianlitp
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    A powerful browser crawler for web vulnerability scanners
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Godotstar image 2474dotUpdated: 1 y agodot License: Strong Copyleft (GPL-3.0)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              t
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              thalby emadehsan
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Getting started with Puppeteer and Chrome Headless for Data Mining
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              JavaScriptdotstar image 2362dotUpdated: 1 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        i
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        instagram-scraperby realsirjoe
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        scrapes medias, likes, followers, tags and all metadata. Inspired by instagram-php-scraper,bot
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Pythondotstar image 2204dotUpdated: 3 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  J
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  JSFinderby Threezh1
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  JSFinder is a tool for quickly extracting URLs and subdomains from JS files on a website.
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Pythondotstar image 2091dotUpdated: 1 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            m
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            metascraperby microlinkhq
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Get unified metadata from websites using Open Graph, Microdata, RDFa, Twitter Cards, JSON-LD, HTML, and more.
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            HTMLdotstar image 2049dotUpdated: 1 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      g
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      google-play-scraperby facundoolano
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Node.js scraper to get data from Google Play
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      JavaScriptdotstar image 1979dotUpdated: 1 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                I
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                InstaLooterby althonos
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Another API-less Instagram pictures and videos downloader.
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Pythondotstar image 1871dotUpdated: 1 y agodot License: Strong Copyleft (GPL-3.0)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          f
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          facebook-scraperby kevinzg
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Scrape Facebook public pages without an API key
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Pythondotstar image 1763dotUpdated: 1 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    b
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    blackbirdby p1ngul1n0
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    An OSINT tool to search for accounts by username in social networks.
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Pythondotstar image 1697dotUpdated: 1 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              J
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              JobFunnelby PaulMcInnis
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Scrape job websites into a single spreadsheet with no duplicates.
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Pythondotstar image 1655dotUpdated: 1 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        d
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        dirbotby scrapy
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Scrapy project to scrape public web directories (educational) [DEPRECATED]
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Pythondotstar image 1627dotUpdated: 1 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  n
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  news-pleaseby fhamborg
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  news-please - an integrated web crawler and information extractor for news that just works
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Pythondotstar image 1626dotUpdated: 1 y agodot License: Permissive (Apache-2.0)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            u
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            uptonby propublica
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            A batteries-included framework for easy web-scraping. Just add CSS! (Or do more.)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            HTMLdotstar image 1613dotUpdated: 1 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      e
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      event-collectby fossasia
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      event website listing to Open Event format scraper and converter
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Pythondotstar image 1510dotUpdated: 1 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                l
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                loklak_scraper_jsby fossasia
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Scrapers for loklak in javascript
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                JavaScriptdotstar image 1476dotUpdated: 1 y agodot License: Weak Copyleft (LGPL-2.1)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          s
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          scraperby causal-agent
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          HTML parsing and querying with CSS selectors
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Rustdotstar image 1407dotUpdated: 1 y agodot License: Permissive (ISC)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    r
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    rvestby tidyverse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Simple web scraping for R
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Rdotstar image 1400dotUpdated: 1 y agodot License: Proprietary (Proprietary)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              M
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Metamorphosisby killme2008
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              A high available,high performance distributed messaging system.
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Javadotstar image 1320dotUpdated: 1 y agodot License: Permissive (Apache-2.0)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        w
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        wombatby felipecsl
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Lightweight Ruby web crawler/scraper with an elegant DSL which extracts structured data from pages.
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Rubydotstar image 1281dotUpdated: 1 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  O
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  OpenWPMby openwpm
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  A web privacy measurement framework
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Pythondotstar image 1254dotUpdated: 1 y agodot License: Proprietary (Proprietary)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            w
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            web-scraper-chrome-extensionby martinsbalodis
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Web data extraction tool implemented as chrome extension
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            JavaScriptdotstar image 1212dotUpdated: 1 y agodot License: Weak Copyleft (LGPL-3.0)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      s
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      search-script-scrapeby stanfordjournalism
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      101 real world web scraping exercises in Python 3 for data journalists
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Pythondotstar image 1206dotUpdated: 1 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                q
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                quotesbotby scrapy
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                This is a sample Scrapy project for educational purposes
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Pythondotstar image 1191dotUpdated: 1 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          l
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          linkedin_scraperby joeyism
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          A library that scrapes Linkedin for user data
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Pythondotstar image 1158dotUpdated: 1 y agodot License: Strong Copyleft (GPL-3.0)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    t
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    toutatisby megadose
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Toutatis is a tool that allows you to extract information from instagrams accounts such as e-mails, phone numbers and more
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Pythondotstar image 1157dotUpdated: 1 y agodot License: Strong Copyleft (GPL-3.0)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              t
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              trafilaturaby adbar
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Python & command-line tool to gather text on the Web: web crawling/scraping, extraction of text, metadata, comments
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Pythondotstar image 1105dotUpdated: 1 y agodot License: Strong Copyleft (GPL-3.0)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        a
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        artooby medialab
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        artoo.js - the client-side scraping companion.
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        JavaScriptdotstar image 1086dotUpdated: 1 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  p
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  pjscrapeby nrabinowitz
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  A web-scraping framework written in Javascript, using PhantomJS and jQuery
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  JavaScriptdotstar image 1006dotUpdated: 1 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            m
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            metainspectorby metainspector
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Ruby gem for web scraping purposes. It scrapes a given URL, and returns you its title, meta description, meta keywords, links, images...
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Rubydotstar image 990dotUpdated: 1 y agodot License: Permissive (MIT)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      o
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      osi.igby th3unkn0n
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Information Gathering Instagram.
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Pythondotstar image 986dotUpdated: 1 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Reuse
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                m
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                mlscraperby lorey
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                🤖 Scrape data from HTML websites automatically by just providing examples
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Pythondotstar image 975dotUpdated: 1 y agodotno licences License: No License (No License)
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Support
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Quality
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    Security
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      License
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Reuse